GPU Failures

Forums: GPU Failures
A public forum for discussing and asking questions about the Octane for Unity Alpha

Moderator: ChrisHekman

GPU Failures

Postby helloaifi » Tue Jan 30, 2018 10:08 pm

helloaifi Tue Jan 30, 2018 10:08 pm
I am getting this issue on a very regular basis and am running out of ideas to resolve it. My drivers are up to date, so I don't think this is an issue. I am receiving the GPU failed error upon rendering at times that are difficult to predict. This seems to cause more issues when using multiple unity materials on the same object, but I cannot tell if this is the cause.

Here are my specs:

Processor: Intel® Core™ i7-6850K CPU @ 3.60GHz
NVIDIA GeForce GTX 1080 Ti (x4)
Memory: 128 GB
OS: Windows 10
helloaifi
 
Posts: 3
Joined: Mon Jan 29, 2018 9:34 pm

Re: GPU Failures

Postby ChrisHekman » Fri Feb 02, 2018 10:13 am

ChrisHekman Fri Feb 02, 2018 10:13 am
Do you have errors? Could you share the octane log? (Go to octane->settings->octane log)

As this is diffucult to debug from our end. Would it be possible for you to share your scene?
ChrisHekman
OctaneRender Team
OctaneRender Team
 
Posts: 968
Joined: Wed Jan 18, 2017 3:09 pm

Re: GPU Failures

Postby helloaifi » Sat Feb 10, 2018 3:24 am

helloaifi Sat Feb 10, 2018 3:24 am
I am getting the following errors:

CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to deallocate device memory
CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to unload module
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to bind device to current thread
device 2: failed to initialize resources
helloaifi
 
Posts: 3
Joined: Mon Jan 29, 2018 9:34 pm

Re: GPU Failures

Postby fis715 » Thu Jan 10, 2019 9:02 pm

fis715 Thu Jan 10, 2019 9:02 pm
im Having the same problem and im up to date, in fact, my computer updated yesterday Octane or Unity need to return my money im done I'm going back to c4d for my animations I find Unity is not worth the headache
fis715
Licensed Customer
Licensed Customer
 
Posts: 7
Joined: Thu Nov 02, 2017 4:21 am

Re: GPU Failures

Postby Lewis » Thu Jan 10, 2019 9:20 pm

Lewis Thu Jan 10, 2019 9:20 pm
I had/have my fair share of those errors in octane 4.x so i can sympathize.

In general i never had so much crashes/render failures in octane v 2.x or 3.x as of V4. I wish OTOY spends more time on polishing v4 instead rushing in next version.

Also i think there should be officially supported/tested drivers so we have certified drivers that work at least 99% sure/stable with Octane. This constant driver changing (win10 updates/nvidia updates) is really really hard to track/maintain stability and we are then having tons of reports/problems for DEVs trying to solve is it driver issue or hardware issue or octane issue (and by that we are wasting precious time for DEVs trying to get answer for each individual PC/window driver for each report).

Would be good to make official test of 4xx.xx driver that really works best with octane and set that as "recommended" driver for users, then we will know is it Octane BUG or just nvidia driver problems.

But yes i get those "..an illegal memory access ...." really easy in octane v4.x and it is NOT hardware issue 'coz it happens on all 4 of my GPUs occasionally and they all are working fine, brand new PSU, 2 new GPus, 2 older GPUs but all are fully functional GPUs and I even wen that far and bought one of most expensive PSUs and most precise PSU components currently available in PSU market (Corsair AX1600). I tested GPUs intensive stuff and all go fine but more importantly they work fine in 3.08.5 octane without those memory issues so it has to be Octane v4 problem or Octane v4 + nvidia driver combo.

Please Otoy step back from BETA 2018.x, 2019,x versions and put full power into stabilizing v4 final first.

Thanks
--
Lewis
http://www.ram-studio.hr
Skype - lewis3d
ICQ - 7128177

WS AMD TRPro 3955WX, 256GB RAM, Win10, 2 * RTX 4090, 1 * RTX 3090
RS1 i7 9800X, 64GB RAM, Win10, 3 * RTX 3090
RS2 i7 6850K, 64GB RAM, Win10, 2 * RTX 4090
User avatar
Lewis
Licensed Customer
Licensed Customer
 
Posts: 1068
Joined: Tue Feb 05, 2013 6:30 pm
Location: Croatia

Re: GPU Failures

Postby fis715 » Fri Jan 11, 2019 1:20 am

fis715 Fri Jan 11, 2019 1:20 am
YES VERY WELL PUT THANK GOD 4 YOU.
fis715
Licensed Customer
Licensed Customer
 
Posts: 7
Joined: Thu Nov 02, 2017 4:21 am

Re: GPU Failures

Postby ChrisHekman » Fri Jan 11, 2019 10:35 am

ChrisHekman Fri Jan 11, 2019 10:35 am
fis715 wrote:im Having the same problem and im up to date, in fact, my computer updated yesterday Octane or Unity need to return my money im done I'm going back to c4d for my animations I find Unity is not worth the headache


Would it be possible for you to share your scene that has this crash so we can find out what is going on?
ChrisHekman
OctaneRender Team
OctaneRender Team
 
Posts: 968
Joined: Wed Jan 18, 2017 3:09 pm

Re: GPU Failures

Postby Lewis » Mon Jan 14, 2019 2:00 pm

Lewis Mon Jan 14, 2019 2:00 pm
Hi guys,

It's not scene related at all, it's just v4.xxx that has some nasty bug somewhere :(

I just had that memory error again on scene where i'm working/rendering for last 2 days but now suddenly frame 1 rendered fine and then frame 2 it got crash/illegal memory erorr (which i rendered fine previously).


00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 2: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 2: path tracing kernel failed
00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 1: path tracing kernel failed
00:12:14 (0734.66) | ... Finish Rendering ...
00:12:14 (0734.66) |
00:12:14 (0734.66) | ... reset image callback ...
00:12:14 (0734.66) | ... wait for the getPreviewImage function ...
00:12:14 (0734.67) * OCTANE API MSG: CUDA error 700 on device 3: an illegal memory access was encountered
00:12:14 (0734.67) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.67) * OCTANE API MSG: device 3: path tracing kernel failed
00:12:14 (0734.76) | Close and free scene, free buffer: 1, reset scene: 1
00:12:14 (0734.76) | ... freeBuffers
00:12:14 (0734.76) | ... setRenderTargetNode(NULL)
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
--
Lewis
http://www.ram-studio.hr
Skype - lewis3d
ICQ - 7128177

WS AMD TRPro 3955WX, 256GB RAM, Win10, 2 * RTX 4090, 1 * RTX 3090
RS1 i7 9800X, 64GB RAM, Win10, 3 * RTX 3090
RS2 i7 6850K, 64GB RAM, Win10, 2 * RTX 4090
User avatar
Lewis
Licensed Customer
Licensed Customer
 
Posts: 1068
Joined: Tue Feb 05, 2013 6:30 pm
Location: Croatia

Re: GPU Failures

Postby ChrisHekman » Mon Jan 14, 2019 2:32 pm

ChrisHekman Mon Jan 14, 2019 2:32 pm
Lewis wrote:Hi guys,

It's not scene related at all, it's just v4.xxx that has some nasty bug somewhere :(

I just had that memory error again on scene where i'm working/rendering for last 2 days but now suddenly frame 1 rendered fine and then frame 2 it got crash/illegal memory erorr (which i rendered fine previously).


00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 2: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 2: path tracing kernel failed
00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 1: path tracing kernel failed
00:12:14 (0734.66) | ... Finish Rendering ...
00:12:14 (0734.66) |
00:12:14 (0734.66) | ... reset image callback ...
00:12:14 (0734.66) | ... wait for the getPreviewImage function ...
00:12:14 (0734.67) * OCTANE API MSG: CUDA error 700 on device 3: an illegal memory access was encountered
00:12:14 (0734.67) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.67) * OCTANE API MSG: device 3: path tracing kernel failed
00:12:14 (0734.76) | Close and free scene, free buffer: 1, reset scene: 1
00:12:14 (0734.76) | ... freeBuffers
00:12:14 (0734.76) | ... setRenderTargetNode(NULL)
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info


This is for v4 in general? Also for standalone?
Could you export the unity scene as an orbx and load it in standalone and see if the error persists?
ChrisHekman
OctaneRender Team
OctaneRender Team
 
Posts: 968
Joined: Wed Jan 18, 2017 3:09 pm

Re: GPU Failures

Postby Lewis » Mon Jan 14, 2019 2:37 pm

Lewis Mon Jan 14, 2019 2:37 pm
ChrisHekman wrote:
Lewis wrote:Hi guys,

This is for v4 in general? Also for standalone?
Could you export the unity scene as an orbx and load it in standalone and see if the error persists?


Hi Chris

Sorry i should explain it better, this is not really Unity only related, happens to me in LW plugin and in standalone. I think it's something in general octane core (i can't make 100% reproducible scene, sometime happens 5 times a day sometime works 5 days no problem) what triers that error (as you can see on forums there is several of users from various apps/dcc plugins have that error so it's not really strictly to unity only.

Sorry one more time for confusion thinking it's unity-Octane only issue.
--
Lewis
http://www.ram-studio.hr
Skype - lewis3d
ICQ - 7128177

WS AMD TRPro 3955WX, 256GB RAM, Win10, 2 * RTX 4090, 1 * RTX 3090
RS1 i7 9800X, 64GB RAM, Win10, 3 * RTX 3090
RS2 i7 6850K, 64GB RAM, Win10, 2 * RTX 4090
User avatar
Lewis
Licensed Customer
Licensed Customer
 
Posts: 1068
Joined: Tue Feb 05, 2013 6:30 pm
Location: Croatia
Next

Return to Octane for Unity


Who is online

Users browsing this forum: No registered users and 4 guests

Thu Mar 28, 2024 9:22 am [ UTC ]