Page 1 of 2

GPU Failures

PostPosted: Tue Jan 30, 2018 10:08 pm
by helloaifi
I am getting this issue on a very regular basis and am running out of ideas to resolve it. My drivers are up to date, so I don't think this is an issue. I am receiving the GPU failed error upon rendering at times that are difficult to predict. This seems to cause more issues when using multiple unity materials on the same object, but I cannot tell if this is the cause.

Here are my specs:

Processor: Intel® Core™ i7-6850K CPU @ 3.60GHz
NVIDIA GeForce GTX 1080 Ti (x4)
Memory: 128 GB
OS: Windows 10

Re: GPU Failures

PostPosted: Fri Feb 02, 2018 10:13 am
by ChrisHekman
Do you have errors? Could you share the octane log? (Go to octane->settings->octane log)

As this is diffucult to debug from our end. Would it be possible for you to share your scene?

Re: GPU Failures

PostPosted: Sat Feb 10, 2018 3:24 am
by helloaifi
I am getting the following errors:

CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to deallocate device memory
CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to unload module
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to bind device to current thread
device 2: failed to initialize resources

Re: GPU Failures

PostPosted: Thu Jan 10, 2019 9:02 pm
by fis715
im Having the same problem and im up to date, in fact, my computer updated yesterday Octane or Unity need to return my money im done I'm going back to c4d for my animations I find Unity is not worth the headache

Re: GPU Failures

PostPosted: Thu Jan 10, 2019 9:20 pm
by Lewis
I had/have my fair share of those errors in octane 4.x so i can sympathize.

In general i never had so much crashes/render failures in octane v 2.x or 3.x as of V4. I wish OTOY spends more time on polishing v4 instead rushing in next version.

Also i think there should be officially supported/tested drivers so we have certified drivers that work at least 99% sure/stable with Octane. This constant driver changing (win10 updates/nvidia updates) is really really hard to track/maintain stability and we are then having tons of reports/problems for DEVs trying to solve is it driver issue or hardware issue or octane issue (and by that we are wasting precious time for DEVs trying to get answer for each individual PC/window driver for each report).

Would be good to make official test of 4xx.xx driver that really works best with octane and set that as "recommended" driver for users, then we will know is it Octane BUG or just nvidia driver problems.

But yes i get those "..an illegal memory access ...." really easy in octane v4.x and it is NOT hardware issue 'coz it happens on all 4 of my GPUs occasionally and they all are working fine, brand new PSU, 2 new GPus, 2 older GPUs but all are fully functional GPUs and I even wen that far and bought one of most expensive PSUs and most precise PSU components currently available in PSU market (Corsair AX1600). I tested GPUs intensive stuff and all go fine but more importantly they work fine in 3.08.5 octane without those memory issues so it has to be Octane v4 problem or Octane v4 + nvidia driver combo.

Please Otoy step back from BETA 2018.x, 2019,x versions and put full power into stabilizing v4 final first.

Thanks

Re: GPU Failures

PostPosted: Fri Jan 11, 2019 1:20 am
by fis715
YES VERY WELL PUT THANK GOD 4 YOU.

Re: GPU Failures

PostPosted: Fri Jan 11, 2019 10:35 am
by ChrisHekman
fis715 wrote:im Having the same problem and im up to date, in fact, my computer updated yesterday Octane or Unity need to return my money im done I'm going back to c4d for my animations I find Unity is not worth the headache


Would it be possible for you to share your scene that has this crash so we can find out what is going on?

Re: GPU Failures

PostPosted: Mon Jan 14, 2019 2:00 pm
by Lewis
Hi guys,

It's not scene related at all, it's just v4.xxx that has some nasty bug somewhere :(

I just had that memory error again on scene where i'm working/rendering for last 2 days but now suddenly frame 1 rendered fine and then frame 2 it got crash/illegal memory erorr (which i rendered fine previously).


00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 2: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 2: path tracing kernel failed
00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 1: path tracing kernel failed
00:12:14 (0734.66) | ... Finish Rendering ...
00:12:14 (0734.66) |
00:12:14 (0734.66) | ... reset image callback ...
00:12:14 (0734.66) | ... wait for the getPreviewImage function ...
00:12:14 (0734.67) * OCTANE API MSG: CUDA error 700 on device 3: an illegal memory access was encountered
00:12:14 (0734.67) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.67) * OCTANE API MSG: device 3: path tracing kernel failed
00:12:14 (0734.76) | Close and free scene, free buffer: 1, reset scene: 1
00:12:14 (0734.76) | ... freeBuffers
00:12:14 (0734.76) | ... setRenderTargetNode(NULL)
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info

Re: GPU Failures

PostPosted: Mon Jan 14, 2019 2:32 pm
by ChrisHekman
Lewis wrote:Hi guys,

It's not scene related at all, it's just v4.xxx that has some nasty bug somewhere :(

I just had that memory error again on scene where i'm working/rendering for last 2 days but now suddenly frame 1 rendered fine and then frame 2 it got crash/illegal memory erorr (which i rendered fine previously).


00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 2: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 2: path tracing kernel failed
00:12:14 (0734.65) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.65) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.65) * OCTANE API MSG: device 1: path tracing kernel failed
00:12:14 (0734.66) | ... Finish Rendering ...
00:12:14 (0734.66) |
00:12:14 (0734.66) | ... reset image callback ...
00:12:14 (0734.66) | ... wait for the getPreviewImage function ...
00:12:14 (0734.67) * OCTANE API MSG: CUDA error 700 on device 3: an illegal memory access was encountered
00:12:14 (0734.67) * OCTANE API MSG: -> failed to wait for event
00:12:14 (0734.67) * OCTANE API MSG: device 3: path tracing kernel failed
00:12:14 (0734.76) | Close and free scene, free buffer: 1, reset scene: 1
00:12:14 (0734.76) | ... freeBuffers
00:12:14 (0734.76) | ... setRenderTargetNode(NULL)
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> failed to unregister pinned memory
00:12:14 (0734.76) * OCTANE API MSG: CUDA error 700 on device 1: an illegal memory access was encountered
00:12:14 (0734.76) * OCTANE API MSG: -> could not get memory info


This is for v4 in general? Also for standalone?
Could you export the unity scene as an orbx and load it in standalone and see if the error persists?

Re: GPU Failures

PostPosted: Mon Jan 14, 2019 2:37 pm
by Lewis
ChrisHekman wrote:
Lewis wrote:Hi guys,

This is for v4 in general? Also for standalone?
Could you export the unity scene as an orbx and load it in standalone and see if the error persists?


Hi Chris

Sorry i should explain it better, this is not really Unity only related, happens to me in LW plugin and in standalone. I think it's something in general octane core (i can't make 100% reproducible scene, sometime happens 5 times a day sometime works 5 days no problem) what triers that error (as you can see on forums there is several of users from various apps/dcc plugins have that error so it's not really strictly to unity only.

Sorry one more time for confusion thinking it's unity-Octane only issue.