Page 1 of 1

CUDA error 700 on third GPU

Posted: Fri Jan 13, 2017 10:45 pm
by jejeaudio
Hello,

I have 3xGTX1080 installed in my case.
Cinema4D R16 and latest octane version.

I have the third card wich make the render fail.
With only 2 cards, everything works perfect, but as soon as I ask the third to render with the others, octane make a cuda error 700.

I have tried on very simples scenes, and It works with the 3x GPU.

If I try to render with only the third, It don't works, even simple scenes.
Here is the log:

"
Scene created in plugin version -1
Export materials time= 21.135 ms
Collect objects time= 53.166 ms
Mesh creation time = 41.373 ms.
Triangles:0 Disp.triangles:0 Hairs:0 Meshes:0
VRAM used/free/max:0Kb/0Kb/0Kb
Out-of-core used:0Kb total used RAM:6.968Gb
Total export Time = 143.463 ms
CUDA error 700 on device 2: an illegal memory access was encountered
-> kernel execution failed(report)
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to launch kernel(ptBrdf2)
device 2: direct light kernel failed
device 2: detected an error on render device! trying to recover...
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to deallocate device memory
CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to deallocate device memory
CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> could not get memory info
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to unload module
CUDA error 700 on device 2: an illegal memory access was encountered
-> failed to bind device to current thread
device 2: failed to initialize resources
device 2: failed to recover - giving up and switching device to failed
----------------------------------------------------------------------------------------
<<< Render failure detected!!! >>>
Please check render statistics to solve the problem.
Triangles:12 Disp.triangles:0 Hairs:0 Meshes:1
Device:0 TotMem:8Gb rtData:0Kb film:0Kb geo:0Kb node:0Kb tex:0Kb unavailable:0Kb temperature:44
Device:1 TotMem:8Gb rtData:0Kb film:0Kb geo:0Kb node:0Kb tex:0Kb unavailable:0Kb temperature:44
Device:2 TotMem:8Gb rtData:0Kb film:0Kb geo:0Kb node:0Kb tex:0Kb unavailable:964Mb temperature:50
"


Any idea ?
Is it broken ?

Re: CUDA error 700 on third GPU

Posted: Fri Jan 13, 2017 11:28 pm
by aoktar
On last days i've seen a lot of similar cuda errors. And mostly on 1080s. See another posts. I'll point this topic to core devs.
Btw what's your brand/models? Links for cards? Maybe overclocked?

Re: CUDA error 700 on third GPU

Posted: Sat Jan 14, 2017 7:40 am
by jejeaudio
Gigabyte G1 gaming.

They are not overcloked, they are slightly downclocked.


But figure what: I've just tried to make the third gpu perfectly stock (no over or down clock, power at 100%)
-> it works... (for now)

I'll let you know if something change !

Re: CUDA error 700 on third GPU

Posted: Sat Jan 14, 2017 1:28 pm
by aoktar
jejeaudio wrote:Gigabyte G1 gaming.

They are not overcloked, they are slightly downclocked.


But figure what: I've just tried to make the third gpu perfectly stock (no over or down clock, power at 100%)
-> it works... (for now)

I'll let you know if something change !
That's great!
I think Cuda is much more sensitive to errors than game engines.

Re: CUDA error 700 on third GPU

Posted: Mon Apr 03, 2017 6:56 pm
by Gwrath
I have also been getting this error suddenly as well as error 999, 799, and a few others, I have everything from 1080s to 1060s 980ti and 980 machines getting these errors. I get them on something as simple as a new scene with a box.

Any help is greatly appreciated as I can't render anything on my farm and have a project due soon.

I am on windows 10 on all computers involved, geforce driver 378.92 ( as well as tried several older drivers, all had same issue)

thanks

Re: CUDA error 700 on third GPU

Posted: Tue Apr 04, 2017 5:36 am
by slepy8
In my opinion Cuda error 700 could be related to a faulty card.

I had them too. With multiple cards setups.
Finally my system works stable without issues, after I threw away 780Ti.
My system is now 2 X 980Ti and a 780 6GB. So I'm mixing Kepler and MAxwell (it's been said that it's not good to mix different architectures).

Now I can run thousands of frames over night without any problem.

Also do not overclock or downclock. Use MSI Afterburner and set everything at stock level.