Nvidia OpenGL driver Error code 3 - render crash!

[slepy8]

Yo!

This is the tough one.

I have a pretty complex scene. Lots of objects, and generators.
There is a yacht, the water underneath, sky hdri, daylight and some more stuff.

After running rendering, it goes for a few secconds than it slows down, screen becomes black and I get "Nvidia OpenGL driver. Error code 3".

I have 1X 780Ti (Zotac AMP! edition) and 2X 660 (company OC).

So I tried different things.
1. render with less objects turned on for rendering - worked.
2. collapsed all objects yacht is built of into one object - worked (I could even turn on more objects - like for example an island behind the scene).
3. After reading some stuff on the web I turned off 660's from system and downclocked (!) 780 to it's basic clocking values - worked for a few times. Sometimes it crashed tho.
4. Turned on all GPUs and downclocked all of them - didn't work

With all GPUs on, I still have 820Mb VRAM free. (660 are 2GB so the scene is using about 1GB VRAM).
computer RAM is not overloaded during scene export - RAM reaches 10GB during export (out of 16GB I have installed).

So this is entirely Octane problem or GPUs problem or some system problem.

Do you have any ideas?

PS: I can't give a scene for testing unfortunatelly :/

[slepy8]

seams I figured out whats causing the crash.
It's a water cloner. I've set up 300 X 300 clones on a gigantic area of 24X24 kilometers..

In fact if I turn of other elements and keep only water - it renders.
So I'll just have to make it in more passes for the animation...

But here's the question to the Otoy team: Is this a normal software/hardware limitation and this means I'm a crazy moron trying to overload everything? Or maybe is it some sort of a bug?

[augdalmo]

I have encountered a similar problem, the error reported is the same. I have tried to export the scene in standalone but also this crashed, the error was the same.
I have discovered the problem was in some object with Hypernurbs applied. Probably the number of polygons had exceeded the maximum...
The solution in my case was to don't apply Hypernurbs...

[aoktar]

slepy,
converted your scene to standalone. after 500 samples, it locks my comp. and have to reset system.
I'll give it to Octane team to investigate.

[slepy8]

Thank you Aoktar. I'm looking forward for the info what's wrong.

[slepy8]

It's getting more crazy....

--------------------------- EXPORT LOG ---------------------------
Export materials time= 187.688 ms
Collect objects time= 28.441 ms
Meshes creation time = 2681.547 ms
Octane voxelization time = 0 ms
Sending to Octane engine time = 27.334 ms
Total export Time = 8085.857 ms
CUDA error 999 on device 2: An unknown internal error occurred.
-> Kernel execution failed (ctonemap)
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to copy 2D memory from device
CUDA device 2: Failed to download tone map result
CUDA device 2: Failed to run tone mapping
<<< Render Failure >>>
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device array
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to deallocate device memory
CUDA error 999 on device 2: An unknown internal error occurred.
-> Could not get memory info
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to unload module
CUDA error 999 on device 1: An unknown internal error occurred.
-> Failed to bind device to current thread
CUDA device 1: Failed to initialize resources
CUDA error 999 on device 2: An unknown internal error occurred.
-> Failed to bind device to current thread
CUDA device 2: Failed to initialize resources
CUDA error 999 on device 0: An unknown internal error occurred.
-> Failed to bind device to current thread
CUDA device 0: Failed to initialize resources
Tried to access pin via an invalid ID P_GEOMETRY (52)
Tried to access null node pin
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
NO LINK 12
Tried to access pin via an invalid ID P_GEOMETRY (52)
Tried to access null node pin
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
NO LINK 9
Tried to access pin via an invalid ID P_POSITION_TYPE (115)
Tried to access null node pin
Can't write int to static pin 'unknown' (P_UNKNOWN, type PT_UNKNOWN) of node '' (NT_UNKNOWN, 000000000D2818B0)
Tried to access pin via an invalid ID P_POSITION_TYPE (115)
Tried to access null node pin
Can't write int to static pin 'unknown' (P_UNKNOWN, type PT_UNKNOWN) of node '' (NT_UNKNOWN, 000000000D2818B0)

Is something wrong with this god damn scene????
with 780Ti only it renders.
When I run it with 2 X 660 it gives Octane Render failure and the quoted message in the log...

What is wrong with this renderer that I can't render a simple scene??!!

[aoktar]

memory problem i think, hide some part of objects and check it. also disable log output, you don't need this output

[slepy8]

Man! this scene is using less than 400mb VRAM!!

What should I hide? The whole scene??

Run it. Check it. It's simple small forest scene with fog.

Aoktar, I'm geting confused with all those errors all the time. I'ts not working as it should after that long time of development!

Wee need to work, not strugle with errors...

[aoktar]

a tree with one million faces, is this a simple scene? my laptop creeps totally in viewport, but octane does not gives any error. If there is a error, this is not a plugin issue. try with simple meshes to see different.
If you get any Render Failure errors, these are mostly not a plugin problem. Cuda errors. So please don't blame me. I can work much more complex scenes without any problem. I think you have some HW problems.

Check this my workflow video with hundreds complex objects
www.aoktar.com/temp/workflow.mp4

[slepy8]

oh! So know I know there is a tree faces limitation in Octane?

HW problem would be possible if I always got errors while using all 3 cards.
But I get errors only while using heavy cloners.
When I decrease thic cloner here to 30 clones it works, but with 50 clones it doesn't.

Don't you see there is some sort of Octane problem when calculating many light bounces??