Page 1 of 2

Crashes

Posted: Thu Jul 21, 2016 9:00 pm
by coilbook
After rendering for 12 hours animation sequence with 5 video cards only 2 continued rendering other ones stopped (netstor Turbo box we use 2 cards quit rendering and 2 continued )_. Then when I clicked cancel rendering 3ds max just closed with crash error . Version 2 never had this problem

Re: Crashes

Posted: Fri Jul 22, 2016 1:21 am
by coilbook
ok. While rendering if one of the slaves dies and rendering stop where it is almost finished with the frame if you hit cancel max just crashes and closes. In version 2 we could just cancel frozen render and restart

Still lots of Cuda 719 errors that we never had in version 2

Re: Crashes

Posted: Fri Jul 22, 2016 3:52 am
by azen
Hi,

It sounds like these could be recently introduced bugs from the standalone.

To clarify:

- What are your hardware specs for the slave and master machines? (interested in the GPUs in particular, to confirm whether the CUDA 8.0 RC issues could be behind this).
- Are you doing the network rendering from 3ds Max or from the standalone?

Re: Crashes

Posted: Fri Jul 22, 2016 10:34 am
by Lewis
Azen

Hi

I had/have similar experience with OR 3.0 plugin. Sometime it crashes single GPU and just throws Error message and rendering stops.

Why can't it Continue rendering without that GPU or continue with decreased speed (on my machine usually when it happens GPU just goes to slow model with decreased speed to 400-500 MHz and i cant' bring it back to full speed unless i reboot machine but if i start new render process it still renders on that GPU, just lot slower due smaller clock but GPU obviously still works).

We already have algorith to loose smaples when we uncheck some GPU form the IPR so why cant' that be made work that way from final render also ?

IF any of GPUs fail ok, just loose those samples and continue rendering until last GPU is rendering (last man standing ;)) but don't drop complete rendering due 1 , 2, 3.. gpu failure/bug/glitch. That's very problematic for production. When i had 5 GPUs and i leave 5 renders overnight to be done (like 2h/frame) and i get up in the morning and i realize i have NOTHING 'coz for some reason GPU 1 (or 2 or 3 doesn't matter 'coz hardware is OK and render in 2.25 fine) reported CUDA 7xx error and ALL stopped rendering :(.

I must say that after introduction of Tonemap option i have much more unstable Octane 3.x and i never had BSOD with 2.25 while in 3.0 and tonemap combo+extenders BSOD is very easy to get :(.

Thanks

Re: Crashes

Posted: Fri Jul 22, 2016 8:26 pm
by coilbook
azen wrote:Hi,

It sounds like these could be recently introduced bugs from the standalone.

To clarify:

- What are your hardware specs for the slave and master machines? (interested in the GPUs in particular, to confirm whether the CUDA 8.0 RC issues could be behind this).
- Are you doing the network rendering from 3ds Max or from the standalone?

Hi We got main PC with two old titans and netstor box with gtx 780tis. Then we got slave with two 780 tis. We only use 3ds max 2015 and octane for 3ds max. We have 362 drivers since newer ones give us BSOD. We use windows 7 ultimate.

Our rendering is for animation AO 1080P. I think most problems are related to scattering medium in our scenes (we use it for light fog effect). Seems like it slows down system a lot and crashes more often with it.

I attached a video where you can see it stopped rendering and I hit cancel and max just crashes so I cannot look up log to see what the problem is. (Sorry video is upside down )

Re: Crashes

Posted: Fri Jul 22, 2016 8:36 pm
by coilbook
looks like this error slaves were getting with version 3 I never had this in version 2

Re: Crashes

Posted: Fri Jul 22, 2016 8:44 pm
by coilbook
Here is recent slave error

Re: Crashes

Posted: Fri Jul 22, 2016 8:59 pm
by coilbook
Lewis wrote:Azen

Hi

I had/have similar experience with OR 3.0 plugin. Sometime it crashes single GPU and just throws Error message and rendering stops.

Why can't it Continue rendering without that GPU or continue with decreased speed (on my machine usually when it happens GPU just goes to slow model with decreased speed to 400-500 MHz and i cant' bring it back to full speed unless i reboot machine but if i start new render process it still renders on that GPU, just lot slower due smaller clock but GPU obviously still works).

We already have algorith to loose smaples when we uncheck some GPU form the IPR so why cant' that be made work that way from final render also ?

IF any of GPUs fail ok, just loose those samples and continue rendering until last GPU is rendering (last man standing ;)) but don't drop complete rendering due 1 , 2, 3.. gpu failure/bug/glitch. That's very problematic for production. When i had 5 GPUs and i leave 5 renders overnight to be done (like 2h/frame) and i get up in the morning and i realize i have NOTHING 'coz for some reason GPU 1 (or 2 or 3 doesn't matter 'coz hardware is OK and render in 2.25 fine) reported CUDA 7xx error and ALL stopped rendering :(.

I must say that after introduction of Tonemap option i have much more unstable Octane 3.x and i never had BSOD with 2.25 while in 3.0 and tonemap combo+extenders BSOD is very easy to get :(.

Thanks
same with gpu speeds.if i open octane preview and mess with medium like daylight scattering (delete it and then undelete it or values) gpu speed will drop. and then when i start rendering it will render at 5 min per frame instead of 2 min per frame like it did just because i did some changes with octane viewport open. Only solution is to restart max or PC

Re: Crashes

Posted: Sat Jul 23, 2016 2:52 pm
by coilbook
seems like if it is rendering for 3-4 hours and you hot cancel it crashes max all the time with octane version 3

Re: Crashes

Posted: Sun Jul 24, 2016 10:59 pm
by azen
Hi,

Thank you for the feedback. I am looking through the materials you provided and seeking to duplicate the issue.