Network Render Stopped Working Tweet

[laszlosomogyi]

Hi All,

I'm hoping someone can help,

I've been working with a master computer (Windows 10 up to date, C4D R17 up to date, Octane 3.08 up to date, 2x 1080Ti FTW3 drivers up to date) and a slave (Windows 10 up to date, Octane Slave Daemon 3.08 up to date, 4x 980Ti FTW2 drivers up to date) for over a year now without a single problem.

Today the slave started giving me the following error:

CUDA error 2 on device 3: out of memory
CUDA error 2 on device 0: out of memory
CUDA error 2 on device 2: out of memory
-> failed to launch kernel(kernel31)
device 3: path tracing kernel failed
-> failed to launch kernel(kernel48_tex)
device 2: path tracing kernel failed
Failed to allocate page-locked memory
-> kernel execution failed(kernel25)
CUDA error 2 on device 1: out of memory
-> failed to deallocate device memory
CUDA error 2 on device 1: out of memory
-> could not get memory info
device 1: failed to upload data texture 28 of context 0
CUDA error 2 on device 0: out of memory
-> failed to launch kernel(kernel26)
device 0: path tracing kernel failed

Now before someone states the obvious that I'm out of memory, I don't think that is the main problem as I've been using this setup for over a year on very similar projects with no problems at all. (Exterior scene with a single building, single light source bit of grass and a few trees)

What seems suspicious to me is the "could not get memory info" message. How is this even possible?

Please help as I'm on a deadline and without the slave being fully operational I won't be able to deliver the project that I have already spent a week on...

Your help would be much appreciated,

Many thanks in advance,

Laszlo

[aoktar]

Possibilities
1- problems with driver
2- corruption on gpus
3- out of memory

Solutions
1-Fix drivers or OS
2-Check with single gpu for testing
3-Check with simplier scenes

[laszlosomogyi]

Hi Aoktar,

Thank you for your quick reply,

Will do the checks and get back to you about my findings,

Many thanks,