2 cards good, 3 cards bad... Tweet

[kopperdrake]

I have a 64Gb system running Windows 10, with one 3080 Ti in it. I then have an external GPU rig (Amfeltec Corp) with two more 3080 Ti cards on.

I'm rendering a pretty hefty scene, and was finding the PC would stop rendering, either through a crash or just stop. Raising the out-of-core values to 15Gb RAM usage limit and 1024MB GPU head room seemed to stop that, although I'm not really sure what they do (I assume they're reasnoably self-explanatory - RAM usage limit means Octane can only take that 15Gb maximum from the system RAM, and GPU head room leaves 1024Mb in the card free, for other stuff?)

Regardless, the scene was taking about 8.5 minutes to render a frame with all three GPUS active. But turning OFF the GPU in the PC actually had a frame rendering in 7 minutes!

Any pointers would be really useful! I have quite a few more frames to render and a speed increase would be so helpful!

If it makes any difference, the third GPU on the GPU rig replaced a 2080 Ti, and I made no changes to studio drivers.

[kopperdrake]

Update,

I turned the monitors off, went to bed, woke up and the frame render time had dropped to 4m 40s - pretty much what I'd hoped to expect!

So what's happening? Why the initial high render times? Is just 'not using' the control PC causing some sort of rebalance?

[NemesisCGI]

The problem is the speed the external cards send & receive data. The bottleneck is the cable. It's only a x1
Also, don't use those cards for tone mapping or denoising, this will slow things down too.

[Lewis]

Another thing what could it be is OutOfCore RAM usage, since your PC GPU is connected to monitors it has less free VRAM (windows and just on monitor eats about 350-400 MB and more resolution/monitors and layout OGL itself it can easily go to 2 GB Vram usage which is not then available for Octane and then the more OCC render uses the slower render it gets.

Margin of slowdown with increased OCC is very, very big.
I tested this few years ago and after 30-40% of OCC render can slow down like 5x or more...

So in your case those GPUs in external have more FREE VRAM, so it uses less OCC, and thus it can almost be faster than using all 3 GPUs since then OCC is bottlenecking all GPUs.


check my results with OCC:
Test 1 (4*2080 Ti) - Denoiser ON, Parallel samples=16, Max Tile samples=32
_____________________
GPU headroom = 300 MB
Out of Core = 0 MB
Rendertime = 1m 8sec (68sec)


Test 2 (4*2080 Ti) - Denoiser ON, Parallel samples=32, Max Tile samples=64
_____________________
GPU headroom = 300 MB
Out of Core = 768 MB
Rendertime = 1m 18sec (78sec) = 15% slower


Test 3 (4*2080 Ti) - Denoiser ON, Parallel samples=32, Max Tile samples=64
_____________________
GPU headroom = 512 MB
Out of Core = 1024 MB
Rendertime = 1m 32sec (92sec) = 35% slower


Test 4 (4*2080 Ti) - Denoiser ON, Parallel samples=32, Max Tile samples=64
_____________________
GPU headroom = 1024 MB
Out of Core = 1792 MB
Rendertime = 2m 04sec (124 sec) = 82% slower


Test 5 (4*2080 Ti) - Denoiser ON, Parallel samples=32, Max Tile samples=64
_____________________
GPU headroom = 2048 MB
Out of Core = 2816 MB
Rendertime = 2m 57sec (177 sec) = 160% slower


Test 6 (4*2080 Ti) - Denoiser ON, Parallel samples=32, Max Tile samples=64
_____________________
GPU headroom = 4096 MB
Out of Core = 4989 MB
Rendertime = 16m 03sec (963 sec) = 1315% slower