OTOY Forums

Hello, I would like to use more than 21GPUs with Nuke, to reduce as much as possible the render times.

I would linke to use Tesla P100 or V100 when available.

I'm not sure if it is there the limit of 20/21 GPUs, If so:

1- Is there any plan to increase that limit?
2- Is there any technical reason why that limit is there?
3- Is there any workaround to avoid the limit?

Thanks

mrfactory wrote:Hello, I would like to use more than 21GPUs with Nuke, to reduce as much as possible the render times.

I would linke to use Tesla P100 or V100 when available.

I'm not sure if it is there the limit of 20/21 GPUs, If so:

1- Is there any plan to increase that limit?
2- Is there any technical reason why that limit is there?
3- Is there any workaround to avoid the limit?

Thanks

It's the same since V3 was released. The limit is there for support reasons. Network rendering is one of the more costly support issues we handle daily, and minimizing the issues that can go wrong is really important (it used to be 12 in V2, and maybe will be different after V3, but too early to say).

We do have a white glove enterprise version of V3 with up to 100+ GPU supported (with HW dongle) but that requires Linux + 10GBE+ (pretty much a VCA type box). You can contact sales if this is really what you need. However, once we support Volta in 3.09, I would guess that 20x V100, you will have about as much rendering power as you can get short of going to a DC with proper cooling.

Thanks Goldorak , that's are very good news for me.

We are planing to render a whole movie in 2019 with virtual sets and matte painting...., with octane render.

We have a realtime PreViz workflow with UNREAL (the Octane Render plugin for UNREAL is essential for us, to avoid the baking process), and we want to do the final render of the same scene we did the pre-visualization with UNREAL, with NUKE, thus we need to speed up the render times as much as we can.

----------------------

Sample Test:: http://212.123.219.101/file/vfx/IMG_3792.mp4

These are the very firsts tests of realtime tracking and chromakey with an Octane Render scene baked in UNREAL, we are going to get much more precission with the realtime trackers, and much better 4K chroma key, this is just the very first a test.

We record in realtime in 4K from camera trigger, everything in sync and TimeCode (from RED, Alexa, Blackmagic...cameras):

Reference Composition: http://212.123.219.101/file/vfx/CLIP0000343_B.mp4
FG: http://212.123.219.101/file/vfx/CLIP0000343_A.mp4
Matte: http://212.123.219.101/file/vfx/CLIP0000343_C.mp4
BG:http://212.123.219.101/file/vfx/CLIP0000343_D.mp4
and also we record all the data of the realtime camera tracking with zoom and focus (depth of field) every thing sycronized and with TimeCode,.

These 5 pieces are record in a folder as a package is for pre-edit purposes, since the final composition is going to be rendered with NUKE, using the realtime FBX camera tracking files and realtime the 4K Matte files if possible (garbage matte and holdout mattes, when needed), the FG will be the RAW camera files, allways for final composition.

----------------------

We are going to use 200Gbit INFINIBAND NETWORK, and at least 10 NVLINK II 8x Tesla V100 servers, ideally with NVLINK.

We need to get less than 30sec of the average frame (4K medium complexity) with the NUKE rendering with Octane Plugin.

I will ask the sales department to ask the pricing of the 100x License.

Regards.

mrfactory wrote:Thanks Goldorak , that's are very good news for me.

We are planing to render a whole movie in 2019 with virtual sets and matte painting...., with octane render.

We have a realtime PreViz workflow with UNREAL (the Octane Render plugin for UNREAL is essential for us, to avoid the baking process), and we want to do the final render of the same scene we did the pre-visualization with UNREAL, with NUKE, thus we need to speed up the render times as much as we can.

----------------------

Sample Test:: http://212.123.219.101/file/vfx/IMG_3792.mp4

These are the very firsts tests of realtime tracking and chromakey with an Octane Render scene baked in UNREAL, we are going to get much more precission with the realtime trackers, and much better 4K chroma key, this is just the very first a test.

We record in realtime in 4K from camera trigger, everything in sync and TimeCode (from RED, Alexa, Blackmagic...cameras):

Reference Composition: http://212.123.219.101/file/vfx/CLIP0000343_B.mp4
FG: http://212.123.219.101/file/vfx/CLIP0000343_A.mp4
Matte: http://212.123.219.101/file/vfx/CLIP0000343_C.mp4
BG:http://212.123.219.101/file/vfx/CLIP0000343_D.mp4
and also we record all the data of the realtime camera tracking with zoom and focus (depth of field) every thing sycronized and with TimeCode,.

These 5 pieces are record in a folder as a package is for pre-edit purposes, since the final composition is going to be rendered with NUKE, using the realtime FBX camera tracking files and realtime the 4K Matte files if possible (garbage matte and holdout mattes, when needed), the FG will be the RAW camera files, allways for final composition.

----------------------

We are going to use 200Gbit INFINIBAND NETWORK, and at least 10 NVLINK II 8x Tesla V100 servers, ideally with NVLINK.

We need to get less than 30sec of the average frame (4K medium complexity) with the NUKE rendering with Octane Plugin.

I will ask the sales department to ask the pricing of the 100x License.

Regards.

very interesting workflow! FYI your entire parent directory is exposed just by deleting the /vfx/videoname.file

Not sure how more gpus makes a more costly support. If program is stable it should work fine with 30, 40 gpus etc. plus more problems more bugs we can find for you guys. And you can sell more licenses.

20 best gpus are barely enough for AO rendering at 1080p for animations averaging 2-4 min per frame (not even night scene ) I cannot imagine full GI quality noise free with path tracing or pmc that will be 10-15 min per frame. I think limit should be increased to be used professionally

coilbook wrote:20 best gpus are barely enough for AO rendering at 1080p for animations averaging 2-4 min per frame (not even night scene ) I cannot imagine full GI quality noise free with path tracing or pmc that will be 10-15 min per frame. I think limit should be increased to be used professionally

That’s why we have the enterprise option and related support system to help with customers that are serious about power and cooling needed to reliably set up GPU racks. 20 GPUs is ~5000-6000 Watts, plus another ~1000 Watts for server and then cooling (which you need at this density). Note that nothing is stopping you from getting all the licenses/nodes you want and running multiple frames in parallel with 20 GPUs each from which is even more efficient in some cases (and also how orc scales to thousands of frames).

When we revisit network rendering in future releases, we may adjust the limits for the standard version.

Goldorak wrote:

coilbook wrote:20 best gpus are barely enough for AO rendering at 1080p for animations averaging 2-4 min per frame (not even night scene ) I cannot imagine full GI quality noise free with path tracing or pmc that will be 10-15 min per frame. I think limit should be increased to be used professionally

That’s why we have the enterprise option and related support system to help with customers that are serious about power and cooling needed to reliably set up GPU racks. 20 GPUs is ~5000-6000 Watts, plus another ~1000 Watts for server and then cooling (which you need at this density). Note that nothing is stopping you from getting all the licenses/nodes you want and running multiple frames in parallel with 20 GPUs each from which is even more efficient in some cases (and also how orc scales to thousands of frames).

When we revisit network rendering in future releases, we may adjust the limits for the standard version.

Thanks Is it possible to upgrade standard license to enterprise?

coilbook wrote:

Goldorak wrote:

coilbook wrote:20 best gpus are barely enough for AO rendering at 1080p for animations averaging 2-4 min per frame (not even night scene ) I cannot imagine full GI quality noise free with path tracing or pmc that will be 10-15 min per frame. I think limit should be increased to be used professionally

That’s why we have the enterprise option and related support system to help with customers that are serious about power and cooling needed to reliably set up GPU racks. 20 GPUs is ~5000-6000 Watts, plus another ~1000 Watts for server and then cooling (which you need at this density). Note that nothing is stopping you from getting all the licenses/nodes you want and running multiple frames in parallel with 20 GPUs each from which is even more efficient in some cases (and also how orc scales to thousands of frames).

When we revisit network rendering in future releases, we may adjust the limits for the standard version.

Thanks Is it possible to upgrade standard license to enterprise?

Sure - just be aware of the requirements listed and contact sales
or info to set up a follow up call to dicuss.