External Graphics Cards PC Tweet

[itou31]

Hi,
With octanerender v3, a lot of traffic occured on the PCIe lanes and exchange with RAM so avoid cluster with 1X PCIe or 4X PCIe for 4GPUs.
I have 2 Clusters 4x2 GPUs with an 4X PCIe adapter and one 4X PCie splitter (for 4GPUs) : loss about 30% of speed of GPU in average in big scenes with many textures.
Thats why I go to 7GPU rig + 4GPUs rig for network rendering. It seems adding a 4GPU with network render is better than adding a 4 GPUs cluster with 4X adapter. RAM traffic is better in each node.

[Tutor]

Hi guys.
Help me, please.
What motherboard I need for 8 external GPU and 2 internal? I use amfeltec clusters.

Thanks.

Sorry for delay. Here's what Amfeltec told me:

"The motherboard limitation is for all general purpose motherboards. Some vendors like ASUS supports maximum 7 GPUs, some can support 8. All GPUs requesting IO space in the limited low 640K RAM. The motherboard BIOS allocated IO space first for the on motherboard peripheral and then the space that left can be allocated for GPUs. To be able support 7-8 GPUs on the general purpose motherboard sometimes requested disable extra peripherals to free up more IO space for GPUs. The server type motherboards like Super Micro (for example X9DRX+-F) can support 12-13 GPUs in dual CPU configuration. It is possible because Super Micro use on motherboard peripheral that doesn’t request IO space."

I have three Supermicro X9DRX+-Fs [ https://www.supermicro.com/products/mot ... drx_-f.cfm & http://www.superbiiz.com/detail.php?name=MB-X9DRXFB ] based systems. The second to last sentence in the above quote (that contains "12-13 GPUs") applies to an X9DRX+-F system running Windows. Linus OSes, like Linux Mint, support even more GPUs with the X9DRX (or X10DRX). Whether the GPUs are internally or externally placed doesn't really significantly affect the IO space limitation, except that it is possible to place more GPUs internally in a system with a LIan LI D8000 case with an X9DRX motherboard.

@ Tutor: I would appreciate it very much if you could provide some information regarding the gpus that your are using. I am thinking about ordering that mobo but frightened of incompatibilities. Do you think I will be able using my ordinary AMD gaming gpus for it?
Sorry for low level question, new to server hardware.

AMD GPUs (gaming or otherwise) don't currently work with Octane and many other CUDA-based 3d renderers. The Nvidia GTX GPUs that I use for 3d rendering are 6x480s, 5x580s, 23x590s, 5x680s, 8xTitans, 1x690, 1xTitan Black, 12x780 6Gs, 13x780 TIs, 6xTitan Zs, 3x980s, 5x980 TIs and 20x1070s */. I also have 5 AMD cards that I use for OpenCL rendering with my four vintage/self upgraded/maintained MacPros: 3x2007 and 1x2009. My other video cards are used just for video display in some of my much more ancient systems. Those systems span back to the mid-1980s and include Dell Precisions, Deskstations, Power Computing, Apple Power and G#s, Commodore Amigas and Video Toasters, and Atari systems that I maintain and use on a more limited basis for particular effects; some of which when purchased were then used for then V1s - V2s of CPU rendering apps that are now at V17-18.

*/ I also have 7xGT 640 4G Nvidia cards that I use to maintain interactivity with each of my seven main systems while rendering. I'll be soon considering replacements with newer hardware with more memory for interactivity as the 4096x2160 display resolution of the GT 640Gs becomes a real limitation on their special use.