Multiple GPU Setups? Tweet

[RyDarling]

First time posting...

I’m interested in replacing my existing render farm (Mental Ray Based) with 10+ GTX 1080's. However, I have some questions regarding how people are setting up 4 or 5 plus GPU’s without using a large number of computers. Rather I’d like to build a rack of 12 NVidia GTX 1080’s to run my farm. I have no idea where to get started regarding how to set up the hardware. Ideally I’d like to have two CPU’s running 12 GPU’s. I’m hoping to break it down like this:

(CPU1[2 GPU’s] + CPU2 [10 GPU’s]).

How would you recommend I move forward with this? I'm not really interested in a server blade solution.

Also, is this something I should look into? I've never heard of anyone using a setup like this. Then again I have no idea how to set up 5 or more GPU's with one CPU.

PC World:

http://www.pcworld.com/article/2099945/ ... iasts.html

[glimpse]

hi there =)

one thing I would recommend would be reading through this forum, there ton of useful information =)

if You wish, I have few articles as well:

7GPU workstation for inspiration, Value build for information & for alternative - Going on air wide open =)

hope that could be any useful.

[archiform3d]

I am about to embark on exactly the same path and am thinking about one master machine with either 1 GPU in it and then three rack mount PCI expansion chassis with each 4 GPUs in it, so a total of 13 GPUs.

I have been wondering whether that many GPUs running off one machine is wise or whether I should break it down to 4 machines with 4 GPUs in each using the Chenbro 4U rackmount case for each.

It is damn hard to find any real knowledge on this kind of system. It is real easy to find people that want to change you a fortune for something that may or may not work.

I am also assuming that the systems that can pack in 8 video cards may lack the cooling for our kind of work. 3D rendering is heavy duty stuff - way heavier than most bitcoin miners realise.

I too would appreciate any insights.....

[glimpse]

A lot depends on Your needs @archiform3d - & for each of us those are different. So even if You find some information (including some provided below), take with a grain of salt & always put that in Your own perspective to see how that fits.

There are a lot of limitations & roadblocks, that You need to take into consideration. As You start building solutions You have to evaluate what You're gaining & what You're loosing while picking one or the ther route.

3x 4GPU nodes are fine, but just because You are going to have less cards per node, doesn't automaticaly mean those cards are not going to throttle down. Plus You have to evaluate additional licences, maintainance, upgradability, etc.

The best way is to have propper cooling (at least hybrid solution with AIO) or even full loop (money will come back in performance). However, if You still do not want to mess with any of that.. go with reference design cooler equiped GPUs if they are stacked close to each other or non reference coolers if You have extra space between cards.

Lan speed matters if You ever going to do fast previz or animations. Since OctaneRender allows up to 20GPUs to be utilised, getting nodes with more than 4 GPUs might make sense as well. Also consider externalGPU route.

2x 7-8GPU builds with good cooling would set Youback ~20k$. But You would get close to 2000 in OctaneBench (with ability to expand adding one more node giving in total 1 screen card + 20 GPUs for render). You could use both (or three) systems for single job or as separate workstations. Also as main + backup.

Maintaining "render farm" with 4GPU nodes would cost more & have more downsides. But if that's somethign You're into, it's doable. Upside is that starting this requires less investment as You can start from smaller/cheaper nodes.

Good luck =)

[RyDarling]

Since OctaneRender allows up to 20GPUs to be utilised.

Can you clairfy? By 20GPU's Does that mean total GPU's on a network or per node?

[rappet]

Since OctaneRender allows up to 20GPUs to be utilised.

Can you clairfy? By 20GPU's Does that mean total GPU's on a network or per node?

20gpu in total for OctaneRender to be used as limitation; so maximum of 20 gpu's on master and gpu's on slave(s) in total.

[Flint_Ironstag]

OP, any time you're looking to cram that much GPU power in a single enclosure, there is tons of tweaking (power & cooling) to be done before you have a 24/7 100% load stable system.

The rig you linked is cool as hell, but for getting real work done I'd look to one of these suppliers:

http://www.onestopsystems.com/compute-accelerators
16 GPU appliance here (you'd need to supply a server or workstation via PCIE)

or

https://sagitta.pw/hardware/gpu-compute-nodes/brutalis/
8 GPU but can cluster. While optimized for password cracking, they can handle any load.