10 GPU open rig (inexpensive)

[Kandori07]

Hello all.. this is my first time posting. I've been reading hundreds of posts and learning lot.. thank you all

I purchased Octane two weeks ago and I'm falling in love with it (so much faster than MR even with a single 780)
So, after doing a lot of research I've decied to build myself a 10gpu rig as inexpensive yet function as I can.
Please give me advice and tips since I'm totally novice at this.

GPU: My scenes are not that big, so I've decided to go with 10 x used 780s 3Gb (around $300 a pop); I could get used 780Tis ($350ish).. do the extra 500 cudas help? I heard that Ti cards tend to get hotter than the regular ones.. (so I choose 780s) because I don't want to lose much performance.

Splitter: Two of amfeltec 4way 4x to 16x splitter (I chose this over the GPU cluster due to cost and bus speed [1x vs 4x])
http://www.interworldna.com/images/amfeltec/3_adjC.jpg

MB: ASRock EXTREME9 I chose this one because is quite cheap and has 5 Pcieslots..but mainly because I want to put 2 GPUs on the MB on slot 1 and slot 5 (put them as far away as possible from each other). On slots 2 and 4 I will put the splitters (for total of 8 GPUs on them)
http://www.amazon.com/ASRock-EXTREME9-D ... B00KZNUB74

Open air Frame: It took a while to find these guys but I think for around $100, this is a good deal. I can have them build me an aluminum case to tailor my needs.. in my case a 8 GPU frame. Their standard is 6 GPU set up.
http://bitcoinware.net/collections/gpu- ... ing-frames


Now I need some advice on the following:

PSU: 10 GPUs will require 3300W (according to what I read in the posts. 300w per GPU +300). Will Two w1600 do the job? which ones do you recommend and how do I connect the two PSU?

CPU: I think the 8 GPUs connected via splitters will have 4x on each of them..(32 lanes) plus two on board (4x each) will be total of 40 lanes.. Now will the Main display Gpu connected on slot 1 "require" a 16 lane? or can I somehow make that first slot use only 4x lanes? Which CPU do you recommend that has 40 lanes and is the most inexpensive?

OS: Do I need to go Win 8.1 to be able to use all 10 GPUs? (can't I use win 7?) or should I try win 10?

Mixed GPUS: is it okay to put different GPUS from several brands?
============================================================================================================
Just a thought.. can I put a FIREPRO w5000 in the first slot (to drive the display for maya viewport) without driver conflicts?
This guy put up a review about this Firepro w5000 card for maya on newegg:

"This review is from: AMD FirePro W5000 100-505842 2GB 256-bit GDDR5 PCI Express 3.0 x16 CrossFire Supported Workstation Video Card
Pros: This is the best video card I have ever owned. Bar none. In Maya I can rotate around a fully textured 100 million polygon scene at 20 fps!
The color fidelity is indescribably better than any consumer card. Nearly silent when fully loaded and doesn't even need a separate power cord.
This thing puts every consumer card to shame in professional apps. Even a GTX 680 looks gimp in comparison.
The right tool for the job."
=========================================================================================================


Let me know your thought.. : )

[smicha]

PSU: 2x 2000W superflower.
Use exactly the same 10 cards.
Amfeltec may work fine. There is also a supermicro mobo for 11 pcie slots.

There is a widely covered post by Tutor about exceeding gpu limits - tons of useful links there.

[Kandori07]

Thanks for your reply Smicha.

I think 2 x w2000 will be over my budget.. So I've decided to go with 6 gpu rig instead (and then later on build me another 6 gpu rig)
in which case I could use 2 x 1200w.

I've looked into the 11pcie slot board you mentioned. (they are 10 x 2.0 8x + 1 4x which is perfect). But I can't find 4x to 16x cables.. (at least not on amfeltec site), so I'll have to go with either 1x to 16x usb cables or the 4x to 4 x 16x splitters.

[smicha]

2x 1600W may not be enough for 10 gpus + cpu + all other stuff. My 4 gpu comp peaks at 1500W with 1100W OC. Difference in price between 2000W and 1600W titanium is 50 EUR.

As for splitters/cables - Tutors shows how to connect gpus to the Supermicro on its post.

[Kandori07]

I found a new frame for my open rig (6 gpu acrylic frame) today and assembled it just few minutes ago (took me 2 hours!). Now I'm ready to start building it. : )



The spacing between the cards will be around 3 inches after putting the cards in..(hope that's enough), so the 4x to 16x Four way splitter won't work.. (due to it's shorter cables). I'll have to use 1x to 16x usb cables instead. : (


Does anyone have real life experience in using the 1x to 16 usb powered cables as far as transfer speed? (loading scenes). Is it really annoyingly slow for decent size scenes? (any examples?)

[joeydbooth]

hey man, does octane recognise 4 cards with the 4-way splitter? im thinking of building one of my own! if so, any slow downs that you've noticed?
thanks!

[Kandori07]

I'll let you know after I try them. I"m still in the middle of getting my parts together.. From what I heard from tutor, they do work with the splitters. I want to use two of them to connect 3 or 4 on each splitter. I'm sure someone who's done it can tell you more from their actual experience. : )


What are you trying to build for yourself later on?

[asher]

the real question is: does your motherboard recognize 10 GPUS???? to my knowledge that motherboard will do 7, maybe 8.
BIos issues....

[asher]

you may need 3 Power supplies. 2 x 1400W or over, and one for the motherboard + gpu attached to that motherboard

[joeydbooth]

I'll let you know after I try them. I"m still in the middle of getting my parts together.. From what I heard from tutor, they do work with the splitters. I want to use two of them to connect 3 or 4 on each splitter. I'm sure someone who's done it can tell you more from their actual experience. : )


What are you trying to build for yourself later on?

my motherboard (x79 deluxe) is too small to accomodate my cards atm (770, titan x, 980 ti), one of them is always sandwiched in between the two others and over heats at 100 degrees, making octane stop the render...

i want to get the splitter, open up my case and split everything on this thing: http://amfeltec.com/products/gpu-oriented-cluster/

eventually, i would get another 980 ti to add to the external rig

OR

either that or get a big ass e-atx like the rampage 4 (i have a lga-2011 cpu 4930k)... but that would be more costly as i would need a new case and on and on...