Best Practices For Building A Multiple GPU System

[glimpse]

That's more than I spent to better outfit my Tyan to higher specs (i.e., I have lots more memory, storage and faster CPUs) and for the Titan Zs.

it doesn't suprice me at all =) as they are only doing bussines - trying to get something cheaper sell for more =) pluss, it's old news =) so don't compare with current prices or technologies =)

we all know that DIY route is going to be Cheaper* (if You have knowledge & time)

[Tutor]

I couldn't find where it says that they got 116 GPUs working. Where is that? I could find where they said a year later that they had 8 GPUs working with Octane [ http://www.renderstream.com/everything/ ... u-vdactr8/ ].

"At RenderStream we are providing solutions with two to eight GPU’s, and we just introduced a sixteen GPU development system (Linux only). Our solutions are made using Nvidia Geforce, Quadro and Tesla cards. For even more speed we announced at the March 2010 ACM Siggraph meeting in Austin our VDACTr8 with eight single GPU cards and VDACTr8x2 solution using eight dual GPU GTX295 cards." (source)

I do have two 32-core 4 CPU SuperMicros, but they have internally only 4 single wide (so its 2 double wide x16 V3) PCIe slots on the main motherboards. Amfeltecs???

check the post before - gave a link in the end about alternative solution (sort of =)

I'll be away from this system for a while installing Linux on my Tyan to see what happens. Thanks Glimpse.

[Tutor]

Thanks again Glimpse for giving me a swift kick in the rump. I needed it. I haven't given up because of your insistence that I look further for a solution.

[glimpse]

Thanks again Glimpse for giving me a swift kick in the rump. I needed it. I haven't given up because of your insistence that I look further for a solution.

hehe, Nice to Hear, Man. I really wish You to find the solution. Maybe contacting those Guys, Trenton, Maybe Your motherboard vendor would give You some ideas how to solve this issue - I'm sure You're not the first for them with such, multi GPU, issues so they might be nice & turn You to right direction.

Anyway, thanks for documenting all this in forums as this surely help for others building a piece =)

[Hydra]

"My goal was to use Titan Z cards to exceed 8 GPU processors and to have 14 GPU processors in one system. Having tried bios mods, Regedit hacks and various card stack configurations, the hardware will not recognize more than 8 GPU processors."

Umm... In reading what you were trying to do, you wanted to create the fastest computer on earth... which could also render. In describing that you tried BIOS hacks, it sounds to me like you should go to the BIOS manufacturer. You might be surprised but they may be interested in helping you create a world record PC. I'd of course wonder whether Windows has a similar limitation.

IRL I'm an engineer and I've had some dealings with manufacturers over industrial PCs. It may be slow to work with them, but they would have the definitive answer for you.

[Tutor]

"My goal was to use Titan Z cards to exceed 8 GPU processors and to have 14 GPU processors in one system. Having tried bios mods, Regedit hacks and various card stack configurations, the hardware will not recognize more than 8 GPU processors."

Umm... In reading what you were trying to do, you wanted to create the fastest computer on earth... which could also render. In describing that you tried BIOS hacks, it sounds to me like you should go to the BIOS manufacturer. You might be surprised but they may be interested in helping you create a world record PC. I'd of course wonder whether Windows has a similar limitation.

IRL I'm an engineer and I've had some dealings with manufacturers over industrial PCs. It may be slow to work with them, but they would have the definitive answer for you.

Thanks Hydra. I'll look into that.

[sikotik13]

Tutor, a teeny, tiny part of me just wants to crawl into your server/computer room and gaze about in slack-jawed awe. Not that this desire is particularly helpful in your pursuit of multi-GPU glory... but the desire remains nonetheless. While personally pleased that my needs/desires for hardware are much easier to achieve than many others, I do consider myself a half-hearted (while striving to be a budget-conscious, non-traditional college student, which is hard) hardware enthusiast in general.

Luckily, my purposes will be sufficed when I can acquire another 2 780 6GB cards, since I primarily am focused on stills for personal pleasure and the learning experience, and just about every poster in this thread has provided some insight I have used to get this far (so a belated thanks to pretty much everyone here for your valuable insights). Should I work up the courage to display my feeble first attempt at a water-cooled rig (feeble in the sense of scope, not necessarily execution, as it does function quite adequately for me), I'll do so here, most likely.

I will chime in my additional fraction of currency in wishing you luck in whatever future endeavors you aspire to, hardware or otherwise.
This applies to everyone in this thread, for whatever that may be worth.

[glimpse]

Should I work up the courage to display my feeble first attempt at a water-cooled rig (feeble in the sense of scope, not necessarily execution, as it does function quite adequately for me)..

Please drop a line, create a thread & tell You story - this will benefit to community as others might learn something new =) in the end we are all in the same boat =)

[glimpse]

Tutor, check this out! =)

500x1000px-LL-afb265f8_DSC00001.jpeg

remember I've told You 'bout aqua freezers? Well the Guy use one with Aquacomputer passive rad =) do a reverse (image search) if You're interested =)..don't remember where from I got this image, but as I found it accidentally on my PC, though of sharing this with You =)

[Tutor]

Tutor, a teeny, tiny part of me just wants to crawl into your server/computer room and gaze about in slack-jawed awe. Not that this desire is particularly helpful in your pursuit of multi-GPU glory... but the desire remains nonetheless. While personally pleased that my needs/desires for hardware are much easier to achieve than many others, I do consider myself a half-hearted (while striving to be a budget-conscious, non-traditional college student, which is hard) hardware enthusiast in general.

Luckily, my purposes will be sufficed when I can acquire another 2 780 6GB cards, since I primarily am focused on stills for personal pleasure and the learning experience, and just about every poster in this thread has provided some insight I have used to get this far (so a belated thanks to pretty much everyone here for your valuable insights). Should I work up the courage to display my feeble first attempt at a water-cooled rig (feeble in the sense of scope, not necessarily execution, as it does function quite adequately for me), I'll do so here, most likely.

I will chime in my additional fraction of currency in wishing you luck in whatever future endeavors you aspire to, hardware or otherwise.
This applies to everyone in this thread, for whatever that may be worth.

Thanks for your post. No need for crawling. If you're every in my neck of the woods, I'll let you see, examine and test my network so long as you give me advance notice. My network is housed acrossed the many rooms of my studio.

Those GTX 780 6Gs are extremely good GPUs if you can find them at a reasonable price. I'm in the process of consolidating my 109 GPUs processors, 13 of which are ATI/AMD processors. I have seven GTX 780 6Gs that I intend to attach soon to one of my two 32-core Supermicro Servers, using Amfeltec clusters [ http://amfeltec.com/products/gpu-oriented-cluster/ ] for assistance. I intend to attach seven original Titans to my other Supermicro Server the same way.

Best of luck and accomplishment in school and in all of your other future endeavors. If you ever need assistance, we're here to serve you.

P.S. For more information on my systems, visit my thread here - http://forums.macrumors.com/showthread.php?t=1333421 , beginning at about post #500 when my GPU focused conversion began to occur.