Best Practices For Building A Multiple GPU System

[rappet]

So might not around this time next year be a good time to purchase Titan Zs for great prices?

I think around this time next year there will be no Titan Z left to buy.
The first quarter 2015 might be a good time to snatch the last few Titan Z if you can wait, but there is always a risk to end empty handed when waiting too long.
There is just a small market for Titan Z and My guess is there are not that many Titan Z produced and that the ones that are stored for sale might be dumped soon if new cards arrive.
I started a thread about a 990 will come?
http://render.otoy.com/forum/viewtopic. ... 90#p215642
For Octane users the new cards will probably be less interesting then for gamers.
The 980 cannot beat the Titan Black, so can a 990 or other new card beat the Titan Z (in Octane)? Probably not.

I am keeping an eye on the prices of Titan Z, and if the price might drop a few 100€ I might go for one,
but if they are sold little by little right now, the prices won't drop anymore... Anyway, imo around this time next year imho there won't be Titan Z on the market... probably not even second hand after only one year of use.

Holiday Greetz,

[Tutor]

So might not around this time next year be a good time to purchase Titan Zs for great prices?

I think around this time next year there will be no Titan Z left to buy.
The first quarter 2015 might be a good time to snatch the last few Titan Z if you can wait, but there is always a risk to end empty handed when waiting too long.
There is just a small market for Titan Z and My guess is there are not that many Titan Z produced and that the ones that are stored for sale might be dumped soon if new cards arrive.
I started a thread about a 990 will come?
http://render.otoy.com/forum/viewtopic. ... 90#p215642
For Octane users the new cards will probably be less interesting then for gamers.
The 980 cannot beat the Titan Black, so can a 990 or other new card beat the Titan Z (in Octane)? Probably not.

I am keeping an eye on the prices of Titan Z, and if the price might drop a few 100€ I might go for one,
but if they are sold little by little right now, the prices won't drop anymore... Anyway, imo around this time next year imho there won't be Titan Z on the market... probably not even second hand after only one year of use.

Holiday Greetz,

Thanks Rappet for a well reasoned analysis.

[rappet]

Another minddrop about the Z
If the 980 can not optimized for Octane to outperformance the 780 and Titan Black, then probably the Titan Z will be the best option for a long time, especially being placed in the bottom slot of the mobo (in aircooled system)...
I have to note that all depends on the total config ofcourse. I.e. not all cards can be placed just above the TitanZ, because of the plate of the Titan Z.. in my case the 580 classified did not enough space, and a 780ti barely (in fact not really).
When you do not want to have one-slot open space between your cards, this Titanz might be a problem. Also with some mobo's it is not possible to stick the Z in the topslot, because of the plate... When the systen ram is very close to the pci slot ther is not much space.
So in aircooled systems the Titan Z can be a joy but bothersome.
Cheers and best wishes for 2015

[FrankPooleFloating]

A pair of 580 3GB HydroCoppers went for US$205 (total - not each) with free shipping on eBay last week... those are the types of deals I am talking about. For those who are cheap-asses, or down on their luck etc, deals like that are effing insane.

[glimpse]

Another minddrop about the Z
If the 980 can not optimized for Octane to outperformance the 780 and Titan Black, then probably the Titan Z will be the best option for a long time, especially being placed in the bottom slot of the mobo (in aircooled system)...
I have to note that all depends on the total config ofcourse. I.e. not all cards can be placed just above the TitanZ, because of the plate of the Titan Z.. in my case the 580 classified did not enough space, and a 780ti barely (in fact not really).
When you do not want to have one-slot open space between your cards, this Titanz might be a problem. Also with some mobo's it is not possible to stick the Z in the topslot, because of the plate... When the systen ram is very close to the pci slot ther is not much space.
So in aircooled systems the Titan Z can be a joy but bothersome.
Cheers and best wishes for 2015

hi, Man.

Just for mind blowing =) check this out! http://amfeltec.com/products/gpu-oriented-cluster/

in theory, You get 4x TitanZ on air (with couple PSUs) from single PCIe - in open air =DDD..

[rappet]

Another minddrop about the Z
...
So in aircooled systems the Titan Z can be a joy but bothersome.
Cheers and best wishes for 2015

hi, Man.

Just for mind blowing =) check this out! http://amfeltec.com/products/gpu-oriented-cluster/

in theory, You get 4x TitanZ on air (with couple PSUs) from single PCIe - in open air =DDD..

Hey,
Yes mindblowing indeed... Do I understand this correctly and is there one mobo where more mobo's with multi gpu are plugged into?
There is another thread regarding to maximum gpu of 7 or 8 definied in the mobo and or OS.
Does this setup has a workaround that?
Greetz from my blowed mind

[smicha]

I saw their products some time ago, but such expansion boards are very expensive
https://www.thedebugstore.com/Amfeltec/ ... _page=9999
For 650 EUR one can get 2 great mobos z97 easily.

[glimpse]

I saw their products some time ago, but such expansion boards are very expensive
https://www.thedebugstore.com/Amfeltec/ ... _page=9999
For 650 EUR one can get 2 great mobos z97 easily.

hi, Smicha =) I do agree with You in a way. But I see a point fur such cluster too =) actually buying from manufacturer the price seems to be lower (I talked with a Guy last week who purchased two of them for ~ that sum =) in this case it's not so bad for 8 slots & open air scenario (what means You don't need watercooling =)

That 8 GPU limmit is a bit of mistory for me..- some do report 8GPUs as max, other say it isn't..when guys build Fastra II they need a custom bios & hacked Linux to utilise more GPUs) Maybe uefi changed something in that respect?

Anyway you can build a relativelly small & noiseless (mITX or even NUC) system & hook lets say 8 GPUs in other room as 10feet cable is available =) & enjoy the silence. (For those hydrofobs, but probably not for Smicha =)

As GPU the same way whole package (rig) has it's own customer - user with preference =)

[Tutor]

I saw their products some time ago, but such expansion boards are very expensive
https://www.thedebugstore.com/Amfeltec/ ... _page=9999
For 650 EUR one can get 2 great mobos z97 easily.

hi, Smicha =) I do agree with You in a way. But I see a point fur such cluster too =) actually buying from manufacturer the price seems to be lower (I talked with a Guy last week who purchased two of them for ~ that sum =) in this case it's not so bad for 8 slots & open air scenario (what means You don't need watercooling =)

That 8 GPU limmit is a bit of mistory for me..- some do report 8GPUs as max, other say it isn't..when guys build Fastra II they need a custom bios & hacked Linux to utilise more GPUs) Maybe uefi changed something in that respect?

Anyway you can build a relativelly small & noiseless (mITX or even NUC) system & hook lets say 8 GPUs in other room as 10feet cable is available =) & enjoy the silence. (For those hydrofobs, but probably not for Smicha =)

As GPU the same way whole package (rig) has it's own customer - user with preference =)

Thanks to Rappet and you for the lead. With over 80 GPUs, for me these would be great for system condensing, reducing non-essential electrical power requirements and maximizing cooling. It appears that they'd still support water-cooling, albeit open air water-cooling.

Here's what I was quoted from the manufacturer direct [ N.B. the first paragraph which could help better explain the 7-8 GPU quandary that we and others have been discussing lately - seems like it may be computer motherboard related ]:

"Please note that the general purpose motherboard can supports maximum 7-8 GPUs.
If you are planning to have more than 7-8 GPUs on your mother board we recommend
to confirm with motherboard vendor regarding support this numbers of GPU.

... .

Quotation (Prices in USD)
----------------------------------------------------------------------------------------------------------------
SKU Item Qty Unit Price
----------------------------------------------------------------------------------------------------------------
SKU-078-22 GPU-oriented PCIe Cluster (5ft cable) 1 $ 578.78 USD
[2 Clusters (up to 8 GPU) + 2 channel Host board]

SKU-078-44 GPU oriented PCIe Cluster (5ft cable) 1-2 $ 1030.69 USD
[4 Clusters (16 GPU) + 4 channel Host board]

For 10ft cable for SKU-078-xx please add $12.75 USD per PCIe Cluster

(Shipping not included)
-----------------------------------------------------------------------------------------------------------------

Quantity discounts are available.

Stock
--------
We have this product in stock
It can be shipped the next business day following the customer’s payment."

The prices seem extremely reasonable to me, especially when you consider that the eight GPU cluster costs less than 1/5 of the retail price of one 8 GPU Tyan Barebones Server.

[glimpse]

The prices seem extremely reasonable to me, especially when you consider that the eight GPU cluster costs less than 1/5 of the retail price of one 8 GPU Tyan Barebones Server.

Exactly! The price is very reasonable =) for some users (who preffer to go o air it should be very tempting =)
yesterday watched 'bout ASUS Poseidon GPU with custom cooler. It performs close to strix & on open air rig should not only run higher clock but also lower temps emitting less noise =) now the good point is that You can hook a loop without adding waterblocks =)

Smichas route, going watercooling is good, Your route is good, & this one too - just every case has it's own user & if it would be my money (considering I already have a system, & my needs - rendering stills) I'd go this expander route with 10feet cable to place th cluster in other room =) the tempting point is ability to minimise Your rig =) as even mITX would work just fine =) & there are crazy cases for those..even passivelly cooled =)