Need Wisdom, 6 GPU server?

Generic forum to discuss Octane Render, post ideas and suggest improvements.
Forum rules
Please add your OS and Hardware Configuration in your signature, it makes it easier for us to help you analyze problems. Example: Win 7 64 | Geforce GTX680 | i7 3770 | 16GB
User avatar
Bulwerk
Licensed Customer
Posts: 238
Joined: Sun Aug 01, 2010 8:04 pm

Guys thank you for all your input! I really appreciate it!

I am looking at getting this mobo http://www.newegg.com/Product/Product.a ... -_-Product now and using this case http://www.ebay.com/itm/TheRockCrusher- ... 1152456518) with 16x and 8x risers.
Should work right?

I am afraid I will have to go with 980 4GB. We mostly handle motion graphics type animations and do not really need a ton of Vram. Plus our IT dept. would not feel good about buying used parts...

Man this thing will be so loud!
mib2berlin
Licensed Customer
Posts: 1194
Joined: Wed Jan 27, 2010 7:18 pm
Location: Germany

Bulwerk, other CUDA render engines report very good performance with GTX 980/70 or
have workarounds for better performance.
So it is only matter of time I hope.

Maxwell:
http://render.otoy.com/forum/posting.ph ... 9&p=204788

Blender Cycles:
http://www.blenderartists.org/forum/sho ... Benchmarks

Cheers, mib
Opensuse Leap 42.3/64 i5-3570K 16 GB
GTX 760 4 GB Driver: 430.31
Octane 3.08 Blender Octane
User avatar
glimpse
Licensed Customer
Posts: 3740
Joined: Wed Jan 26, 2011 2:17 pm
Contact:

Yeah, if You don't need very much of vRam then maxwell might be a good fit (we just need to wait a bit how come the code will be optimised for those new cards..) nVidia claims 5tflops of SP performance - if that's doable then these cards beasty, though as pointed out You should not expect that efficiency..

as for open air case it seems very practical (but those fans are plain porngraphy - as it looks like Xmas tree =)

apart from that You have to be careful to choose certain Ribon cables, as some of them might just be too short. As these minig cases are made for miners who usually use powered 1x (just because they are cheap & so..usually they are a bit longer..) One thing I don't understand why they left brackets for slotted (plugged into motherboard) cards if they make space for six on top - doesn't make sense..- keep these things in mind

if You care bout noise, & looking to space cards (a rig like this) better go for non reference coolers like wind force or ACX (I think) - this type is better for open air cases (compared to regular shroud type reference coolers). If You provide enough airflow they keep good temps, even with slight over clock..- it's just not advisable to stack these cards on a motherboard as the space is too little, especially if cards have backplates fitted..

hope that helps =) make a good choose & drop some pics later!

cheers
User avatar
Bulwerk
Licensed Customer
Posts: 238
Joined: Sun Aug 01, 2010 8:04 pm

You all have been so helpful! thanks for all the advice.
Whatever I end up going with I will Post some pics!
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

980 is 40-50% slower than 780 :)
6x980 = 3x780

4x780 on water = 6x780 on air

Don't go with 980, get titan or 780 6gb, 780ti if 3gb is enough.

Two days ago I put 780 6gb asus strix next each other on stock air cooling. The upper one hit 80C quickly and core clock was reduced below 900mhz and was so noisy, very loud. So 4x780 run at 836mhz core / 6000mhz mem while 4x780 on water run at 1250 mhz/7000mhz = 49% faster overall, instead of putting 6 get 4 watercooled gpus or build second asus ws mobo and render via network.

http://render.otoy.com/forum/viewtopic.php?f=9&t=42540

enjoy :)
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
gueoct
Licensed Customer
Posts: 402
Joined: Mon Jul 11, 2011 3:10 pm

Bulwerk wrote:.....I am looking at getting this mobo http://www.newegg.com/Product/Product.a ... -_-Product now and using this case http://www.ebay.com/itm/TheRockCrusher- ... 1152456518) with 16x and 8x risers.
Should work right?....Man this thing will be so loud!
Exactly; this thing will be soooo loud!

I´ve been running a watercooled machine for almost 4 years now and i am SO happy!

AND NOW THE BEST THING:

I have seen an 8-slot motherboard with 8 watercooled GPUs (which are actually only one slot thick)!
And it will be silent compared to the air-cooled madness....
Last edited by gueoct on Wed Oct 01, 2014 3:19 pm, edited 1 time in total.
Intel i7-970 @3,20 GHz / 24 GB RAM / 3 x EVGA GTX 580 - 3GB
User avatar
Bulwerk
Licensed Customer
Posts: 238
Joined: Sun Aug 01, 2010 8:04 pm

Can someone ball park what the cost of water cooling 6x780ti ?
I could use a normal case if I water cooled then.... hmmm

I have never built a water cooled rig before and it kinda scares me :|
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Give me the model of 780 ti and I come up with something for you.

780 ti are dual slot cards - you'll have to go with raisers for 6 of them.

If you have budget for 8x780ti (in one chassis!) and watercool stuff I can also recommend you something :) Just let me know.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
gueoct
Licensed Customer
Posts: 402
Joined: Mon Jul 11, 2011 3:10 pm

smicha wrote: 780 ti are dual slot cards - you'll have to go with raisers for 6 of them.
Most Air-cooled cards are dual slot, but with a watercooling they are 1 slot!
Intel i7-970 @3,20 GHz / 24 GB RAM / 3 x EVGA GTX 580 - 3GB
User avatar
glimpse
Licensed Customer
Posts: 3740
Joined: Wed Jan 26, 2011 2:17 pm
Contact:

Bulwerk wrote: I could use a normal case if I water cooled then.... hmmm|
actually You can't Use "normal" case as 6x dual slot cards is more than any regular case & mobo can provide (spacing is typically 7-8 expansion slot width) - in order to house more than 4x dual slot cards You need server case & mobo (or some type of riser system) or custom case.

if You still want to house 6, 7 or even 8 GPUs in "regular" case You need to modify 780s to single slot (with water-cooled blocks).. or go for TitanZs (that are expensive) - EK has water blocks for them to get 3slot card into 2slot - that way You can have 8GPUs (2per card) running in a regular case with compatible ATX motherboard. Sort of a solution, but You pay extra for saved space, less spending on a loop & so..

In the end You can always ask someOne to build a rig for You =) no headache & Your custom beast will be planed, built & ready to render =)
Post Reply

Return to “General Discussion”