Page 2 of 6

Re: Need Wisdom, 6 GPU server?

Posted: Tue Sep 30, 2014 2:58 pm
by Bulwerk
Guys thank you for all your input! I really appreciate it!

I am looking at getting this mobo http://www.newegg.com/Product/Product.a ... -_-Product now and using this case http://www.ebay.com/itm/TheRockCrusher- ... 1152456518) with 16x and 8x risers.
Should work right?

I am afraid I will have to go with 980 4GB. We mostly handle motion graphics type animations and do not really need a ton of Vram. Plus our IT dept. would not feel good about buying used parts...

Man this thing will be so loud!

Re: Need Wisdom, 6 GPU server?

Posted: Tue Sep 30, 2014 4:29 pm
by mib2berlin
Bulwerk, other CUDA render engines report very good performance with GTX 980/70 or
have workarounds for better performance.
So it is only matter of time I hope.

Maxwell:
http://render.otoy.com/forum/posting.ph ... 9&p=204788

Blender Cycles:
http://www.blenderartists.org/forum/sho ... Benchmarks

Cheers, mib

Re: Need Wisdom, 6 GPU server?

Posted: Tue Sep 30, 2014 5:46 pm
by glimpse
Yeah, if You don't need very much of vRam then maxwell might be a good fit (we just need to wait a bit how come the code will be optimised for those new cards..) nVidia claims 5tflops of SP performance - if that's doable then these cards beasty, though as pointed out You should not expect that efficiency..

as for open air case it seems very practical (but those fans are plain porngraphy - as it looks like Xmas tree =)

apart from that You have to be careful to choose certain Ribon cables, as some of them might just be too short. As these minig cases are made for miners who usually use powered 1x (just because they are cheap & so..usually they are a bit longer..) One thing I don't understand why they left brackets for slotted (plugged into motherboard) cards if they make space for six on top - doesn't make sense..- keep these things in mind

if You care bout noise, & looking to space cards (a rig like this) better go for non reference coolers like wind force or ACX (I think) - this type is better for open air cases (compared to regular shroud type reference coolers). If You provide enough airflow they keep good temps, even with slight over clock..- it's just not advisable to stack these cards on a motherboard as the space is too little, especially if cards have backplates fitted..

hope that helps =) make a good choose & drop some pics later!

cheers

Re: Need Wisdom, 6 GPU server?

Posted: Tue Sep 30, 2014 6:43 pm
by Bulwerk
You all have been so helpful! thanks for all the advice.
Whatever I end up going with I will Post some pics!

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 2:05 pm
by smicha
980 is 40-50% slower than 780 :)
6x980 = 3x780

4x780 on water = 6x780 on air

Don't go with 980, get titan or 780 6gb, 780ti if 3gb is enough.

Two days ago I put 780 6gb asus strix next each other on stock air cooling. The upper one hit 80C quickly and core clock was reduced below 900mhz and was so noisy, very loud. So 4x780 run at 836mhz core / 6000mhz mem while 4x780 on water run at 1250 mhz/7000mhz = 49% faster overall, instead of putting 6 get 4 watercooled gpus or build second asus ws mobo and render via network.

http://render.otoy.com/forum/viewtopic.php?f=9&t=42540

enjoy :)

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 2:59 pm
by gueoct
Bulwerk wrote:.....I am looking at getting this mobo http://www.newegg.com/Product/Product.a ... -_-Product now and using this case http://www.ebay.com/itm/TheRockCrusher- ... 1152456518) with 16x and 8x risers.
Should work right?....Man this thing will be so loud!
Exactly; this thing will be soooo loud!

I´ve been running a watercooled machine for almost 4 years now and i am SO happy!

AND NOW THE BEST THING:

I have seen an 8-slot motherboard with 8 watercooled GPUs (which are actually only one slot thick)!
And it will be silent compared to the air-cooled madness....

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 3:14 pm
by Bulwerk
Can someone ball park what the cost of water cooling 6x780ti ?
I could use a normal case if I water cooled then.... hmmm

I have never built a water cooled rig before and it kinda scares me :|

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 3:17 pm
by smicha
Give me the model of 780 ti and I come up with something for you.

780 ti are dual slot cards - you'll have to go with raisers for 6 of them.

If you have budget for 8x780ti (in one chassis!) and watercool stuff I can also recommend you something :) Just let me know.

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 3:22 pm
by gueoct
smicha wrote: 780 ti are dual slot cards - you'll have to go with raisers for 6 of them.
Most Air-cooled cards are dual slot, but with a watercooling they are 1 slot!

Re: Need Wisdom, 6 GPU server?

Posted: Wed Oct 01, 2014 3:24 pm
by glimpse
Bulwerk wrote: I could use a normal case if I water cooled then.... hmmm|
actually You can't Use "normal" case as 6x dual slot cards is more than any regular case & mobo can provide (spacing is typically 7-8 expansion slot width) - in order to house more than 4x dual slot cards You need server case & mobo (or some type of riser system) or custom case.

if You still want to house 6, 7 or even 8 GPUs in "regular" case You need to modify 780s to single slot (with water-cooled blocks).. or go for TitanZs (that are expensive) - EK has water blocks for them to get 3slot card into 2slot - that way You can have 8GPUs (2per card) running in a regular case with compatible ATX motherboard. Sort of a solution, but You pay extra for saved space, less spending on a loop & so..

In the end You can always ask someOne to build a rig for You =) no headache & Your custom beast will be planed, built & ready to render =)