Best Practices For Building A Multiple GPU System

Discuss anything you like on this forum.
Post Reply
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

The ASROCK RACK 3U8G-C612 8-WAY GPU SERVER reminds me a lot of my eight GPU Tyan server, except all of my GPUs in the old Tyan are in the rear.
How do the temps run with these, it doesn't appear that you would watercool it (or maybe yes?). I liked looking at the ASROCK RACK 3U8G-C612 manual where it talked about air flow, it never really talks of max temp, although it lists these cards as part of its QVL (Qualified Vendors List):

1 Intel Intel Xeon phi Coprocessor 5110P
2 NVIDIA NVidia GEFORCE GTX TITAN BLACK
3 NVIDIA NVidia Tesla K40
4 NVIDIA NVidia Tesla K80
5 NVIDIA NVidia Tesla C2050
6 NVIDIA NVidia Tesla C2075
7 NVIDIA NVidia Tesla P100
8 NVIDIA NVidia GTX1080
9 MSI MSI Twin Forzer GTX680 (NVidia GeForce GTX680 )
10 AMD AMD W600
11 AMD AMD R5000
12 AMD AMD W9100
13 AMD AMD S9100
14 AMD AMD S9150
15 AMD AMD S9170
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
philliplakis
Licensed Customer
Posts: 79
Joined: Wed Nov 19, 2014 2:27 am
Location: Sydney, Australia

Hey guys,

Sorry to chime in late. Does the X10DRX have open ended PCIe slots? So we could effectively insert 11 - 16x slot cards?
Win10 - 6700k - 32gb Ram - GTX Titan X (Pascal) + GTX 1080
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

philliplakis wrote:Hey guys,

Sorry to chime in late. Does the X10DRX have open ended PCIe slots? So we could effectively insert 11 - 16x slot cards?
Only 8 if them are open - the last ones.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
rappet
Licensed Customer
Posts: 1962
Joined: Fri Apr 06, 2012 3:57 pm
Location: The Netherlands
Contact:

Huh?... Can this mobo have 6 dual-slotted gpu on it..?

I see those small pci slot on the mobo, Does the gpu needs only that first small slot?

If so, then this mobo can i.e. run 6x 1080 watercooled without having (Smicha :P ) to cut them and make them singleslotted?

4090+3089ti & Quad 1080ti
ArchiCAD25, ofcourse Octane & OR-ArchiCAD plugin (love it)
http://www.tapperworks.com
http://www.facebook.com/pages/TAPPERWOR ... 9851341126
http://www.youtube.com/user/Tapperworks/videos
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

rappet wrote:Huh?... Can this mobo have 6 dual-slotted gpu on it..?

I see those small pci slot on the mobo, Does the gpu needs only that first small slot?

If so, then this mobo can i.e. run 6x 1080 watercooled without having (Smicha :P ) to cut them and make them singleslotted?
Only 4 - I was thinking about it but there is another reason - there is only Lian li case that can handle this mobo...it has to have 11 pcie IO brackets.
Last edited by smicha on Wed Feb 22, 2017 9:18 am, edited 1 time in total.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

here you go
You do not have the required permissions to view the files attached to this post.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
rappet
Licensed Customer
Posts: 1962
Joined: Fri Apr 06, 2012 3:57 pm
Location: The Netherlands
Contact:

Aha, those brackets and even cases can be a bottleneck when looking at the whole picture
... everytime it is a puzzle and you have the knowledge pieces :)
We do not need wikipedia, we have Wikismicha :lol:

4090+3089ti & Quad 1080ti
ArchiCAD25, ofcourse Octane & OR-ArchiCAD plugin (love it)
http://www.tapperworks.com
http://www.facebook.com/pages/TAPPERWOR ... 9851341126
http://www.youtube.com/user/Tapperworks/videos
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

rappet wrote:Aha, those brackets and even cases can be a bottleneck when looking at the whole picture
... everytime it is a puzzle and you have the knowledge pieces :)
We do not need wikipedia, we have Wikismicha :lol:
:)

There is another option I am thinking about now: open top pcie ports - simply cut them (I know something about cutting :)) and install 11 gpus on water single slotted. Just thinking about 11 sli water terminal now - 7+7 with 3 outlets blocked. And surely a custom case.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
rappet
Licensed Customer
Posts: 1962
Joined: Fri Apr 06, 2012 3:57 pm
Location: The Netherlands
Contact:

Then the OR 20 gpu limit can be reached with one master and one slave PC... That would be topnodge!

4090+3089ti & Quad 1080ti
ArchiCAD25, ofcourse Octane & OR-ArchiCAD plugin (love it)
http://www.tapperworks.com
http://www.facebook.com/pages/TAPPERWOR ... 9851341126
http://www.youtube.com/user/Tapperworks/videos
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Guys,

Do you know anything about pricing of the Klimax 210?
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
Post Reply

Return to “Off Topic Forum”