4 GPU Build. Need your opinion

[xavier.altius]

Hi Octane people!

In my company we are looking for a powerful pc ONLY for GPU rendering, with 2 GPUs at the beginning, and later we will include two more.

For tree weeks I was looking for knowledge about PCI lanes, GPU's, temps, CPU...

And this is the computer I think is more suitable for us:

CPU- AMD Threadripper 1920X 400€
CPU Cooler- EVGA CLC 240mm All-In-One RGB LED CPU Liquid Cooler, 2x FX12 120mm PWM Fans 120€
CASE- Thermaltake Core X9E-ATX Cube Chassis 170€
MOTHERBOARD- MSI X399 SLI PLUS 320€
GRAPHICS CARD- 2x EVGA RTX 2080Ti XC Hybrid 2800€
RAM- G.Skill Trident Z RGB DDR4 3600 PC4-28800 64GB 4x16GB CL17 859€
STORAGE- Samsung SSD 970 Pro NVMe PCI-E M.2 512 GB 173€
PSU- EVGA SuperNOVA 1600 T2, 80+ TITANIUM 1600W, Fully Modular 500€
OS- Windows 10 Home 110€
FANS- 2x Noctua NF-A20 PWM 60€

TOTAL= 5512€

The idea is to put 4 hybrid gpus inside with all the radiators at the top of the Case (The thermaltake core X9 is perfect for this setup)
The only thing that i'm not sure is the CPU, the Threadripper 1920x, this cpu has 64 PCIe Gen3 Lanes, it's perfect for 4 GPU but everybody uses Intel... And I don't know why...

Do you think this Setup is balanced?
I need you opinion guys.

Thank you !!

[Lewis]

GPUs will be too close to each other, regardless of them being cooled by Liquid they still have FAN for VRAM and maybe other components and when you stack them 1,3,5,7 GPUs 3 ad 5 are gonna be very hot/throttle and GPU 1 (top) will also be too hot. Only first left GPU (GPU 7 on MB) will have sufficient airflow for the fan.

In such combo only full Liquid cooling will work or you need 1 slot apart for Airflow/fans but then you need MB with at least 5 PCI-E full sized slots so you can use slots 1,4,7.

[FrankPooleFloating]

Lewis, I just don't have the problem you speak of. My middle hybrid is just cool as the first. The third is maybe a couple degrees cooler than the others. I simply never have any heat problems whatsoever. Keep in mind bro that they (hybrids at least) are designed to bring in air at the front end of the GPU, and exhaust it out the back. Having two 200mm fans in the front of my X9 with no drive bays present likely helps, but I am not 100% sure it is even necessary to have the low temps I do.

Xavier, doesn't Win10Pro have some bells and whistles that make it worth the extra bucks for workstations and servers (even render nodes)?... And if space is not an issue the X9 is absolutely your guy as you know.. but you prolly could easily get away with an X5. Dude, I swear, if my case was empty and the mobo tray was out, my 5'2" 104lb wife could possibly scrunch herself into a ball and fit in this damn case. It is effing huge.

[Lewis]

Lewis, I just don't the problem you speak of. My middle hybrid is just cool as the first. The third is maybe a couple degrees cooler than the others. I simply never have any heat problems whatsoever. Keep in mind bro that they (hybrids at least) are designed to bring in air at the front end of the GPU, and exhaust it out the back. Having two 200mm fans in the front of my X9 with no drive bays present likely helps, but I am not 100% sure it is even necessary to have the low temps I do.

Hi Frank,

check this video if you don't understand what I talk about, that describes important difference in multiGPU setup.

https://www.youtube.com/watch?v=0domMRFG1Rw

I've recently had user who installed 2*2080Ti near each other and TEMPS are 80 for bottom and 88C for top GPU and Top GPU has no enough "air" to breath properly (since it's not blower style anymore) and it's rendering fine but clock of that GPU downlocked itself (automatically Nvidia adjust clocks with temperature) to 1250 MHZ while one what has enough Air is at 1750 MHz which is BIG difference of 500Mhz (lot less OBpoints when GPU is low clock).

My 2080Ti work on 1850-2050 MHZ depending on location/airflow position in case but my have separate slots/more air and they are spaced between themself and temps go form 68-79C depending on location in case (and i have huge double-width Thermaltake W200 tower)..

[FrankPooleFloating]

How many of yours are hybrids Lewis? You understand that the vast majority of my three GPUs heat is going out the rads at the top of my case, right? And that bloke in video did not have with him or mention hybrids at all, from what I could tell, skimming through real quick. From what I have gathered about hybrids, all heat from the core is taken care of by pump/rad and the fan easily takes care of VRAM and VRMs...

[Lewis]

How many of yours are hybrids Lewis? You understand that the vast majority of my three GPUs heat is going out the rads at the top of my case, right? And that bloke in video did not have with him or mention hybrids at all, from what I could tell, skimming through real quick.

As i said in my first post, if is not full water cooler it still suffers from lack of fresh AIR 'coz that fan has no room to suck enough amount of air (even if is just for VRAM and not for chip alone VRAM heats same tmeperatures form wha ti hav eon my readings and i tested with IC Heat gun also). Sure hybrid is cooler than regular AIR only but those which he posted have no back GPU cutout for pulling AIR in (like old Blower style GPUs) so they still need air on side of GPU so stacking 4 of those each to other is not great solution and temps will be higher and MHZ will downclock.

I've built dozens of multiGPU systems and tested in 7-8 different cases and always better to have one slot empty/for freesh air unless is full WaterCooled system then it's great cool and quiet.

[FrankPooleFloating]

Not out to argue with anyone, and just giving my testimony to my experience... but with EVGA's 2080Ti Hybrid's Dedicated Memory Plate "A dedicated memory plate that makes direct contact with the water cooling block. Giving the lowest memory temperatures possible", I wouldn't hesitate to slap four in beside each other. If they occasionally throttled down, so be it.. being the alternative is to be left with a mere two GPUs on a four-gpu mobo, or doing a full loop, which is crazy expensive and time-consuming. Yeah, I'd personally take my chances. I trust EVGA.

[senorpablo]

How many of yours are hybrids Lewis? You understand that the vast majority of my three GPUs heat is going out the rads at the top of my case, right? And that bloke in video did not have with him or mention hybrids at all, from what I could tell, skimming through real quick.

As i said in my first post, if is not full water cooler it still suffers from lack of fresh AIR 'coz that fan has no room to suck enough amount of air (even if is just for VRAM and not for chip alone VRAM heats same tmeperatures form wha ti hav eon my readings and i tested with IC Heat gun also). Sure hybrid is cooler than regular AIR only but those which he posted have no back GPU cutout for pulling AIR in (like old Blower style GPUs) so they still need air on side of GPU so stacking 4 of those each to other is not great solution and temps will be higher and MHZ will downclock.

I've built dozens of multiGPU systems and tested in 7-8 different cases and always better to have one slot empty/for freesh air unless is full WaterCooled system then it's great cool and quiet.

I have 3 hybrids next to each other. They always run substantially cooler than a single air cooled card, like 60%. It makes no difference if they run for 5 minutes or 24 hours.

Think about it. The water cooling takes all the GPU heat away and out of the case(if you have your radiators correctly blowing outward), which is the vast majority of the heat produced. Then, the blower fan--which cools everything on normal air cooled cards--is left to cool everything else. There's more cooling than is needed by a large margin and it makes no difference how many cards you stack next to each other.

[xavier.altius]

Thank you all guys!

In a couple of weeks I will know if we buy this Build.

The only thing that i'm still not sure is the CPU, AMD 1920x looks nice (it have more than 60pci lanes) but everybody uses intel, WHY?

I always bought intel CPU's, always, but the PCI lanes the 1920x have instead of an intel CPU...

I need you opinion with the CPU.

[smicha]

Xavier,
What Frank and Lewis are saying are great remarks. What I may add (as I produce now custom machines with up to 11 GPUs):

1. Platform you choose is the most important for long run stable operation. Years ago I used gaming components and these worked more or less good, sometimes more, sometimes less And some heavy changes into BIOS were required to make them stable. So don't be tempted by price point - it is better to have 1 GPU for now less and better mobo, pro ssd, pro ram... than pushing to a limit cheaper stuff. Shortly speaking since I moved to Supermicro platform with Xeons and ECC RAM support I don't have issues

2. Be careful with PCIe lanes count. TH4 does not give you all of them for GPUs. Only some PLX mobo solutions will give you PCIe increase, while on dual xeons we have true 96 lanes and ability to run 11 GPUs at 8x or some Supermicro support 4 GPUs at 16x bandwidth, rest at 8x.

3. Case X9 quality (I built many machines on it till I started pure alloy production cases) is rather cheap, especially at the mobo tray area and SSD mounts. If you put 4 GPUs and hybrids rads on its top you'll have limited access to mobo itself. For the money it's ok but there is one more question I always pose to myself - how much harm have TT done to Case Labs that it went bankrupt... Competition is good for a market, unless it is honest.

4. Heat is the main issue, especially for RTX 2080ti - with 4 of them you'll draw 1000W easily and there are 3 ways: water (silent and cool), hybrids (cool, some buzz on their built-in makes me crazy, but rather quiet if you mod their rads fans to control their speed), and air but with extra spacing between gpus and superior case airflow....