Best Practices For Building A Multiple GPU System

Discuss anything you like on this forum.
Post Reply
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Tutor wrote:
rappet wrote:... .Funny thing is that few years back I made switch from CPU-rendering to GPU-rendering
and I thought CPU did not matter that much anymore when going for GPU-rendering.
About that lanes is over my head to understand at this point...just do not know the technical aspect of it.
I was just wondering about what minimum CPU is needed to get multi-GPU work properly. ... .,
Here're a few easy read, technical references re aspect of lanes...
What a joy to read. Thank you so much, Tutor.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Tutor,

If you (or any US citizen knowledgeable in US current and circuits) could help. Namely this 2000W power supply does work in EU (Poland, 230v, 50Hz current) but it does not work in USA - tested.
http://www.super-flower.com.tw/products ... 19&lang=en

As you see in the specification tab it says
Model SF-2000F14HP (230V only)
but why it says: ?
Input Voltage Full Ranage - 100V~240V
and in the overview tab there is:
AC INPUT 200Vac~ 240Vac 15A 50Hz

Does all of the above specs prove that this PSU is not designed to work in the US? If we look closely at specs of another 1600W SF PSU
http://www.super-flower.com.tw/products ... 21&lang=en
it has inputs far different and IMO this is compatible with US current, isn't it?
AC INPUT 115Vac~ 240Vac 17~10A 60HZ/50Hz

On Tuesday we'll be connecting EVGA Supernova 1600W P2 sold on US market (which in fact is rebranded SF 1600W)
http://www.evga.com/Products/Product.as ... P2-1600-X1
and has AC INPUT 115Vac~ 240Vac 17~10A 60HZ/50Hz

I was given such an opinion on the PSU if it works without any problems in the US. His remarks:
Should be fine assuming they aren't plugging it into a 15A circuit with lots of other things on it (and it's drawing the full 1600W) - in the right circumstances a 500W could have problems if there are a lot of things on the circuit.

That PSU is designed for the US market though so will be fine in the vast majority of normal house wiring circumstances as it will run on a standard 110V circuit.
Tutor, does this mean you in USA have different circuit standards? 15A or 20A? Could you please give some hints about your PSUs you are using and how they are connected? BTW We must avoid voltage regulators http://www.220converters.com/Simran-Sym ... former.htm due to space restrictions and safety.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Hi Smicha, just throwing in my 1 cent-
in US most household outlets will be 15 AMP and then some are 20 or even 30 AMP for appliances like oven, refrigerator, Central Air AC/Heater, depending on how demanding the structured appliance is.
In general, you can put in as long as you have a circuit breaker which supports it. I believe a circuit breaker panel has a max amps it can allow total simultaneous.
If I was building a house I would have an electrician put in a 30 amp circuit, or 20 if there was some coding issue in my area of.

Now, the most single GPU I had running on a 15 AMP was 6 stock air cooled Titan Z cards with a Titan X on mobo, idle.
(1) EVGA 1600 P2 - PC, Titan X, 3 Titan Z, and (2) EVGA 1600 T2 - 3 Titan Z, all on same outlet via a power strip.
When I started rendering I would run the risk of tripping, but it did not always trip. Whenever I ran OBench it did trip, always, after about one minute.
Just FYI - It never tripped with 5 Titan Z on Obench with everything, only with 6. So, I got the extension chord.
But at 3200 Watts potential, it was fine at idle, everything powered and ran, and could be run 1 by 1, or even up to 5 Titan Z, no issue.
The bottleneck of the circuit ampage came into play when I ran it all at once in a rendering session > 1 minute or so.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Notiusweb,

Thank you so much. The most important and promising from your post is that you have EVGA 1600W P2 which is working fine under heavy load on 15A circuit. Do you have any other devices connected on the same circuit, like AC, ovens,...?
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Hi Smicha,
I don't have anything else on the circuit. But when I first was getting into it and I had 3 Titan Z and 1 Titan X, I actually tripped the breaker with a 3 Titan Z render when I had a space heater going in another room on the same circuit.
The 1600W PSU never tripped or shut down with power still on, so I think they are just fine as long as circuit is not overloaded. Better to have a clear circuit while rendering, but it's okay to have other things plugged in and such, just leave powered off while rendering to avoid tripping. Also, it shouldn't matter 110 vs 220 line overall in house/apt. I think the comment you got from the guy is really just saying what I am saying, you are fine as long as you don't trip the breaker.

From google
Most modern residential circuits are 15 or 20 amps, so we're looking at a max load of either (15A x 120V =) 1800 watts or (20A x 120V =) 2400 watts before the breaker trips. The breaker will be labeled either 15 or 20.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Thank you so much.

So do you think that these labels proof that

AC INPUT 200Vac~ 240Vac 15A 50Hz - is not for USA (SF 2000W PSU)?
AC INPUT 115Vac~ 240Vac 17~10A 60HZ/50Hz - works in USA (1600W P2)...
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

smicha wrote:Thank you so much.

So do you think that these labels proof that

AC INPUT 200Vac~ 240Vac 15A 50Hz - is not for USA (SF 2000W PSU)?
AC INPUT 115Vac~ 240Vac 17~10A 60HZ/50Hz - works in USA (1600W P2)...
Maybe someone else knows this better than me, but I took a look at the Leadex Superflower 2000W product, and I noted (1) in the images it does not have a US outlet prong, (I think a EU plug?), and (2) there don't seem to be reviews from people in US, or listings for it in US stores, or even Amazon, Ebay, etc. This would lead me to believe that it won't work out of box in US. Now maybe it could work with with some adapter, but look at what we are talking about - a PSU which should have proper safety elements for the electrical line (ie grounding) as well as for the devices attached (GPU), so I would be interested what safeguards are either reduced or met with an adapter.

As for the EVGA P2 I know for a fact it works in US as I have one in the US. I actually have the T2 as well. They never failed, no issues, plug is US, works great. Now, I don't know, but maybe someone outside of US has rigged these (P2 or T2 1600W) to a European outlet with an adapter and can attest to what an adapter can do in these situations. This would give some insight as to what could be accomplished. Keep in mind I look to mod a situation to work for me with this sort of stuff, so I would not count out anything given pursuance.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

EVGA P2, T2 and all SF works in EU without any problems. SF 2000W even with a US plug converter does not work in US.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

smicha wrote:EVGA P2, T2 and all SF works in EU without any problems. SF 2000W even with a US plug converter does not work in US.
The vast majority of electrical outlets in US homes */ will safely support at best <1,800 W PSUs at sustained levels, for they were conceived with a vision of running only normal household appliances at full load. But, in the typical US detached home, there's usually one or two or three average-consumer accessible 230 VAC main power connectors that were conceived to support large power hungry appliances such as refrigerators, freezers, washing machines and dryers. The plug points and plates are typically much larger (and the assembly protrudes significantly out from the wall) and is supported by a larger diameter electrical wire/supply line.

"The Super Flower Leadex Platinum 2000 W retails for 311 euro without VAT, and in Europe, you will find it at Caseking.de and Overclockers.co.uk. You won't find it anywhere in the US since it can't be run off 115 VAC mains."... "The Leadex Platinum 2 kW is currently the strongest desktop PSU, and its high capacity only makes it compatible with 230 VAC mains since a standard 115 V socket can deliver no more than 15 amperes, which is 1725 W. During the full load test, the PSU pulled 2194.15 W out of the AC socket, which clearly shows that a normal 115 VAC socket cannot feed it. This is one of the reasons behind its limited availability as it is only sold by a few online shops in the EU, Caseking.de being one since they are Super Flower's official EU reseller. Only a handful of people obviously need such high capacity PSUs, and I am referring to professional-class overclockers that apply huge overclocks in their efforts to conquer the HWbot charts. Only by connecting three AMD Radeon R9 295X2 VGAs will you push this unit close to its official maximum power rating. But my tests also show that it had no problem delivering up to 2.2 kW over prolonged periods at high operating temperatures. This PSU's only major downside is its fan's noise output as it spins at high speeds at even light loads. I believe Super Flower could provide a more relaxed fan profile for light-mid loads, but they apparently sought to stay on the safe side, which I can't fault. This PSU is for those that will most likely run with much noisier system components than this unit's fan, so Super Flower thought it best to provide adequate cooling under all circumstances to ensure the PSU's proper operation and longevity." [ Emphasis added] [ https://www.techpowerup.com/reviews/Sup ... HP/11.html ]. For the above reasons, most suppliers of PSUs to US consumers, top out their offerings at 1,600 W.

*/ What electrical power needs can be supported at U.S. business establishments may differ, however, depending on what electrical needs were forecast at conception and the steps taken to meet those needs. The power needs of a data center running lots of "Big Iron, for instance Supermicro Fat Twin Servers and/or Super Blades" would surely be greater than those at children's daycare centers where the support needs consist of powering mainly lighting, and normal household type appliances such as TVs and standard consumer computer systems running current teaching applications.
Last edited by Tutor on Sun Oct 02, 2016 4:04 pm, edited 1 time in total.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

I felt in my bones something of that greatness will come soon, Tutor :)

Thank you both of you. I just needed a confirmation that problems with running 2000W from a USA wall is on the PSU side. Here in my house the PSU drew on 7 GPUs 1550W at peak so switching to 1600W P2 is fine. I'll post some details on a build-log soon.

Thank you again so so much.
You do not have the required permissions to view the files attached to this post.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
Post Reply

Return to “Off Topic Forum”