Best Practices For Building A Multiple GPU System

Discuss anything you like on this forum.
Post Reply
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

itou31 wrote:Thanks Tutor for the description.
I ckecked the X8DTH-IF that have 2 controller 5520 chipset, so up to 72 PCIe lanes (40 PCIe lanes on my P9X79 3930K). So the board is not bad !
I could have the complete rig (used) X8DTH-IF + 2 X5570 + 48Go RAM ECC for 350€. A good price I think ?
For a complete X8DTH-IF Supermicro system (if it's in good condition), 350€ or about ($389) is an excellent price.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

Notiusweb wrote:Tutor, can the supermicro boards run Windows 7, 8, 10 fine, or are there certain compatibility issues that are known?
Since itou31 has most recently asked about the X8DTH-IF Supermicro and Smicha has asked about the X10DRX, to connect with their interests also, I'll answer your question with the help of resources applicable to the X8DTH-IF Supermicro and the X10DRX. Generally, all Supermicro systems will run Windows 7, 8, and 10 fine, but the quad CPU systems and higher CPU count systems require Windows Server. The systems' manuals are the best that I've seen and Supermicro's web site is excellent for exploring the various CPU/OS options. In the case of the X8DTH-IF Supermicro, the manual states, in relevant part: "After you've installed the Windows Operating System, a screen as shown below will appear. You are ready to install software programs and drivers that have not yet been installed. To install these software programs and drivers, click the icons to the right of these items. (To install the Windows OS, please refer to the Windows OS Installation Guide posted at http://www.supermicro.com/support/manuals/.)" Moreover, this applies to most Supermicros: "Select Enabled to use the Virtualization Technology to allow one platform to run multiple operating systems and applications in independent partitions, creating multiple "virtual" systems in one physical computer. The options are Enabled and Disabled. Note: If there is any change to this setting, you will need to power off and restart the system for the change to take effect. Please refer to http://www.intel. com for detailed information." Emphasis added. For the newer Supermicro Systems such as the X10DRX, the web site resources page for the particular system will most often have an OS Compatibility link [ http://www.supermicro.com/support/resou ... _list2.cfm ] that charts and provides comment about OSes tested by Supermicro, but you need to keep in mind that there are so many variants of Linux that it would be extraordinarily difficult/onerous for Supermicro to test them all.
Notiusweb wrote:PS - how does it feel to have to parent us crazy kids :D
The following response shows how I interpret your question. For 38 years I've researched the solutions to problems for and provided advice to people and entities to earn a living. In those paid-for-service interactions, the street/flow of ideas was most often just one way. I wasn't given breathe and thought to provide just paid services. Moreover, these "crazy kids" (a group in which I count myself a member) help each other. So here, the streets are two way and there are no parents, but just kids of different ages. I feel more fulfilled and alive at age 62 (now) than I did at age 24 (when I completed my formal education). In sum, I feel that my interactions with my friends/distant cousins are worth more than gold to me and I wish the same curse on all of you.
Last edited by Tutor on Tue Jun 07, 2016 7:21 am, edited 1 time in total.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Off subject, but as we are all imagining best practicing with 20 GPU...
I'm seeing a lot of login and sign on bugs with the official V3 release :? It was like they introduced the new login system right at the end with the Betas and its been a problemo ever since.
I hope those admin pieces don't hog up too much from the true engine's development.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Notiusweb wrote:Off subject, but as we are all imagining best practicing with 20 GPU...
I'm seeing a lot of login and sign on bugs with the official V3 release :? It was like they introduced the new login system right at the end with the Betas and its been a problemo ever since.
I hope those admin pieces don't hog up too much from the true engine's development.
Just to argue with myself,
Notius, I just installed the V3 for standalone and for some plugins, and there were no issues on my end. It probably won't impede anything.
-Notiusweb
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

Notiusweb wrote:
Notiusweb wrote:Off subject, but as we are all imagining best practicing with 20 GPU...
I'm seeing a lot of login and sign on bugs with the official V3 release :? It was like they introduced the new login system right at the end with the Betas and its been a problemo ever since.
I hope those admin pieces don't hog up too much from the true engine's development.
Just to argue with myself,
Notius, I just installed the V3 for standalone and for some plugins, and there were no issues on my end. It probably won't impede anything.
-Notiusweb
Thanks for the update. I'll probably start using V3 after projects in progress and my other chores settle down.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

I think the only real point of interest with V3 is Volumetrics, which if using Houdini, Lightwave, or C4D is a big deal.
I do like the now doubled environment option and will be looking forward to the toon shader. Although there are many ways of arriving at that effect.
image.jpg
image.jpg
I employed no post, I just whacked the GPUs really hard and the image changed on 2nd render.
You do not have the required permissions to view the files attached to this post.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Here is a Best Practices question. Do graphic cards go stale if unused for a while?
Does it help overall performance to have them running on a regular (ie daily) basis? In my case this may be more if a scenario because I'll operate in 2 modes:
1) design mode - only main GPU active, others powered off
2) render mode - all GPUs active, all on

This may be a question involving water-cooling too, does water go stale and bother the ridge chamber where it passes the GPU chip?
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

I am back guys, f...g back. With something completely new. Stay tuned for the log and valuable advice. Thanks for all, for your support.

Image
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Seekerfinder
Licensed Customer
Posts: 1600
Joined: Tue Jan 04, 2011 11:34 am

Sounds like you cracked it! Well done, Smicha...
Seeker
Win 8(64) | P9X79-E WS | i7-3930K | 32GB | GTX Titan & GTX 780Ti | SketchUP | Revit | Beta tester for Revit & Sketchup plugins for Octane
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Seekerfinder wrote:Sounds like you cracked it! Well done, Smicha...
Seeker
Yes, on a new mobo ;)
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
Post Reply

Return to “Off Topic Forum”