Best Practices For Building A Multiple GPU System

[smicha]

Thanks. Yes - I did google

[Notiusweb]

Tutor, how is using the Linux build of Octane with many GPUs, do you find it to be stable? The reason I ask is because a lot of the stuff I read online says that Linux is not always optimized or stable for applications inherently designed for Windows OS. */

*/ Citations cannot be provided due to fact that poster's internet was down for a few seconds while searching for comparisons between Linux and Windows OS, and poster did not conduct search once said internet was restored. As evidence for poster's internet issue, please refer to historical citation of Windows internet issues http://www.pcworld.com/article/204423/w ... rvers.html

[Tutor]

Circumstance prevails; So it depends ... .

Tutor, how is using the Linux build of Octane with many GPUs, do you find it to be stable? The reason I ask is because a lot of the stuff I read online says that Linux is not always optimized or stable for applications inherently designed for Windows OS. */

*/ Citations cannot be provided due to fact that poster's internet was down for a few seconds while searching for comparisons between Linux and Windows OS, and poster did not conduct search once said internet was restored. As evidence for poster's internet issue, please refer to historical citation of Windows internet issues http://www.pcworld.com/article/204423/w ... rvers.html

My research before resorting to running Linux builds of creative software with both Windows and Linux builds indicated that Linux was the preferred OS by most professional creative studios in that scenario mainly because of Linus' rock hard stability. Most of the software that I run on Linux is professional grade and thus has a Linux build (and that includes, but isn't limited to, Blender). Accordingly, I don't think it's surprising that I prefer running creative software (with both Linux and Windows builds) on Linux. My experience with Linux builds is that they aren't just on par with the Windows builds, but usually the Linux build surpasses the Windows build in stability (and, in a few cases, the quality of special features). Moreover, running applications that have Linux, MacOS and Windows builds has led me to conclude that, in many cases, the best OS for creative professionals, IF THE SOFTWARE HAS LINUX, MAC AND WINDOWS BUILDS, is first - Linux, secondly - MacOS, and thirdly - Windows (coming in last place). However, there are downsides with running MacOS and Linux. Running only MacOS has the downside of the GPU limit in pre-Sierra OSes [version 10.12, released recently in 2016]. Those older MacOS versions had a five/four/two GPU limit, depending on the OS version. Moreover, Apple's Sierra OS does not support nearly as many GPUs as does Windows running on an excellent motherboard like the one that you have. On the other hand, Linux supports running more GPUs than does either MacOS or Windows. I have not had any problems running the Linux build of Octane with many GPUs.*/ Also, the downside with running only Linux (or only MacOS) is that quite a few creative applications that I use don't have either MacOS or Linux builds. Some software designed only for Windows runs fine on the Mac if one installs a quality fake/sudo Windows environment like Parallels or VMware; but in this sort of scenario the current offerings for running Windows-only applications on Linux are hit and miss (with a lot of misses). So, in the end, a lot depends on what creative application one is using and on what OS(es) the application was designed to run. In most cases, that comes down to Windows being the most dominantly used of the three major OSes when it comes to coverage of most creative applications. For these reasons, all of my systems run multiple OSes (as I've long indicated in my signature).

*/ That may be because I don't like being a beta-tester/guinea pig with paying projects. I run software for my paying projects only after the software has had time to have had virtually all of the kinks worked out. I haven't even started using Octane V3 for any of my paying projects.

[Notiusweb]

Some more recent Nvidia talk:

Nov 2016
https://www.google.com/amp/s/www.nextpl ... highs/amp/

Sep 2016
http://www.redgamingtech.com/dont-buy-y ... a-in-2018/

[smicha]

Thanks for sharing Notiusweb.

[Tutor]

Thanks for GPU news update, Notiusweb.

[Tutor]

Company Focus Matters

Of the gamer board companies, here's why I rank AsRock the highest - http://www.asrock.com/news/events/ContentCreation/ .

[smicha]

Company Focus Matters

Of the gamer board companies, here's why I rank AsRock the highest - http://www.asrock.com/news/events/ContentCreation/ .

AsrockRack (a separate Server/WS division of Asrock) had great support. I used http://www.asrockrack.com/general/produ ... ifications and http://www.asrockrack.com/general/produ ... ifications with 2620,30,40 V4 Xeons.

Today I measured power draw on latter one and 2x1070 - 275W entire machine, and 1070 draws 85W in Octane only.

[Tutor]

...
AsrockRack (a separate Server/WS division of Asrock) had great support. I used http://www.asrockrack.com/general/produ ... ifications and http://www.asrockrack.com/general/produ ... ifications with 2620,30,40 V4 Xeons.

Today I measured power draw on latter one and 2x1070 - 275W entire machine, and 1070 draws 85W in Octane only.

I recently purchased a pair of E5-4650 ES/QBED (V.1) processors for a total of $300. As I've posted earlier, on non-quad CPU systems, those processors are seen as slightly overclocked E5-2680s. The Supermicro X9DRX mobo*/ is getting harder to find, but luckily I found two of them for $429 (USD) each; they're still backordered. Waiting in my lab (for my next X9DRX mobo to arrive) are fifteen GTX 1070's (for about $400 ea. **/ ) to begin my next system project. I've yet to decide how I will max out the 20 GPU processor per system Octane license allotment.


*/ The Supermicro X10DRX mobo costs about $200 more than does the X9DRX ( for the same no. [eleven] of PCIe slots).

**/ $400 (USD) is 1/3 the price of a Titan X Pascal [or 5,760 Pascal CUDA cores (in 3x1070) for the price of 3,584 CUDA cores in one Titan X Pascal (also, most likely lower clocked too)]. Supermicro + Linux are wonderful.

[smicha]

...
AsrockRack (a separate Server/WS division of Asrock) had great support. I used http://www.asrockrack.com/general/produ ... ifications and http://www.asrockrack.com/general/produ ... ifications with 2620,30,40 V4 Xeons.

Today I measured power draw on latter one and 2x1070 - 275W entire machine, and 1070 draws 85W in Octane only.

I recently purchased a pair of E5-4650 ES/QBED (V.1) processors for a total of $300. As I've posted earlier, on non-quad CPU systems, those processors are seen as slightly overclocked E5-2680s. The Supermicro X9DRX mobo*/ is getting harder to find, but luckily I found two of them for $429 (USD) each; they're still backordered. Waiting in my lab (for my next X9DRX mobo to arrive) are fifteen GTX 1070's (for about $400 ea. **/ ) to begin my next system project. I've yet to decide how I will max out the 20 GPU processor per system Octane license allotment.


*/ The Supermicro X10DRX mobo costs about $200 more than does the X9DRX ( for the same no. [eleven] of PCIe slots).

**/ $400 (USD) is 1/3 the price of a Titan X Pascal [or 5,760 Pascal CUDA cores (in 3x1070) for the price of 3,584 CUDA cores in one Titan X Pascal (also, most likely lower clocked too)]. Supermicro + Linux are wonderful.

Tutor,

These are exactly my thoughts. I am building 2x 7x1080 now but planning to get an 18PCI backplane from Trenton to expand 6 single slotted 1070s on a asrockrack mobo on further 14 gpus in a custom case. Waiting for quotes from Trenton.