Now that we've reached page 30, I think a Best Practices recap is in order for newcomers. So here it is:
09/23/2015
Top Ten Honey Bees
Honey, Be Curious
1) Try to stay abreast of the current and upcoming technological developments involving motherboards, PSUs (power supplies), CPUs (central processors) and GPUs (Graphics Processors). For example, for CPUs you can get their specs from
http://ark.intel.com for details concerning matters such as the number of PCIe lanes supported. You can compare specifications of released GPUs on
https://en.wikipedia.org/wiki/List_of_N ... sing_units and you get a glimpse of what GPUs are planned for the future here -
http://videocardz.com and here -
http://wccftech.com . You can get information from the website of the manufacturer or seller of your motherboard, including the number of PCIe lanes a motherboard supports.
Honey, Be Knowledgeable
2) Download, convert to PDF and read the manual(s) before purchasing any component, especially the motherboard, CPU(s) and GPUs, paying particular attention to (a) any schematics and diagrams for the motherboard’s layout of the PCIe slots and lanes and (b) especially all PCIe bios settings (and above all else an option to allocate more resources to PCIe devices such as “Above 4G” functionality - this functionality is very important to have).
Honey, Be Analytical
3) Assess your current (and try to assess your future) needs before you purchase any hardware. The more systems you have to use for 3d rendering the more software licensing costs rise. Keep in mind that OctaneRender’s license has a 12 GPU processor limit per licensed system. The word, “processor,” is key because because a dual processor video card such as a GTX 590, GTX 690 or GTX Titan Z counts as two GPU processors.
Honey, Be Observant
4) As to GPU potential, note the number of PCIe slots on the motherboard and don’t forget to count the smaller slots, such as those x4 and x1 slots - the more slots there are, the more likely the motherboard will support more GPUs, but that isn’t always the case; there isn’t necessarily a one-to-one correspondence between total number of card slots and the number of GPUs supported. Many times, slot number is just the minimum number of GPUs supported; then at other times, the number of GPUs slots of all sizes is more than the number of GPUs supported, particularly if you select a CPU that doesn’t support as many lanes as the motheroard supports. So you ask the question, "How do I connect more GPUs than there are slots?" my answer is "Honey, be patient and don't ignore the footnote, below."
Honey, Be For Real
5) For the near future, GPUs must be connected to a motherboard. Currently, Supermicro [
http://www.supermicro.com/index_home.cfm ] and Tyan [
http://www.tyan.com ] server motherboards support larger GPUs installations than gamer/enthusiast motherboards. Supermicro server motherboards are now the better of those two brand named motherboards in terms of supporting more GPUs, but this may not always be the case.
Honey, Be Consistent
6) Try to use all GPUs of the same type/model number. Some GPUs in one families, such as Maxwell, may have higher Input Output (IO) memory requirements than some GPUs in another family, such as Kepler. Higher IO Memory ("IOMem") requirements mean fewer successful GPU installs. This isn’t to say that you can’t mix it up; it just that mixing it up may have its own baggage which, at times, can get very heavy to carry as you add more and more GPUs.
Honey, Be Cool
7) GPUs, as well as other computer system components, emit heat. The more GPUs that you install, the more heat that will be emitted. Do not neglect to take into account the cooling needs (including peaks) of your system, including especially those of your GPUs. Water cooling is more effective, but costlier than air cooling and there’re other pros and cons such as leaks and noise level. Adequate GPU card spacing is important. Consider using backplates on your GPUs, particularly where the back of the card has memory modules, and assess the impact of backplates on GPU spacing. Also, don’t neglect environmental cooling options such as air conditioning because whether you cool GPUs by air or water, that hot air will ultimately get emitted somewhere nearby.
Honey, Be Both Safe and Powerful
8) GPUs, as well as other computer system components, consume electricity. The more GPUs that you install, the more electricity will be consumed/needed. Do not neglect to take into account the power needs (including peaks) of your system, including especially those of your GPUs. Depending how you define “massive” may like require the use of more than one power supply and another circuit into which the supplementary PSU(s) will be plugged. Although if you use only Octane and do not over clock your GPUs those power needs are easier to determine because using Octane alone will not usually require your GPUs to exceed the manufacturer’s Thermal Design Power (TDP). Thus, you need to know the TDP of your video cards. You can find the TDP here -
https://en.wikipedia.org/wiki/List_of_N ... sing_units if its not otherwise available to you. If you use your GPUs for other applications or you over clock them you should try determine how that affects power requirements by using power meters. Also, assess your structure’s electrical system’s capabilities and related factors such as the need for surge protectors, ground fault protection and certain the wire gauges in electrical cables. Be aware when selecting wire gauges that lower numbers are better than higher numbers, e.g., 12 gauge provides more protection than 14 gauge.
Honey, Be Agnostic/Flexible or Receptive
9) Generally, Linux will support more GPUs than Windows. Windows will support more GPUs than MacOS and the GPUs in Windows systems can be over clocked using EVGA Precision X or MSI AfterBurner utilities. In contrast, urrently only the display card’s GPU can be over/under clocked if you’re using only Linux. If you’re using MacOS, none of the GPUs can currently be over/under clocked. Emulators may, however allow you, for example, to run Windows applications on a Linux system, but as with any emulation there are pros and cons. However, even if you’re bound to one particular OS, at least be open to (however, I recommend that you seek) recommendations from other users of that OS who do (or are able to) share optimizations.
Honey, Be Creative
10) Try to maintain a level of creativity, i.e., think outside the box and I mean that literally as applied to your system’s case. Learn as much as you can about GPU connection aids such as riser cables and cards, splitters, and external chassis
*/ [see, e.g.
http://amfeltec.com/# ] (and their power needs, placement restrictions/options and other limitations/advantages) and use them where you have satisfied yourself that they will aid your attainment of your system goal(s).
*/ Devices such as splitters and external chassis allow one to exhaust a system's GPU capacity when a motherboard's slots aren't enough to do so.
N.B. Honey Bees - these best practices aren’t cast in stone - they'er just intended to make honey gathering a bit easier.
P.S. See my post(s) on page 31 for more and revised "Best Practices."
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
