Best Practices For Building A Multiple GPU System

[Tutor]

PS. There is only 1st and 2nd powered by cables. Is there any way to get them all (7) in device manager only from 75W pcie power, without cables?

Not likely, particularly if there're IO space issues. Try my suggestion in post just before this one, with all GPUs fully powered on at all times.

Make sure that "above 4g" is enabled and for registry hack make sure to use 32 (not 64) bit words.

[smicha]

So here is the problem: at the moment I cannot power cables to all gpus - no one is there in an office. With 2 gpus powered (1st,2nd slot) (and with 5 gpus powered recently, 1st, 3rd, 5th, 7th, 2nd slots) there is no more than 4 gpus visible in regedit. 2nd slot (4th, 6th) - x8 slots - are not visible. Tomorrow I will power all of gpus and see if it is changed.


PS. No driver is installed for the image.

[Tutor]

So here is the problem: at the moment I cannot power cables to all gpus - no one is there in an office. With 2 gpus powered (1st,2nd slot) (and with 5 gpus powered recently, 1st, 3rd, 5th, 7th, 2nd slots) there is no more than 4 gpus visible in regedit. 2nd slot (4th, 6th) - x8 slots - are not visible. Tomorrow I will power all of gpus and see if it is changed.


PS. No driver is installed for the image.

If you convert time you want me to stay at this with you to US Central time (it's now 4:42 pm) and tell me when you need for me to be available, I'll stick with you through all of this my friend.

[smicha]

Thank you so much. I'll try to tweak some things more for about 1h. It's +7h in Poland. 11:45 PM now. I'll send you a screenshot of how it looks with nvidia driver installed (1st, 2nd slot powered b y cables, all 7 in pcie slots). I would like to see 2nd gpu in the 2nd slot working (visible in regedit).

[Tutor]

Thank you so much. I'll try to tweak some things more for about 1h. It's +7h in Poland. 11:45 PM now. I'll send you a screenshot of how it looks with nvidia driver installed (1st, 2nd slot populated). I would like to see 2nd gpu in the 2nd slot working (visible in regedit).

Just keep in mind that how slots are arrayed in the registry may not exactly match how they are physically arrayed on the motherboard.

Also see, https://msdn.microsoft.com/en-us/librar ... s.85).aspx and see this first https://msdn.microsoft.com/en-us/librar ... s.85).aspx .

[smicha]

So here is the situation after installing nvidia - although the 2nd gpu in the 2nd slot is powered with cables is not visible at all, neither 4th, nor 6th (not powered with cables)

[smicha]

Is this important to have a primary gpu as 0000 - in my personal machine with 4 gpus the main gpu is marked as 0003.

[Tutor]

So here is the situation after installing nvidia - although the 2nd gpu in the 2nd slot is powered with cables is not visible at all, neither 4th, nor 6th (not powered with cables)

In registry for each video card (even those not fully powered) Add two 32 bit dword values by right clicking on, for example, 0000 and selecting add 32 bit dword:
DisplayLessPolicy
{then repeat process and add}
LimitVideoPresentSources
and set each value to 1 {double click on each entry {but do this one at a time} that appears in right side within larger window and enter "1" (without the quotation marks) in popup window that opens; then close each window after entering "1." This is done serially for each 32 bit dword.}
5) Once these have been added to each of the video cards, shut down Regedit and then reboot.

[smicha]

And last observation from gpuz - if 1st gpu works at x16 it takes some speed (lanes?) form other gpus preventing them from being visible? May this be true/possible? Hence, auto settings in bios for pcie speed shall be changed manually to gen2?

[smicha]

So here is the situation after installing nvidia - although the 2nd gpu in the 2nd slot is powered with cables is not visible at all, neither 4th, nor 6th (not powered with cables)

In registry for each video card (even those not fully powered) Add two 32 bit dword values by right clicking on, for example, 0000 and selecting add 32 bit dword:
DisplayLessPolicy
{then repeat process and add}
LimitVideoPresentSources
and set each value to 1 {double click on each entry {but do this one at a time} that appears in right side within larger window and enter "1" (without the quotation marks) in popup window that opens; then close each window after entering "1." This is done serially for each 32 bit dword.}
5) Once these have been added to each of the video cards, shut down Regedit and then reboot.

Doing it now and report it in minutes.