Best Practices For Building A Multiple GPU System

Discuss anything you like on this forum.
Post Reply
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Tutor wrote:
Notiusweb wrote: ... .
Thanks for update.
Notiusweb wrote:PS - Any more Pascalian rumours? ;)
Yes. But, I'm not taking the bait. Check you PMs for a glimpse of what's to come & when.
Ha! LOL...
Well let me ask this...have you ever been to a Pascalian restaurant? They have great food...

BTW, Tutor, do you see a less smooth maneuvering of a scene when Post Processing is enabled, for a high res scene (ie 1080p).
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

Notiusweb wrote:...

BTW, Tutor, do you see a less smooth maneuvering of a scene when Post Processing is enabled, for a high res scene (ie 1080p).
Only to run your test file at your request have I used V3. So no because I haven't had a basis to judge it at all.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Tutor wrote:
Notiusweb wrote:...

BTW, Tutor, do you see a less smooth maneuvering of a scene when Post Processing is enabled, for a high res scene (ie 1080p).
Only to run your test file at your request have I used V3. So no because I haven't had a basis to judge it at all.
Thanks Tutor. Actually, it exists in V2 as well, only less pronounced.
So, if you ran my red car file at a res of 3840 x 2160, on V2, and try maneuvering the scene with the mouse, I am curious if you will see a difference in maneuvering speed/choppiness when Post Processing is enabled vs not enabled.
I am expecting you will, as it happens to me with just the Titan X at 16x. But if you do not, man, there is definitely something that my PC may need (i.e. a CPU upgrade, etc)
YouWill`.jpg
You do not have the required permissions to view the files attached to this post.
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Or does anyone else notice if you take any scene, put the res really high to like 3840 x 2160, and then maneuver the scene with your mouse. Do you see a difference in maneuvering speed/smoothness when post processing is enabled vs disabled. My experience is that it exists in both V2 and V3, just is more pronounced in V3. (V2 requires a higher resolution to see the difference).
If you see it that's what I'm expecting, just interested if everyone comes back and says, "No, I don't see that at all..." :shock:
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

Notiusweb wrote:
Tutor wrote:
Notiusweb wrote:...

BTW, Tutor, do you see a less smooth maneuvering of a scene when Post Processing is enabled, for a high res scene (ie 1080p).
Only to run your test file at your request have I used V3. So no because I haven't had a basis to judge it at all.
Thanks Tutor. Actually, it exists in V2 as well, only less pronounced.
So, if you ran my red car file at a res of 3840 x 2160, on V2, and try maneuvering the scene with the mouse, I am curious if you will see a difference in maneuvering speed/choppiness when Post Processing is enabled vs not enabled.
I am expecting you will, as it happens to me with just the Titan X at 16x. But if you do not, man, there is definitely something that my PC may need (i.e. a CPU upgrade, etc)
YouWill`.jpg
I've used post-processing in V2 previously with high rez ( 3840x2160 ) and didn't experience any difference in maneuvering. Using your sample scene in V2, I also didn't notice any speed/choppiness issues when maneuvering. For this test, and as is and has been my past practice when enabling post processing, I used my Supermicro SuperServer SYS-8047R-TRF+ Quad LGA2011 (with four E5-4650 V1 ES - each equivalent to E5-2680 V1). On this occasion it was run with three GTX Titans (all three on Amfeltec x4 Splitter) for rendering and a GT 640 for interactivity. OS and Octane installed on 960GB OCZ 350 PCIe SSD ( http://ocz.com/breakaway ).

P.S. Same result when using rez 4096x2160.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Tutor wrote:
I've used post-processing in V2 previously with high rez ( 3840x2160 ) and didn't experience any difference in maneuvering. Using your sample scene in V2, I also didn't notice any speed/choppiness issues when maneuvering. For this test, and as is and has been my past practice when enabling post processing, I used my Supermicro SuperServer SYS-8047R-TRF+ Quad LGA2011 (with four E5-4650 V1 ES - each equivalent to E5-2680 V1). On this occasion it was run with three GTX Titans (all three on Amfeltec x4 Splitter) for rendering and a GT 640 for interactivity. OS and Octane installed on 960GB OCZ 350 PCIe SSD ( http://ocz.com/breakaway ).

P.S. Same result when using rez 4096x2160.

I am very surprised at this, but I am not surprised since it is XEON you have. I think this is a CPU thing. Yours has 8 cores 20Mb cache, mine has 6 cores 12Mb cache, thinking it's the CPU cache. :?:
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
Tutor
Licensed Customer
Posts: 531
Joined: Tue Nov 20, 2012 2:57 pm
Location: Suburb of Birmingham, AL - Home of the Birmingham Civil Rights Institute

Notiusweb wrote: ... .
I am very surprised at this, but I am not surprised since it is XEON you have. I think this is a CPU thing. Yours has 8 cores 20Mb cache, mine has 6 cores 12Mb cache, thinking it's the CPU cache. :?:
Each of my Supermicro SuperServer SYS-8047R-TRF+s has 32 cores (4 Xeons x 8 cores with each CPU having 20Mb of cache) and each system has 128G of ram. The differences in our experiences may certainly be affected by our systems' differences such as, but not necessarily limited to, our CPUs.
Because I have 180+ GPU processers in 16 tweaked/multiOS systems - Character limit prevents detailed stats.
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

Tutor wrote:
Notiusweb wrote: ... .
I am very surprised at this, but I am not surprised since it is XEON you have. I think this is a CPU thing. Yours has 8 cores 20Mb cache, mine has 6 cores 12Mb cache, thinking it's the CPU cache. :?:
Each of my Supermicro SuperServer SYS-8047R-TRF+s has 32 cores (4 Xeons x 8 cores with each CPU having 20Mb of cache) and each system has 128G of ram. The differences in our experiences may certainly be affected by our systems' differences such as, but not necessarily limited to, our CPUs.
Tutor, does the board stack CPU processing, as in x2 the cache and speed (barbell curls). Or is it only giving you additional cores and CPU bandwidth to handle tasks (dumbell curls).

128Gb DRAM? I am sure C4D appreciates that!
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
User avatar
smicha
Licensed Customer
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Guys,

Any advice when there are only 4 gpus visible in windows and there are more on board? Is there anything I can do apart from clean reinstall of nvidia drivers?

They (7) were visible all when connected via DP cable to the highest card. When connected to lowest via DVI only 4 of them are visible. When HDMI cable was used same thing. Now DP cable is connected to the 1st cards and only 4 are visible... Any clues?
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
Notiusweb
Licensed Customer
Posts: 1285
Joined: Mon Nov 10, 2014 4:51 am

smicha wrote:Guys,

Any advice when there are only 4 gpus visible in windows and there are more on board? Is there anything I can do apart from clean reinstall of nvidia drivers?

They (7) were visible all when connected via DP cable to the highest card. When connected to lowest via DVI only 4 of them are visible. When HDMI cable was used same thing. Now DP cable is connected to the 1st cards and only 4 are visible... Any clues?
Hey Smicha, Device Manager recognizes only 4 or all 7? Or does it recognize all 7, but 3 have a yellow exclamation point?
Win 10 Pro 64, Xeon E5-2687W v2 (8x 3.40GHz), G.Skill 64 GB DDR3-2400, ASRock X79 Extreme 11
Mobo: 1 Titan RTX, 1 Titan Xp
External: 6 Titan X Pascal, 2 GTX Titan X
Plugs: Enterprise
Post Reply

Return to “Off Topic Forum”