External Graphics Cards PC

[Notiusweb]

Hello Notiusweb,
Is that 1,197.89 OctaneBench score for 6 Titan Zs for your system? If so, congratulations!

Hi Tutor, thanks

Yes, Bench for a while was not allowing new scores, and is still freezing up when a higher overclock is used with 10 or more cores. So, I can run a much higher overclock and use the software fine, but I can only overclock so much in Octane Bench in order to compute and upload a score like I did with the 12 Zs. Also, still investigating a 14 GPU boot

Interesting, the 3 watercooled cards seem to pick up the brunt of processing, because the 3 Amfeltec card temps do not go as high as they did when all cards were air cooled. It's like they all go out to dinner, but the watercooled cards take care of the tip.

I wonder if there was ever any communication from the parties that posted the 9x 980 Ti, or the 8x Titan X scores. Curious if they had an inclination to add more GPU cores to their rigs. 12 Titan X at posted score could yield 1600+. (What's that,1 Volta? )

Regards!

[rwilliams23]

Hi Octaners,
Newbie here. I'm trying to get my mind around using 8-12 external graphic cards with Octane and am amazed at what you guys are doing. Maybe its posted somewhere, but I haven't seen it: Is there data showing the changes in Octane performance increase as you add more GPUs? Is there a "sweet spot" for the number of graphics cards? I mean, do you start suffering diminishing returns as you add more cards? Or is the performance increase more or less linear as you add them? If this has been answered elsewhere, my apologies. Could you point me where I can find the answer?

Thank you!

[glimpse]

Hi Octaners,
Newbie here. I'm trying to get my mind around using 8-12 external graphic cards with Octane and am amazed at what you guys are doing. Maybe its posted somewhere, but I haven't seen it: Is there data showing the changes in Octane performance increase as you add more GPUs? Is there a "sweet spot" for the number of graphics cards? I mean, do you start suffering diminishing returns as you add more cards? Or is the performance increase more or less linear as you add them? If this has been answered elsewhere, my apologies. Could you point me where I can find the answer?

Thank you!

hi, if You take alook into menu, under Resources You will find OctaneBench. That's a tool for testing performance =) from this & some data gathered before we could say simply that GPUs scales linearly, the only issues You may run into is boot problem that might start after 7 cards..running anything more might be a bit tricky, but it isn't impossible =)

performance of soem rigs with multiple cards get lower speed per card, ot because of scaling issues, but rather because of temperature - if You can't get all the heat out of the thermal throttling appears & cards start to run at lower speeds =) however if you manage to deal with that for instance using watercooling, You will not loose, but actually gain performance =)

Sweet spot in my eyes would be around 7-8 cards per system..You can seat them inside propper motherboard or use GPU splitter, Clsuters or other gear in order to be able to hook them, but still out of problems with boot, special motherboards, etc. however it's my opinnion only & others might have different perspective on this matter =)

[rwilliams23]

Sweet spot in my eyes would be around 7-8 cards per system..You can seat them inside propper motherboard or use GPU splitter, Clsuters or other gear in order to be able to hook them

This is terrific information, tomGlimps--something for me to shoot for. BTW--I really like your web site. Extremely informative and your work is inspiringly elegant.
Ric

[Tutor]

Hi Octaners,
Newbie here. I'm trying to get my mind around using 8-12 external graphic cards with Octane and am amazed at what you guys are doing. Maybe its posted somewhere, but I haven't seen it: Is there data showing the changes in Octane performance increase as you add more GPUs? Is there a "sweet spot" for the number of graphics cards? I mean, do you start suffering diminishing returns as you add more cards? Or is the performance increase more or less linear as you add them? If this has been answered elsewhere, my apologies. Could you point me where I can find the answer?

Thank you!

In addition to savoring Glimpse's sage advice, you might, when you have the free time, read this entire thread {using 8 or more GPU cards with a single system will very likely will entail going external unless you have a very, very fat purse or wallet}, if you haven't already done so, and read this thread - viewtopic.php?f=40&t=43597 - too. If your quest is to use 8-12 graphics cards, then I'd suggest that you consider using a Supermicro motherboard such as the MBD X9DRX [ http://www.supermicro.com/products/moth ... DRX_-F.cfm ] or the MBD X10DRX [ https://www.supermicro.com/products/mot ... x10drx.cfm ] or as Noticeweb has so masterfully and spectacularly done use a high end enthusiast motherboard with a plethora of PCIe slots such as the ASRock X79 Extreme 11 [ http://www.asrock.com/mb/Intel/X79%20Extreme11/ ]. A motherboard's having very many PCI-e slots such as 7 or more of them is usually a dead-give-away that it'll support 8 to 12 GPU processors. I used the term "processors" to account for those dual GPU processor cards such as the GTX 590s, GTX 690s and Titan Zs, etc. Each GPU processor counts towards Octane's 12 GPU per machine license limit. Thus, for example, six Titan Z cards would be the most you could use to render a project with OctaneRender on a single system per your Octane License.

[Notiusweb]

Postby rwilliams23 » Fri Oct 30, 2015 1:02 am
Hi Octaners,
Newbie here. I'm trying to get my mind around using 8-12 external graphic cards with Octane and am amazed at what you guys are doing.

Hello Ric,
I found you will have a linear progression as so long as you can work through any limits of your hardware/motherboard (like Tutor mentioned) and heat/system stability (like Tom G mentioned). In my case, I found that at 10 GPU, Octane Bench will crash during its script if I overclock to a certain point, whereas the standalone and plugin I am using will not crash. So in usage, I have been able to overclock higher than what Bench has captured for me at this point (1200 with 12 Zs). So, the sweetspot will depend on your hardware limitations, as well as ingenuity.

What is your current rig like, are you contemplating expanding, or are you in the process with any pending decision to make?

[Notiusweb]

Postby glimpse » Thu Sep 10, 2015 7:54 pm
so, now it's time to put those under water.. =DDD rough math would lead to ..12 x 110 ~> 1320???.. =DDD

Hello! I completed the waterblock on the 3 Amfeltec-connected cards and now have all 6 Titan Zs watercooled.

It was tricky...The Amfeltec did not want to cooperate with the pump at first, as it positions the cards at start with challenging vertical air locks, and I wanted to arc the tube connections so they never ran with the potential to drip over the center PCI board, which meant I had to run 12"+ tubing SLI connections between each card in orbital paths. (As opposed to my 1" USB riser Swiftech SLI connections directly between each card...) I was getting little to no flow and I was considering I would have to return to air. What I did to overcome was to fill all 3 of the waterblocks with water before-hand manually with a funnel through the main tubing line, and then connected the main tubes to the Aquaduct, and then ran the pump to prime the loop. Since there was water already throughout the toughest parts of the loop, the head pressure was able to push the liquid without fighting any giant air pockets, and I was in business. Not only that, but I was able to add a supplemental XSPC AX480 Quad Fan Radiator on the loop to boost cooling. I had already added one for the 3 USB Riser cards which dropped my coolant by up to 10 degrees and the cards themselves by up to 20 degrees in extreme rendering cases, so I wanted one for the 3 Amfeltec cards as well. Low and behold, the Amfeltec cards run cooler than the USB riser cards by 10 degrees at times. I got to hand it to the Amfeltec, a PIA to set up, but it takes care of its cards and is a great rendering weapon.

Side of the 3 USB 3.0 Riser Titan Z loop

Side of the 3 Amfeltec Titan Z loop

2 Aquaduct 720 XT Mark V, 1 per loop

2 supplemental XSPC AX480 Quad Fan Radiators, 1 per loop

3 Watercooled Titan Zs on the Amfeltec 4-Way GPU Cluster - (Swiftech Komodo Waterblock).
Tubing runs outside and under the cluster.

[glimpse]

Thanks for sharing Man. That's impresive to say the least!

(You probably do not have cat =DDD Yesterday I open my box for inspection soon after my furry friend went to check if all of wires are needed with interntion make some "hardware changes =DDD"

[Notiusweb]

Heading for some colder weather now, we'll probably see some higher Bench scores

[Seekerfinder]

Heading for some colder weather now, we'll probably see some higher Bench scores

We are going to see some extreme bench scores over the next next 8 weeks or so, despite heading into mid summer... in the southern hemisphere!

Seeker