I don't know either how the excess power above the TDP is distributed. I do know that in a perfect world that when a PCIe card has two 8-pin PCIe connectors that those are there to allow the card to draw at least 375 watts (75 watts from the slot and 150 watts through each of the two 8-pin power connectors, but what happens when the TDP is exceeded - I'm now completely in the dark about. BTW - I tried MSI AfterBurner as you recommend, after reading a review about the top 5 GPU over-clocking utilities. The author said that MSI AfterBurner was the gold standard and that EVGA Precision X (which I use uniformly) was 2nd in the ranking. That being said, while the MSI utility may be great for the vast majority of uses and users, it has one critical flaw at which, for this endeavor, make it a bit tarnished. It gave me really crazy numbers for the power draw of one of the GPU processors on the 705 MHz Titan Z that I was testing. As for the second GPU, the numbers were completely in line with what GPU-Z was telling me. However, for the first GPU processor it was giving me numbers in the 700 to 900 % above TDP range. Thus, I think that for judging TDP that I'll stick with GPU-Z, although I do like the other features of the MSI utility better.Tutor wrote:Then The Titan Z Is The Real Deal.
smicha wrote:Tutor,
I have no knowledge how the power draw is distributed - whether this is via a pcie slot or power connectors. The (#gpus+1)*300W rule allows me for rough approximation of PSU nominal power I shall get to run certain number of gpus safely. For example, for 7 Titans Z I would go with 3x1500W PSUs (15x300W), although entire system while rendering in Octane may draw about 3000-3300W (14x200W for gpus + 200-500W for the rest). So with 3x1500W PSUs there should be plenty of safety/stability reserves (even for heavy overclocking).
PS. Sorry if I missed it - if there is a limit of 12 gpus in Octane how will you make use of 7 Zs? Will you render on 6 and excessive 1 will handle the system?
I believe that there are some important points that we haven't discussed:
(1) Imagine being in a 40 m deep hole; now imagine it's 80 m deep. Would it normally require your exerting the same amount of energy to climb out of each hole?
(2) The Titan Z card was to leap frog Nvidia so far ahead of AMD in performance as to stop AMD dead in its tracks. Little did Nvidia know what AMD had up it's sleeve until shortly after the Titan Z was announced. Then upon realization that AMD had a very fast GPU answer, even a water-cooled only dual GPU answer, Nvidia held off releasing the Titan Z for about two months while it recovered from the shock and further tweaked the Titan Z and intentionally didn't didn't give samples to a lot of reviewers. The previous dual processor Nvidia cards had not been down clocked as significantly as the Titan Z was eventually down-clocked as to its base speed */. I read some reviews where the reviewer thought that the Titan Z was a 500 watt TDP card (i.e., 2x a Titan Black), but it wasn't and yet its then was cost $2K (US) more. At its standard/base clocking, its a lot less powerful than two Titan Blacks [ In reality, less powerful than two of my original Titans]. I'm sure that there's been some binning madness to find lower base clocked GK110Bs that have a lot of boost potential under the right environment, but the standard/base clock rate is a lot lower than most imagined that it would be. Two GK110Bs fully clocked as Titan Blacks are clocked, would have made the Titan Z (non-Hydro Copper], probably, about a four or five slot card on air cooling - which would have been completely unacceptable. In sum, I believe that the Titan Z can perform like two Titan Blacks ONLY IF one can get around the heat issue and power issues moving from an extremely low base clock. And if one does get around the heat issue by using a really effective water-cooling method one will be far along in the process of reaping the benefits of having two Titan Black processors on the same card, but here's the rub. The chips that excel at that mission ( reaping the benefits of having two Titan Black processors ) probably were thrown only into the Titan Black bin and a Titan Black has a TDP of 250 watts. I'll bet that to get that kind of performance that the higher TDP numbers shown by GPU-Z and MSI (as to CPU2) for blowing past the 375 watt TDP aren't uncommon. So, if you're satisfied simply with performance greater than that of one Titan Black, for a card that has only recently dropped in price from about $2,900 (US) to about $1,500 (US) {and strangely enough, that's what the dual-295 AMD cards used to cost} then you'll likely stay within the 375 watts TDP. But if you want to fully unlock the full potential of those two GK110B's, then you're going to have to break a few eggs by blowing past that TDP of 375 even if you're running Octane Render. To be sure, you will probably not necessarily have to double the TDP of one Titan Black to achieve that goal because all of the chips requiring some power aren't doubled on the Titan Z, but most of the biggest power consumers are - more memory chips and an additional GPU processor and there're a few chips that weren't needed when all that you had was one processor on the card. Also, I recognize that the processors in the Titan Z likely have a lot better boost range than those in the Titan Black, but higher boost range isn't a purely equal substitute for a higher base clock. Thus, I believe that because the Titan Z began in such a deep base clock hole, that I'll have to continue to exceed the stated Titan Z TDP because I want maximum performance from both of my GK110Bs and staying within that 375 watt TDP doesn't appear likely to get me there.
3) I'd bet you that any two of the GPUs in the first line of your signature would out performance any one of my GTX Titan Zs if it was running only on air-cooling, at stock setting and staying within its TDP. Moreover, I wouldn't be shocked if that were the case even when I finish the mod. Sometimes, something can be so below standard that its deficiencies cannot be made up in a particular allotment of time, regardless of the amount of energy expended to make up for its low beginnings.
*/ In fact, even within the 7xx series, only some, but not all, GT series cards have such low base clocks [ http://en.wikipedia.org/wiki/List_of_Nv ... 700_Series ]. Even more significant is that the same comparison applies to the 6xx series[ http://en.wikipedia.org/wiki/List_of_Nv ... 600_Series ], to the 5xx series [ http://en.wikipedia.org/wiki/List_of_Nv ... 500_Series ] and to the 4xx series [ http://en.wikipedia.org/wiki/List_of_Nv ... 400_Series ] ( and I know that the earlier ones didn't have boost capability). So we're talking about an epically low base clock for the Titan Z. So if you were to say to me, "Keep it short, Tutor, and tell me in one sentence how low is that 705 MHz base clock for the least expensive Titan Z?" I'd respond: "It's so low that only some (but not all) of the cheapest video cards that Nvidia has made in the last 5 years are clocked that low."
Few cents?.. When I say great deals, I am thinking more like $150 for 580 3GB etc... there will be plenty (more than usual) along those lines on eBay. And the GPU prices on eBay always get much better when supply is high...
