Have to say, You got quite a kit =) powerful beast with a lot of possibilities.
My budget... yeah what can I say. I rather not spend too much. But I rather spend a little bit more if it's significantly better.
without any other upgrades (just purely looking to GPU) You can have up to 3x GPUs (considering You already have one, so two more - let's say TitanBlacks, 780s or single TitanZ - as it's dual GPU card - all of them are based on the same technology, having GK100 chips under the hood)
as mentioned, 780 6GB (for 400-500$) will give You the best value (but hard to find), then TitanZ (if You manage to find it around 1500$) and last but not the least of these will be TitanBlack for around $1000.
Titan Black might look least attractive as it offers less value compared to something like 780, but on the other side it has reference design blower type cooler, that emits all the heat out of the case. In Multi GPU rigs with air cooling that's a benefit, because if GPU reach certain temperatures You start loosing performance because of thermal throttling (read more in that long article). Non reference design circulate all the heat inside the case (for this You need extra airflow to get rid) & then TitanZ split - half of the heat is dumped through rear IO while the half still stays inside the case..
- what ever You choose keep in mind to get better fans in front (20-50$) investment into pair of fans will give You better temps and performance with controllable noise levels =)
Personally I'd go for TitanZ & to max out Your current config. Without any additional investment (maybe only into pair of fans, though You should check if Your PSU can handle load stably) this simple upgrade would let You to render 3x faster than You do now on single Titan Black.
Spacing cards out so both of them Would have plenty of space between & would not startve from fresh (cold air is a good practice).
but why Z, You may ask. Wwll, because later You with simple upgrade if You ever choose watercooling will allow You to have dual GPU card inside Your rig while taking only two slots (in simple words power density is twice better than what Titan Black or 780 on air could offer).
Let's jump to software, we're done with hardware =)
I wonder if it's better to have one expensive card or two less expensive ones (Does this apply: the chain is as strong as the weakest link?)
The way OctaneRender works is that You have to fit scene into vRAM of every card =) so if You have let's say 3GB & 6GB card Your usable amount of vRAM is 3GB - so in this case yeah, Your concert is straight to the point..
I mentioned out-of-the-core texture feature - that allows You not to be limited by vRAM, but that comes in expense of speed drop..(because of slow interface & additional process required to refresh data needed for rendering process)
While rendering Your whole scene should fit inside vRAM (Geometry,Textures & let's say "frameBuffer") - geometry is not so heavy (You'll likely get into program limmit of 19mln tris, if I remember good, before You'll reach vRAM ceiling). Textures is another thing that eats up ram, but it purelly repends at what resolutions You're working (& with out-of-core textures You can find a way around).
What is going to eat a lot of space is resolution : let's say high res, 8K (square) -
see here- will eat You up to 2.1GB - jsut to be able to render the output =) so..if You ask me..I would not go for less than 6GB..considering limitations & the fact You probably don't want to buy new gear once more demanding project is going to arrive =)
BTW really interesting reads on the links you gave me!
It does make my head spin though to dive that deep into hardware and cooling. But as I read it, it can surely pay off to have watercooling for better performance
Yeah, as Sebastian nicely puts - 40% difference in terms of performance is really justified, especially when You can spend less.
I wouldn't call that a necessity, more like a high end solution in a case when You need more power but could not physically fit more hardware =) or, when You want silence - that's costly..but as we nailed down that topic - totally worth the effort!
so in the end,
I'd say TitanZ makes sense for me in Your case..
if You wan't to have less spending get 780 6GB =)
cheers
If I may recommend something - try to acquire the same cards as you have (if you don't want to sell them). If you reach 4 of them it may happen that you are not be happy with their temps and then watercooling comes to play. And bridging/joining different cards with water connectors may be difficult or even impossible.
valid point from Smicha
(the same Sebastian from article =)