Blackouts, help!

[Rico_uk]

Having a major issue with machines blacking out and rebooting when they're under heavy load in Octane. They occur most frequently when I adjust something like a material colour, or rotate the camera. If it's rendering and I leave the scene alone it's quite stable. It's only when I change something.

The odd thing is that it is happening on two of our machines which have identical hardware, so this really rules out any hardware failures as it's happened on both machines at the same time. It started a few months ago, before that the machines have both been stable since we made them 18 months before.

Things we've tried:

1) BIOS - increasing VCore voltage, and disabling any power saving features.
2) Plugging in via a UPS.
3) Had an electrician out to monitor mains voltages, everything was fine and constant, no drops in voltage. We're also plugged straight into the mains, tried different sockets, etc.
4) Rolled back NVIDIA drivers to 358.50, and of course tried latest drivers.
5) Tried Octane 3.02 and 3.03.
6) Fresh install of Windows 10
7) Stress testing with OCCT and Furmark, system is stable running these.
8) Using the machines as network render nodes only, and they still crashed.
9) Tried Max 2014 and 2016.

Only things I can think of that are left to try are:

10) Install Windows 8.1 (we upgraded to Windows 10 a few months ago, and the problems happened shortly after).
11) I've ordered a new PSU (1200w which is the same as we have now which is more than enough). I also doubt this will solve it as we have two machines with the same problem.
12) Export the scene from Max to Octane Standalone to see if it still crashes there. I imagine it will as I can sometimes get a crash from using GPU accelerated rendering in Premiere.

Hopefully someone can help and suggest anything else we can try, or even some way to monitor what is happening when the crash happens? This has really hindered us as we use Octane as our primary renderer, we've got 2 other machines that are still working, but we've essentially halved our render power by having these two machines out of action / unreliable.

[Zay]

Sounds like there's not enough power for some reason. If you have more than 1 card then try to take 1 out and see if it runs without rebooting.

[Rico_uk]

One machine is water cooled so taking out cards is a real pain, but on the other machine we've gone back to one card and it still would turn off, although it would take a lot longer of spinning around and changing materials before we'd get a crash.

Just managed to get Octane Standalone to make the machine blackout and reboot.

Also just realised I made a mistake in the first post, the PSUs are 1500w and not 1200w.

Also just reformatted one of the machines and did a clean install of Windows 8.1, and it still crashed on the same scene.

So it's either:
- a hardware issue that happens to be a coincidence that it's occurred on two machines at the same time after 18 months of solid use.
- an Octane issue from a recent update (I'll install an Octane 2 version and test now.
- something else that I can't think of?!

[mark0spasic]

Maybe if you have displacement in your scene try to disable it. I had a couple of times problems with displacement.

[smicha]

When you install the new PSU please send feedback if the problem still persists.

Please try to connect 1 and 2 gpus (out of 3) and see if the blackouts happen.