Previously I reported a crash for this condition and suggested some better error handling was needed.
I note now it says 'Failed to Allocate GPU Mesh Object. Please reduce scene complexity' when this is encountered however from that moment the render is hung at the previous res and with the vram showing the mem occupied by the new image setting but not the geometry. Then its not possible to change the resolution (you would of course like to change to something smaller)...perhaps it is not possible to go back once the vram is over allocated? although before it just gave a black screen and a strangely fast render speed until you reduced it...
I also note that it seems to take a very long time with a big mesh to display the image in the viewport when first clicking on the node to get it rendering. perhaps it needs a 'loading and confirming license' message in place of the hourglass icon
