After testing I found out that it's because of motion blur enabled. As soon as I disable motion blur in the camera and the render settings, the camera renders fine with animation (but of course without motion blur).
Motion blur only works if you render straight to picture viewer but not if you render with the "Render All Marked Takes to PV" command for takes and also does not render when you render over deadline with takes.
I'm totally pissed.

v3.08.2 R2.1 (there was nothing in the v3.08.3 release notes that made me think this bug could have been fixed)
Win10