Batch render from command line crash

[sampakinpa]

Hi,
I'm using a batch file to render several maya animation files. after a while, completely randomly, the batch crashes and a popup message appears: "MayaBatch application has stopped working".
when I cancel the popup window it continues to the next file in the batch file queue.
As I said it is completely random, it can happen on any file and on any frame.
I'm using OctaneRender for Maya v1.11b beta and OctaneRender standalone v1.10

Please help me.

[sampakinpa]

I paste bellow the content of the CMD text after I enabled the verbose mode in the plugin setting, it is only the end part because I can not find the full log file anywhere.
The render crashed on frame 24:



Number of UV indices: 171944
0:04 ...done setting Mesh data in 0:00
0:04 Voxelizing...
0:04 ...voxelized in 0:00
0:04 ...done creating meshes
0:04 Octane mesh data is set to render engine.
0:04 Octane kernel set: Path Tracing

****************************** THIN LENS CAMERA ***
*
Name: cameraShape1
Orthographic: 0
Film width: 35.999928, Focal length: 45, calculated
Aperture: 0.03
Auto-focus: 1
Focal depth: 2.376271
Stereo: 0
Stereo distance: 0.001
Near clip depth: 1e-007
Camera: X: -1.894747, Y: 3.952579, Z: 0.290888
Target: X: 0.773788, Y: 1.149091, Z: -0.1129
Up: X: 0.712307, Y: 0.693543, Z: -0.107783
IMAGER:
Exposure: 0.8
FStop: 2.8
Iso: 100
Gamma: 0.7
Response: 400
Vignetting: 0.3
Saturation: 1
Hot pixel removal: 1
Premultiplied alpha: 0
Min display samples: 1
Dithering: 0
White saturation: 1
POSTPROCESSOR:
Enabled: 1
Bloom power: 20
Glare power: 10
Glare ray count: 4
Glare angle: 31.263159
Glare blur: 0.0491789

****************************** TEXTURE ENVIRONMENT
Sun/Sky Node: octaneSunSky
Power: 2.2
Rotation: X 0.5, Y -0.2

0:06 Octane engine: Target node is set...
0:06 Octane: Scene export complete.
0:00 ...done
Octane: Refreshed image, 490 samples per pixel of 1
Octane: Refreshed image, 1065 samples per pixel of
Octane: Refreshed image, 1613 samples per pixel of
Octane: Refreshed image, 2103 samples per pixel of
Octane: Refreshed image, 2725 samples per pixel of
Octane: Refreshed image, 3511 samples per pixel of
Octane: Refreshed image, 3982 samples per pixel of
Octane: Refreshed image, 4512 samples per pixel of
Octane: Refreshed image, 5109 samples per pixel of
Octane: Refreshed image, 5780 samples per pixel of
Octane: Refreshed image, 6533 samples per pixel of
Octane: Refreshed image, 7384 samples per pixel of
Octane: Refreshed image, 8339 samples per pixel of
Octane: Refreshed image, 9384 samples per pixel of
Octane: Refreshed image, 10425 samples per pixel of
Octane: Refreshed image, 11470 samples per pixel of
Octane: Refreshed image, 12510 samples per pixel of
Octane: Refreshed image, 13552 samples per pixel of
Octane: Refreshed image, 14597 samples per pixel of
Octane: Refreshed image, 16000 samples per pixel of
Octane: Frame 23 is rendered, saved "maya" image to
ing/images/Ring_8825_Turn02a.023.iff"


Octane: starting animation of frame 24...
Octane: Scene scale factor: 1
0:00 Device "1: GeForce GTX 690 (compute model 3.0
0:00 Device "2: GeForce GTX 690 (compute model 3.0
0:00 Device "3: GeForce GTX 690 (compute model 3.0
0:00 Device "4: GeForce GTX 690 (compute model 3.0
0:00 Device "5: GeForce GTX 690 (compute model 3.0
0:00 Device "6: GeForce GTX 690 (compute model 3.0
0:00 Device "7: GeForce GTX 690 (compute model 3.0
0:00 Device "8: GeForce GTX 690 (compute model 3.0
0:03 Octane engine: creating meshes...
0:03 Gathering Mesh objects data...
Octane: Object "Ring_8825RND:polySurfaceShape1" is
Octane: Object "Ring_8825RND:polySurfaceShape2" is
Octane: Object "polySurfaceShape1" is not visible,
Octane: Object "polySurfaceShape2" is not visible,

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:D_Whit
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:G_Whit
****************************** MESH DATA **********
Number of vertices: 10775
Number of normals: 10775
Number of UVs: 11538
Number of polygons: 10688
Number of vertex indices: 42752
Number of normal indices: 42752
Number of UV indices: 42752
Number of material indices: 10688
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:G_Whit
****************************** MESH DATA **********
Number of vertices: 12800
Number of normals: 12800
Number of UVs: 12800
Number of polygons: 12288
Number of vertex indices: 49152
Number of normal indices: 49152
Number of UV indices: 49152
Number of material indices: 12288
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:Prong_
****************************** MESH DATA **********
Number of vertices: 18650
Number of normals: 18830
Number of UVs: 19620
Number of polygons: 18688
Number of vertex indices: 74752
Number of normal indices: 74752
Number of UV indices: 74752
Number of material indices: 18688
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00

0:03 Setting Mesh data of mesh: Ring_8825RND:Solite
****************************** MESH DATA **********
Number of vertices: 217
Number of normals: 752
Number of UVs: 657
Number of polygons: 161
Number of vertex indices: 752
Number of normal indices: 752
Number of UV indices: 752
Number of material indices: 161
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:03 ...voxelized in 0:00
0:03 ...done,
0:03 Gathering Subdiv objects data...
0:03 ...done,
0:03 Gathering Octane Light objects data...
0:03 ...done,
0:03 Gathering Instancer objects data...
0:03 ...done,

0:03 Setting Mesh data of general mesh
****************************** GENERAL MESH DATA **
Number of vertices: 43758
Number of normals: 47693
Number of UVs: 48573
Number of polygons: 42797
Number of material indices: 42797
Number of vertex indices: 171944
Number of normal indices: 171944
Number of UV indices: 171944
0:03 ...done setting Mesh data in 0:00
0:03 Voxelizing...
0:04 ...voxelized in 0:01
0:04 ...done creating meshes
0:04 Octane mesh data is set to render engine.
0:04 Octane kernel set: Path Tracing

****************************** THIN LENS CAMERA ***
*
Name: cameraShape1
Orthographic: 0
Film width: 35.999928, Focal length: 45, calculated
Aperture: 0.03
Auto-focus: 1
Focal depth: 2.375821
Stereo: 0
Stereo distance: 0.001
Near clip depth: 1e-007
Camera: X: -1.894747, Y: 3.952579, Z: 0.290888
Target: X: 0.773788, Y: 1.149091, Z: -0.1129
Up: X: 0.712307, Y: 0.693543, Z: -0.107783
IMAGER:
Exposure: 0.8
FStop: 2.8
Iso: 100
Gamma: 0.7
Response: 400
Vignetting: 0.3
Saturation: 1
Hot pixel removal: 1
Premultiplied alpha: 0
Min display samples: 1
Dithering: 0
White saturation: 1
POSTPROCESSOR:
Enabled: 1
Bloom power: 20
Glare power: 10
Glare ray count: 4
Glare angle: 31.263159
Glare blur: 0.0491789

****************************** TEXTURE ENVIRONMENT
Sun/Sky Node: octaneSunSky
Power: 2.2
Rotation: X 0.5, Y -0.2

0:06 Octane engine: Target node is set...
0:06 Octane: Scene export complete.
0:00 ...done

[JimStar]

It seems that it is not the plugin crash, as in this case it could be seen in this plugin log. But as I see in this log, the plugin did the work correctly, and just started the render (has set the render-target to the engine)... So, it seems something goes wrong when the engine tries to load the scene to all your cards, as in the last frame the render has even not started (the plugin have never got the image-buffer of 24th frame from the engine)...

Can you please after the crash to look into the directory c:\Users\<User Name>\AppData\Local\Temp\ for the last files MayaCrashLog*.dmp and MayaCrashLog*.log. These may be very helpful to get understanding what is going on.

[sampakinpa]

I will, thanks.

[sampakinpa]

I cannot find those specific files anywhere on my local machine. Please direct me on how can I produce those files while render crashing.

[JimStar]

I cannot find those specific files anywhere on my local machine. Please direct me on how can I produce those files while render crashing.

If you can't find them - it is the sign that Maya was not able to create them, and this in turn is the sign that the crash happens outside the plugin, in the engine module...
It seems, that you got some platform-dependent issue, as I can't reproduce it on my machine...

Have you checked the GPUs for overheating?..
Can you please try to check the same render with only one GPU core enabled, will you get the same crashes in this case?..

[sampakinpa]

I'll try this ASAP, thanks.

[sampakinpa]

OK, so the gpu temprature are not over heating when the crashes happened, but I did find the AppCrash_mayabatch.exe reports that I included in a zip file attached to this post.
Please take a look at them.
Thanks

crash_report.zip

(4.22 MiB) Downloaded 228 times

[JimStar]

sampakinpa
Thank you.
Don't forget please about trying to render the same on only one GPU enabled...

[sampakinpa]

Hi
It took more then seventeen hours to render one and a half animation files on one GPU (out of 8) until it stopped, but this time it did not crash but it froze on frame 35 on the second anim sequence. what happened?
I pasting the verbose text on the next post. no crash file this time.

BTW, the only way I could enable one GPU to render, is by disabling all GPU's in the command line switches (-gpu1 0 -gpu2 0...) because those switches apparently support only 4 GPUs' and I have 8. so by disabling all 4 it actually disables all of the 8. then it select automatically one GPU as a default to render.