Help Understanding Error Log - CUDA Error 700

[CalTrago]

Hi,

I am urgently trying to get a draft render out for a client review but can't successfully get a consistent render going without crashes after 9-20 frames.

I have 1,3m polys in my scene, using C4d 2024.3.2, and the latest Octane. I have tried on my Macbook M1, and my Windows machine, but I can't manage to get more than a handful of frames exported before crashing.

Windows - RTX 3080, GTX 1080ti

The main issue seems to be 'CUDA error (700) on device 1: an illegal memory access was encountered'. Is this just a case of allowing more memory in the Device 1 OOC?

If anyone could help me understand the following error log, and also guide me in to running successful renders on this particular issue & project, I would be incredibly helpful.


----

Started logging on 23.04.24 10:31:33

OctaneRender Studio+ 2023.1.2 (13000200)

VRAM used/free/max:0Kb/10Gb/10Gb Out-of-core used:0Kb RAM used:18.664Gb total:63.883Gb OpenGL free/total:0/0

FRAME:0 fps:25
--------------------------- EXPORT LOG ---------------------------

Export materials time= 158236.229 ms
Collect objects time= 9.823 ms
Mesh creation time = 558.569 ms.
MB:0/0 ST/MOV:0/68 Nodes:197 Tris:3.633m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:3.465Gb/3.028Gb/10Gb Out-of-core used:0Kb RAM used:26.766Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=0sec. updateTM=0sec. renderTM:3.497sec. totalTM:166.201sec.

FRAME:1 fps:25
MB:0/0 ST/MOV:0/104 Nodes:198 Tris:6.189m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:3.929Gb/2.503Gb/10Gb Out-of-core used:0Kb RAM used:26.243Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=0.701sec. updateTM=1.309sec. renderTM:4.085sec. totalTM:6.182sec.

FRAME:2 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_2.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_2.png
MB:0/0 ST/MOV:0/143 Nodes:270 Tris:9.213m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:4.47Gb/1.761Gb/10Gb Out-of-core used:0Kb RAM used:28.338Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=1.214sec. updateTM=2.141sec. renderTM:5.079sec. totalTM:8.474sec.

FRAME:3 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_3.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_3.png
MB:0/0 ST/MOV:0/183 Nodes:348 Tris:12.393m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:5.039Gb/1.131Gb/10Gb Out-of-core used:0Kb RAM used:30.307Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=1.679sec. updateTM=2.841sec. renderTM:5.924sec. totalTM:10.482sec.

FRAME:4 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_4.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_4.png
MB:0/0 ST/MOV:0/225 Nodes:428 Tris:15.884m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:5.664Gb/505Mb/10Gb Out-of-core used:0Kb RAM used:32.862Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=2.259sec. updateTM=4.038sec. renderTM:7.063sec. totalTM:13.407sec.

FRAME:5 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_5.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_5.png
MB:0/0 ST/MOV:0/269 Nodes:512 Tris:19.688m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:6.345Gb/0Kb/10Gb Out-of-core used:0Kb RAM used:41.838Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=2.808sec. updateTM=6.344sec. renderTM:7.936sec. totalTM:17.129sec.

FRAME:6 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_6.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_6.png
device 0: disabling RTX acceleration as mesh data cannot be placed into VRAM
MB:0/0 ST/MOV:0/315 Nodes:600 Tris:23.803m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:5.668Gb/539Mb/10Gb Out-of-core used:384Mb RAM used:44.361Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=3.572sec. updateTM=7.575sec. renderTM:25.657sec. totalTM:36.844sec.

FRAME:7 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_7.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_7.png
MB:0/0 ST/MOV:0/363 Nodes:692 Tris:28.23m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:6.75Gb/0Kb/10Gb Out-of-core used:1.125Gb RAM used:43.04Gb total:63.883Gb OpenGL free/total:0/0
mblurTM=0sec. createTM=4.223sec. updateTM=22.331sec. renderTM:15.845sec. totalTM:42.438sec.

FRAME:8 fps:25
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_8.png
FILE is not found:P:\\Projects\Nichols\SS24\Land\Land 1 Project\Land 1 Main\tex\HN Land Plate 1 v2 000_8.png
device 0: disabling RTX acceleration as mesh data cannot be placed into VRAM
MB:0/0 ST/MOV:0/413 Nodes:788 Tris:32.969m DispTris:0 Hairs:0 Meshes:1k
Textures Grey8/16:11/0 Rgb32/64:50/0
VRAM used/free/max:5.687Gb/481Mb/10Gb Out-of-core used:1.591Gb RAM used:44.905Gb total:63.883Gb OpenGL free/total:0/0
device 0: disabling RTX acceleration as mesh data cannot be placed into VRAM
CUDA error (700) on device 1: an illegal memory access was encountered
-> failed to synchronize event
device 1: path tracing kernel failed
Check geometry subdivisons.
Picture viewer uses 'Render' subdivisions. But Live Viewer uses 'Editor' subdivision values.
Try to decrease 'render/editor' subdivison values.
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to synchronize event
device 0: path tracing kernel failed
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to destroy CUDA event
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to destroy CUDA event
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to destroy CUDA event
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to deallocate pinned memory
CUDA error (700) on device 0: an illegal memory access was encountered
-> could not get memory info
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to deallocate pinned memory
CUDA error (700) on device 0: an illegal memory access was encountered
-> could not get memory info
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to deallocate device memory
CUDA error (700) on device 0: an illegal memory access was encountered
-> could not get memory info
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to deallocate device array

CUDA error (700) on device 1: an illegal memory access was encountered
Failed to free page-locked memory
-> failed to deallocate device array
Failed to free page-locked memory

CUDA error (700) on device 0: an illegal memory access was encountered
CUDA error (700) on device 1: an illegal memory access was encountered
-> could not get memory info
-> failed to deallocate device memory
CUDA error (700) on device 0: an illegal memory access was encountered
CUDA error (700) on device 1: an illegal memory access was encountered
-> could not get memory info
-> failed to deallocate device memory
CUDA error (700) on device 1: an illegal memory access was encountered
CUDA error (700) on device 0: an illegal memory access was encountered
-> could not get memory info
-> failed to deallocate device memory
CUDA error (700) on device 0: an illegal memory access was encountered
CUDA error (700) on device 1: an illegal memory access was encountered
-> failed to deallocate device memory
-> could not get memory info
CUDA error (700) on device 0: an illegal memory access was encountered
-> could not get memory info
CUDA error (700) on device 1: an illegal memory access was encountered
CUDA error (700) on device 0: an illegal memory access was encountered
-> failed to deallocate device memory
-> failed to deallocate device memory

OptiX error for device 0: [PIPELINE] Error synching on OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
Error destroying OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
OptiX API error for device 0: CUDA error
OptiX error for device 0: [PIPELINE] Error synching on OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
Error destroying OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
OptiX API error for device 0: CUDA error
OptiX error for device 0: [PIPELINE] Error synching on OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
Error destroying OptixPipeline event (CUDA error string: an illegal memory access was encountered, CUDA error code: 700)
OptiX API error for device 0: CUDA error
Break rendering...
**** RENDER FINISHED **** stat=3

[bepeg4d]

Hi,
in general, CUDA 700 error is caused by corrupted or not correctly installed Nvidia driver.

Please, try to perform a clean installation of 552.22 Nvidia Studio driver:
https://www.nvidia.com/download/driverR ... 156/en-us/

To perform a clean installation, please follow these steps:


After reboot, if not enough, try to connect the monitor to GTX 1080, and use RTX 3080 for octane only.

Keep us informed.

ciao,
Beppe

[Tim_Twisted]

For me this sounds like not enough VRAM to handle the scene. Especially looking at the log

Frame 8:

VRAM used/free/max:5.687Gb/481Mb/10Gb Out-of-core used:1.591Gb RAM used:44.905Gb total:63.883Gb OpenGL free/total:0/0

481Mb shouldn't crash but is close enough for me to remove some stuff in the scene and see if the crash persists.