Project data locality performance penalty
Posted: Tue Jun 27, 2017 11:27 pm
Hey aoktar and everyone,
We've been unsuccessfully trying to figure out how to eliminate a major performance penalty we're seeing with using Octane when storing/opening C4D projects from a NAS. Even with 10G networking to the workstations and flash-based storage in the NAS, we're seeing massive performance hits when rendering a project stored on the NAS, often 5-10x performance drops vs copying the whole project to the render master, and network rendering it from there. These performance penalties appear to go away completely with using Octane by itself, and are hugely reduced when using C4D by itself (based on some testing with their respective benchmark tools). It's only when using the Octane Plugin for C4D that we see such huge drops.
We're wide open to suggestions around best practices to make shared storage work - with our workflow it's a major impedance for a project to live on the workstation of the artist instead of centrally. In my crazy head I see the plugin having to convert each C4D file (everything from lightmap pixels and up) to a format Octane can work on, which would right there create the bottleneck, and that we might be able to reduce this by, I don't know, perhaps running a big ramdisk for that conversion's filesystem to live? But I"m *completely* guessing and need guidance.
Do other people have success using centralized storage for projects? Is there any way we can work around it?
Thanks
We've been unsuccessfully trying to figure out how to eliminate a major performance penalty we're seeing with using Octane when storing/opening C4D projects from a NAS. Even with 10G networking to the workstations and flash-based storage in the NAS, we're seeing massive performance hits when rendering a project stored on the NAS, often 5-10x performance drops vs copying the whole project to the render master, and network rendering it from there. These performance penalties appear to go away completely with using Octane by itself, and are hugely reduced when using C4D by itself (based on some testing with their respective benchmark tools). It's only when using the Octane Plugin for C4D that we see such huge drops.
We're wide open to suggestions around best practices to make shared storage work - with our workflow it's a major impedance for a project to live on the workstation of the artist instead of centrally. In my crazy head I see the plugin having to convert each C4D file (everything from lightmap pixels and up) to a format Octane can work on, which would right there create the bottleneck, and that we might be able to reduce this by, I don't know, perhaps running a big ramdisk for that conversion's filesystem to live? But I"m *completely* guessing and need guidance.
Do other people have success using centralized storage for projects? Is there any way we can work around it?
Thanks