Project data locality performance penalty

[tangelo-otoy]

Hey aoktar and everyone,

We've been unsuccessfully trying to figure out how to eliminate a major performance penalty we're seeing with using Octane when storing/opening C4D projects from a NAS. Even with 10G networking to the workstations and flash-based storage in the NAS, we're seeing massive performance hits when rendering a project stored on the NAS, often 5-10x performance drops vs copying the whole project to the render master, and network rendering it from there. These performance penalties appear to go away completely with using Octane by itself, and are hugely reduced when using C4D by itself (based on some testing with their respective benchmark tools). It's only when using the Octane Plugin for C4D that we see such huge drops.

We're wide open to suggestions around best practices to make shared storage work - with our workflow it's a major impedance for a project to live on the workstation of the artist instead of centrally. In my crazy head I see the plugin having to convert each C4D file (everything from lightmap pixels and up) to a format Octane can work on, which would right there create the bottleneck, and that we might be able to reduce this by, I don't know, perhaps running a big ramdisk for that conversion's filesystem to live? But I"m *completely* guessing and need guidance.

Do other people have success using centralized storage for projects? Is there any way we can work around it?

Thanks

[RahiSan]

This was also an issue for us, thats why we now use dropbox for shared project. All the data is backed up, synchronized and still stored locally on every machine. We are a small company though, so this might not be a solution for you.

[aoktar]

This feedback is requiring more details. First plugin will not reload textures on next frames. Where how is differences of render times between locally stored textures or NAS stored for a sequence?

[tangelo-otoy]

Hi aoktar, thanks for your reply, and I'm sorry about the delay in mine.

The difference is about 5x between rendering with the project files on the NAS vs on the render master's local drive.

I have some excel files with the results of various tests, but basically when the data isn't local it's incredibly slower. This is despite 10G networking all through the stack.

Is there any way to configure networked storage so that c4d renders via octane plugin is not so slow? Or an other combination of workflows we can do to make this process faster? Any best practices for a highly collaborative, centralized process?

Thanks kindly,

~Micah

[tangelo-otoy]

Hey everyone,

Just wanted to follow up on this. I'm going back on-site to the ofice on Thursday and I'd love to be able to implement some change(s) that may make this faster.

I'm keen to get any suggestions from the community around keeping projects on central storage without sacrificing 80% of our render speed. I'd be very keen to know if you all can replicate a significant performance penalty on your own systems by, for example, putting a project on a network share and trying to render it from there. The differences in our case are striking.

At this point I'm probably going to try and implement syncthing to keep the NAS and the workstations' active project folders in sync. This is going to help us all keep synced but is massively less desirable than just keeping things in one place, due to the operational, computational and space requirements that model would impose.

Any help or suggestions greatly appreciated!

~Micah