It truly works and whenever I'm in the process of material caching I can move around freely in the viewport and actually work on stuff! I've noticed there is a grey mesh, then white mesh and then loaded material, probably some hack to make it work?

There is a small issue with it's tendency to show black material either on scene load or on Remove duplicate material. The Remove duplicate material one happens a lot.
These are extremely simple Glossy materials with only Diffuse channel loaded.
