Hi all,
This week we had three major merges of separate development branches which means that things still need some work and need to mature a bit more we can release a first alpha of v3. But since I promised a development update earlier this week I would like to give you an idea where we are and where we are going to.
(And apologies for not posting some sexy pictures. Didn't have time to do them yet...)
EDIT: In case you are interested in later updates: Here is the update from 14/11 and here the one from 27/11.
What was done the last months?
We did some refactoring of the render kernels and the render framework and in parallel we were working new features:
Overhaul of the integration kernels
To support future features like OSL and OpenCL, we had to completely overhaul the way how the integration kernels work. Since the beginning of Octane the integration kernels calculated complete samples of large parts of the image in big work chunks. This had advantages and disadvantages. Since the disadvantages started to outweigh the advantages we decided to restructure the kernels in a way that they do smaller work chunks, but better optimized and trying to keep the GPU threads more occupied.
The main problem of the new system is that we need to store additional information between the kernel calls, which takes time to read/write and requires additional memory. To solve the first part of the problem, we spent quite some time to optimize the way this data is read and written. I did some initial testing on various scenes on modern GPUs (Keplers/Maxwell 2s) and got the following preliminary results:
- Direct lighting with coherent ratio has about the same speed as direct lighting in 2.24.2 or is slightly faster. Depending on the scene complexity. The more complex the better the new kernels perform compared to the old kernels. With coherent ratio set to 0.3, the old kernels are usually slightly better than the new ones.
- Path tracing with a coherent ratio of 0 usually performs now significantly better than in 2.24.2, often 10-30% faster - depending on the scene complexity. With a coherent ratio of 0.3, the difference is not as big anymore, but still most of the times 5-20% faster.
- PMC with short path lengths (<6) has about the same speed as in 2.24.2, but with longer path length is often 5-20% faster. PMC still requires some work before it can be released and I hope there is still room for speed improvements (e.g. currently for some reason the GPU usage seems to stay between 50-60%...).
Again, the above results are preliminary and should be taken with a grain of salt.
Moved film buffers to the host and tiled rendering
The second major refactoring in the render core was the way we store render results. Until v3 each GPU had its own film buffer where part of the calculated samples were aggregated. This has various drawbacks: For example, a CUDA error usually means that you lose the samples calculated by that GPU or a crashing/disconnected slave means you lose the samples. Another issue is that large images mean a large film buffer, especially if you enable render passes.
To solve the above (and more) issues we decided to move the film buffer into host memory. Doesn't sound exciting, but has some major consequences. The biggest one we had to fight with is finding a way to deal with the huge amount of data the GPUs produce. Especially in multi-GPU setups or when network rendering is used.
As a solution, we introduced tiled rendering for all integration kernels except PMC and added a bunch of additional options to allow you to tweak the way how integration work is distributed. One side effect is that we (hopefully) have solved the problem that slaves got starved off work while they were sending very large results back to the master (like stereo panos for the GearVR). Another side effect is that info passes are now rendered in parallel.
Volume rendering
As you already have seen in the v3 preview, we added volume rendering of OpenVDB files. Of course, plugins or Lua scripts can also feed volumetric data directly into Octane without going through files. Since the preview, various improvements got implemented, including the support of hierarchical grids.
There is still lots of work to be done, especially on the way how you define the medium appearance, but it's already quite useful for many cases.
Deep pixel rendering
Back when we announced the support of render passes, one the first questions that came up was the support for deep pixel rendering. Today the branch for deep pixel rendering got merged into the main development branch and it really looks good so far. There is still some more tweaking planned and we need to integrate it into network rendering, but other than that it's good to go. Unfortunately, as far as we know the only tool that is able to do something useful with deep pixel images is Nuke, so if you don't use Nuke, this feature is probably not of much use for you.
Texture baking
We also started working on texture baking. Currently we've got a first version running to bake diffuse surfaces. The way it works is that you set a "baking camera" as camera node and partition your geometry using baking IDs. In the baking camera you can then pick the ID you want to bake and it just renders the texture out. It also extrapolates the baked texture to avoid black edges. And of course, you can use the render passes to bake out various channels and stuff. Please be aware though, that the UV map needs to provided by the geometry and is not created by Octane.
Raw/filter passes
We added the possibility to render raw and filter beauty passes, which can then be multiplied during compositing, which allows you to modify the colour of objects in post (of course, reflection of changed object will still show the old colour). The calculation of the raw passes is done during tonemapping which means it works only if you use the linear response curve with a gamma of 1.
Photoshop workflow plugin
We also developed a Photoshop plugin which allows you to load a bunch of render passes and set them up automatically in a Photoshop project. To make this work you have to export a compositing file while you save the render passes. The compositing file is used by the plugin to find, load and arrange the render passes. The plugin also includes an OpenEXR loader, allowing you to load OpenEXRs without any other plugins.
Environment medium
Setting up participating media has been quite painful so far: You had to create a volume geometry, which encloses the whole scene. The you had to create a "bubble" around the camera, with the normals pointing inwards, so that camera rays enter the medium. And this bubble had to be animated with the camera...
Now you can just specify a medium in the environment nodes, specify some size/thickness and that's it. Quite handy.
Other changes
I'm probably forgetting a few, but there have been various smaller improvements, too, like:
- Improved bump at the terminator, reducing "terminator acne" caused by excessive bump strengths and hard lights.
- Implemented motion blur for panoramic cameras.
- Static noise is now as static as possible. Unfortunately there are still some numerical differences between different compute architectures, but on the same hardware you can get now the same results, even after restarting the render.
- Added neutral response option which disables the tinting caused by various camera response curves.
- Added a tangent normal info pass and an opacity info pass.
Where are we now?
We are currently getting really close to a first alpha release, since all these developments are coming together, but as mentioned above there are still some issues left, which we want to fix before we release, like:
- PMC requires more work, to solve various bottlenecks.
- PMC doesn't work with network rendering yet.
- Deep pixel rendering doesn't work over the network yet.
- The UV lookup structure of texture baking requires some more work and it's at the moment not clear if it will be finished with the first alpha release.
There are also some features that got broken and will be fixed after the first release(s), like region rendering.
What is going to happen next?
We hope to release an initial alpha version out as soon as possible, to get feedback as early as possible. The current plan is that during the alpha and beta phase you will be able to use your v2 license to run v3 builds and when the beta phase has ended you will need to upgrade to a v3 license to be able to run v3 builds.
After the initial alpha release, we plan to have frequent development releases until most of the bugs got squashed and some more features added. Here is a list of some of the features we will be working on next (order may change):
- Adding functionality that makes rendering more robust tries to recover failed GPUs or slaves.
- Adding the support of save+resume of renderings.
- Adding advanced texture baking, like light fields.
- Introducing "specialized kernels" to remove the 2GB limit we currently have for geometry primitives.
- OSL
- OpenCL
- Various other stuff like object handles to translate/rotate/scale instances...
And of course, there are hundreds of feature requests and we also have many more ideas, i.e. I don't think we will get bored any time soon
Cheers,
Marcus