v3 dev update

Forums: v3 dev update
Generic forum to discuss Octane Render, post ideas and suggest improvements.
Forum rules
Please add your OS and Hardware Configuration in your signature, it makes it easier for us to help you analyze problems. Example: Win 7 64 | Geforce GTX680 | i7 3770 | 16GB

v3 dev update

Postby abstrax » Fri Oct 30, 2015 2:24 pm

abstrax Fri Oct 30, 2015 2:24 pm
Hi all,

This week we had three major merges of separate development branches which means that things still need some work and need to mature a bit more we can release a first alpha of v3. But since I promised a development update earlier this week I would like to give you an idea where we are and where we are going to.

(And apologies for not posting some sexy pictures. Didn't have time to do them yet...)

EDIT: In case you are interested in later updates: Here is the update from 14/11 and here the one from 27/11.


What was done the last months?

We did some refactoring of the render kernels and the render framework and in parallel we were working new features:


Overhaul of the integration kernels

To support future features like OSL and OpenCL, we had to completely overhaul the way how the integration kernels work. Since the beginning of Octane the integration kernels calculated complete samples of large parts of the image in big work chunks. This had advantages and disadvantages. Since the disadvantages started to outweigh the advantages we decided to restructure the kernels in a way that they do smaller work chunks, but better optimized and trying to keep the GPU threads more occupied.

The main problem of the new system is that we need to store additional information between the kernel calls, which takes time to read/write and requires additional memory. To solve the first part of the problem, we spent quite some time to optimize the way this data is read and written. I did some initial testing on various scenes on modern GPUs (Keplers/Maxwell 2s) and got the following preliminary results:

- Direct lighting with coherent ratio has about the same speed as direct lighting in 2.24.2 or is slightly faster. Depending on the scene complexity. The more complex the better the new kernels perform compared to the old kernels. With coherent ratio set to 0.3, the old kernels are usually slightly better than the new ones.

- Path tracing with a coherent ratio of 0 usually performs now significantly better than in 2.24.2, often 10-30% faster - depending on the scene complexity. With a coherent ratio of 0.3, the difference is not as big anymore, but still most of the times 5-20% faster.

- PMC with short path lengths (<6) has about the same speed as in 2.24.2, but with longer path length is often 5-20% faster. PMC still requires some work before it can be released and I hope there is still room for speed improvements (e.g. currently for some reason the GPU usage seems to stay between 50-60%...).

Again, the above results are preliminary and should be taken with a grain of salt.


Moved film buffers to the host and tiled rendering

The second major refactoring in the render core was the way we store render results. Until v3 each GPU had its own film buffer where part of the calculated samples were aggregated. This has various drawbacks: For example, a CUDA error usually means that you lose the samples calculated by that GPU or a crashing/disconnected slave means you lose the samples. Another issue is that large images mean a large film buffer, especially if you enable render passes.

To solve the above (and more) issues we decided to move the film buffer into host memory. Doesn't sound exciting, but has some major consequences. The biggest one we had to fight with is finding a way to deal with the huge amount of data the GPUs produce. Especially in multi-GPU setups or when network rendering is used.

As a solution, we introduced tiled rendering for all integration kernels except PMC and added a bunch of additional options to allow you to tweak the way how integration work is distributed. One side effect is that we (hopefully) have solved the problem that slaves got starved off work while they were sending very large results back to the master (like stereo panos for the GearVR). Another side effect is that info passes are now rendered in parallel.


Volume rendering

As you already have seen in the v3 preview, we added volume rendering of OpenVDB files. Of course, plugins or Lua scripts can also feed volumetric data directly into Octane without going through files. Since the preview, various improvements got implemented, including the support of hierarchical grids.

There is still lots of work to be done, especially on the way how you define the medium appearance, but it's already quite useful for many cases.


Deep pixel rendering

Back when we announced the support of render passes, one the first questions that came up was the support for deep pixel rendering. Today the branch for deep pixel rendering got merged into the main development branch and it really looks good so far. There is still some more tweaking planned and we need to integrate it into network rendering, but other than that it's good to go. Unfortunately, as far as we know the only tool that is able to do something useful with deep pixel images is Nuke, so if you don't use Nuke, this feature is probably not of much use for you.


Texture baking

We also started working on texture baking. Currently we've got a first version running to bake diffuse surfaces. The way it works is that you set a "baking camera" as camera node and partition your geometry using baking IDs. In the baking camera you can then pick the ID you want to bake and it just renders the texture out. It also extrapolates the baked texture to avoid black edges. And of course, you can use the render passes to bake out various channels and stuff. Please be aware though, that the UV map needs to provided by the geometry and is not created by Octane.


Raw/filter passes

We added the possibility to render raw and filter beauty passes, which can then be multiplied during compositing, which allows you to modify the colour of objects in post (of course, reflection of changed object will still show the old colour). The calculation of the raw passes is done during tonemapping which means it works only if you use the linear response curve with a gamma of 1.


Photoshop workflow plugin

We also developed a Photoshop plugin which allows you to load a bunch of render passes and set them up automatically in a Photoshop project. To make this work you have to export a compositing file while you save the render passes. The compositing file is used by the plugin to find, load and arrange the render passes. The plugin also includes an OpenEXR loader, allowing you to load OpenEXRs without any other plugins.


Environment medium

Setting up participating media has been quite painful so far: You had to create a volume geometry, which encloses the whole scene. The you had to create a "bubble" around the camera, with the normals pointing inwards, so that camera rays enter the medium. And this bubble had to be animated with the camera...

Now you can just specify a medium in the environment nodes, specify some size/thickness and that's it. Quite handy.


Other changes

I'm probably forgetting a few, but there have been various smaller improvements, too, like:

- Improved bump at the terminator, reducing "terminator acne" caused by excessive bump strengths and hard lights.
- Implemented motion blur for panoramic cameras.
- Static noise is now as static as possible. Unfortunately there are still some numerical differences between different compute architectures, but on the same hardware you can get now the same results, even after restarting the render.
- Added neutral response option which disables the tinting caused by various camera response curves.
- Added a tangent normal info pass and an opacity info pass.


Where are we now?

We are currently getting really close to a first alpha release, since all these developments are coming together, but as mentioned above there are still some issues left, which we want to fix before we release, like:

- PMC requires more work, to solve various bottlenecks.
- PMC doesn't work with network rendering yet.
- Deep pixel rendering doesn't work over the network yet.
- The UV lookup structure of texture baking requires some more work and it's at the moment not clear if it will be finished with the first alpha release.

There are also some features that got broken and will be fixed after the first release(s), like region rendering.


What is going to happen next?

We hope to release an initial alpha version out as soon as possible, to get feedback as early as possible. The current plan is that during the alpha and beta phase you will be able to use your v2 license to run v3 builds and when the beta phase has ended you will need to upgrade to a v3 license to be able to run v3 builds.

After the initial alpha release, we plan to have frequent development releases until most of the bugs got squashed and some more features added. Here is a list of some of the features we will be working on next (order may change):

- Adding functionality that makes rendering more robust tries to recover failed GPUs or slaves.
- Adding the support of save+resume of renderings.
- Adding advanced texture baking, like light fields.
- Introducing "specialized kernels" to remove the 2GB limit we currently have for geometry primitives.
- OSL
- OpenCL
- Various other stuff like object handles to translate/rotate/scale instances...

And of course, there are hundreds of feature requests and we also have many more ideas, i.e. I don't think we will get bored any time soon ;)


Cheers,
Marcus
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
User avatar
abstrax
OctaneRender Team
OctaneRender Team
 
Posts: 5483
Joined: Tue May 18, 2010 11:01 am
Location: Auckland, New Zealand

Re: v3 dev update

Postby whersmy » Fri Oct 30, 2015 2:39 pm

whersmy Fri Oct 30, 2015 2:39 pm
*grabs a big cup of coffee
Octane 2022.1.1 nv535.98
mac pro g5| pentium g2030 iGPU| maximus extreme V| 2x gtx590 - 8gb - SSD - win7-x64- 1500W Silverstone|
x201t - gtx580 - egpu ec
Dell G5 - 16GB - dgpu GTX1060 - TB3 egpu @ 1060 / RTX 4090

Octane Render experiments
User avatar
whersmy
Licensed Customer
Licensed Customer
 
Posts: 723
Joined: Thu Aug 30, 2012 7:40 am

Re: v3 dev update

Postby smicha » Fri Oct 30, 2015 2:43 pm

smicha Fri Oct 30, 2015 2:43 pm
reading...

Thanks for you work, guys!
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Licensed Customer
 
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Re: v3 dev update

Postby Refracty » Fri Oct 30, 2015 2:44 pm

Refracty Fri Oct 30, 2015 2:44 pm
Thanks a lot Marcus for the detailed and realistic update.
User avatar
Refracty
Licensed Customer
Licensed Customer
 
Posts: 1598
Joined: Wed Dec 01, 2010 6:42 pm
Location: 3D-Visualisierung Köln

Re: v3 dev update

Postby RobSteady » Fri Oct 30, 2015 3:38 pm

RobSteady Fri Oct 30, 2015 3:38 pm
Sounds great!
Also thanks for the roadmap, this is what users want to see ;)

A question for the plugins:
Is there any plan to acquire more developers for this?
I think it's obvious that the workload is too much for one person to handle on multiple plugins; JimStar for example who is doing a great job but of course the progress is very slow.
Can he please get a V-Ray license? I can't believe that he hasn't one yet.
We need a working material converter; 99% of the models you can buy are for V-Ray and the manual conversion of assets is so time consuming.
Corona users don't have this problem...

This would make the software more expensive, yes. But I think a lot of people are willing to pay for it if it speeds up the progress.
Last edited by RobSteady on Fri Oct 30, 2015 5:28 pm, edited 2 times in total.
Octane for 3ds Max v2.21.1 | i7-5930K | 32GB | 1 x GTX Titan Z + 2 x GTX 980 Ti
User avatar
RobSteady
Licensed Customer
Licensed Customer
 
Posts: 618
Joined: Thu Jun 07, 2012 7:33 am

Re: v3 dev update

Postby moritzw » Fri Oct 30, 2015 4:54 pm

moritzw Fri Oct 30, 2015 4:54 pm
Glad to read updates about v3.
I'm wondering what's the status of 2.25, which you guys stated (via twitter) will include Octane Render Cloud?
User avatar
moritzw
Licensed Customer
Licensed Customer
 
Posts: 7
Joined: Sat Apr 11, 2015 7:47 pm
Location: Stuttgart, Germany

Re: v3 dev update

Postby xcaseyx » Fri Oct 30, 2015 5:26 pm

xcaseyx Fri Oct 30, 2015 5:26 pm
this is great, I wish you guys would have dates, but I guess that already got you in trouble when you announced 'june' earlier this year.

can you shed some light on orc please? or at least free up the part in the license terms that forbids us to create our own solutions in amazons AWS. Its currently possible but breaks our EULA.


please consider allowing visualization, or give us an update on orc.
xcaseyx
Licensed Customer
Licensed Customer
 
Posts: 115
Joined: Wed Feb 18, 2015 9:04 pm

Re: v3 dev update

Postby glimpse » Fri Oct 30, 2015 5:48 pm

glimpse Fri Oct 30, 2015 5:48 pm
Thanks for an update! =) seems like a hell of a work went into this..- take time & make the best out of it!
User avatar
glimpse
Licensed Customer
Licensed Customer
 
Posts: 3715
Joined: Wed Jan 26, 2011 2:17 pm

Re: v3 dev update

Postby Rafael-Vallaperde » Fri Oct 30, 2015 6:01 pm

Rafael-Vallaperde Fri Oct 30, 2015 6:01 pm
Feels like Christmass!!

Thank you mate.

Question1:
Anything on the polygon cap? are we still bound to the 19kk Polys?

Question1:
How about light caching on separate buffers?, before we could add lights to a group and then render these as separete outputs. very cool to do some post work.
No performance hit as long as it fits memory on the GPU.

Are we still gpoing to be able to do the same ? or its going to have a hit on performance or even worse, compute as a whole new render?

Again, thank you so much for the great work you guys have been doing!!
User avatar
Rafael-Vallaperde
Licensed Customer
Licensed Customer
 
Posts: 200
Joined: Mon Nov 25, 2013 1:07 am

Re: v3 dev update

Postby pinochio » Fri Oct 30, 2015 6:13 pm

pinochio Fri Oct 30, 2015 6:13 pm
Thanks a bunch for the hard work guys, especially for the texture baking feature!
v3 is looking very attractive. I`m also curious about the OpenCL implications.
pinochio
Licensed Customer
Licensed Customer
 
Posts: 28
Joined: Mon Jan 11, 2010 6:57 am
Next

Return to General Discussion


Who is online

Users browsing this forum: No registered users and 17 guests

Fri Mar 29, 2024 12:44 am [ UTC ]