How to use the render layer passes.

Forums: How to use the render layer passes.
Tutorials for OctaneRender - demonstrations and howto's.

How to use the render layer passes.

Postby stratified » Thu Jan 08, 2015 2:45 am

stratified Thu Jan 08, 2015 2:45 am
Hi everybody!

In this short write-up I will explain how to use the render layer passes introduced in OctaneRender 2.20. These new render passes are a useful tools to composite a render on a backplate in post. With my limited artistic skills I will try to compose a wine glass on top of a picture of a desk. The wine glass is a free model from Turbo Squid which I subdivided in Octane because the original was modelled very crude. The picture of the desk is a random picture I took from the internet. The hardest part of this exercise was trying to match the camera and the lighting.
backplate.jpg
Random picture from the internet.


The new render passes are named shadows, black shadows, coloured shadows and reflections. These passes only work in conjunction with the render layers. If you enable them in your project without any further configuration, you will only see empty renders for these passes. More formally, these passes capture the "effect" from geometry part of the active render layer onto geometry part of all the other render layers. The first thing that needs to be done is configure object layers for the geometry. You do this by connecting your scene geometry to an object layer map and connect object layer nodes to each pin of the object layer map. Like this:
object_layer_map.png
Geometry needs to be connected to an object layer map to work with render layers.


In the object layer node, it's possible to configure a render layer ID. The render layer ID is number of the render layer the current object belongs to. In this example, the wine glass is in render layer 1 and the light and ground plane are in render layer 2. You can visualize which objects are on which layer via the Render Layer ID info channel kernel or render pass. The render target node has an extra pin in 2.20 to connect it with a render layer node. In the render layer node you can configure if render layers are active and which render layer is rendered (the active render layer). There's only 1 active render layer at all times. The best way to render out multiple layers is to configure multiple render target nodes and use the batch render script that comes with the standalone.
object_layer_node.png
The render layer id is configured in the object layer node.

render_layer_node.png
Which layer to render is configured in the render layer node.


During rendering, objects that aren't on the active render layer are ignored by camera rays. This means that all objects in the active layer are isolated in the beauty pass. There is no need anymore to isolate objects from the render via masks. The main advantage is that you can easily isolate out of focus or motion blurred objects. This is impossible with masks in some cases. If you still like to use masks and do DOF and motion blur in post, you can still render out an alpha mask of the render layer with the render layer mask info channel or render layer mask pass. Lets quickly look at an example where masks fail. When trying to mask out the blurry blue sphere, it's impossible to avoid leaking through the yellow sphere. With render layers the yellow sphere was never in the render to begin with. You could of course render out the scene without DOF and motion blur and do them in post. But I think that with the fast render times of Octane this is a bit of an obsolete work flow in most cases.

full.png
Full scene.

masked.png
It's virtually impossible to mask out the blue sphere without leaking in the yellow sphere.

ball_isolated.png
With the ball isolated in a layer there's no problem.


When everything is fully set-up, you can enable the render layer passes:
passes_node.png
The render layer passes are configured in the render passes node.



  • Black shadows: Captures black shadows, i.e points on the non-active layer geometry where light is fully blocked by objects on the active layer. If light is blocked, shadows are always captured regardless of the material that receives the shadow. It's assumed that the object that receives the shadows has a white diffuse material. e.g. shadows cast on a polished mirror like surface would not be visible in the render but we capture them in the shadow pass anyway. This pass only uses the alpha channel and should be composed in via the normal blend mode (regular alpha blending).
  • Colored shadows: Captures colored shadows cast by objects on the active layer geometry. Only objects with a specular material with fake shadows enabled can cast colored shadows. (TIP: when enabling fake shadows make sure that the kernel has alpha shadows enabled, otherwise it won't work). This pass doesn't have an alpha channel and should be composed in via the multiply blend mode.
  • Shadows: This pass is there for convenience. It combines black shadows (in the alpha channel) with colored shadows (in the RGB channels) in a single image. The blend mode is multiply. It captures the same shadows as the matte material with the difference that the matte materials captures all shadows in the alpha channel and hence doesn't keep color information.
  • Reflections: Captures light reflected off of objects on the active layer on objects on the non-active layers. This pass respects the materials so the look of the reflections really depends on the materials used.

To get some nice reflections on the glass from the desk, we map the background image onto the ground plane. For this mapping we use the same transformation as the camera. This can be achieved with a script written by Roeland that you can get here. It's also very important to tick the affect alpha option of the specular material of the glass. Otherwise the glass will be opaque in the composite. The rendered passes should look something like this in the end. We only have black shadows because we didn't enable fake shadows. In the reflections we get a little bit of caustics:
scene_wo_passes_1.png
The scene without render layers enabled. Notice the background plate mapped onto the ground plane to get some nice reflections in the glass.

wineglass_Beauty.png
The beauty pass with render layers enabled.

wineglass_Black shadows.png
Black shadows cast from the wine glass onto the ground plane.

wineglass_Layer reflections.png
The reflections from the wine glass onto the ground plane.


All these results can be composed together onto a background plate.
final.png
The final result (not perfect but it demonstrates the workflow).


Please feel free to ask any questions if something isn't clear. We're always very interested to see some nice composites made with Octane.

Attached are the GIMP project (I will replace try to do this exercise in Photoshop as well) and the Octane project.

cheers,
Thomas
Attachments
gimp-project.zip
(4.69 MiB) Downloaded 572 times
render_layers_example.orbx
Octane package
(425.45 KiB) Downloaded 570 times
User avatar
stratified
OctaneRender Team
OctaneRender Team
 
Posts: 945
Joined: Wed Aug 15, 2012 6:32 am
Location: Auckland, New Zealand

Re: How to use the render layer passes.

Postby Tugpsx » Thu Jan 08, 2015 4:22 am

Tugpsx Thu Jan 08, 2015 4:22 am
Wow! thanks for explanation and project files. This will help in getting the setting correct.
Dell Win Vista 32 | NVIDIA Quadro NVS135M | Core2 Duo T7500 2.20GHz | 4GB
CyberPowerPC Win 7 64| NVIDIA GTX660Ti (3G) GTX480(1.5G) | 16GB
Tugpsx
Licensed Customer
Licensed Customer
 
Posts: 1055
Joined: Thu Feb 04, 2010 8:04 pm
Location: Chicago, IL

Re: How to use the render layer passes.

Postby Lewis » Thu Jan 08, 2015 10:46 am

Lewis Thu Jan 08, 2015 10:46 am
Thanks for walk-trough Thomas. Really nice addition :)
--
Lewis
http://www.lewis.tomsoft.hr
Skype - lewis3d
ICQ - 7128177

WS Dual Xeon E5-2698v4, 128GB RAM, Win10, TitanX(M) + 1080 + 2x1080Ti
RS1 Dual Xeon E5-2690, 64GB RAM, Win8, 3x1080Ti
RS2 i7 6850K, 64GB RAM, Win10, 2x1080Ti + 1080
User avatar
Lewis
Licensed Customer
Licensed Customer
 
Posts: 568
Joined: Tue Feb 05, 2013 6:30 pm
Location: Croatia

Re: How to use the render layer passes.

Postby glimpse » Thu Jan 08, 2015 11:00 am

glimpse Thu Jan 08, 2015 11:00 am
Thank You, Thomas! =)
tomGlimps | [email protected] | 32GB | TitanBlack
Available for Freelance, Coaching & Mentoring
User avatar
glimpse
Licensed Customer
Licensed Customer
 
Posts: 3464
Joined: Wed Jan 26, 2011 2:17 pm

Re: How to use the render layer passes.

Postby georgesemmanuelarnaud » Thu Jan 08, 2015 2:32 pm

georgesemmanuelarnaud Thu Jan 08, 2015 2:32 pm
Thank you so much , you are a life saver . . .
georgesemmanuelarnaud
Licensed Customer
Licensed Customer
 
Posts: 16
Joined: Sat Mar 29, 2014 4:27 pm

Re: How to use the render layer passes.

Postby linvanchene » Fri Jan 09, 2015 2:20 am

linvanchene Fri Jan 09, 2015 2:20 am
First thank you all for your efforts to give us more options how to composite CG elements with live action footage.

- - -

removed duplicate questions because they are included in the answer to save some space.
Last edited by linvanchene on Mon May 11, 2015 7:15 am, edited 2 times in total.
User avatar
linvanchene
Licensed Customer
Licensed Customer
 
Posts: 750
Joined: Mon Mar 25, 2013 10:58 pm
Location: Switzerland

Re: How to use the render layer passes.

Postby stratified » Fri Jan 09, 2015 3:28 am

stratified Fri Jan 09, 2015 3:28 am
linvanchene wrote:First thank you all for your efforts to give us more options how to composite CG elements with live action footage.

- - -

stratified wrote:
Please feel free to ask any questions if something isn't clear. We're always very interested to see some nice composites made with Octane.



The shorter version:


What would help me would be either

- a written step by step tutorial featuring a lot of screenshots with a tutorial kit to recreate this image example by scratch from an empty scene

or

- a video tutorial with voice over that shows the complete process starting at an empty scene until the finished composition

- - -
- - -

The longer version:


I feel like there might be a huge gap between "casual" plugin users that rarely use the standalone and advanced users familiar with lua scripts.
In order to prevent that this gap increases it may help to have step by step guides with a lot of images or video tutorials that show complete workflows from scratch to finish.


Specific points that i would like to have more detailed information about:

- How do I work with scripts?

I tried to follow the description in the topic

Apply camera mapping to a texture

viewtopic.php?f=73&t=37574

How do I work with scripts.jpg


What would help me would be a step by step guide with images or a video showing how I use the lua script to map the background image onto the ground plane in this specific example.


- Which layer mode is suggested to use for the reflection pass in photoshop?

I tried with the layer mode "screen"

test reflection - layer type screen v1001.jpg


I also tried the layer mode "Difference"

test reflection - layer type difference v1001.jpg


By default I expected the layer mode "screen" to be the go to type for this kind of compositing but somehow I prefer the result I got from "Difference".
My subjective impression is that the strong shadow may sell the effect better even if it may be less realistic depending on the actual light situation.

Maybe a scene example with a light situation that causes stronger caustic effects may provide additional insight.

Can anyone share their results?


- - -

- What is the suggested layer order in photoshop?

- Shadow
- Reflection
- Beauty
- Backplate

or

- Reflection
- Shadow
- Beauty
- Backplate

- - -


Hi,

The Lua scripts on the forums are community driven. We (the OctaneRender team) only provide official support for scripts that ship with Octane itself (i.e turntable animation, daylight animation, animation render & batch render). Some of the scripts on the forum are written by us but that was mostly in our free time. Expecting tutorials for scripts is not realistic because good tutorials are very time consuming. That said most scripts have very helpful authors and they will help you out when you ask questions. The most effective way is learning a bit of Lua and trying to understand what the scripts does but we understand that not everybody is willing to do that.

Don't let my write-up mislead you. You don't need Lua scripting to make use of the compositing features in Octane. You might as well put a wood grain texture on the ground plane and catch nice reflections in the wine glass.

In Photoshop you would have the following layer stack and layer modes:

1) Beauty (normal layer mode)
2) Black shadows (normal layer mode) / Colored shadows (multiply layer mode) / Shadows (multiply layer mode)
3) Reflections (linear dodge (add) layer mode)
4) Backplate

If I get my hands on Photoshop, I will post an example file. Of course you need to tweak it afterwards because the shadow as they are capture in Octane likely don't match the ones in your picture (The shadows in Octane are cast onto a white diffuse material regardless of your real material set-up).

Does that answer your questions?

cheers,
Thomas
User avatar
stratified
OctaneRender Team
OctaneRender Team
 
Posts: 945
Joined: Wed Aug 15, 2012 6:32 am
Location: Auckland, New Zealand

Re: How to use the render layer passes.

Postby Oleg » Fri Jan 09, 2015 8:36 am

Oleg Fri Jan 09, 2015 8:36 am
Thats great, but what if the material of the table will be different? Some shiny reflective surface such as plastic or lacquered
x58 chipset | i7 [email protected] | 24Gb DDR3 | 2xGTX 780Ti
User avatar
Oleg
Licensed Customer
Licensed Customer
 
Posts: 51
Joined: Thu Nov 07, 2013 3:47 pm

Re: How to use the render layer passes.

Postby stratified » Fri Jan 09, 2015 9:33 am

stratified Fri Jan 09, 2015 9:33 am
Oleg wrote:Thats great, but what if the material of the table will be different? Some shiny reflective surface such as plastic or lacquered


That shouldn't be a problem. You can still catch the reflections but you will have to make the shadows less strong in post.

cheers,
Thomas
User avatar
stratified
OctaneRender Team
OctaneRender Team
 
Posts: 945
Joined: Wed Aug 15, 2012 6:32 am
Location: Auckland, New Zealand

Re: How to use the render layer passes.

Postby linvanchene » Fri Jan 09, 2015 2:33 pm

linvanchene Fri Jan 09, 2015 2:33 pm
stratified wrote:
In Photoshop you would have the following layer stack and layer modes:

1) Beauty (normal layer mode)
2) Black shadows (normal layer mode) / Colored shadows (multiply layer mode) / Shadows (multiply layer mode)
3) Reflections (linear dodge (add) layer mode)
4) Backplate

If I get my hands on Photoshop, I will post an example file. Of course you need to tweak it afterwards because the shadow as they are capture in Octane likely don't match the ones in your picture (The shadows in Octane are cast onto a white diffuse material regardless of your real material set-up).

Does that answer your questions?

cheers,
Thomas


Thank you. It seems the "theory" behind the steps without lua are now clear.

- - -

I made some in between shots between the different layers to get a better understanding how each pass adds to the final result.

Layer reflections and Background.jpg
Layer reflections and Background


- - -

Black Shadows - Layer reflections and Background.jpg
Black Shadows - Layer reflections and Background


- - -

Beauty - Black Shadows - Layer reflections and Background.jpg
Beauty - Black Shadows - Layer reflections and Background


- - -

The glass and table example seem like a good and simple setup for some further testing.

At the moment I am under the impression that the Black Shadows on top of the Layer Reflections causes the caustics to disappear into the shadows.

Will try to catch a moment when the sun shines on the table to capture a picture and experiment with a version of a real glass causing caustics compared to a CG glass next to it...


It would be interesting to see some examples how other users are compositing with the render passes...
Attachments
composition v1001.zip
test composition .psd - 12mb
(8.8 MiB) Downloaded 440 times
Win 10 Pro 64bit | Rendering: 2 x Asus GTX 1080 Ti FE | Display: Asus GTX 1080 STRIX A8G | Intel Core i7 5820K | ASUS X99-E WS| 64 GB RAM
FAQ: OctaneRender for DAZ Studio - FAQ link collection
User avatar
linvanchene
Licensed Customer
Licensed Customer
 
Posts: 750
Joined: Mon Mar 25, 2013 10:58 pm
Location: Switzerland
Next

Return to Tutorials


Who is online

Users browsing this forum: No registered users and 1 guest

Sat Jan 20, 2018 5:23 am [ UTC ]