Machine Learning panel labeling using Render Layer ID Tweet

[MrMetatron]

Hello all

I am working on setting up a pipeline for ML training with synthetic data sets generated with C4D and Octane.

The beauty shots are all good but when I need to go and set up a the INFO passes as AOVs I have an issue.

I'm masking out some parts of the geometry, in this case the wheels of a car, with a simple geometry on top of it that is not rendered out on the beauty pass by setting it's layer ID to a different number. That enables me to have the beauty pass, without any other masking geometry visible. (See image 1.)

When I add the object layer ID to the Render AOV group as a separate layer, all the geometry shows up color coded as it should, which is great. The problem starts from there. I don't have an obvious way to control the color of the Object Layer ID (see image 3.). It seams it's generated internally and I need to figure out how to assign a specific color (designated by the ML research team)to each layerID. (see image 2.)

Now mind you, this my way of doing it. If there is an obvious solution to this I'm missing, please let me know.

I need to have flexibility of color color management for those layers or maybe find a way to use the Display color of the object to drive the Layer ID colors.

If anyone has a solution to this I appreciate the help.

Thank you.

[MrMetatron]

^^ BUMP ^^

Anyone?

[jobigoud]

I'm not sure about the specifics in C4D but in standalone there is a Render AOV named "Object layer color AOV", under the Info category. This renders the object in the color you set in Object layer > Color.

[MrMetatron]

Thank you for the response.

It is available and it would do the job of customizing the color, but it doesn't solve the visibility problem. I need to have a beauty pass without the masking objects visible and an info pass where some things are occluded for the AI to group them into one label. That means I need the whole rim of the wheel to be occluded without any transparent parts or objects with other labels visible through it.

The Render Layer ID solves this pretty elegantly as I can add different label groups to a new Render layer and not worry about having them rendered in the beauty pass. The only thing missing is a way to assign a color to those layers. A checkmark that says Use Display color would solve the problem. Not sure if it can be scripted in somehow or add some custom node that can pipe in the desired color. I'll try to figure out other methods as well but if I can't solve this, ultimately I might need move on from Octane as solution.

I have added a screenshot to show the result of the "Object layer color AOV"

[jobigoud]

The Render Layer ID solves this pretty elegantly as I can add different label groups to a new Render layer and not worry about having them rendered in the beauty pass. The only thing missing is a way to assign a color to those layers.

If I understand what you want to do, you could just assign the same layer color to all the objects that are on a given layer?
Or even share an object layer node between all the objects that need to be grouped, this way they all get the same layer id and layer color.

To clarify, what you want in your last screenshot is for both elements of the wheel to be one single color, say red. Correct?

[MrMetatron]

Thank you for your response. Unfortunately English is not my first language and I'm not sure if I understood your formulation.

I will try and explain the issue more clearly.

We would like to render out Beauty passes of objects that are complex to teach AI to recognize those objects in real life. To do that we need to color code parts of the objects and group complex parts into a more general shape that can be then labeled by the research team by color grouping.

In 3D we would create a proxy shape that would represent the complex group with one color in the AOV pass. The proxy object would only be visible in the AOV pass and not the Beauty pass.

I need to find a way to render the whole object as it would look like in real life but have the proxy parts as well as other parts of the object color coded specifically to the research team's color specs.

I realized over the weekend that the Layer ID is good for hiding the proxies from the main render layer but with multiple objects it would be impossible to render out everything I need on layer 1 but with different color coded parts.

I need a way to hide the proxy objects from the main render (but show up in an AOV) and to control the color parts of the object by grouping geometry into specific color groups (in the AOV).

I hope this is clearer now.
Thank you

[jobigoud]

Would this work: render the scene once on render layer 1 (normal geo) and save the beauty pass, then render the scene on render layer 2 (proxy geos) and save the Object layer color AOV.

Solution 2, simpler but maybe there are too many objects: don't use render layers or proxy objects, go into the object layer of each part and assign the color corresponding to its category. Then save beauty and object layer color.

[MrMetatron]

Would this work: render the scene once on render layer 1 (normal geo) and save the beauty pass, then render the scene on render layer 2 (proxy geos) and save the Object layer color AOV.

Solution 2, simpler but maybe there are too many objects: don't use render layers or proxy objects, go into the object layer of each part and assign the color corresponding to its category. Then save beauty and object layer color.

Solution No1. would double the render time because we would need to render the scenes twice. When we are rendering for AI those takes thousands of images to train the AI on.
Solution No2. This is on the right track but I need the proxy geos as a way to cover all the complexity in the Beauty pass and group it under one color. I am going to experiment with that going forward but I need a way to hide the proxy geo in the beauty pass.

A tip for the devs that would change the game for AI dataset generation workflow: Add a checkbox that allows the LayerID color to be set by the Object Display color. This would allow to hide the proxies but still be seen in the AOVs.

[jobigoud]

I don't think it will double the render time because the proxy objects should be very fast to render. You can even use the Info channels kernel for this, it has the object layer color mode, and very few samples per pixels, maybe even 1 spp.
So render time should be negligible, it will take longer to save the images to disk.
As long as you use render layers you will have to render twice anyway, as far as I know.

[jobigoud]

Another solution that might be closer to what you are trying to do: use a null material on the proxy objects. This way they won't show up in the main beauty render, but they will still show up in the Object layer color AOV.
In this case there is no need for render layers, you can save the beauty and the segmentation map in one render.