Finding correct settings for rendering layer masks Tweet

[SundancePioneer]

Perhaps you can upload a cropped version of your photoshop file.
It looks to me like your mixup is there rather than your render.

Sure thing, here it is:

alpha-fail_cropped-1.psd (33.83 MiB) Viewed 1790 times

I used the Beauty pass from Octane and the OCol infopass - i used the method explained here to make use of the Object Color Id pass as a layer mask in PS: https://vimeo.com/119850717

If this is not the right method, i'd be thrilled to know what i need to do

Thanks so much for your help!

Greetings

Sundance

[niestudio]

It looks like you're using the C4D save file rather than the render layers you have setup.
Have you tried those?
This is just two of the layers, but you should be able to add all you need.
They are grouped into one folder in PS, and then that folder is masked with the OCol Layers image data.

This is the basic setup I use, and I don't ever see that fringe like you are getting.
Perhaps there is a way to use the C4D Save file, but someone else would have to advise on that as I don't even keep that checked.

[SundancePioneer]

It looks like you're using the C4D save file rather than the render layers you have setup.
Have you tried those?
This is just two of the layers, but you should be able to add all you need.
They are grouped into one folder in PS, and then that folder is masked with the OCol Layers image data.

That looks very good - what settings did you use to get the layers that way? Or did you set it up yourself in PS?

I actually used the files that were put out by Octane Render (render layer mask and Ocol, tried both) - i only had the C4D Save checked to get a flat beauty pass.

Did you render my C4D file without any changes to the settings?

Thanks for your kind help!

Greetings

Sundance

[niestudio]

Did you render my C4D file without any changes to the settings?

Yes, Those were the layers rendered from the file as you uploaded it.

[aoktar]

Did you render my C4D file without any changes to the settings?

Yes, Those were the layers rendered from the file as you uploaded it.

Thanks for pointing the correct way. Great help!

[SundancePioneer]

Yes, Those were the layers rendered from the file as you uploaded it.

Thanks indeed, i can see now where the error in my workflow was!

Did you use any bending method for the layers? Is there a guide to the correct layer stacking order and blending modes when working with multiple passes?

Thanks again for taking the time to look into it! much appreciated!!

Regards

Sundance

[niestudio]

this is a great one (all of his are great) https://www.youtube.com/watch?v=jBO7ywXpxCY

[SundancePioneer]

this is a great one (all of his are great) https://www.youtube.com/watch?v=jBO7ywXpxCY

Thank you for the link and for your great help on this matter

Kind regards

Sundance

[SundancePioneer]

Yes, Those were the layers rendered from the file as you uploaded it.

Hi niestudio,

sorry to open this again and bother you, but i seem to have another/new problem in this matter now:

I rendered seperate passes (Refl. Direct, Refl., Diffuse, Beauty, Shadow) - when using the approach you suggested via your PSD file, the mask works absolutely fine for Refl. Direct and Reflection, but the Diffuse Channel within the layer group gives me a white fringe again.

Any idea what could cause this? Is the way i am merging the layers wrong? I just used "Passthrough" in PS like in your file, with the diffuse channel at the very bottom. The reflection passes mask fine, but the Diffuse pass gives me problems.

Any help would be much appreciated!

Thanks in advance

Sundance

[niestudio]

Hi Sundance,

Looking back at the file you uploaded, it looks like you have a couple other things working against you.
1. in your camera imager you have pre-mulitply alpha and Disable partial alpha ticked. This is giving you an aliased edge.
2. your white background is rendered with the product in your current setup, and that is what you are seeing on the edge.

I'd suggest reading up on the Render Layer system and using that method.
https://vimeo.com/119848103 - this is an old version of octane, so the settings look different, but the technique remains similar.
The photoshop end of it will be similar if not the same, but you'll get your object(s) rendered without the background.

If you don't want to use render layer, unticking those camera imager settings should get you a tighter (anti-aliased) edge, and from there you could use Select > Modify > Contract…(1px) in photoshop on your mask selection.