Custom AOV masking errors

[DartFrog]

A. Windows 10
B-D. Rendered on multiple machines with different graphic cards, different RAM capacities.
E. OctaneRender_Enterprise_2021_1_5_win
F. OctaneRender_Enterprise_for_3ds_Max_2021.1.5_-_12.23
G. 3ds Max 2023.1


Hi,
I'm having trouble getting clean black and white masks saved out using Custom AOVs. The masks look correct in the "Kernel OctaneRender viewport" but when saved out, the output images look to have scanline artifacts. Also the mask thru reflection and refraction doesn't work on output --but looks fine in viewport

Just to break down the project a little bit for you...
- Rendering a window configurator that keys materials, hardware, window grille patterns, and a few other things.
- I have 6 AOVs using "Render AOV output" to mask -- frame, hardware, glass, background, grille, plastics
- Custom AOVs are being set in the materials (I've also tried setting on geometry with no luck)

In total, this project will produce thousands of images but I've rendered a small subset to show what the errors are occurring; as well as what I'm seeing in the Kernel OctaneReder viewport --which is what I'm looking to get as outputs.

TLDR - I need some of the masks to work thru refraction/reflections and I need clean masks.
Any ideas what could be causing this and any fixes? Or is it just a bug? Thanks

[neonZorglub]

A. Windows 10
B-D. Rendered on multiple machines with different graphic cards, different RAM capacities.
E. OctaneRender_Enterprise_2021_1_5_win
F. OctaneRender_Enterprise_for_3ds_Max_2021.1.5_-_12.23
G. 3ds Max 2023.1


Hi,
I'm having trouble getting clean black and white masks saved out using Custom AOVs. The masks look correct in the "Kernel OctaneRender viewport" but when saved out, the output images look to have scanline artifacts. Also the mask thru reflection and refraction doesn't work on output --but looks fine in viewport

Just to break down the project a little bit for you...
- Rendering a window configurator that keys materials, hardware, window grille patterns, and a few other things.
- I have 6 AOVs using "Render AOV output" to mask -- frame, hardware, glass, background, grille, plastics
- Custom AOVs are being set in the materials (I've also tried setting on geometry with no luck)

In total, this project will produce thousands of images but I've rendered a small subset to show what the errors are occurring; as well as what I'm seeing in the Kernel OctaneReder viewport --which is what I'm looking to get as outputs.

The attachment AOV Error.jpg is no longer available

TLDR - I need some of the masks to work thru refraction/reflections and I need clean masks.
Any ideas what could be causing this and any fixes? Or is it just a bug? Thanks

Hi DartFrog,
Thanks for the details.
I couldn't reproduce your issue.
Could you post a sample scene ?

How does this sample works for you ? Is it the same thing you are doing ?

AOVMask04.zip

(88 KiB) Downloaded 101 times

Are you saving from the Octane viewport button ? or with render elements and normal render ?
Those artifacts seems to come from a format / size problem.. You could try to change LDR / HRD.. or change the image size, making sure it's a multiple of 16 for example.. save as png / exr / exr with all passes ?

Thanks

[DartFrog]

Hi neonZorglub-

Thanks for taking the time to try and figure this out.

Here's a sample scene. I had to dumb it down for legal purposes but this scene is giving me the same issues.

AOV Test.zip

(61.35 MiB) Downloaded 123 times

Things I've tried to fix the issue with no good results:
- Saving thru the Octane viewport button, normal render button, and a pluggin called Prism/aka LPM (for batch render launch) with render elements.
- Different output sizes from 250x250 to 4096x4096
- Rendered both LDR and HDR
- File types .png, .tga, .tif, .exr (exr compression: ZIP, DWAA, DWAB -- 16 bit and 32 bit)

[neonZorglub]

Hi DartFrog

Thanks a lot for your scene! I could reproduce the issue, and this also happens in Stand Alone..
I can't think of a good work-around for now, but I'll try to have a quick fix soon until the engine team provide an update.

EDIT: in fact, the issue seems to be solved in 2022, and was caused by having multiple GPU doing tone mapping:

> To fix it with 2021.1.5 / 2021.1.6, you need to enable 'Tone' to only 1 GPU, in Render settings / Devices.
and click the Apply button.
(double check by opening the Octane Device Setting window)

Or you could you try to use 2022 for those specific renders :
viewtopic.php?f=80&t=80238

Thanks

[DartFrog]

Hi neonZorglub

That did fix the scanline looking output errors but it did not fix masking thru reflections and refractions. Unfortunately I cant get the whole office to update to the 2022 version. Maybe once the full stable version is released we will be able to but I have a work around for the time being.

Thanks for figuring out the issue.

[neonZorglub]

Hi neonZorglub

That did fix the scanline looking output errors but it did not fix masking thru reflections and refractions. Unfortunately I cant get the whole office to update to the 2022 version. Maybe once the full stable version is released we will be able to but I have a work around for the time being.

Thanks for figuring out the issue.

Hi DartFrog,

Glad to hear that the work around partially works for you.

I'm not sure what you mean with masking thru reflection / refraction..

In you sample, I only see the pass Denoised reflection direct, that seems to work same in viewport, and saved to file..

Are you using a Composite AOV output, with one of the reflection pass as INPUT, and one of the custom AOV as Input of a Render AOV output for the MASK ?
This also seems to work in viewport and saved to file..

Do you have a sample scene for this ?

Thanks

[DartFrog]

Hi neonZorglub

I am not using the Composite AOV output. Every time I tried that route, Max would crash. I just used Render AOV output. You can see in the screen shot, AOV 05 is rendered thru a glass material. I set in the Custom AOV 5 to Refraction (at the bottom of the Custom AOV settings). In the kernel Octane Viewport render it works perfect. In the actual output (when just clicking the Render button), there's nothing - just a black image.

screenshot:

Thanks for taking the time!

[DartFrog]

Sample Scene Link:

AOV Test Scene for Octane Team.zip

(677.61 KiB) Downloaded 89 times

[neonZorglub]

Sample Scene Link:

AOV Test Scene for Octane Team.zip

Hi Dartfrog,
Thank you for the sample scene and details !
I see; that's right, the custom AOV parameters are not sent to the renderer in normal render mode.
Sorry about that.
That will be fixed in the next release.
Thanks