Lots of Denoiser Crashes

Forums: Lots of Denoiser Crashes
Houdini Integrated Plugin

Moderator: juanjgon

Lots of Denoiser Crashes

Postby labmeta » Wed May 08, 2019 8:56 pm

labmeta Wed May 08, 2019 8:56 pm
Hi Jaunjo,

I'm getting tons of denoiser crashes at the moment - Comes with an error message that there is not enough memory (needs approx 400Mb).
When i look at my system monitor i have over 32GB out of 64GB system memory free, and only 6.8GB out 8GB video card memory is being used.

The frame size it's trying to denoise is a fairly modest 3000x1000 ish - Any ideas? How much headroom does the denoiser need beyond this?

Best,

Paul
labmeta
Licensed Customer
Licensed Customer
 
Posts: 24
Joined: Wed Jul 30, 2014 7:27 pm

Re: Lots of Denoiser Crashes

Postby juanjgon » Wed May 08, 2019 9:04 pm

juanjgon Wed May 08, 2019 9:04 pm
Hi,

The denoising is done on the GPU, so perhaps you don't have enough free VRAM to execute it. If you have several GPUs, you could configure only one for denoising, and never the same GPU used for the monitors. You could also try to enable the out of core feature, to see if it leaves more free VRAM for the denoising.

Thanks,
-Juanjo
Computer Graphics Software Developer
Octane plugin development and support: LightWave and Houdini
User avatar
juanjgon
Octane Plugin Developer
Octane Plugin Developer
 
Posts: 7558
Joined: Tue Jan 19, 2010 12:01 pm
Location: Spain

Re: Lots of Denoiser Crashes

Postby labmeta » Wed May 08, 2019 9:34 pm

labmeta Wed May 08, 2019 9:34 pm
Hi Jaunjo,

Thanks for the fast reply. This is quite curious - If i enable out of core textures for the render node (I'm running everything through deadline here), deadline gives me a caught exception and the frame gets failed. De-enabling allows the frame to render but it then fails at the denoiser stage. I wonder if this caught exception could this be related to my previous support request about deadline that we never got to the bottom of?

Good idea to reserve one card for denoising ( this machine has x2 2080's) but sadly this didn't help. Further to this I have another machine with the exact same specs (x2 2080's) and it's rendering the same scene/frames without problem.

Any other thoughts?

Thanks,

Paul
labmeta
Licensed Customer
Licensed Customer
 
Posts: 24
Joined: Wed Jul 30, 2014 7:27 pm

Re: Lots of Denoiser Crashes

Postby juanjgon » Wed May 08, 2019 9:48 pm

juanjgon Wed May 08, 2019 9:48 pm
Hmm, it is weird that this problem only happens in one machine if you have another one with the same specs. Are you sure that you are denosing only with the GPU not used for display? Usually, this GPU has more free VRAM. Do both machines have the same drivers?

Thanks,
-Juanjo
Computer Graphics Software Developer
Octane plugin development and support: LightWave and Houdini
User avatar
juanjgon
Octane Plugin Developer
Octane Plugin Developer
 
Posts: 7558
Joined: Tue Jan 19, 2010 12:01 pm
Location: Spain

Re: Lots of Denoiser Crashes

Postby labmeta » Thu May 09, 2019 8:33 am

labmeta Thu May 09, 2019 8:33 am
Yes same drivers, same brand of RTX cards too. I've just tried a driver update across all machines but it hasn't changed anything.

There's no other software running on either of the machines while rendering either.

The out of core switch crashing deadline nodes seems repeatable here, I'm reporting this to thinkbox as a bug to see what they make of it - I'm curious if it's repeatable for yourself or anyone else?

Thanks for you help so far,

Paul
labmeta
Licensed Customer
Licensed Customer
 
Posts: 24
Joined: Wed Jul 30, 2014 7:27 pm

Re: Lots of Denoiser Crashes

Postby labmeta » Thu May 09, 2019 8:34 am

labmeta Thu May 09, 2019 8:34 am
and to answer your other question - I've tried reserving both card 1 and card 2 for denoising separately just to make sure - same error crops up.
labmeta
Licensed Customer
Licensed Customer
 
Posts: 24
Joined: Wed Jul 30, 2014 7:27 pm

Re: Lots of Denoiser Crashes

Postby juanjgon » Thu May 09, 2019 8:57 pm

juanjgon Thu May 09, 2019 8:57 pm
What a weird issue, I can't figure what could be the problem, to be honest. And do you have this problem with smaller scenes? Perhaps this scene is just in the limits to leave enough vram for the denoising process, and somehow this system some a bit less available free vram.

About your problem with Deadline, how are you enabling the out of core feature? Does deadline have an option for it?

Thanks,
-Juanjo
Computer Graphics Software Developer
Octane plugin development and support: LightWave and Houdini
User avatar
juanjgon
Octane Plugin Developer
Octane Plugin Developer
 
Posts: 7558
Joined: Tue Jan 19, 2010 12:01 pm
Location: Spain

Re: Lots of Denoiser Crashes

Postby labmeta » Fri May 10, 2019 9:48 am

labmeta Fri May 10, 2019 9:48 am
Hey Jaunjo,

Thanks for the input on the denoiser issue - I'll work around it for now, it is a strange one. When i have more time i'll look into it some more.

For the out of core memory issue - There is no deadline option tick box or anything . As far as i can see it picks it up from the octane preference file for that machine. I'm enabling it from within the Octane Options from the Octane shelf in Houdini then closing the program. I'll test some more across the other nodes and see whats up.

Hope we can figure this one out!

Thanks,

Paul
labmeta
Licensed Customer
Licensed Customer
 
Posts: 24
Joined: Wed Jul 30, 2014 7:27 pm

Re: Lots of Denoiser Crashes

Postby juanjgon » Mon May 13, 2019 9:28 pm

juanjgon Mon May 13, 2019 9:28 pm
The plugin has a couple of hscript commands to set the OOC parameters and to get the OOC status, as you can see in the attached image. They can be used also from python, and probably they should be included in the rendering script (for example in the Deadline hython rendering script) to configure the feature at the render time.

In any case, as you say, the OOC could also be configured from the preferences file, so I'm not sure why Octane crashes when it is enabled ...

Thanks,
-Juanjo
Attachments
2019_05_13-23_21_32.jpg
Computer Graphics Software Developer
Octane plugin development and support: LightWave and Houdini
User avatar
juanjgon
Octane Plugin Developer
Octane Plugin Developer
 
Posts: 7558
Joined: Tue Jan 19, 2010 12:01 pm
Location: Spain

Return to Houdini


Who is online

Users browsing this forum: No registered users and 4 guests

Tue May 26, 2020 6:31 pm [ UTC ]