Problem with projecting onto a live action plate.

[noisyboyuk]

Hi!

First off, apologies for the long post, I just want to cover as much as I can right off the bat.

I have a problem with a "projected" video not playing back in sync with my background plate (of the same video).

I'm doing a test scene for an upcoming job and in my test I just want to place a glossy sphere onto a ground plane and then project my plate onto the plane so that it reflects the floor up onto the sphere.

I first tracked the plate in Nuke and added a ground plane (just a card). I then exported the tracked camera and the plane out of Nuke with the footage and then imported the scene into Cinema. I then created a Cinema 4D material (not an Octane material) which had my background plate in the color channel, then I hit calculate in the animation tab and added it to a Background object. I then turned on the Alpha channel in Octane's settings and ran the Live View. So far so good - I can see my plate and the plane in Live View and it plays back when I hit play.

Next I added the same material to the ground plane and set the material projection method to Camera Mapping and then selected the tracked camera to project from. I refreshed the live viewer and boom, I can now see the ground plane with the video projected onto it and it perfectly lines up with the Background plate. If I move the playhead to any other frame however it all goes out of whack but as soon as I hit refresh on the Live View it all pops back into place again and lines up fine.

The real problem arises when I hit play. For some reason, even though the first frame lines up fine, if I hit play on the timeline the image projected onto the plane and the background plate start to move apart visually. The same happens if I try and render a sequence to the picture viewer although in the Viewport everything looks how I want it to look.

I should note that to try and narrow down the cause of the issue, I have tried this method with the standard Cinema 4D renderer and also Arnold and the technique works fine. In fact when I tried it with the standard renderer, for consistency I literally used the exact same scene and just changed the renderer and it worked fine.

The camera track is good too but that shouldn't even matter as being as the footage is projected from the point of view of the camera it should always line up to the back plate. I also can't see any frame skipping or frame doubling which makes me believe that there isn't a framerate mismatch anywhere.

If anyone has any thoughts on this I would be very grateful as I've been at it for hours now and can't seem to find much documentation about it online either. I can always go back and do this with Arnold but I could really do with Octane's speed on this one.

Here is a project file: https://www.dropbox.com/s/0cbhfyll6j8yq ... 1.rar?dl=0

It's about 500MB due to the video clip included but if someone wants a pop at it please feel free! (note that the frame range is set from 50-350 rather than 0-300 due to the way Nuke initially exported the scene and it didn't seem worth correcting at this point).

Oh and one last thing... Nuke exported a super tiny scene which revealed another issue. Even though I have set the F-Stop to Infinity and turned "Filter" all the way down in Octane kernel settings, I still have a little bit of depth of field blurring. Is there any way to turn DoF off entirely?

Thanks guys and gals!

[bepeg4d]

Hi noisyboyuk,
it seems that you have forgot to set the lenght of the sequence:

after that, it works as expected in Picture Viewer.
ciao beppe

[noisyboyuk]

Hi noisyboyuk,
it seems that you have forgot to set the lenght of the sequence:

The attachment IMG_0128.JPG is no longer available

after that, it works as expected in Picture Viewer.
ciao beppe

Thanks for the reply

I apologise, I must have forgotten to save it with the frame length set but I have been setting it in my tests. In fact just to be sure I just went back in and set it but I am still getting the same behaviour. One thing I would mention is that when I set it up myself, I set it to "Frame Range", rather than "Exact Frame" so I can set it to start at frame 50 (which is where the camera tracking data starts from the Nuke export).

These settings should be correct as it works fine if I render with C4D's Renderer instead of Octane.

Just for clarity, the issue isn't getting it to play. The problem is that if you render the sequence out to the picture viewer and play it back, the projected material on the plane appears to float around rather than match up perfectly with what you can see on the Background object. If on your end it seems to be working fine, would you mind sending the project file back to me so I can check it out at all?

Also, here is a version of the test using just the C4D renderer. If you render from this project to the picture viewer and then render from the original Octane version, you will see the difference if you compare them (the C4D version works as expected):

Test_with_assets_v002_(C4D_Renderer).rar

(210.03 KiB) Downloaded 157 times

You will have to re-link the image sequence from the original upload.

Hopefully it's just something silly I've missed so if that is the case it would be awesome if you could help me work this out.

Cheers Beppe!

[aoktar]

I'm checking your issue.
It's a detection problem on uvw of plane. Difference than other renderers we're generating/updating datas when we detect changes on objects/materials/etc... Object is not resent to renderer on next frames so generated uvs by camera mapping that stays same over animation.
Workaround is
1- to apply deprecated method: Send all data or animate somethings over plane object or its tags.
2- keying something in texture tag as i did in attached scene. I advice this way.

[noisyboyuk]

Thank you!!! Your scene appears to be a good workaround!

I noticed that the material's Offset-V is keyframed from 0.055% at the start to 1% at the end. Are these the same values I should use each time I use this method in a project or is this a value I need to work out manually every time I do this ? If I have to work it out each time, may I ask how you came to those particular values and how I work it out myself? I'm just concerned about accuracy if I use this method to project onto geometry when reconstructing a scene for a VFX sequence.

Also is there anything else that was adjusted to make this work or was it just Offset-V?

Thanks again for your help guys! It was a long day and night trying to get this far!

[aoktar]

You can trick it with another parameters. I hope you see the problem. We're just trying force recreation on plane object. You can use another texture tag before your material and key it like this.
Btw i'll look for a better solution on plugin side and put a fix for this issue.

[noisyboyuk]

I'll take another look at it after some sleep as I'm still struggling to grasp the mechanics of it (probably due to being crazy tired) but I can at least work off of the scene you sent over for now and above all else I'm just happy that the issue will be looked into further

Thanks again to you and the team! I really appreciate it!

Kraig.

[noisyboyuk]

Just to say, I think I get it now.

Because Octane currently only updates when there is a change, you are just keyframing values so there is a change on every frame which forces Octane to update on every frame. As you suggested I just added a new tag before my material and just keyframed the v-offset on that, but I get that it can actually be any parameter.

Just as a final follow up, in Arnold you have several parameters for compositing: visible to camera, visible in reflections, visible in refraction etc. This allows me to make the plane visible in the reflections of my sphere but not show up in the camera. Is there a way to do this with Octane?

Thanks again!

EDIT: I realised that there isn't, so I had to live without in the Live Viewer and just exported separate passes for compositing.

[aoktar]

Pls check lates test build(3.06-test3.1). And you need to add Octane Object tag to your camera mapped objects. It's a temporary solution for this build.