NVLINK Mismatch Error - Mismatching barriers, 5 and 2

[leehenshall]

I'm using x 2 2080ti with a matching NVLink Adapter attached with the system correctly configured in SLI with Nvidia Studio Driver 442.92 installed and RTX enabled. I am also using Octane 2020.1-R2

I enable "peers 1 and 2" to allow for Octane to use NVLINK.

When I start the LV render, the image starts as expected. When I move the Camera for a while I get an error.

The Log then says "Mismatching barriers, 5 and 2"

I've read that users in the MAX forums are having similar issues:

viewtopic.php?f=81&t=74535&p=382233&hil ... 39#p382233

I have tried reinstalling the driver. But still getting this issue.

If I disable Peers 1 and 2 the rendering works fine so I'm fairly sure this is related to NVLINK or at least my understanding of how to use it effectively.

Could it be that the NVLINK adapter is struggling with bandwidth somehow? Could Parallel samples and Max tiles be a factor regarding NVLINK bandwidth?

Here are my IPR settings:

Also finally. Is there a way to see how much out of core is going to NVLINK and how much is going to System Memory? Can the two be used together or is it one or the other?

Maybe my scene is maxing out my NVLINK and cannot then go on to system RAM?

Not sure if I understand correctly. Any Help would be greatly appreciated

[mojave]

Thank you for reporting this,

We have internally fixed a couple of issues with the latest version when using NVlink, could you share some scene that can be used to reproduce this issue so we can look into it?

Best,

[leehenshall]

I can send OTOY the full production scene and assets.

Do you have a process for this that ensures the assets will be kept private and for testing purposes only?

[mojave]

I can send OTOY the full production scene and assets.

Do you have a process for this that ensures the assets will be kept private and for testing purposes only?

We might not need the full scene, if you could reproduce it with a subset of it that would actually help us too. Either way you may share it by any means you prefer via PM, we will not share any assets outside our internal test environment.

Thank you.

[leehenshall]

Apologies for the delay in responding.

I will collect a simplified version of the scene and send via WeTransfer or Dropbox link and PM you.

[leehenshall]

Okay so I finally got round to trying to replicate this in a simple scene.

I grabbed a single asset from my complex production scene and duplicated it to spike the VRam usage and managed to replicate the issue.

Here is a link to the test scene: (download link will expire in 1 week)

https://we.tl/t-Zqq4nq6kG3

The error reads "Mismatching barriers, 5 and 2" it only occurs when peers are set to use NVLink in the device settings and only occurs when the scene is heavy enough to use all of the primary GPU VRam thus requiring NVLink out of core.

[mojave]

Thank you for sharing this, we will look into it.

[leehenshall]

Looks like this issue is fixed in the latest build.

I'm guessing it is probably one of these line items that has helped.

I have re enabled Peer 2 Peer and seems to be working fine. I'll let you know if it happens again.

Thanks

[leehenshall]

Unfortunately I have encountered the issue again in the latest build. This scene is a very memory intensive so I think my use case is more out of the ordinary.

Here is a screenshot:

It is much more stable in this new build however but still seems to be a problem.

I am unable to see any feedback in the C4D plugin as to whether I am using P2P/NVLink. In SA I can see that there is a P2P bar in blue, presumably to indicate when and how much the scene is in P2P/NVLink.

[rclassenphoto]

Hi,

i have the Same problem. I use NVlink with two Titan RTX and i get this errors in Octane 2020 with C4D r21 and the new octane Plugin:
.----------------------
Started logging on 13.06.20 20:11:01

OctaneRender Enterprise 2020.1 (8010000)

CUDA error 719 on device 0: unspecified launch failure
-> failed to wait for event
device 0: path tracing kernel failed
CUDA error 700 on device 1: an illegal memory access was encountered
-> failed to wait for event
device 1: path tracing kernel failed
.----------------------

Started logging on 14.06.20 10:43:11

OctaneRender Enterprise 2020.1 (8010000)

Mismatching barriers, 5 and 1
could not request a tonemap because we don't have any renderthreads available.
Mismatching barriers, 5 and 1

.----------------------
But i test it out to turn off NVlink and reinstall the driver without the NVlink bridge it does the Same. The errors Are Still be there.

This errors only Happend if i move a objekt in C4d with octane Live Viewer fired on.

I dont know why?


----------------------

Started logging on 15.06.20 20:13:31

OctaneRender Enterprise 2020.1 (8010000)

CUDA error 700 on device 0: an illegal memory access was encountered
-> failed to wait for event
device 0: path tracing kernel failed
CUDA error 700 on device 1: an illegal memory access was encountered
-> failed to wait for event
device 1: path tracing kernel failed
--------------------