slave not rendering w/ picture viewer

Sub forum for help and tutorials.

Moderators: ChrisHekman, aoktar

garytyler
Licensed Customer
Posts: 234
Joined: Mon Nov 10, 2014 7:40 am
Location: Portland, OR

aoktar wrote:Ok thanks. i'll check it. My idea is still same. If it's working on LV, should work on Picture Viewer. Anyway i need to make some renderings to test it again. I'm away from my network. It's possible to make these tests a few hours later. Even i'm thinking add some log output for getting and printing messages from clients.
Thank you very much for looking into it. In the past, working in LV has worked in Picture Viewer but I must reiterate this again, this time in all capital letters because it needs to be understood somehow: IT WORKS IN LIVE VIEWER BUT NOT IN PICTURE VIEWER.

I could produce an elaborate video proving to you that this is happening or you could believe me based on the logical evidence I have provided. Please just believe me. I am an experienced Octane for c4d user. I will continue posting any findings but I believe there is plenty of evidence in this thread to warrant considering this issue seriously.

If there aren't any findings, I will put a warning in the General Discussion thread that other users compare their render times because I believe this has been happening on my setup for the past week and I didn't even notice it until I started getting annoyed at the long render times. I turned off network rendering and everything got faster.
Win 8.1 / C4D R16 / Maya '16 / 5x GTX 980 Ti
User avatar
aoktar
Octane Plugin Developer
Posts: 16066
Joined: Tue Mar 23, 2010 8:28 pm
Location: Türkiye
Contact:

Here is a picture to show my results. As you can see, network rendering is definitively faster. Master has a 780, a client has also 780, other one has 680.
Secondly, I couldn't reproduce your issue here. Works flawless. Also i'm sending a test version to check. I have made a small change on network function. I hope that will be better.
Attachments
a1.jpg
Octane For Cinema 4D developer / 3d generalist

3930k / 16gb / 780ti + 1070/1080 / psu 1600w / numerous hw
garytyler
Licensed Customer
Posts: 234
Joined: Mon Nov 10, 2014 7:40 am
Location: Portland, OR

Capture.PNG
Capture.PNG
I was performing a similar test for you and received this error message on the slave after rendering 1 frame:

The GPUs referred to are both regular Titans that perform well on their own as of today.

PS - I tried updating to the new 353.30 Nvidia driver shortly before this. It runs great on the master. I may try restoring my slave right about now.
Win 8.1 / C4D R16 / Maya '16 / 5x GTX 980 Ti
User avatar
bepeg4d
Octane Guru
Posts: 10356
Joined: Wed Jun 02, 2010 6:02 am
Location: Italy
Contact:

hi gary,
i would like to help you, but as I already said,we need more detailed information about your setup for both master and slaves:
machines specification
GPUs specifications
system specifications
a scene that show the issue
thanks,
ciao beppe
User avatar
bepeg4d
Octane Guru
Posts: 10356
Joined: Wed Jun 02, 2010 6:02 am
Location: Italy
Contact:

about the frame buffer:
in general we work with lower resolution in LV and then, launch the final render in PV with the desired resolution.
But the frame buffer has a huge impact in VRAM consumption, so it could happen that what is easy to calculate in LV, it does not in PV because the VRAM is not enough ;)
ciao beppe
User avatar
abstrax
OctaneRender Team
Posts: 5509
Joined: Tue May 18, 2010 11:01 am
Location: Auckland, New Zealand

Hi Garytyler,

Sorry for the late reply but we are currently very deep in version 3 development...

Although network rendering looks simple and usually just "magically" does it's work, it's quite complex inside. So there are many things that can go wrong here. At the moment, if a slave crashes or fails there are usually no good error messages on the master. So you would have to check the console output of the slave process on the slave computer. And we try to make sure that the slave doesn't crash. In version 3 we plan to make things more robust. We are right now working in that area.

Anyway, to give you an idea what happens internally, this is the flow on a high level when an image is rendered via network rendering (assuming the master <-> slave connection is already established):

1) The master sends the current render data (geometry, textures, etc.) to the slave.
2) The master assigns some work to the slave.
3) The slave starts working through the assigned work.
4) After some work is finished, it sends a result back to the master and continues working through the assignment.
5) After receiving the result the master assigns more work to the slave unless all samples are done.
6) The master uses the result of this slave, other slaves and the local GPUs to create the current render result, which is passed on to the plugin.
7) go back to 4)

Ok, so what are the things that can be a problem here?

a) Sending render data to the slave takes very long or doesn't "get through" to the slave.
b) Sending assignments to the slave takes very long or doesn't "get through" to the slave.
c) The slave start working on the assignment, but its GPUs crash causing it to never return any results to the master.
d) Sending back results to the master takes too long, causing the slave to run out of assigned work long before it gets new work, i.e. it's sitting there most of the time waiting for new work.
e) Something else... maybe the plugin does something special which causes the network rendering to stall.

Your last screenshot indicates a GPU crash on the slave. Does it happen reliably? I.e. if you stop the daemon via CTRL-C and then restart it using the _run_installed_daemon.bat and then using it for network rendering, does it immediately crash?

On the other hand, you are saying that it works in the Live Viewer. There is probably one major difference between rendering in the Live and the Picture Viewer: In the Live Viewer images are usually rendered as LDR results, i.e. they are only 1/4 the size than the HDR results which you are probably rendering in the Picture Viewer. Another difference is that you probably don't have any render passes in the Live Viewer, but do have them in the Picture Viewer. Render passes require more memory on the GPUs and take longer to transmit to the master.

Btw, which OS and which version are you using? What is the resolution you are rendering? Is it an animation or just a single frame? Does the rendering finish, i.e. reach the maximum samples/pixel?

One thing you can do to help me understand the problem better is to enable logging on the slave and the master, which is described here: http://render.otoy.com/forum/viewtopic. ... 75#p234275

The same way you can enable logging for the Standalone (master), you can do it for the plugin (master) and for the slave. You just have to copy the octane_log_flags.txt file into the directory where the octane.dll is. Make sure that this is in a directory where you have write permissions (otherwise Octane can't write the log files). To kick off logging the application needs to be restarted (because the flags file is only read once in the beginning).

That's it for a start. I hope with the log files I will see what's going on.
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
garytyler
Licensed Customer
Posts: 234
Joined: Mon Nov 10, 2014 7:40 am
Location: Portland, OR

Wow. Thank you so much for this information. Truly wonderful! I'm sure more users will find it useful in the future as well.

I will not be able to implement logging for another day or so but I certainly will. I did read over everything and it makes very good sense. I'm sorry my details and and screengrabs are so sporadic. I am generally trying to get work done, troubleshoot, and post at the same time.

And bepeg4d, thank you for the explanation of frame buffer.

When I have another day to troubleshoot this, I will report back with anything I find.

Thank you! :)
Win 8.1 / C4D R16 / Maya '16 / 5x GTX 980 Ti
Post Reply

Return to “Help / Tutorials”