Estimate off by 3x (Solved) and lack of support

Forums: Estimate off by 3x (Solved) and lack of support
Discussion forum for OctaneRender Cloud

Estimate off by 3x (Solved) and lack of support

Postby Renderwiz » Fri Oct 06, 2017 11:44 am

Renderwiz Fri Oct 06, 2017 11:44 am
Edit: It appears the largest part of the estimate inaccuracy was being caused by animated alembic cache files. If you are going to use these, be aware they may slow down your frame initialization significantly on orc. Your estimates will be wrong if your local machine does not experience this slow initialization. Once I remove the alembic files, the estimates seem to be just about spot on for large renderings, and much closer for animation frames.

Suggestion: The calculator could take the time per frame into consideration as well as the orbx filesize and apply a factor to the estimate for animation frames. If OB/hr is going to be a trusted unit of measurement, it would make sense to avoid unnecessary confusion when using it.

-----------

The estimate provided is around 1/3 of what I am being charged. Also based on my understanding of the pricing model the reality seems to be about 3x the cost i was expecting. Does anyone agree with orc's estimates for animation?

This is purely based on their calculator and my math....I do not see a Run an Estimate button in the interface as the help docs show.

I have read other responses that point to loading/saving time being the culprit. If this is the case, surely otoy can take this into account when estimating (and make that it clear when quoting pricing)

I have sent an email to support and have not heard back in over 24 hours. Is this typical?
Last edited by Renderwiz on Tue Oct 17, 2017 7:45 pm, edited 7 times in total.
Renderwiz
Licensed Customer
Licensed Customer
 
Posts: 8
Joined: Mon Aug 07, 2017 10:34 am

Re: Estimate off by 3x and lack of support

Postby Renderwiz » Wed Oct 11, 2017 11:17 am

Renderwiz Wed Oct 11, 2017 11:17 am
Good news I hope. I have heard just back from support, with an apology that this is not the norm.

They are looking into it further. I will report back when I figure out what can be expected of the orc service.
Last edited by Renderwiz on Mon Oct 16, 2017 11:27 pm, edited 2 times in total.
Renderwiz
Licensed Customer
Licensed Customer
 
Posts: 8
Joined: Mon Aug 07, 2017 10:34 am

Re: Estimate off by 3x and lack of support

Postby Renderwiz » Mon Oct 16, 2017 11:25 pm

Renderwiz Mon Oct 16, 2017 11:25 pm
Ok, not great news. I still have not received a good answer about why the estimates are so wrong. Also have not received a response to this post, which has now been up over a week, and has almost 200 views. I wanted to allow otoy support plenty of time to respond to my questions before sharing my review of orc. I have waited long enough at this point.

I should point out that obviously they cannot be held to exactly what the estimate says.... but to be off by 3-4x (and yes I double checked the OB scores on my machine) makes the calculator useless.

My tests show that there is significant non-gpu overhead in the ORC system... in some cases 75% of the time is spent on initialization and/or saving files. This does not exist typically on local machines, so it causes a problem for estimating correctly. You will need to apply a significant multiplier factor in the case of using ORC for animation.

To explain this a bit better.... when you render locally, close to 100% of the overall time is gpu time spent rendering a frame. On the other hand, with ORC it seems this percentage can vary, and is as little as 25% in some cases. So you will be paying for every minute of the machine time, but in some cases only 25% of the time is actually rendering.

For very complex or large frames (ie. longer than 10 min per frame on a single 1080ti) the overhead seems to be better, but my estimates were still off by around 50% even on longer frames. For high resolution stills I imagine it gets closer to the estimate provided, but I haven't tested this yet.

On animation frames that take less time (around 1-2 min per frame on a single 1080ti) my tests show that as little as 25% of the time is actually spent rendering. The calculator provided by otoy does not factor in this overhead, so if you are rendering this type of animation on orc, you should plan 2x - 4x the cost estimated by the calculator.

What is worse is that we have no way of choosing how fast we would like frames rendered. The faster the nodes applied to the animation the more expensive it will be. I ran the same 11 frames multiple times, sometimes it took aroun 13 minutes to finish at a cost 300% of the estimate. Other times it finished in around 6 minutes, at a cost of 400% of the estimate.

I do believe these are worst case for orc as they were fairly fast rendering frames (~1 min on a single 1080ti) For high resolution stills, I believe the estimates should be much closer to reality. This is not much of a consolation since many people look at render farms primarily for handling animation frames.

Hopefully this can be improved in the future. For the time being, buyer beware.
Renderwiz
Licensed Customer
Licensed Customer
 
Posts: 8
Joined: Mon Aug 07, 2017 10:34 am

Re: Estimate off by 3x (Solved) and lack of support

Postby fuzzybro » Tue Oct 17, 2017 10:43 pm

fuzzybro Tue Oct 17, 2017 10:43 pm
Hi Chris,

I see that the issue was resolved via our helpdesk email ([email protected] and [email protected]).
Please let us know if there is anything else we could do.

Best,

Vladimir
ORC Team
User avatar
fuzzybro
OctaneRender Team
OctaneRender Team
 
Posts: 562
Joined: Mon Jun 03, 2013 7:10 am
Location: Los Angeles, California

Re: Estimate off by 3x (Solved) and lack of support

Postby Renderwiz » Wed Oct 18, 2017 2:12 pm

Renderwiz Wed Oct 18, 2017 2:12 pm
Vladimir,

Thanks for the reply.

The issue is ongoing.... as far as I know the calculator still gives incorrect estimates.

Your comment is also misleading about the type of support I received.... I figured out myself what was causing my estimates to be wrong (spent around $25 in credits doing so). Support did not respond in a timely manner (3-4 days is too long to wait for replies) and then also ignored details I provided in my emails.

As to what else you can do.... I would like you to either 1) provide us free credits for running estimates before each job 2) fix the calculator 3) make it clear on your calculator which types of jobs will cause estimates to be wrong.

I would also like to be refunded for the money I spent helping you debug the system, and maybe throw in a few extra credits for my trouble :-)

But mainly I would like for support and communication to be improved. It is the only thing keeping me from being all in for octane/Otoy. This is meant as constructive criticism, and I truly hope it will improve.

Regards,

Chris
Renderwiz
Licensed Customer
Licensed Customer
 
Posts: 8
Joined: Mon Aug 07, 2017 10:34 am

Return to OctaneRender Cloud


Who is online

Users browsing this forum: No registered users and 1 guest

Sun Dec 17, 2017 7:35 pm [ UTC ]