Page 1 of 1

csv error

Posted: Thu Sep 19, 2013 12:53 pm
by v-cube
Hello,

during my latest test I tried to attache a csv file to a block and came across this error :
screen11.jpg
since the csv was quite massive I wonder if there is some kind of limit on how big these kind of files can get...

best regards

Andy

Re: csv error

Posted: Thu Sep 19, 2013 10:51 pm
by face_off
Was the CSV file definitely comma separated? How many transforms as it? I've tested with 50,000 at some point and it was OK.

Would it be possible to send me the csv file pls?

Paul

Re: csv error

Posted: Fri Sep 20, 2013 10:20 am
by v-cube
face_off wrote:Was the CSV file definitely comma separated? How many transforms as it? I've tested with 50,000 at some point and it was OK.

Would it be possible to send me the csv file pls?

Paul
Hi Paul, well it was a quite massive one... I think something like 700 000 done with phantom scatter as part of my ongoing grass experiments ;-)

I will sent you an dropbox link by pm

Andy

Re: csv error

Posted: Fri Sep 20, 2013 11:43 am
by face_off
700,000 transforms! There are 2 issues 1) there will be a finite number of transforms you can load into RAM (I don't know what that number is, but I've only previously been able to load 50,000, so 700,000 is way out there), and 2) you need to multiply the number of transforms with the number of polygons in the OBJ you are scattering to get a total polygon count that Octane has to render. If you have 10,000 polygons in your OBJ, that's 70,000,000,000 polygons, and you probably need to expect some problems with that.

Paul

Re: csv error

Posted: Fri Sep 20, 2013 11:48 am
by v-cube
yes I know it was ... ambitious :oops:

but it was "only" a grasleave object with approximately 30 polygons or so... concerning ram usage i did not have the impression that it was critical but i will do another test run here...

thanks for the info so far

Andy


face_off wrote:700,000 transforms! There are 2 issues 1) there will be a finite number of transforms you can load into RAM (I don't know what that number is, but I've only previously been able to load 50,000, so 700,000 is way out there), and 2) you need to multiply the number of transforms with the number of polygons in the OBJ you are scattering to get a total polygon count that Octane has to render. If you have 10,000 polygons in your OBJ, that's 70,000,000,000 polygons, and you probably need to expect some problems with that.

Paul

Re: csv error

Posted: Fri Sep 20, 2013 12:11 pm
by face_off
The issue is that the plugin has an internal limit set for transforms of 10,000, and the exception was being raised when you hit this limit. I have fixed the exception, and increased the limit to 1 million transforms. I was able to get your 700,000 transforms onto a 70,000 polygon object, and the viewport opened!

The above change will go through with the material integration release. If you need a big grass transform prior to then, I suggest 700 blades of grass transformed 10,000 times.

Hope that helps.

Paul

Re: csv error

Posted: Fri Sep 20, 2013 12:43 pm
by v-cube
face_off wrote:The issue is that the plugin has an internal limit set for transforms of 10,000, and the exception was being raised when you hit this limit. I have fixed the exception, and increased the limit to 1 million transforms. I was able to get your 700,000 transforms onto a 70,000 polygon object, and the viewport opened!

The above change will go through with the material integration release. If you need a big grass transform prior to then, I suggest 700 blades of grass transformed 10,000 times.

Hope that helps.

Paul
Thanks Paul, I have the feeling I should buy you a beer sometime ;-)

700 000 X 70 000 = 49 000 Million Polygons ??? unbelievable ! Screenshot!

Re: csv error

Posted: Wed Sep 25, 2013 7:39 am
by newske
When this change goes through... it's going to be a fun time testing how many blades of grass we can do with RGScatter...

Setting up now to do this kind of thing. Did a test just now - took about 1 minute to scatter 500,000 individual blades. Can't render it yet, though, given the current limit.

The .csv files are 64.4MB for that many xforms, though. Not sure how that gets interpreted in vram.

Newske.