OctaneBench

Forums: OctaneBench
Read about News and Updates regarding Octane Render in this forum.
Forum rules
For new users: this forum is moderated. Your first post will appear only after it has been reviewed by a moderator, so it will not show up immediately.
This is necessary to avoid this forum being flooded by spam.

OctaneBench

Postby abstrax » Wed Mar 11, 2015 3:01 am

abstrax Wed Mar 11, 2015 3:01 am
Hi all,

Remember the OctaneBench competition from 2013? Last year we got around to actually implement it, but then a lot of things changed in the render engine and we decided to wait until things have settled a bit, to make sure that the benchmark results are still relevant for later version 2 releases.

After some more polishing and tweaking we can finally release the OctaneBench based on version 2.17. It allows you to run a standardized test on one or more GPUs and store and upload the results to the OctaneBench result data base. OctaneBench can also be called via command line which allows you to automate testing. And you can take the OctaneBench resources and run OctaneBench on any OctaneRender Standalone version 2.21 or higher.

NOTE: When you run OctaneBench and upload the results to the database, it's important to not only benchmark the full system, but also sub-configurations and single GPUs. This way we get better statistics of the performance of each GPU model, which is relevant for most people who use the OctaneBench results to make a decision about which GPU to purchase. Thanks a lot in advance for participating and helping to create a representative overview of GPU hardware and its performance.

You can find the archive, more details and a link to the results page on the OctaneBench page. Here are some screenshots:

bench1.png

bench2.png

bench3.png


Happy benchmarking!
Your OTOY NZ Team
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
User avatar
abstrax
OctaneRender Team
OctaneRender Team
 
Posts: 5483
Joined: Tue May 18, 2010 11:01 am
Location: Auckland, New Zealand

Re: OctaneBench

Postby glimpse » Wed Mar 11, 2015 5:39 am

glimpse Wed Mar 11, 2015 5:39 am
Pressing on this thread I though to myself..
"not again, another benchmark thread.."
- haven't noticed a fact that this was written by devs =)
but my face brighten up instantly seeing Julio Cayetano scene!!! =)

Thanks Guys!!! This is very nice addition to Octane tool-set! =)

P.S. great to see average & maximum scores! =)
User avatar
glimpse
Licensed Customer
Licensed Customer
 
Posts: 3715
Joined: Wed Jan 26, 2011 2:17 pm

Re: OctaneBench

Postby smicha » Wed Mar 11, 2015 7:48 am

smicha Wed Mar 11, 2015 7:48 am
I am downloading it now and I will send some feedback soon. Thank you so much for this.
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Licensed Customer
 
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Re: OctaneBench

Postby glimpse » Wed Mar 11, 2015 8:05 am

glimpse Wed Mar 11, 2015 8:05 am
smicha wrote:I am downloading it now and I will send some feedback soon. Thank you so much for this.


I guess Your result will be "the max" on 780s =DDD..qurious 'bout it man!
User avatar
glimpse
Licensed Customer
Licensed Customer
 
Posts: 3715
Joined: Wed Jan 26, 2011 2:17 pm

Re: OctaneBench

Postby smicha » Wed Mar 11, 2015 8:44 am

smicha Wed Mar 11, 2015 8:44 am
First tests done :)

Marcus, if this is possible please add some more statistics to OctaneBench:

1. Core and memory clocks for each of gpus - these parameters affect strongly final scores.
2. GPU Voltage (under rendering only) - its level gives us a general look of how hard the gpu is used.
3. Cuda cores for each of gpu + total cuda cores. A 'final score per cuda core' would be really nice. If someone has 3k vs 20k it would be great to look at relative scores.
4. GPU temperatures recorded right at the end of tests - average or starting temps are irrelevant.
5. Option to select by a user if the system is water or air cooled.
6. For those who have power draw meters an option to enter real power consumption of entire system would be nice.

So the final score would look like this:


Octane version:
2.17
Operating system:
Windows 7 64-bit
Devices:
GeForce GTX TITAN @ 2688 cuda cores, 1293mhz core, 7000mhz memory, 1.2V, 37C, water cooled
GeForce GTX 780 @ 2304 cuda cores, 1293mhz core, 7000mhz memory, 1.213V, 39C, water cooled
GeForce GTX 780 @ 2304 cuda cores, 1293mhz core, 7000mhz memory, 1.213V, 40C, water cooled
GeForce GTX 780 @ 2304 cuda cores, 1293mhz core, 7000mhz memory, 1.213V, 41C, water cooled

____________________Scene,__________Kernel,_________Ms/s,__GTX980_Ms/s,____Ratio,___Weight,________Score
_Idea_(by_Julio_Cayetaño),___info_channels,___383.246626,________81.25,____4.717,_______10,____15.722939
_Idea_(by_Julio_Cayetaño),_direct_lighting,____80.690374,________18.64,____4.329,_______35,____50.503632
_Idea_(by_Julio_Cayetaño),____path_tracing,____70.125203,________15.99,____4.386,_______35,____51.164938
_Idea_(by_Julio_Cayetaño),_____________PMC,____10.081340,_________2.88,____3.500,_______20,____23.336435
ATV_(by_Jürgen_Aleksejev),___info_channels,___147.458088,________32.48,____4.540,_______10,____15.133219
ATV_(by_Jürgen_Aleksejev),_direct_lighting,____51.359056,________12.40,____4.142,_______35,____48.321692
ATV_(by_Jürgen_Aleksejev),____path_tracing,____41.799553,________10.02,____4.172,_______35,____48.668808
ATV_(by_Jürgen_Aleksejev),_____________PMC,____14.865587,_________3.77,____3.943,_______20,____26.287510
___Box_(by_Enrico_Cerica),___info_channels,___264.969876,________59.18,____4.477,_______10,____14.924517
___Box_(by_Enrico_Cerica),_direct_lighting,____51.228626,________11.76,____4.356,_______35,____50.822049
___Box_(by_Enrico_Cerica),____path_tracing,____51.779837,________11.74,____4.411,_______35,____51.456397
___Box_(by_Enrico_Cerica),_____________PMC,____14.306461,_________3.83,____3.735,_______20,____24.902455


Total score
421.244591

Total score per cuda core
421.244591/9600 = 0.4387
# of cuda cores 9600

Total power draw
955W

PS. From the test it seems that 980 is really great for PMC kernel - much better than 780 or titan. I'll run separate tests soon
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Licensed Customer
 
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Re: OctaneBench

Postby valters » Wed Mar 11, 2015 9:28 am

valters Wed Mar 11, 2015 9:28 am
I would like also to change GPU for comparison..not to see only 980 ;)
valters
Licensed Customer
Licensed Customer
 
Posts: 79
Joined: Sun Sep 07, 2014 3:18 pm

Re: OctaneBench

Postby smicha » Wed Mar 11, 2015 9:36 am

smicha Wed Mar 11, 2015 9:36 am
valters wrote:I would like also to change GPU for comparison..not to see only 980 ;)


+1
3090, Titan, Quadro, Xeon Scalable Supermicro, 768GB RAM; Sketchup Pro, Classical Architecture.
Custom alloy powder coated laser cut cases, Autodesk metal-sheet 3D modelling.
build-log http://render.otoy.com/forum/viewtopic.php?f=9&t=42540
User avatar
smicha
Licensed Customer
Licensed Customer
 
Posts: 3151
Joined: Wed Sep 21, 2011 4:13 pm
Location: Warsaw, Poland

Re: OctaneBench

Postby abstrax » Wed Mar 11, 2015 9:43 am

abstrax Wed Mar 11, 2015 9:43 am
smicha wrote:First tests done :)

Marcus, if this is possible please add some more statistics to OctaneBench:

1. Core and memory clocks for each of gpus - these parameters affect strongly final scores.
2. GPU Voltage (under rendering only) - its level gives us a general look of how hard the gpu is used.
3. Cuda cores for each of gpu + total cuda cores. A 'final score per cuda core' would be really nice. If someone has 3k vs 20k it would be great to look at relative scores.
4. GPU temperatures recorded right at the end of tests - average or starting temps are irrelevant.
5. Option to select by a user if the system is water or air cooled.
6. For those who have power draw meters an option to enter real power consumption of entire system would be nice.

Thanks for your feedback. These are all valid points and we try to attack them in some future version. Unfortunately, there is currently no way to map a CUDA device to a device in the NVAPI.

OctaneBench is mainly meant to 1) allow people testing the render performance of their system and compare it to results in the database and 2) to give others at least some indication what to expect from a certain GPU. Our hope is that the current version does that good enough. It's certainly not perfect and with version 3 of Octane we will likely update OctaneBench again and then try to implement your and other suggestions.
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
User avatar
abstrax
OctaneRender Team
OctaneRender Team
 
Posts: 5483
Joined: Tue May 18, 2010 11:01 am
Location: Auckland, New Zealand

Re: OctaneBench

Postby Seekerfinder » Wed Mar 11, 2015 9:44 am

Seekerfinder Wed Mar 11, 2015 9:44 am
smicha wrote:Total score per cuda core

I think this could be very helpful.

Seeker
Win 8(64) | P9X79-E WS | i7-3930K | 32GB | GTX Titan & GTX 780Ti | SketchUP | Revit | Beta tester for Revit & Sketchup plugins for Octane
User avatar
Seekerfinder
Licensed Customer
Licensed Customer
 
Posts: 1600
Joined: Tue Jan 04, 2011 11:34 am

Re: OctaneBench

Postby abstrax » Wed Mar 11, 2015 9:50 am

abstrax Wed Mar 11, 2015 9:50 am
valters wrote:I would like also to change GPU for comparison..not to see only 980 ;)

Using a GPU as reference was just a way to give the scores some meaning. Otherwise the numbers would have been very abstract.

In theory, we could recalibrate the score for some other device, but that would need to be done on the results page and I'm not sure how much work that is and how much stress this would put on the web server. I will ask Tim and Chris, what they think.
In theory there is no difference between theory and practice. In practice there is. - Yogi Berra
User avatar
abstrax
OctaneRender Team
OctaneRender Team
 
Posts: 5483
Joined: Tue May 18, 2010 11:01 am
Location: Auckland, New Zealand
Next

Return to News & Announcements


Who is online

Users browsing this forum: No registered users and 11 guests

Thu Mar 28, 2024 9:25 pm [ UTC ]