Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Unity Benchmark v0.1

Discussion in 'General Discussion' started by Arowx, Sep 20, 2012.

  1. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    I don't think there was any change in the app, it's very simple.

    A later release of version of 4.3 did improve the performance. However it never quite caught up with 4.2 for simple cube drawing.

    There was quite a lot of changes in the graphics area of Unity, Culling, 2D ect. http://unity3d.com/unity/whats-new/unity-4.3

    It could just be a case of some streamlining for 2D or Improved culling, has an impact on this simple benchmark which throws lots of cubes around, but improves the performance within 2D and 3D games.

    What do you get in 4.5?
     
  2. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    I get 8625 in the webplayer with PS, Unity 4.6.9, C4D, Skype, 5 webpages open, FilterForge, Unitron, iPhoto, iTunes and Premier open. this is a 2013 basic iMac 22" with 4GB of RAM and nothing special..out-of-the-box cheapest Mac I could get with a compatible display when the former one died when I smacked a fly on the screen.
     
  3. Bradamante

    Bradamante

    Joined:
    Sep 27, 2012
    Posts:
    300
    I have no idea if I am using this correctly or not. I did "All" and then let it run for a while. When it did it's thing for a while and I saw no more progress I went back to the menu ...
     

    Attached Files:

  4. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Your score is in the top left: 47,341 cube marks.
     
  5. Kyohiro

    Kyohiro

    Joined:
    May 22, 2014
    Posts:
    1
    Using the version on the website I get 78108 cubes.
    Got the Nvidia Geforce TITAN
    Intel Core i7 3.60 GHz
    16GB Ram
    Windows 8 64-bit
     
    Last edited: Jun 11, 2014
  6. NomadKing

    NomadKing

    Joined:
    Feb 11, 2010
    Posts:
    1,461
    With the web version I get:

    2D Mark: 36440
    3D No Colliders: 45647
    3D Colliders: 10703
    Cube Mark: 92790
     
  7. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    I need to implement a global high score system to verify the hardware and cube marks some people are getting.
     
  8. inafield

    inafield

    Joined:
    Feb 26, 2013
    Posts:
    281
    This is weird. Chrome 35.0.1916.144m is blocking the file cubemark45.zip file as malicious. Scanned with AVG though as safe.
     
  9. inafield

    inafield

    Joined:
    Feb 26, 2013
    Posts:
    281
    Web Player:
    2D: 26945
    3D NoC: 36280
    3D w/Coll: 9710
    CubeMark: 72935

    Windowed Application:
    2D: 34079
    3D NoC: 45193
    3D w/Coll: 10780
    CubeMark: 90052

    GeForce 660 GTX 2GB
    i7-4770S
    16GB RAM

    Lightroom, Chrome, OneNote, dual screens (1 pano, 1 portrait), Steam
     
  10. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    No problems here anyone else getting this problem?
     
  11. NomadKing

    NomadKing

    Joined:
    Feb 11, 2010
    Posts:
    1,461
    I got the same thing, then my Trend Micro did the same too, but I didn't think anything of it. Certainly not the first time an anti-virus has thrown a false positive at a custom executable.
     
  12. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  13. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Last edited: Jul 13, 2014
  14. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Quick update the 4.5.2 webplayer version was crashing this was due to the addition of Oculus Rift features and the required dll. As you know webplayer does not support external dlls (at least I do not believe it does).

    Anyway the updated version should work fine now.
     
  15. inafield

    inafield

    Joined:
    Feb 26, 2013
    Posts:
    281
    i7-3630QM, GeForce 670MX

    Unity 4.5.0 / 4.5.2 :: percentage of 4.5.0, larger is better

    CubeMark:
    • Web: 63588 / 75840 :: 119%
    • Exe 720p windowed: 81877 / 80780 :: 99%
    • Exe 1080p f/s: 63650 / 63809 :: 100%

    2D Mark:
    • Web: 23673 / 21713 :: 91%
    • Exe 720p windowed: 31915 / 31462 :: 98%
    • Exe 1080p full screen: 63650 / 63809 :: 100%
    3D No Colliders:
    • Web: 30948 / 39392 :: 127%
    • Exe 720p windowed: 39989 / 39226 :: 98%
    • Exe 1080p f/s: 25882 / 25807 :: 100%
    3D Colliders:
    • Web: 8967 / 9275 :: 103%
    • Exe 720p windowed: 9973 / 10092 :: 101%
    • Exe 1080p f/s: 9972 / 10125 :: 101%
     
  16. pete1061

    pete1061

    Joined:
    Oct 14, 2013
    Posts:
    67
    Spent all day working on this:

    I have:
    i5 3570 @ 3.4Ghz
    16G ram
    Nvidia 660 GTX w/2G

    I ran the test for 6 rounds, at 10 different settings.

    1280x720 exe windowed (fastest,good & fantastic)
    1920x1080 exe fullscreen (fastest,good & fantastic)
    1920x1200 exe fullscreen (fastest,good & fantastic)
    web player

    here's some graphs of the results:

    exe



    web



    and here's the averages:

    1280x720 exe windowed (fastest/good/fantastic):
    2D:.............. 32601 / 32456 / 35205
    3D no coll:...... 46317 / 45013 / 41669
    3D w/coll:....... 11033 / 11054 / 11046
    Cube mark:....... 89951 / 88860 / 87919

    1920x1080 fullscreen (fastest/good/fantastic):
    2D:.............. 31748 / 33961 / 35494
    3D no coll:...... 45975 / 45084 / 41751
    3D w/coll:....... 11043 / 11086 / 11078
    Cube mark:....... 88815 / 90127 / 88323

    1920x1200 fullscreen (fastest/good/fantastic):
    2D:.............. 34720 / 34432 / 35088
    3D no coll:...... 46773 / 45695 / 42440
    3D w/coll:....... 11074 / 11093 / 11081
    Cube mark:....... 92567 / 91220 / 88609

    Web Player
    2D:.............. 30942
    3D no coll:...... 41994
    3D w/coll:....... 10115
    Cube mark:....... 83050

    It looks to me like I get best performance at my monitors native resolution (1920x1200)

    It also looks to me like I have waaaay too much free time on my hands ;P
     
  17. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,350
    @Arowx you could add some option at the end to post results on your server? (as in highscores)
     
  18. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  19. pete1061

    pete1061

    Joined:
    Oct 14, 2013
    Posts:
    67
    I noticed variation in results from one run to the next, so I decided to do a more substantial number of runs and calculate an average. One thing led to another, and my OCD side took over. I'm a bit of a data & statistics junkie. I should try to find a job testing software because I have a habit of testing things six ways to sunday.
     
    inafield likes this.
  20. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    So an ideal version of Cube Mark would run multiple tests, at various resolutions and provide nice graphs and a scoreboard?
     
  21. inafield

    inafield

    Joined:
    Feb 26, 2013
    Posts:
    281
    Yes, I was thinking the same thing. I was thinking of also writing my own scene as a tech demo and to learn Unity better, things like particles and different lighting and shadows.
     
  22. spraycanmansam

    spraycanmansam

    Joined:
    Nov 22, 2012
    Posts:
    254
    Web Player
    Mac OSX 10.8.5, 8GB, GTX680M 2GB

     
  23. pete1061

    pete1061

    Joined:
    Oct 14, 2013
    Posts:
    67
    There is always room for new features to add to any software.
    I'm not foaming at the mouth for any of those features, but they certainly would be welcome.

    The amount of variation from one test to the next is a curiosity, I'd look into why that happens.
    Maybe for starters, you could just have it run multiple times at whatever display mode it is currently set at and average the results. It would make the testing process take a while, but it would be a workaround for the variation. You could just add an extra menu option to run "x" number of times and average, won't even need fancy graphs & like.
     
    inafield likes this.
  24. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    OK Noticed a DX11 performance improvement in the 4.5.2_p2 patch of Unity.

    So had to give it a whirl and my previous cube mark @ 2560x1600 on Fantastic Full Screen

    Previous CubeMark 53,441

    New CubeMark 67,656

    Wow that's an improvement of 26.6%, well done Unity!

    Cube Mark PC DX11 - Unity 4.5.2.p2
     
    inafield likes this.
  25. Dustin-Horne

    Dustin-Horne

    Joined:
    Apr 4, 2013
    Posts:
    4,568
    Eh, my laptop didn't fair too badly. Running the latest 4.5.2.p2 DX11 build, 1920 x 1080, Full Screen, Fantastic and Cube Mark 70654.

    cubemark.png
     
  26. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,350
    with "Cube Mark PC DX11 - Unity 4.5.2.p2":

    2D mark: 26778
    3D No coll: 34706
    3D coll: 9833
    Cubemark: 71317

    pc: (win7, fullscreen fantastic, 1680x....))
    AMD FX-8350, AM3+, 4,0GHz, 8-core
    Asus AMD Radeon R9 290, 4GB GDDR5
     
  27. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  28. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Last edited: Sep 16, 2014
  29. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194


    OK some people seem to like data and graphs so I thought I would see how the performance of Unity has changed over time. Cube Marks are in thousands truncated to one decimal place and are derived from my PC at 2560 x 1600 @ 30fps with up to date drivers and AMD GPU/CPU.

    It should be noted that though there was a definite drop in performance from 4.2 to 4.3 there is also a slight difference in the benchmark between versions (you can see the difference in quad test).

    But please test on your PC/Browser and I recommend running the benchmark multiple times to omit blips and obtain an average.

    PS this data is based on a single run of each version.
     
  30. Bradamante

    Bradamante

    Joined:
    Sep 27, 2012
    Posts:
    300

    Attached Files:

  31. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    OK I've been playing around with the current build of Cube Mark and it now has a Run All x 10 and x 100 options.

    Cube Mark 4.5.4 with Batched Runs

    So got 30.4k @ 60fps and 62.8k @ 30fps on 10 x runs.

    Thinking of adding a batched quality mode, so you can set it away to do 10 runs at the default Unity quality settings, what do you think?
     
  32. inafield

    inafield

    Joined:
    Feb 26, 2013
    Posts:
    281
    Run All x 10 is probably a good idea. Not sure what your architecture is like, but with new 5x features coming out, you might want to make it "plugginable" to add new and/or different tests. Number of lights on static mesh, number of lights on dynamic mesh, number of lights on mechanim, GI, fog, bloom, pbr shader.
     
  33. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194


    OK it's starting to come together and provides a selectable batch run with combination of fps targets and quality settings (see above screenshot). Also couldn't resist playing with the main menu screen.

    The plan is to upload the web version onto itchi.io as free and the PC downloadable version for a nominal charge. If I get enough sales I might then be able to afford 5.x and get more motivated to add more tests/settings and maybe an online database of scores.

    B4N Back soon.
     
    inafield likes this.
  34. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    OK new webplayer version is up on itch.io http://arowx.itch.io/unity-cube-mark

    Update and you can download the PC version for the extortionate minimum price of £1 (or donate more if you must) on the above link.

    Thank You.
     
    Last edited: Sep 19, 2014
  35. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Oops re-worked with bug fix, same links as above and first post.
     
  36. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    inafield likes this.
  37. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  38. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  39. NTDC-DEV

    NTDC-DEV

    Joined:
    Jul 22, 2010
    Posts:
    593
    That's actually a pretty massive drop... 10% is not negligible.
     
  40. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Update with 4.6 build 52,127 @ 30fps and 41,672 @ 60 fps - Same test as above.
     
  41. lmbarns

    lmbarns

    Joined:
    Jul 14, 2011
    Posts:
    1,628
    Test Unity 5 vs 4.x yet?
     
    Arowx likes this.
  42. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    No still haven't made enough money with 4.x to buy 5! Will give it a run once it's open beta or released as Free.

    Although Unity have done a nice little benchmark of their own on their blog but it's mostly WebGL focused. I think the biggest gain will be they new multi-threading job system with additional improvements once they have all builds going to native compiled code.

    Also there are some concerns regarding game engine rendering pipeline performance on the next generation of screen resolutions 4k+. Especially on mobile platforms and the new VR headsets.
     
  43. lmbarns

    lmbarns

    Joined:
    Jul 14, 2011
    Posts:
    1,628
    I was recently lucky enough to go to FB offices and test drive the internal VR prototype and talk to them for a while and they were very anti unity because of performance. They're using UE4 for their demos and acted surprised somebody would use anything else. They were also saying the mobile VR isn't going to have performance problems but I don't see how. They were very adamant about if you can't maintain 75fps, don't develop for their product, citing 1 bad experience is a lost customer.
     
  44. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    They do have a point on performance, you really do need high and consistent fps to maintain the impression of immersion.

    I should add a flexible fps target for Cubemark, although they do say for true immersive VR the ideal is 120fps!

    I previously had CubeMark working with the Rift DK1, and people actually tried it with the HMD on. But you just end up feeling like you're in a cosmic cube washing machine. Barf bags required.
     
  45. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    That's odd as Playful are making Lucky's Tale and that's using Unity and they have been working closely with Oculus.

     
  46. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    OK Updated http://arowx.itch.io/unity-cube-mark to Unity 4.6.

    Note the UI in the benchmark is still NGUI based, and bitmapped text blurs at low quality settings.

    Plan to rework the UI.
     
  47. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Quick Test with Unity 4.6.1

    Running 3x Fantastic @ 2560 x 1600 full screen on PC.

    4.5 / 4.6 / 4.6.1
    30 fps > 51,716 / 48,853 / 48,254
    60 fps > 42,230 / 39,983 / 39,337

    These results vary from previous as I have upgraded my graphics driver since the last runs, also this is a very simple benchmark so your own game/apps performance gains/loses will vary.

    Just ensure you write an automated benchmark test that throws everything your game uses around in Unity, turned up to 11. They you can optimise your game and test how Unity's latest version impacts performance.
     
  48. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    I'm thinking of doing a kickstarter to bring CubeMark to Unity 5 would anyone be interested?
     
  49. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Quick Test with Unity 4.6.2

    Running 3x Fantastic @ 2560 x 1600 full screen on my PC.

    4.5 / 4.6 / 4.6.1 / 4.6.2
    30 fps > 51,716 / 48,853 / 48,254 / 48,681
    60 fps > 42,230 / 39,983 / 39,337 / 39,490

    Run (Webplayer Version) or Download and run from http://arowx.itch.io/unity-cube-mark
     
  50. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Very Quick test with Unity 4.6.3 (it's late or very early now).

    30 fps > 49,835
    60 fps > 40,723

    So it looks a bit faster, will run older benchmarks and double check tomorrow, as well as upload new benchmark as standalone and web player.
     
    NTDC-DEV likes this.