Speed comparison: 2d Unity vs Monkey

Dear all, being both a Unity3d and a Monkey (language! lol) lover I just made a quick test regarding 2d performance of both and would like to share the numbers:

I've used 2000 sprites which move randomly over the screen. Results are:

Unity 4.5
APK size: 8,404,358 bytes
FPS: ~20

Monkey v79c
APK size: 67,331 bytes
FPS: ~33 (previously I did a wrong fps calculation)

Unity C# code:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4.  
5. public class Sprites : MonoBehaviour
6. {
7.   public GameObject _sprite;
8.  
9.   private ArrayList _spriteList;
10.  
11.   // Use this for initialization
12.   void Start()
13.   {
14.     _spriteList = new ArrayList();
15.     for( int i = 0; i < 2000; i++ )
16.     {
17.       GameObject sprite = (GameObject)Instantiate( _sprite, new Vector3( Random.Range( -6.0f, 6.0f ), Random.Range( -4.0f, 4.0f ) ), Quaternion.identity );
18.       _spriteList.Add( sprite );
19.     }
20.   }
21.  
22.   // Update is called once per frame
23.   void Update()
24.   {
25.     for( int i = 0; i < _spriteList.Count; i++ )
26.     {
27.       GameObject s = (GameObject)_spriteList[i];
28.       Vector3 position = s.transform.position;
29.       float deltaTime = Time.deltaTime;
30.       position += new Vector3( Random.Range( -5.0f, 5.0f ), Random.Range( -5.0f, 5.0f ) ) * deltaTime;
31.       position.x = Mathf.Clamp( position.x, -6.0f, 6.0f );
32.       position.y = Mathf.Clamp( position.y, -4.0f, 4.0f );
33.       s.transform.position = position;
34.     }
35.   }
36. }
37.  

Monkey code:

Code (csharp):

1.  
2. Strict
3.  
4. Import mojo
5.  
6. Function Main:Int()
7.   New Game()
8.   Return 0
9. End Function
10.  
11. Class Sprite
12.   Field _img:Image
13.   Field _x:Float
14.   Field _y:Float
15.  
16.   Method New( img:Image, x:Float, y:Float )
17.     _img = img
18.     _x = x
19.     _y = y
20.   End Method
21. End Class
22.  
23. Class Game Extends App
24.   Field _sprite:Image
25.   Field _spriteList:List<Sprite>
26.   Field _resX:Float
27.   Field _resY:Float
28.  
29.   'FPS stuff
30.  Field _previousTime:Int
31.  Field _currentTime:Int
32.  Field _passedTime:Int
33.  Field _loopTimeAll:Int
34.  Field _fpsCounter:Int
35.  Field _fpsLastUpdate:Int
36.  Field _fps:Float
37.  
38.  Method OnCreate:Int()
39.    _sprite = LoadImage( "sprite.png" )
40.    _spriteList = New List<Sprite>
41.    _resX = Float( DeviceWidth() )
42.    _resY = Float( DeviceHeight() )
43.    For Local i:Int = 0 Until 2000
44.      Local s:Sprite = New Sprite( _sprite, Rnd( 0, _resX ), Rnd( 0, _resY ) )
45.      _spriteList.AddLast( s )
46.    Next
47.    _currentTime = Millisecs()
48.    SetUpdateRate(60)
49.    Return 0
50.  End Method
51.  
52.  Method OnUpdate:Int()
53.    Return 0
54.  End Method
55.  
56.  Method OnRender:Int()
57.    _previousTime = _currentTime
58.    _currentTime = Millisecs()
59.    _passedTime = _currentTime - _previousTime
60.  
61.    Cls()
62.    For Local s:Sprite = EachIn _spriteList
63.      Local dx:Float = Rnd( -0.1, 0.1 ) * _passedTime
64.      Local dy:Float = Rnd( -0.1, 0.1 ) * _passedTime
65.      s._x += dx
66.      s._y += dy
67.      s._x = Clamp( s._x, 0.0, _resX )
68.      s._y = Clamp( s._y, 0.0, _resY )
69.      DrawImage( s._img, s._x, s._y )
70.    Next
71.  
72.    DrawText( _fps + " FPS", 10, 10 )
73.    updateFPS()
74.    Return 0
75.  End Method
76.  
77.  Method OnSuspend:Int()
78.    Return 0
79.  End Method
80.  
81.  Method OnResume:Int()
82.    Return 0
83.  End Method
84.  
85.  Method updateFPS:Void()
86.    _loopTimeAll += _passedTime
87.    
88.    _fpsCounter += 1
89.    If( ( _currentTime - _fpsLastUpdate ) > 500 )
90.      Local renderTime:Float = Float( _loopTimeAll ) / Float( _fpsCounter )
91.      If( renderTime < 1.0 ) Then renderTime = 1.0
92.      _loopTimeAll = 0
93.      _fpsLastUpdate = _currentTime
94.      _fps = 1000.0 / renderTime
95.      _fpsCounter = 0
96.    End If
97.  End Method
98. End Class
99.  

Sample APKs:
Unity: http://www.leidel.net/dl/monkey/2dtest/unitytest.apk
Monkey: http://www.leidel.net/dl/monkey/2dtest/monkeytest.apk

Project files:
http://www.leidel.net/dl/monkey/2dtest/unityproject.zip
http://www.leidel.net/dl/monkey/2dtest/monkeyproject.zip

Test device was a Sony Xperia S. I'd be interested in more comparison values and stuff how (especially) the Unity code could be done better? Have I done something terribly stupid in my Unity code?

 

Looks interesting, It seems Unity is not that efficient at 2D stuff after all. Although I cannot read your monkey code equivalent, I would say that calling Random.range 4000 times per frame on mobile is why FPS is dropping so far. Maybe you could do a comparison that does not use Random.range as I find always find this to be a Unity choke point in performance.

 

Thanks for that, I'll check it out! Don't get me wrong I don't want to make Unity bad because it's not but maybe I just did something terribly wrong in my code? Something obvious?

 

Nothing that would cause a 10x frame rate difference. Maybe try removing the cast when setting gameobject S.

 

That loop isn't perfect. You'd better create list of transforms and move these directly instead of getting component (transform/gameobject) 2 times inside each loop "tick".

I would suggest something like this:

Code (csharp):

1. public List<Transform> trans;
2.  
3. void Start() {
4.     trans = new List<Transform>();
5.     for( int i = 0; i < 2000; i++ ) {
6.       Transform sprite = (Transform)Instantiate( _sprite, new Vector3( Random.Range( -6.0f, 6.0f ), Random.Range( -4.0f, 4.0f ) ), Quaternion.identity );
7.       trans.Add( sprite );
8.     }
9.   }
10.  
11. void Update() {
12.     for( int i = 0; i < _spriteList.Count; i++ )  {
13.       Transform s = _trans[i];
14.       Vector3 position = s.position;
15.        float deltaTime = Time.deltaTime;
16.       position += new Vector3( Random.Range( -5.0f, 5.0f ), Random.Range( -5.0f, 5.0f ) ) * deltaTime;
17.       position.x = Mathf.Clamp( position.x, -6.0f, 6.0f );
18.       position.y = Mathf.Clamp( position.y, -4.0f, 4.0f );
19.       s.position = position;
20.     }
21.   }
22.  

It's just a draft. It may not work because I didn't tested that.

 

Try moving rigidbody (iskinematic) instead of transform position, I tried it a few years ago, moving rigidbody(iskinematic) is faster than moving transform position. I am not sure if that is still the case tho

 

I think you messed up how you are calculating the fps in Monkey. You seem to be basing it on the time between assigning _loopTimeStart and _loopTimeEnd rather than measuring the frequency of calls to OnRender. I doubt those calls to DrawImage are actually doing much and the real work happens when the OnRender method returns.

I don't know anything about Monkey though so I could be wrong but I see you are calling SetUpdateRate(60). What is the point of that? Surely the FPS would be 60 max? Given the amount of overdraw in your example I wouldn't be surprised to see 20fps on a mobile device. I'd be really surprised to see a mobile device running at 200fps for anything.

Edit: The resolution on a Xperia S is 720x1280 so that is 921,600 pixels. Each frame you are rendering 2000x64x64 or 8,192,000 pixels. So basically over 8 full screens worth of pixels. I find it really hard to believe that Monkey can do that at 200fps.

 

Android phones can't run at 200 FPS, because they are limited to 60 at best, and this can't be changed.

And I am sure there is something wrong with your Monkey project, because there is no mobile device on Earth that can support that much overdraw. You have 41 draw calls, more than 1000 batched dynamically, and repaint 100% of the screen more than 10 times.

 

Thanks for the answers, I'll try these optimizations.

@soldmeadow and mikhail: I know that Android devices can't go beyond 60 fps. SetUpdateRate just says how often the OnUpdate is called per second. I do nothing in the Update loop. I just measure the net render time and compute the fps from that. So actually one OnRender call takes 5ms (which leads to these 200 fps), which doesn't mean that OnRender is more often called than 60 times per second.

I'll rewrite that test to see how many sprites can be displayed per second. That might be more helpful?

 

@Xaron - I don't believe you are measuring the net render time though. As I said before, my guess is that the rendering occurs after OnRender returns. The calls to DrawImage are probably just being used to build up some OpenGL commands that get submitted when OnRender returns, otherwise Monkey can't be using any batching which makes it even less likely it is outperforming Unity. I don't know anything about the architecture of Monkey though. You certainly aren't comparing apples to apples at the moment.

Why don't you try a more realistic test with less sprites or smaller sprites? There is no way you can visually inspect your example so you have to rely entirely on the fps calculation.

 

He is stress testing it, a few sprites makes it difficult to measure difference between them.

 

I said less sprites not a few sprites. 8 screens worth of pixels per frame is totally unrealistic for a mobile device.

 

You are doing it wrong. You must measure FPS, and you are calculating execution time of a function, which actually doesn't render anything.

 

As are some of the more common graphics and CPU benchmarks, its not about being realistic, its about showing the efficiency of each engine.

 

You can't compare engines with the test, because such fillrate is a death to all known mobile GPUs. Instead, a more realistic, but not necessary an easy weight, test should be created and correctly measured.

 

What's causing problems with your frame rate may be Unity 4.5 more than bad code. They're already working on a several fix.

 

Ok I got your points even though they don't convince me yet. For the Unity FPS computation I use the HUDFPS script from here: http://wiki.unity3d.com/index.php?title=FramesPerSecond

Actually it's almost the same stuff I use. frames per second are usually 1/rendertime. I guess you mean that I don't measure the rendertime but just this loop and rendering happens after that, did I get you right?

If so, how about this test: I will render as much as many sprites till I have one Update call (in case of Monkey OnRender) per second. Would that be the same? In that case I would get the numbers of sprite per second.

 

something like:

Code (csharp):

1. previous_time = current_time
2. current_time = Milliseconds()
3. milliseconds_passed = current_time-previous_time
4. if ( milliseconds_passed > 0 ) fps = 1000/milliseconds_passed

 

Thanks for all those answers! I have to admit that I was indeed wrong with my fps counter. Using the code cannon posted, I get 33 fps for Monkey which is more realistic and still 65% faster than Unity. Thanks again and sorry for that! I'm still very satisfied with these results.

Another thing that seems to be not in focus is the APK size. I think it's really something UT should do something about.

 

You can't expect Unity to be much less considering the amount in it, and does anyone care if a game has an extra like 7.5mb to it?

 

Well I do care indeed.

 

Is it really 7.5mb additional? I love Unity but that is a bit high. If a game looks to be worth playing then I'm sure the user will download it, but my overall happiness and feeling is that I like a small download files most especially for those simple but fun and unique type games.

 

Well an empty(!) Unity project is already 8MB, so yes, this is additional.

 

As much I don't like the additional overhead file sizing, i'm not sure if many users will actually notice it too much when they go to download it. As long as its not in the 100s mb, it shouldn't be too much of an issue. But then again, if you have a tic-tac-toe game that is 8mb in size then it maybe in question...

 

Remember, it's at least 8MB. That's not including anything at all that the actually game needs. I've heard that Unity is pretty good at keeping what you throw at it pretty small, but for a small game like Tic Tac Toe, or say... Cube Runner, an extra 8MB is a bit too much. Especially when there are similar games that come in, with everything they need, at 5MB or less. Don't get me wrong, I like Unity, but this is a bit of an issue for me.

Just saw this, for Unity, if you're still using the same code as the first post, couldn't you change

Code (CSharp):

1. for(int i =0; i < _spriteList.Count; i++)
2. //to
3. for(int i =0; i < 2000; i++)

?

Or would that be cheating? And I see you're using a foreach in the monkey code, rather than a for loop. Aren't they a little different?

 

Using the Unity Profiler (in Deep Profile Mode) I managed to take your unity 'benchmark' code from 89.37ms down to 31.21ms.

Note that in deep profile mode there is a big overhead due to the profiler, hence the whopping 89.37ms.

89.37 ms - Original on my PC
75.42ms - Changed from a List of GameObjects to Transforms.
45.37ms - Removed the Vector creation and operations making them operations on the coordinates.
31.21ms - Changed the list to an Array.

Other minor changes shaved off a ms here and there.

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. public class Test : MonoBehaviour
4. {
5.   public GameObject _sprite;
6.   private Transform[] _spriteList; // use transform not gameobject to reduce lookups, moved to basic array to speed up indexing
7.  
8.   void Start()
9.   {
10.     _spriteList = new Transform[2000];
11.  
12.     for( int i = 0; i < 2000; i++ )
13.     {
14.       Transform sprite = ((GameObject)Instantiate( _sprite, new Vector3( Random.Range( -6.0f, 6.0f ), Random.Range( -4.0f, 4.0f ) ), Quaternion.identity )).transform;
15.       _spriteList[i] = sprite;
16.     }
17.  
18.   }
19.   void Update()
20.   {
21.  
22.     float deltaTime = Time.deltaTime; // don't ask for the time in the inner loop
23.     Transform t; // don't initialise variable in the inner loop
24.     Vector3 pos; // ditto
25.    
26.     int c = _spriteList.Length;
27.     int randx = 0;
28.  
29.     for( int i = 0; i < c; i++ )
30.     {
31.       t = _spriteList[i];
32.       pos = t.position;
33.  
34.       // removed vector3 creation and vector3 operations, faster to operate on the coords
35.  
36.       pos.x += (Random.value * 10f - 5f) * deltaTime;  // Random.value faster than range
37.       pos.y += (Random.value * 10f - 5f) * deltaTime;
38.  
39.       if (pos.x < -6f) pos.x = -6f; // faster than calling the clamp method in an inner loop
40.       if (pos.x > 6f) pos.x = 6f;
41.  
42.       if (pos.y < -4f) pos.y = -4f;
43.       if (pos.y > 4f) pos.y = 4f;
44.  
45.       t.position = pos;
46.     }
47.   }
48. }

Tried to build an array of random values to cycle through but that failed as all sprites moved to the outside edge. I suspect that Random, uses the system clock so generating lots of values in the same frame creates a limited pool of numbers with trends.

In the profiler (deep mode) getting and setting the transform on 2000 sprites takes about 12ms a frame, the only way to prevent this would be to try particles or generate our own sprite meshes and move them.

Interested to know how this runs on your device?

@Xaron have you tried my Unity Cube Mark (on Google Play) http://play.google.com/store/apps/details?id=com.arowx.ArowxMark

 

OK Not sure of the impact this will have with Mobile, but on a PC changing the sprite to a quad (Unlit/Transparent shader) took the FPS from from 148 fps Sprites up to 230 fps with Quads.

 

Try this :

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. public class Sprites : MonoBehaviour
4. {
5.     public GameObject _sprite;
6.     private Transform[] _spriteList; // use transform not gameobject to reduce lookups, moved to basic array to speed up indexing
7.  
8.     private Transform t;
9.  
10.     private Vector3 pos;
11.  
12.     void Start()
13.     {
14.         _spriteList = new Transform[2000];
15.      
16.         for( int i = 0; i < 2000; i++ )
17.         {
18.             Transform sprite = ((GameObject)Instantiate( _sprite, new Vector3( Random.Range( -6.0f, 6.0f ), Random.Range( -4.0f, 4.0f ) ), Quaternion.identity )).transform;
19.             _spriteList = sprite;
20.         }
21.  
22.         Application.targetFrameRate = 300;
23.     }
24.  
25.     void Update()
26.     {
27.      
28.         float deltaTime = Time.deltaTime; // don't ask for the time in the inner loop
29.         // don't initialise variable in the inner loop
30.  
31.         for( int i = 0; i < 2000; i++ )
32.         {
33.             t = _spriteList;
34.             pos = t.position;
35.          
36.             // removed vector3 creation and vector3 operations, faster to operate on the coords
37.          
38.             pos.x += (Random.value * 10f - 5f) * deltaTime;  // Random.value faster than range
39.             pos.y += (Random.value * 10f - 5f) * deltaTime;
40.          
41.             if (pos.x < -6f) pos.x = -6f; // faster than calling the clamp method in an inner loop
42.             if (pos.x > 6f) pos.x = 6f;
43.          
44.             if (pos.y < -4f) pos.y = -4f;
45.             if (pos.y > 4f) pos.y = 4f;
46.          
47.             t.position = pos;
48.         }
49.     }
50. }

 

Code (CSharp):

1. Application.targetFrameRate = 300;

LOL tried it and my FPS with quads went up to 270fps.

 

Try the rest of it as well

 

OK It looks like you only moved some variables to private this made very little difference for me in the editor. But I'm not building to platforms the OP was.

 

Actually I moved some work from update, which executes many times per second - it is a good thing to do.
Nice thread btw.

 

I have a Samsung Galaxy S4, and I just tested both sample APK's. My results were:

Unity: 14.19MB Uncompressed @ ~30FPS
Monkey: 188 KB Uncompressed @ Solid 60FPS

2nd Try Unity: 21-40FPS, average ~35FPS

The 2nd time I locked my phone to full power at 1.9GHz, and ran the Unity test with no app but that one open. However, in Unity's defense, it seemed to me like the squares were moving much more, and much faster, than those in monkey. Also, dont know why, but the squares in Unity looked like they were constrained to only some of the screen. None of the squares moved outside of an imaginary rectangle, leaving a thick empty border around the screen.

Arowx, if you posted your code, I'd be willing to change it in the unity project and try it again.

 

Just a shoutout from a fellow Monkey lover
Reading with interest..

 

The size could be solved with build stripping although it won't be as small as the Monkey file. Maybe the fps will rise? I never developed for mobile so I'm not sure build stripping would increase fps.

 

Unity is a 3D engine hacked into a 2D engine, thats why its performance is inferior to Monkey.

 

@Serdnad My code is a few posts back, or you could use @Ippokratis improved version.

Also have you tried replacing the Unity Sprites for Quads, this worked faster, from 140fps to 230fps and @Ippokratis tweaks took it up to 270fps on PC.

@Serdnad the Unity version limits the movement on the X/Y plane to remove the border you can adjust the camera settings.

 

Sorry, just noticed the new posts, didn't get any email. I'll test llpokratis' version when I can, and ill see about the quads and camera. I'll update this post with the results when I can.

Update: After fixing two tiny errors in llpokratis' script, and a miniscule change to the scene, I ran it on my S4 with the same settings as the 2nd Unit test. This time i was getting 39 - 43 FPS, with an average of around 41 or 42 FPS.

 

Firstly, I'd expect monkey to obliterate Unity at 2D performance and file size. Secondly, Monkey is next to useless for any kind of graphically ambitious project. You're not going to be mixing 3D objects of any kind, nor are you going to leverage much other than the basics. So that's complex shaders, better sound, better networking and everything else far beyond Monkey's reach without substantial effort put in.

So if you're making a simpler 2D title, then monkey is by FAR the best tool for the job. And this is about best tool for the job. This kind of comparison is utterly pointless.

Unity can do everything and it comes at a price. Monkey can't, and it does what it does very well.

 

Thanks for the clarification, hippocoder.

 

That is a common misconception people have for some reason. Unity can do ANYTHING and I hate how people believe that the more complex the engine, the more you can do. It may be true in some cases but mostly they are equal.

 

try in android export optimizin textures for powerVR based phones.

 

So... what about hippos words are you disagreeing with?

Anyway, while Unity may indeed be "able to do anything" that's not really an important question. The important questions are how effectively can it be done and how well does the result perform? And while the answers will usually be "very effectively" and "great" if you ask someone who knows what they're doing, there are also some things where it's perhaps not the best tool for the job.

 

the correct answer is: unity is made for 3D games, optimized for it, it have a good 2D, but is relatively new and has to be optimized.

 

@Serdnad

Monkey: 188 KB Uncompressed @ Solid 60FPS

Unity: 14.19MB Uncompressed

1st Try @ ~30FPS
2nd Try Unity: 21 - 40FPS, average ~35FPS
3rd Try 'Optimised' Unity @ ~39 - 43 FPS.

In the Unity profiler on PC the 2000 get and set transform.position calls were using up about 12ms a frame, in Deep mode the profiler 'slows' down the app or game but still all of these calls have an impact.

Which got me thinking that what if Unity had a batch based mechanism for Vector / Quaternion operations, even on larger projects you will be updating the positions, speeds, rotations of lots of objects.

Had an idea

 

Code (CSharp):

1. using UnityEngine;
2. public class Test2 : MonoBehaviour
3. {
4.   void Start()
5.   {  
6.     QualitySettings.vSyncCount = 0;
7.     Application.targetFrameRate = -1;
8.   }
9. }
10.  

Getting about 700 fps on PC.

OK I've set up...

• a Particle system using the sprite as the material
• a box emitter with random direction
• 6 x 3d box colliders around the emitter to limit them to the screen
• turned on Collision with World.

Just make sure you set it to pre-warm, limit to 2000 particles and emit 2000 particles and TA DA!

Unfortunately we don't have a batch update this side of the Unity engine, but I think the particle system will be taking advantage of some optimisations.

 

Hopefully this means my pong clone will be less than 33megs when it's built entirely out of untextured cubes.

 

So...

Have I missed a "not" in here or something?