Unity is factor 10 slower than normal C# - Why?

Hello!

The longer version: https://enzisoft.wordpress.com/2016/03/11/factorio-in-unityc-part2/
The shorter one:
I made 2 test applications. A 1 million for loop that adds 2 floats to a basic vector class.

The results are that the same code runs at a factor 10 slower than as a normal C# console application.

Here you can find the TestProjects:
Unity: http://www.enzenebner.com/vectorAdd/vectorAdd_Unity.rar
C#: http://www.enzenebner.com/vectorAdd/vectorAdd_CSharp.rar

I really want to know what's going on. I thought the measuring was wrong at first, but then I let the program run for 10 seconds and simulate how much 1 million loops it could make. Again, factor 10 popped up.

 

Hm. I can confirm your code with following approximate numbers per 1 million loop (I'm not about to wait 10 seconds, so I removed that part and just passed loop 10 times):
Debug Unity 140-146 ms
Release Unity 63-65 ms
Debug C# 17-21 ms
Release C# 5.6 - 8.8 ms

BUT:
Possible cause for that: GC ran 10 times in Unity (Note that I called that loop 10 times) while it did 0 times in C# console for me. I suspect, C# optimized new Vector2(...) while Unity didn't.
Now for checking suspicion I've changed your Vector2 from class to struct:
Release Unity 19.5-19.6 ms
Release C# 3.8-4.1 ms
Still strange, but at least not such big numbers and I've managed to decrease difference to only 5 times.

^Just another proof to fight allocations...

 

I guess Unity does not just sit there running your loops. There are other sub-systems to manage: graphics, sound, ui, ... even if it is an empty scene, these systems are still active. Whereas the console application does not do that much. Though, I'm just speculating.

 

Unity uses its own C# compiler. I am curious, what the numbers would be, if the Unitiy project would be compiled with the compiler from Visual Studio.

 

It's a valid point if the engine is multi threaded. The test should be run in awake I guess. In any case, I don't want to download a zip to find out.

 

Just checked in Awake, it didn't change anything more than by 0.1ms from what I've posted above. Threading that same code didn't either - still same time(+- errors of stopwatch). I think C# compiler actually optimizes better, but I'm too lazy to dig into bytecode.
Same code with UnityEngine.Vector2 did same loop in 31 ms in Unity release. Comparing that against C# bad code shown above (who the hell allocates 1 million Vector2 as class and compares performance in Debug mode?) Unity actually does better (yeah, I just played statistics trick, I know).

 

Thanks for the feedback guys. I can pinpoint the problems now much better.

Those are:
- allocations: This can be optimized away with a VectorAdd(float x, float y) - 700 loops in the editor
- editor/release difference: This caught me completely off guard. It's quite dramatic. With the above optimization and a relase build Unity pumps out 3000 1 million loops compared to 3800 1 million loops in C#. Results are much closer now.

The big elephant in the room is still the god awful garbage collector. But we all knew that before, right?

It's just a stress test. Nothing wrong with that. Using that in practice would be idiotic.

 

nvm... being slow

 

I'm not sure what you are comparing here. Again Unity is a complex system with a lot going on, that has nothing to do with a plain simple console application.

It seems you're suggesting there's something wrong or badly implemented on the Unity side?

 

class Vector2 { } instead of struct Vector2 { } this is what is wrong.

Also, there are many ways to add two vectors together. You chose the slowest one.

Code (CSharp):

1.             Vector2 a = new Vector2(0, 0);
2.  
3.             for (int i = 0; i < 1000000; i++)
4.             {
5.                 a.Add(new Vector2(.1f, .2f)); // 1598 1 million loops
6.             }

Code (CSharp):

1.             Vector2 a = new Vector2(0, 0);
2.             Vector2 b = new Vector2(.1f, .2f);
3.  
4.             for (int i = 0; i < 1000000; i++)
5.             {
6.                 a.Add(b); // 2385 1 million loops
7.             }

Code (CSharp):

1.             Vector2 a = new Vector2(0, 0);
2.             Vector2 b = new Vector2(.1f, .2f);
3.  
4.             for (int i = 0; i < 1000000; i++)
5.             {
6.                 a.x += b.x; // 3985 1 million loops
7.                 a.y += b.y; //
8.             }

Code (CSharp):

1.             Mono.Simd.Vector2d a = new Vector2d(0, 0);
2.             Mono.Simd.Vector2d b = new Vector2d(.1f, .2f);
3.  
4.             for (int i = 0; i < 1000000; i++)
5.             {
6.                 a = a + b; // 4424 1 million loops
7.             }

Measured in Unity, x86, release.

PS
The same code compiled for .Net shows ~3990 loops in all the cases except the last one with Mono.Simd.

PPS
.Net example that uses System.Numerics.Vector2 also shows ~4400 loops since it's also uses simd instructions.

 

You keep repeating this, but it's irrelevant. The code being tested runs straight through, and would not be interrupted by anything Unity does on its own. All the extra stuff that Unity does, it doesn't do while your own scripts are running*. Unity does its stuff, then it calls all the relevant code, then it does more of its own stuff. Anything that takes place entirely within a single function of your own code (and doesn't use any yielding) should be mostly unaffected by all the extra stuff Unity does.

In other words, when you have a loop contained entirely within a single function, yes, Unity does just sit there and run your loops.

* The exception would be things that are spun off into a separate thread like audio or pathfinding, but most likely these tasks would be offloaded to a different core if the main thread is heavily using the first core... which is largely the point of having them on their own thread in the first place. Unless this code is being run on a single-core machine, which I really doubt.

Other comments in the thread have hit the nail on the head - this is clearly a symptom of garbage collection and using a class instead of a struct.

 

Sorry. My bad. I (wrongly) assumed the exact same code was used in both tests, and (wrongly) assumed the claim that the code was running 10x slower was justified. Then I went on speculating.

 

You weren't wrong in this one. Code is same. But C# GC is less bloated than Unity GC (at least because Unity GC already contains a lot of objects to check while C# is run in empty environment, not sure about their implementation). And test wasn't testing how fast addition is; it was testing how fast allocation+addition is, and of course, GC triggers weren't accounted for. So your point is valid, but in a bit different context.

 

Thanks a lot for this! I never knew Simd is usable in Unity.

 

You can use a lot of Mono libraries as long as you place them into Assets folder. It's not available out-of-box though (at least not works for me in empty project).

 

P*P*P

Portability, Productivity, Performance;
You can't have all three, pick two.