Language speed comparisons in Unity

I'm starting this thread to investigate speed differences among the 3 languages Unity supports. This is an offshoot of a discussion that started here.

It's been tossed about for a long time that the languages are all compiled and end up the same speed in the end, but I've never seen any real data to show this. I recently did a few tests and discovered that there are indeed real speed differences among the languages after compile and that the assumption they all perform equally is incorrect. I converted my game to C# from UnityScript and found I got a 14.5% boost in speed during loading/processing of a level in the standalone build. I want to try to understand where those speed differences lie.

A couple of things upfront:

• I want this thread to be sort of a database of people's findings comparing the speed of the languages.
• This is not the place for bashing or defensive posturing. This should be for data gathering and analysis and nothing else. (Discuss the findings, but please don't get into arguments about which language is "better".)
• This is meant to investigate where speed differences occur among the languages and possibly answer why.
• This is the place for both synthetic and real-world benchmarks.
• Feel free to investigate any variable such as target platform, using compiled DLLs, different compilers, etc.

Please feel free to suggest or use a different format/methodology.

Here is a summary of some of our findings so far:
-------------------------------------------------------------

Real-World Example: Game converted from UnityScript to C#:

I prepared two identical projects of my game, one in C# and one in UnityScript. (Sorry, I don't have time to try Boo.) No special language optimizations were made during the conversion to C#. Here's what I found:

Loading and processing one level:
Editor:
C#: 7.08s = (baseline)
US: 6.20s = 114.19%

Standalone:
C#: 4.33s = (baseline)
US: 5.06s = 87.57%

Each test was run several times and averaged one after the other on the same boot. (The first load was discarded so that disk loading would not be a factor.) I even re-ran the tests again just to make sure there wasn't any issue being run in some particular order. There was very little variance.

Two interesting findings:
- UnityScript version loads in the editor about 14% faster than C# in this test.
- C# version loads in final build about 14.5% faster than UnityScript.

I'm very surprised to see this, especially the 14% margins. There actually are differences in speed among the languages, and not that small either. But what could be causing the difference? It's really strange one would run faster in the editor and the other would run faster in a final build. (I haven't done enough testing to determine why... that will be an ongoing investigation.)

It got me thinking and I decided to do some benchmarks of some very basic processes:

---------------------------------

Synthetic benchmarks
[NOTE: Some of these results below are Editor only results. I will update this post when I rebenchmark them in the Standalone build.]

Write to an int variable

RESULTS:
Editor:
C#: 37ms - 38ms - (baseline)
US: 37ms - 38ms - 100%
Boo: 52ms - 53ms - 71.70%

Standalone:
C#: 8ms - (baseline)
US: 8ms - 100%
Boo: 8ms - 100%

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        var sw = Stopwatch.StartNew();
        int x;
        for(int i = 0; i < 15000000; i++) x = i;
        sw.Stop();
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
    }
}

// UnityScript
#pragma strict
import System.Diagnostics;
function Update () {
    var sw = Stopwatch.StartNew();
    var x : int;
    for(var i : int = 0; i < 15000000; i++) x = i;
    sw.Stop();
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}

// Boo
import UnityEngine
import System.Diagnostics;
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        sw = Stopwatch.StartNew()
        x as int
        for i as int in range(15000000) :
            x = i
        sw.Stop()
        guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"


Write to a string variable

RESULTS:
Editor:
C#: 37ms - 38ms - (baseline)
US: 37ms - 38ms - 100%
Boo: 50ms - 51ms - 74.51%

Standalone:
C#: 8ms - (baseline)
US: 8ms - 100%
Boo: 9ms - 88%

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        var sw = Stopwatch.StartNew();
        string x;
        for(int i = 0; i < 15000000; i++)
            x = "test";    
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
    }
}

// UnityScript
#pragma strict
import System.Diagnostics;
function Update () {
    var sw = Stopwatch.StartNew();
    var x : String;
    for(var i : int = 0; i < 15000000; i++)
        x = "test";
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms"; 
}

// BOO
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        sw = Stopwatch.StartNew()
        x as string
        for i as int in range(15000000) :
            x = "test"
        guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"


Append a string variable

RESULTS:
Editor:
C#: 60ms - (baseline)
US: 82ms - 73.17%
Boo: 82ms - 73.17%

Standalone:
C#: 18ms - (baseline)
US: 37ms - 48.65%
Boo: 37ms - 48.65%

Explanation:
alexzzzz: (see post) C#'s IL code calls System.String.Concat(string, string). US's code calls Boo.Lang.Runtime.RuntimeServices.op_Addition(strin g, string) which then calls System.String.Concat(string, string)

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        var sw = Stopwatch.StartNew();
        string x;
        for(int i = 0; i < 5000000; i++) {
            x = "";
            x += "test";
        }
        sw.Stop();
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
    }
}

// UnityScript
import System.Diagnostics;
#pragma strict
function Update () {
    var sw = Stopwatch.StartNew();
    var x : String;
    for(var i : int = 0; i < 5000000; i++) {
        x = "";
        x += "test";
    }
    sw.Stop();
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}

// Boo
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        sw = Stopwatch.StartNew()
        x as string
        for i as int in range(5000000) :
            x = ""
            x += "test"
        sw.Stop()
        guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"  


Write to a string array

RESULTS:
Editor:
C#: 73ms - 74ms - (baseline)
US: 94ms - 95ms - 77.89%
Boo: 102ms - 72.55%

Standalone:
C#: 35ms - (baseline)
US: 35ms - 100%
Boo: 49ms - 71.42%

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    private string[] sA = new string[100];
    void Update () {
        var sw = Stopwatch.StartNew();
        string[] s = sA;
        for(int i = 0; i < 50000; i++) {
            for(int j = 0; j < s.Length; j++) {
                s[j] = "test";
            }
        }
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms"; 
    }
}

// UnityScript
#pragma strict
import System.Diagnostics;
public var sA : String[] = new String[100];
function Update () {
    var sw = Stopwatch.StartNew();
    var s : String[] = sA;
    for(var i : int = 0; i < 50000; i++) {
        for(var j : int = 0; j < s.length; j++) {
            s[j] = "test";
        }
    }
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms"; 
}

// BOO
import UnityEngine
import System.Diagnostics
class SpeedTestBoo (MonoBehaviour): 
    private sA as (string) = array(string, 100)
    def Update ():
        sw = Stopwatch.StartNew()
        s as (string) = sA
        for i as int in range(50000) :
            for j as int in range(s.Length):
                s[j] = "test"
        guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"


Add to generic string list (old test style, will update)

RESULTS (fps): EDITOR ONLY
C#: 26.5 - 28.0 = (baseline)
US: 26.5 - 28.0 = 100%
Boo: 26.6 - 27.6 = ~100%

Sources:

PHP:

// C# Test
using UnityEngine;
using System.Collections;
using System.Collections.Generic;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        List<string> list = new List<string>();
        for(int i = 0; i < 1200000; i++) {
            list.Add("test");
        }
    }
}

// UnityScript Test
#pragma strict
import System.Collections.Generic;
function Update () {
    var list : List.<String> = new List.<String>();
    for(var i : int = 0; i < 1200000; i++) {
        list.Add("test");
    }
}

// BOO TEST (for)
import UnityEngine
import System.Collections.Generic
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        list as List[of string] = List[of string]()
        for i as int in range(1200000) :
            list.Add("test")

Getting a component and casting to transform (old test style, will update)
RESULTS (fps): EDITOR ONLY
C#: 34.2 - 34.5 = (baseline)
US: 34.2 - 34.4 = ~100%
Boo: 34.2 - 34.3 = ~100%

Sources:

PHP:

// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        for(int i = 0; i < 100000; i++) {
            Transform t = (Transform)GetComponent(typeof(Transform));
        }
    }
}


// UnityScript Test
#pragma strict
function Update () {
    for(var i : int = 0; i < 100000; i++) {
        var t : Transform = GetComponent(Transform);
    }
}

// BOO TEST (for)
import UnityEngine
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        for i as int in range(100000) :
            t as Transform = GetComponent(Transform);
            

Getting a component using generics (old test style, will update)
Interesting note: Using generics is slower than casting. Runs at 84.34% speed compared to casting.
RESULTS (fps): EDITOR ONLY
C#: 28.9 - 29.1 = (baseline)
US: 28.8 - 29.1 = ~100%
Boo: 28.7 - 29.0 = ~100%

Sources:

PHP:

// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        for(int i = 0; i < 100000; i++) {
            Transform t = GetComponent<Transform>();
        }
    }
}

// UnityScript Test
#pragma strict
function Update () {
    for(var i : int = 0; i < 100000; i++) {
        var t : Transform = GetComponent.<Transform>();
    }
}

// BOO TEST
import UnityEngine
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        for i as int in range(100000) :
            t as Transform = GetComponent[of Transform]()


Getting a component by its base class (old test style, will update)
RESULTS (fps): EDITOR ONLY
C#: 40.4 - 40.5 = (baseline)
US: 40.4 - 40.5 = 100%
Boo: - = %

PHP:

// C# TEST
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
    void Update () {
        for(int i = 0; i < 100000; i++) {
            BaseCS b = (BaseCS)GetComponent(typeof(BaseCS));
        }
    }
}

    // BaseCS.cs
    using UnityEngine;
    using System.Collections;
    public abstract class BaseCS : MonoBehaviour {
    }

    // ExtCS.cs
    using UnityEngine;
    using System.Collections;
    public class ExtCS : BaseCS {
        void Start () {
        }
        void Update () {
        }
    }


// UnityScript Test
#pragma strict
function Update () {
    for(var i : int = 0; i < 100000; i++) {
        var b : BaseJS = GetComponent(BaseJS);
    }
}

    // BaseJS.js
    #pragma strict
    function Start () {
    }
    function Update () {
    }
    
    // ExtJS.js
    #pragma strict
    public class ExtJS extends BaseJS {
        function Start () {
        }
        function Update () {
        }
    }


 

Calling an empty method (alexzzzz)

Results (Standalone):
C#: 83ms (private method)
C#: 83ms (public method)
C#: 294ms (public virtual method)

US: 83ms (private method)
US: 295ms (public method)
US: 83ms (public final method)

Boo: 87ms (private method)
Boo: 87ms (public method)
Boo: 296ms (public virtual method)

Explanation:
alexzzzz: (See posts here and here.) The performance difference is caused by inlining. With NoInlining attribute non-virtual function calls become as "slow" as virtual ones.
Declaring the function as final [in US] eliminates the performance difference between the languages

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Start () {
        var sw = Stopwatch.StartNew();
        for (int i = 0; i < 150000000; i++) Test(); 
        sw.Stop();
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms";
    }
    private void Test() {}
}

// US
#pragma strict
import System.Diagnostics;
function Start () {
    var sw = Stopwatch.StartNew();
    for (var i : int = 0; i < 150000000; i++) Test(); 
    sw.Stop();
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms";
}
private function Test() : void {}

//BOO
import UnityEngine
import System.Diagnostics;
class SpeedTestBoo (MonoBehaviour): 
    def Update ():
        sw = Stopwatch.StartNew()
        for i as int in range(150000000) :
            Test();
        sw.Stop()
        guiText.text = "Boo " + sw.ElapsedMilliseconds.ToString() + "ms"       
    
    private def Test() as void:
        pass


 

EDIT: Problem solved. It was an oversight. See post here.

This one is baffling. As part of my effort to reduce my compile times, before I converted my game to C#, I converted it to use mostly externally compiled DLLs instead of script files. Eventually, I converted it to C# and then back to script files. So I essentially have 4 versions: UnityScript in scripts, UnityScript in DLLs, C# in scripts, C# in DLLs. When I was doing the tests where I discovered the performance increase moving from C# (scripts) to UnityScript (scripts), I also discovered that the C# DLL versions did not perform as well as the C# script, US script, and US dll versions. I spent yesterday and part of today tracing down the >2 second performance gap on level load in the C# dll version and eventually found the cause. It was a nested loop. But why would this nested loop perform worse in C# DLLs vs C# scripts and why did UnityScript DLL not have any problem with it either? (The 2nd example below is closer to the real loop in the game and exhibits a much worse result than the 1st.)

I might be doing something wrong because this just seems a bit outrageous, but I haven't been able to find any reason so far. Also note that as the 2nd example shows, there's no speed difference between VisualStudio and MonoDevelop output for C# (in this test).

Nested loops in scripts vs DLLs

C# is 30%/33% slower when using DLLs in this case. US suffers no loss whatsoever.

Results:
Editor
C#: 5085ms
US: 5084ms
C# dll (debug, VStudio2010, .NET 3.5): 7342ms
US dll (debug): 5084ms

Standalone
C#: 1129ms
US: 1137ms
C# dll (debug, VStudio2010, .NET 3.5): 1695ms
US dll (debug): 1135ms

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Start () {
        
        var sw = Stopwatch.StartNew();
        
        int count = 0;
        for(int i = 0; i < 2000; i++) {
            for(int j = 0; j < 1000; j++) {
                for(int k = 0; k < 1000; k++)
                    count++;
            }
        }
        
        sw.Stop();
        guiText.text = count + " C# " + sw.ElapsedMilliseconds.ToString() + "ms"; 
    }
}

// US
#pragma strict
import System.Diagnostics;
function Start () {
    var sw = Stopwatch.StartNew();
    
    var count : int = 0;
        for(var i : int = 0; i < 2000; i++) {
            for(var j : int = 0; j < 1000; j++) {
                for(var k : int = 0; k < 1000; k++)
                    count++;
            }
        }
    
    sw.Stop();
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms"; 
}

Nested loops in scripts vs DLLs (with array access)

This one is astounding. C# DLL takes 2.25X/1.6X as long as the script version. Again, UnityScript is unphased.

Results:
Editor
C#: 10796ms
US: 10797ms
C# dll (debug, MonoDev, .NET 3.5): 24283ms
C# dll (release, MonoDev, .NET 3.5): 24281ms
C# dll (debug, VStudio2010, .NET 3.5): 24281ms
C# dll (release, VStudio2010, .NET 3.5): 24280ms
US dll (debug): 10796ms
US dll (release): 10797ms

Standalone
C#: 2717ms
US: 2697ms
C# dll (debug, MonoDev, .NET 3.5): 4479ms
C# dll (release, MonoDev, .NET 3.5): 4474ms
C# dll (debug, VStudio2010, .NET 3.5): 4481ms
C# dll (release, VStudio2010, .NET 3.5): 4477ms
US dll (debug): 2683ms
US dll (release): 2685ms

Sources:

PHP:

// C#
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Start () {
        var sw = Stopwatch.StartNew();
        
        int numNodes = 10000;
        int maxConn = 8;
        int[] aa = new int[numNodes*maxConn];
        int[] ab = new int[numNodes*maxConn];
        
        sw.Start();
        
        int count = 0;
        for(int i = 0; i < numNodes; i++) {
            for(int j = 0; j < maxConn; j++) {
                for(int k = 0; k < count; k++) {
                    if(aa[k] == 50  aa[k] == 50) {
                        break;
                    }
                }
                count++;
            }
        }
        
        sw.Stop();
        guiText.text = "C# " + sw.ElapsedMilliseconds.ToString() + "ms"; 
    }
}

// US
#pragma strict
import System.Diagnostics;
public var sA : String[] = new String[100];
function Start () {
    var sw = Stopwatch.StartNew();
    
    var numNodes : int = 10000;
    var maxConn : int = 8;
    var aa : int[] = new int[numNodes*maxConn];
    var ab : int[] = new int[numNodes*maxConn];
    
    sw.Start();
    
    var count : int = 0;
    for(var i : int = 0; i < numNodes; i++) {
        for(var j : int = 0; j < maxConn; j++) {
            for(var k : int = 0; k < count; k++) {
                if(aa[k] == 50  aa[k] == 50) {
                    break;
                }
            }
            count++;
        }
    }
    
    sw.Stop();
    guiText.text = "US " + sw.ElapsedMilliseconds.ToString() + "ms"; 
}

EDIT: I just checked the code in ILSpy and the loop code is identical. UnityScript uses a public override void for Start() whereas C# uses a private void, but that's the same for both C# versions.

 

Wow, that's nutty. Please keep it up.

 

Could you quickly describe how you're creating and using the DLL so I can try to replicate?

 

Sure.

C#: (MD or VS)
Create a new solution and project in the IDE
Set project options for Debug build:

Symbols:
DEBUG;TRACE;UNITY_3_5_5;UNITY_3_5;UNITY_EDITOR;ENABLE_PROFILER;UNITY_STANDALONE_WIN;ENABLE_GENERICS;ENABLE_DUCK_TYPING;ENABLE_TERRAIN;ENABLE_MOVIES;ENABLE_WEBCAM;ENABLE_MICROPHONE;ENABLE_NETWORK;ENABLE_CLOTH;ENABLE_WWW;ENABLE_SUBSTANCE

Suppress warnings: 0169
Platform target: .NET 3.5
Compile target: Library

Add references: C:\Program Files (x86)\Unity\Editor\Data\Managed\UnityEngine.dll, System
Create SpeedTestCS_DLL.cs and copy and paste the C# code above into it
Build
Copy the bin/Debug/SpeedTestCS_DLL.dll to yourgameproject\Assets\Plugins
Make a scene and add an empty game object with a GUIText on it and attach the SpeedTestCS_DLL monobehaviour to it. (Open the dropdown arrow in the DLL in the project view in Unity and drag and drop.)


UnityScript: (MD)
Create a new solution and project (can be same solution as above if you made it in MD)
Add references: C:\Program Files (x86)\Unity\Editor\Data\Managed\UnityEngine.dll, System
Open the project.unityproj in a text editor:

Set:
<TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
<OutputType>Library</OutputType>

In the Debug <PropertyGroup> add:
<DefineConstants>DEBUG;TRACE;UNITY_3_5_5;UNITY_3_5;UNITY_EDITOR;ENABLE_PROFILER;UNITY_STANDALONE_WIN;ENABLE_GENERICS;ENABLE_DUCK_TYPING;ENABLE_TERRAIN;ENABLE_MOVIES;ENABLE_WEBCAM;ENABLE_MICROPHONE;ENABLE_NETWORK;ENABLE_CLOTH;ENABLE_WWW;ENABLE_SUBSTANCE</DefineConstants>
<WarningLevel>4</WarningLevel>
<NoWarn>0169</NoWarn>

Create SpeedTestUS_DLL.js and copy and paste the US code above into it
Build
Copy the bin/Debug/SpeedTestUS_DLL.dll to yourgameproject\Assets\Plugins
Make a scene and add an empty game object with a GUIText on it and attach the SpeedTestUS_DLL monobehaviour to it.

The debug symbols and suppress warnings settings are just mimicing what Unity does to its own projects in the default solution. (The US versions seems to ignore a lot of the stuff in the xml file anyway like the target framework and the warnings. The most important one is to change OutputType to Library.)

 

EDIT: Problem solved. It was an oversight. See post here.

The same holds true if you omit UnityEngine.dll reference and just create a simple class in the external DLL and call it from a Monobehaviour script file.

(Note: UnityScript dll compilation seems to require a reference to UnityEngine.dll even if you don't call any unity classes... probably because every script is a monobehaviour even if you put nothing but your own basic class in it.)

Editor:
C# script: 5083ms
US script: 5082 ms
C# dll: 7341ms
US dll: 5085

Standalone:
C# script: 1129ms
US script: 1128ms
C# dll: 1816ms
US dll: 1130

C# Script version sources:

PHP:

// SpeedTestCS.cs
using UnityEngine;
using System.Collections;
using System.Diagnostics;
public class SpeedTestCS : MonoBehaviour {
    void Start () {
        Test_CSscript test = new Test_CSscript();
        long time = test.Go();
        guiText.text = "C# " + time.ToString() + "ms"; 
    }
}    
public class Test_CSscript {
    public long Go() {
        var sw = Stopwatch.StartNew();
        int count = 0;
        for(int i = 0; i < 2000; i++) {
            for(int j = 0; j < 1000; j++) {
                for(int k = 0; k < 1000; k++)
                    count++;
            }
        }
        sw.Stop();
        return sw.ElapsedMilliseconds; 
    }
}

C# DLL version sources:

PHP:

// SpeedTestCS_dll.cs
using UnityEngine;
using System.Collections;
public class SpeedTestCS : MonoBehaviour {
    void Start () {        
        Test test = new Test();
        long time = test.Go();    
        guiText.text = "C# " + time.ToString() + "ms"; 
    }
}

// CS.dll
using System.Collections;
using System.Diagnostics;
public class Test_CS_dll_assembly {
    public long Go() {
        var sw = Stopwatch.StartNew();
        int count = 0;
        for(int i = 0; i < 2000; i++) {
            for(int j = 0; j < 1000; j++) {
                for(int k = 0; k < 1000; k++)
                    count++;
            }
        }
        sw.Stop();
        return sw.ElapsedMilliseconds;
    }
}

UnityScript script version sources:

PHP:

// SpeedTestUS.js
import System.Diagnostics;
function Start () {
    var test : Test_USscript = new Test_USscript();
    var time : long = test.Go();
    guiText.text = "C# " + time.ToString() + "ms"; 
}
public class Test_USscript {
     public function Go() : long {
        var sw = Stopwatch.StartNew();
        var count : int = 0;
        for(var i : int = 0; i < 2000; i++) {
            for(var j : int = 0; j < 1000; j++) {
                for(var k : int = 0; k < 1000; k++)
                    count++;
            }
        }
        sw.Stop();
        return sw.ElapsedMilliseconds; 
    }
} 

UnityScript DLL version sources:

PHP:

// SpeedTestUS_dll.js
function Start () {
    var test : Test_US_dll_assembly = new Test_US_dll_assembly();
    var time : long = test.Go();
    guiText.text = "C# " + time.ToString() + "ms"; 
}

// US.dll
import System.Diagnostics;
public class Test_US_dll_assembly {
     public function Go() : long {
        var sw = Stopwatch.StartNew();
        var count : int = 0;
        for(var i : int = 0; i < 2000; i++) {
            for(var j : int = 0; j < 1000; j++) {
                for(var k : int = 0; k < 1000; k++)
                    count++;
            }
        }
        sw.Stop();
        return sw.ElapsedMilliseconds; 
    }
}

 

I've just been officially mindblown haha

 

I'd kind of expect all of this stuff to perform more or less the same, because it's all stuff that will compile down to largely the exact same bytecode. Your CPU doesn't care what language the machine code it's executing originated in, all it cares is what operations you ask it to do. With simple code like the above I'd expect it to end up compiling to pretty near identical bytecode, and thus perform pretty near identical operations.

 

You would expect so but it does not. The discovery that this long-held assumption was wrong was the entire reason I decided to start this thread. The very first example I noted in the first post shows a real game level loading/processing test I did. I only started investigating this because I saw that real-world difference in speed with my game when converted it to C# (14% difference in level load time). The whole point of this was to investigate where those speed differences that I already established exist came from. (Actually it just made me curious. ) And as for the synthetic bencharks, they are already compiled, and the standalone versions are compiled without debugging code, and yet show differences in many cases. And I know these simplistic benchmarks aren't going to reveal a whole lot, but it's just a start. I hope to expand this and add benchmarks from various portions of my game in C#/US as time permits. (Real-world is always better than synthetic.)

There are even a couple of explanations above pointing out where the differences came from on some tests. You can see in the IL decompiled code that US, C#, and Boo do not always compile to the same code in the end. (String concatenation and string arrays are two examples).

 

Wow. Nerdy, but very interesting Keep it up please. Couldn't see through all tests yet, but I'm looking forward to see your results!
Nice work man!

 

guavaman, try to change script execution order. I have found a strange correlation:

Testing inside the project I'm working on

(0) other scripts
(100) CS dll: 2458ms, 2432ms, 2470ms
(200) CS: 2411ms, 2442ms, 2417ms
(300) US: 2483ms, 2251ms, 2420ms

(-300) CS dll: 2420ms, 2413ms, 2410ms
(-200) CS: 3428ms, 3430ms, 3441ms
(-100) US: 2408ms, 2418ms, 2418ms
(0) other scripts

Awake instead of Start
(-300) CS dll: 2427ms, 2414ms, 2411ms
(-200) CS: 2420ms, 2411ms, 2411ms
(-100) US: 2429ms, 2423ms, 2410ms
(0) other scripts

Testing inside an empty project

(-300) CS dll: 3443ms, 3511ms, 3475ms
(-200) CS: 2421ms, 2432ms, 2455ms
(-100) US: 2472ms, 2454ms, 2454ms

(-300) US: 2439ms, 2421ms, 2427ms
(-200) CS: 2435ms, 2443ms, 2412ms
(-100) CS dll: 2436ms, 2408ms, 2436ms

 

I tried changing the order and it didn't do anything, but I may not be doing it exactly like you are. I'm not sure what your setup looks like, but I had a little trouble earlier because I had both a US.dll and a C#.dll which contained a public Test class. At some point, the C# caller script was using the US.dll's Test class. Could this be happening in your case? I see you don't have a US dll, but maybe it's finding the US script's or the C# script's Test class and using it because of the execution order change? (Unity doesn't always warn you about duplicates in dll's from what I recall.)

I currently have nothing but the C# dll and its calling script (using this format) in the scene and tried setting the execution order on the lone script to -300 and +300 but I still get the slow results. Changing it to Awake doesn't help either (I thought it did before I deleted the other scripts and dll in the scene, but it ended up it was just finding the US dll's Test cass) Best to rename the Test class for the different languages and even for each script/dll test in the same scene.

I'm glad to see that you've replicated the "bug" though and I know I'm not crazy.

Edit: I updated sources in the above post to so the class names don't overlap. (Sorry about that ovesight.) And I retested and found no difference with script ordering.

EDIT: Problem solved. It was an oversight. See post below.

 

Slow DLLs in C# Solved
It ended up being an oversight. If you enable "optimize code" in the Build tab in Visual Studio (Compiler -> Enable Optimizations in MonoDevelop), the C# version runs just as fast as the UnityScript version.

I never set this option because I saw it was not enabled by MonoDevelop in the Unity project solution. However, I compiled a version of the DLL using Unity's compile command line (from the editor log) and found it ran very fast. That told me something was wrong with the DLL compiling.

Unity's command line compiler also uses a platform target of AnyCPU (64-bit preferred).

C# DLL: 5052
C# Scripts 5076

 

Interesting, thank you for clarifying that because I was unable to reproduce the issue (and optimize code is enabled by default for projects created using VS).

 

Strange, my version did not enable it by default (VS 2010). Neither did MonoDevelop. I was basing my options on Unity's own solution file, but apparently Unity doesn't really set up the solution in MonoDevelop with the correct options since they rely on the command line compiler to actually compile it in the editor when it imports it. That's what threw me off.

Next time if you decide to try replicating a test and you can't reproduce it, it would be helpful if you'd let me know.