Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

My adventures with Unity and multi threading

Discussion in 'Community Learning & Teaching' started by RavenLiquid, Oct 7, 2015.

  1. RavenLiquid

    RavenLiquid

    Joined:
    May 5, 2015
    Posts:
    44
    I have been working on a video texture(more on that in a later post hopefully) for a while and one thing lead to another and now I'm working on multi threading.

    First off, this is based on the AsyncBridge and C# 6(made possible by Alexzzz). I did not invent any of this.

    So I would like to talk a little about my experiences with multi threading in Unity and the pitfalls I encountered and what I did get working.
    I'm not going to claim to be an expert or anything regarding both Unity or threading, so if you spot anything plainly wrong or something you think can be better let me know. The blind leading the blind is not going to help anyone.

    So first some background on why not just stick to coroutines.
    Code (CSharp):
    1.  
    2.    private void StartCoroutines()
    3.    {
    4.      var strings = new List<strings>() { "One", "Two", "Three"};
    5.      StartCoroutine(CountToThree(strings));
    6.      StartCoroutine(DoStuff());
    7.    }
    8.  
    9.    IEnumerator CountToThree(List<string> strings)
    10.    {  
    11.      foreach(var s in strings)
    12.      {
    13.        print(s);
    14.        // suspend execution for 1 seconds
    15.        yield return new WaitForSeconds(1);      
    16.      }
    17.      yield return null;
    18.    }
    19.    IEnumerator PrintTime()
    20.    {
    21.      print("Starting " + Time.time);
    22.  
    23.      yield return null;
    24.    }
    25.  
    As you can see it's fairly easy to run code in a coroutine, and you have nice parallel code.
    Well, not exactly. Coroutines do not run on a separate thread but on the main thread, and therefore utilize time slicing instead of multi threading.
    In other words, every time you call yield, control is returned to the main Unity loop. When the main loop has done it's thing, control is returned to your coroutine.

    This works well enough if you have small work packages that run in loops, but there are some big issues when using it for stuff that takes a lot of time or when you call code that runs a loop outside of your control and does not yield.

    Your code runs on the main thread, so when your code runs, Unity does not. This means that until you call yield you are blocking the main thread and Unity is unresponsive. No updates are drawn and input is not registered.

    So the main rule is, do not block the main thread!

    So the issue is that it is not always possible to call yield in a coroutine when needed. However, the one reasons to use coroutines is because you have a task that takes a lot of time and you cannot run it in the main loop.
    So what to do? Well, use another thread besided the main loop!

    There is a reason why Unity uses coroutines instead of multiple threads and I will get to that later.

    Instead of using the Thread class directly I'm going to use the Task Parallel Library, because they make a lot of stuff easier and work well with the async framework.
    Tasks can be run on a scheduler, and this can then create threads for you or run it on a specific thread.

    So, what is a task? Well, it's just a bunch of work. It simply encapsulates a method (named or anonymous), and can then be executed.

    Code (CSharp):
    1.  
    2.    private void DoTaskWork()
    3.    {
    4.      Task t = new Task(TaskWork);
    5.      t.Run();
    6.      
    7.      Task t2 = new Task(() => { //Do Stuff });
    8.      t2.Run();    
    9.    }
    10.    
    11.    private void TaskWork()
    12.    {
    13.      //Do Stuff
    14.    }
    15.  
    Pretty easy right? You can also use the TaskFactory, which is nice because it lets you set so defaults like the scheduler and provides some different options for starting multiple tasks like waiting for them all to complete and then do something else.

    So you can do some work and then update the UI on the progress.

    Code (CSharp):
    1.  
    2.   [SerializeField]
    3.   private Text updateText;
    4.    
    5.    private void DoTaskWork()
    6.    {
    7.      Task t = new Task(TaskWork);
    8.      t.Run();
    9.      
    10.      Task t2 = new Task(() => { //Do Stuff });
    11.      t2.Run();    
    12.    }
    13.    
    14.    private void TaskWork()
    15.    {
    16.      //Do Stuff
    17.      updateText.text = progressText;
    18.    }
    19.  
    Except you cannot. If you run this code, the console in the editor will tell you that updating that UI element has to be done on the tread that it was created on: the main thread.

    So what if we use an event to notify the UI of an update?

    Code (CSharp):
    1.  
    2.   [SerializeField]
    3.   private Text updateText;
    4.    
    5.    public delegate UpdateEventHandler(string progress);
    6.    public event UpdateEventHandler UpdateEvent;
    7.    
    8.    private void DoTaskWork()
    9.    {
    10.      UpdateEvent += (string progress) => { updateText.text = progress; };
    11.      Task t = new Task(TaskWork);
    12.      t.Run();
    13.      
    14.      Task t2 = new Task(() => { //Do Stuff });
    15.      t2.Run();    
    16.    }
    17.    
    18.    private void TaskWork()
    19.    {
    20.      //Do Stuff
    21.      UpdateEvent(progressText);
    22.    }
    23.  
    Well, same deal. The event is dispatched on the background thread, so the callback is also executed on this thread. No dice.

    This seems to me one of the reasons that Unity uses coroutines, as you can easily update GameObjects. This also goes for things like textures and audioclips. You cannot create these on a background thread.

    So, is this totaly impossible? No, what we need to do is invoke the part that cannot be run on the background thread on the main thread. This means we are going to block the task, run our code on the main thread and when it is done continue the task.

    Code (CSharp):
    1.  
    2.   [SerializeField]
    3.   private Text updateText;
    4.    
    5.    private void DoTaskWork()
    6.    {
    7.      Task t = new Task(TaskWork);
    8.      t.Run();
    9.      
    10.      Task t2 = new Task(() => { //Do Stuff });
    11.      t2.Run();    
    12.    }
    13.    
    14.    private void TaskWork()
    15.    {
    16.      bool work = true;
    17.      
    18.      var tf = new TaskFactory(UnityScheduler.MainThread);
    19.      while(work)
    20.      {
    21.        //Do Stuff
    22.        tf.StartNew(() =>
    23.        {
    24.          updateText.text = progressText;
    25.        });
    26.      }
    27.    }
    28.  
    I'm still trying to figure out a nice way to do this, ideally I want to be able to call something like RunOnMain on the UnitySchedular and have it figure out if the current thread is the main thread (to prevent invoking if it is not needed) and invoke the code if it is not, otherwise just run it.

    The idea is kind of the same as a BackgroundWorker and the Dispatcher Invoke in XAML.

    So after this pitfall, let me explain one that is less easy to notice.

    When running code in the editor, and there is an exception in your code you will get a nice red error in the console. The nice thing in Unity is that exception don't halt the application. The execution of the current block is just ended, and the next script gets it's execution time. The down side of this is that the rest of the code will become difficult to predict and behave in unexpected ways.
    When using multiple threads, things get more complicated. Unity does not catch exceptions that occur in other threads than the main thread.
    What this means is that if your thread runs into an exception, it dies and you don't know that it happened or why.

    There is one way of checking if it ran to completion, and that is checking the Task object. You can check if it IsCompleted, IsFaulted and get the Status.
    IsFaulted will be true if it ran into an issue.
    This is difficult to check, because you will need to check it after completion and that is something you usually don't want to because you don't know when that is.
    There are no events to register to, so no callbacks for you.

    Another that that is possible, and more helpful but a bit more work. Surround your code in a try/catch and write the exception to the console in the catch.
    You can use the Debug class from any thread and it will write to the editor console. This is also helpfull if you wan't to log some progress or try to debug an annoying issue.
    Because debugging this has had me scratching my head at times. Because I was unaware of the exceptions not being shown in the editor, I tried placing break points and ran in to a rather unexpected issue.

    The code that I tried to run synchronously (I was starting a task from the background to run on the main thread), was skipping over the task while I was explicitly calling Wait, and started it with RunSynchronously. I had expected it to jump into the task first but it did not.
    So what was happening? Well, the task was running into an exception but did this before the debugger showed it had IsFaulted. I'm still not 100% sure about how to explain this behaviour but it worked as expected after I fixed the issue.

    So it helps if you put some Debug statements in your code to figure out what it is actually doing and how far it got.

    Next topic, Parallel.

    Lets say your task looks like this

    Code (CSharp):
    1.  
    2.    private void DoTaskWorkInLoop()
    3.    {
    4.      var listOfFiles = new List<string> () {"file.blob", "file2.blob", "file3.blob"};
    5.      
    6.      var t = new Task(() =>
    7.      {
    8.        foreach (var f in listOfFiles)
    9.        {
    10.          //Read file f
    11.          //Process file f
    12.          //Save file f
    13.        }
    14.      });
    15.      t.Run();
    16.    }
    17.  
    Or maybe you though to multithread it better and did this

    Code (CSharp):
    1.  
    2.    private void DoTaskWorkInLoop()
    3.    {
    4.      var listOfFiles = new List<string> () {"file.blob", "file2.blob", "file3.blob"};
    5.      
    6.      foreach (var f in listOfFiles)
    7.      {
    8.        var t = new Task(() =>
    9.        {
    10.          //Read file f
    11.          //Process file f
    12.          //Save file f
    13.        });
    14.        t.Run();
    15.      }
    16.    }
    17.  
    While the second example multithreads better, you are going to run into an issue there.
    You are using f to store the current filename, but this is being overwritten with the next filename as the foreach loops.
    In most cases it will contain "file3.blob" for eacht task.
    A quick fix is to store the name before you pass it into the task.

    Code (CSharp):
    1.  
    2.      foreach (var f in listOfFiles)
    3.      {
    4.        var filename = f;
    5.        var t = new Task(() =>
    6.        {
    7.          //do stuff with filename
    8.        }
    9.      }
    10.  
    Then there is an even better method: Parallel.ForEach.

    This lets you replace the foreach and the task with one call. You just give it the collection you want to iterate over, and a method that matches the signature of your collection contents. So, if you have List<string> strings for example, you can do Parallel.ForEach(strings, (s) =>{ Debug.Log(s);}). The system tries to determine it self how many threads (if any) it should create and runs them all.

    Code (CSharp):
    1.  
    2.    private void DoTaskWorkInLoop()
    3.    {
    4.      var listOfFiles = new List<string> () {"file.blob", "file2.blob", "file3.blob"};
    5.      
    6.      var options = new ParallelOptions();
    7.      options.TaskScheduler = UnityScheduler.UnityTaskScheduler.Default; //Run on background thread scheduler
    8.      options.MaxDegreeOfParallelism = 2; //Do max 2 items at once
    9.      Parallel.ForEach(listOfFiles, options, (f) =>  
    10.      {
    11.        var t = new Task(() =>
    12.        {
    13.          //Read file f
    14.          //Process file f
    15.          //Save file f
    16.        });
    17.      }
    18.    }
    19.  
    It is also blocking, so that helps if you need the result of when it has run all the code.

    But here is also something to mind, if you call this from the main thread you are of course blocking the main thread.
    But you can also make it worse, like I accidentally did. I didn't realize I was doing this on the main thread and in the code of the foreach I had a nice callback to the main thread. So what happens then, lets see:

    - Parallel.ForEach starts on the main thread, blocks until completed
    - Various threads are spawned do their thing
    - Background thread invokes the main thread, blocks

    And there you have it, a nice deadlock.
    Your main thread is waiting for the background thread to complete, and your background thread is waiting for the main thread to respond and invoke its work.

    So these are some of the main things I did and ran in to.
    Other things to watch out for is high CPU usage and the Unity editor dying for various reasons. If the main thread blocks, this usually locks up the editor as well.
    Also using Visual Studio seemed to cause some issues here and there. I've seen a lot of out of scope errors when debugging, even if the code executed on the line before. This makes things difficult when you have an issue somewhere.
    So sometimes I wish the editor was more like Visual Studio, so that I can stop debugging if there is an issue instead of the whole editor crashing.
    Maybe multithreading will come in Unity 6 as a more standard feature.

    So leave some comments on how I can make this better or questions if things are unclear.
     
  2. alexzzzz

    alexzzzz

    Joined:
    Nov 20, 2010
    Posts:
    1,447
    Nice overview! I have a couple of notes.

    1. I guess this is the simplest way to run code on the main thread:
    Code (CSharp):
    1.         var task = new Task(() =>
    2.                             {
    3.                                 // do UI stuff
    4.                             });
    5.         // and then call
    6.         task.RunSynchronously(UnityScheduler.MainThreadScheduler);
    7.         // or
    8.         task.Start(UnityScheduler.MainThreadScheduler); // run asynchronously
    9.  
    No matter where you run this code from, the task will be executed on the main thread.

    2. Parallel.ForEach example has a couple of errors. It should look like this
    Code (CSharp):
    1.     private void DoTaskWorkInLoop()
    2.     {
    3.         var listOfFiles = new List<string> {"file.blob", "file2.blob", "file3.blob"};
    4.  
    5.         var options = new ParallelOptions();
    6.         options.TaskScheduler = TaskScheduler.Default; //Run on background thread scheduler
    7.         options.MaxDegreeOfParallelism = 2; //Do max 2 items at once
    8.  
    9.         Parallel.ForEach(listOfFiles, options, f =>
    10.                                                {
    11.                                                    //Read file f
    12.                                                    //Process file f
    13.                                                    //Save file f
    14.                                                });
    15.     }
    16.  
    3. Some of your examples use Task.Run() to execute the task asynchronously. The version of TaskParallelLibrary I use in my C# 6.0 integration project is very old, and at that time this method was called Task.Start().

    4. I've made some changed to my project to make scheduler names a bit more consistent and self-explaining. Now, when you need to specify a scheduler, you can type UnityScheduler.MainThreadScheduler or UnityScheduler.ThreadPoolScheduler.
     
    twobob and DarkFlameX1 like this.
  3. better_walk_away

    better_walk_away

    Joined:
    Jul 12, 2016
    Posts:
    291
    Using plain coroutines doesn't guarantee that the app is deadlock-free though. Sometimes, the problem resides in the engine. For example, there was a bug in Unity 2019.4.22f1, if the app used Resources.Load() while asynchronously loading an asset bundle, there would be a chance that a deadlock would happen on the main thread. This bug was fixed in Unity 2021 and backported to Unity 2019.4.23f1.