Capturing local variables from Coroutine's for-loop woes

Hey everyone,

So as I was working on some code I ran into a bug that I couldn't quite place. After some experimentation I found that the problem lies in capturing the value of a Coroutine's loop-local variable. While it works as expected when the method the binding takes place in is a "regular method" (i.e. void), it seems to always hold the same value when done from within a Coroutine. This value is always the same as the last value it held. I think a code sample to reproduce the problem is probably clearer.

Code (csharp):

1. using UnityEngine;
2. using System;
3. using System.Collections;
4. using System.Collections.Generic;
5.  
6. public class Test : MonoBehaviour {
7.     public List<Action> actions = new List<Action>();
8.  
9.     IEnumerator Start(){ //when changed to "void Start()" the sequence runs as expected
10.         yield return null;
11.         for(int i=0; i<3; i++){
12.            int_i = i;
13.            Debug.Log("Adding DoThing() for index " + _i);
14.            actions.Add(() => DoThing(_i));
15.         }
16.  
17.         Init();
18. }
19.  
20.     void DoThing(int index){
21.         Debug.Log("Doing the thing " + index + "!");
22.     }
23.  
24.     void Init(){
25.         for(int i=0; i<actions.Count; i++){
26.             actions[i]();
27.         }
28.     }
29. }

The expected output here would be:

Code (csharp):

1. Adding DoThing() for index 0
2. Adding DoThing() for index 1
3. Adding DoThing() for index 2
4. Doing the thing 0!
5. Doing the thing 1!
6. Doing the thing 2!

But instead, when using Start() as a Coroutine I get:

Code (csharp):

1.  
2. Adding DoThing() for index 0
3. Adding DoThing() for index 1
4. Adding DoThing() for index 2
5. Doing the thing 2!
6. Doing the thing 2!
7. Doing the thing 2!

Is this a known limitation of Coroutines? Why is this happening? And is there an easy fix for this problem, or would I need to refactor my code so as to completely avoid this scenario?

So many questions, but looking forward to an answer! =)

Thanks,
Patrick

 

That's a pretty good word to describe this with!

Can you provide some steps to take to do/ test this? I've never done anything like it, and I'm not even sure if it's possible for me as I'm on OSX? If it is I'd love to give it a try though!

 

Oof, this board is moving fast! Quoting you real quick so you get a notification. =)

 

Wow, that's functionality you'd expect from Javascript in terms of variables having function scope, and not block scope. I'm not sure why it's working that way in C#? Maybe IEnumerator instances from methods are implemented differently than I had originally expected.

My guess now is that an IEnumerator method call returns an instance that has state saved for every local variable in a customized extension of IEnumerator that Unity provides.

Example of function scope in Javascript:

Code (JavaScript):

1. var a = 1;
2.  
3. function four() {
4.   if (true) {
5.     var a = 4;
6.   }
7.  
8.   alert(a); // alerts '4', not the global value of '1'
9. }

 

Hmm, but then that would mean that (the first) int _i has its state saved as well, instead of being created every iteration? That doesn't sound like the intended behaviour then. Or did you mean something else?

 

This is complete guesstimation on my part as to how Unity handles IEnumerator instances for their methods that has return type IEnumerator and of course some testing of printing the type returned to the screen.

The way your method is probably interpreted after a call to Start() by Unity

(Note: the class is not tested btw)

Code (CSharp):

1.  
2. public class c++_iterator<T> : IEnumerator where T : struct
3. {
4.  
5.     private int i;
6.     private int _i;
7.     private int chunk = 0;
8.     private YieldInstruction _yield;
9.     public object Current { get { return _yield; } }
10.  
11.     // ... code to set values, except chunk
12.  
13.     public bool MoveNext()
14.     {
15.  
16.         switch(chunk)
17.         {
18.  
19.             case 0:
20.                 _yield = null;
21.                 chunk = 1;
22.             return true;
23.      
24.             case 1:
25.      
26.                 for(i=0; i<3; i++)
27.                 {
28.                     _i = i;
29.                     Debug.Log("Adding DoThing() for index " + _i);
30.                     actions.Add(() => DoThing(_i));
31.                 }
32.          
33.                 Init();
34.                 chunk = 2;
35.             return false;
36.      
37.         }
38.  
39.  
40.         return false;
41.     }
42.  
43.     public void Reset(){ throw new InvalidOperationException(); }
44.  
45.     public void Dispose()
46.     {
47.         // possibly some kind of disposal for whatever types needed it
48.     }
49. }
50.  

Since the Anonymous method assignment from actions.Add(() => DoThing(_i)) is using the reference from the specially constructed class, and since it isn't called yet, all DoThing(_i) calls will get the same _i value once the actions execute their functions.

Again, this is purely a guess!!!

 

Code (CSharp):

1. IEnumerator Start(){ //when changed to "void Start()" the sequence runs as expected
2.         yield return null;
3.         int _i = 0;
4.         for(int i=0; i<3; i++){
5.  
6.             Debug.Log("Adding DoThing() for index " + _i);
7.             actions.Add(()=>
8.                         {
9.                 DoThing(_i);
10.                 _i++;
11.             });
12.         }
13.        
14.         Init();
15.     }

 

Ah, but it is a guess that would make sense -- and as I'm typing this, approved by @superpig himself!

I'm trying to remember if I have an IDE installed on my Windows partition right now. Perhaps I can take a peek tonight, but as I'm new to this whole compiling-to-DLL thing I'd appreciate it if either of you could give a whirl as well. =)

Thanks!

 

Interesting; thanks for posting! Unfortunately, while this works for this exampe case, my real-life scenario is a bit more complicated. =/ I guess I could make it work like this, but it would become a bit unwieldly to maintain I'm afraid.

 

So far, I've only learned to use C# with Unity and nowhere else XD

I'm not 100% familiar enough with .Net to do something like what is requested, unfortunately. I spend most of my time studying the Unity documentation, discussions in the discussion section of the forums and working on my game XP

So methods that return an IEnumerator in .Net define this behavior, not specifically Unity? I was under the assumption that Unity was responsible for the custom c++_iterator<> instance... unless there are other workings happening under-the-hood with the compiler that I'm not quite understanding for this scenario.

I could have sworn that the representitive for this Unity video about Coroutines said that it was behavior specific to Unity, but upon rewatching the part I thought I remembered hearing it, he simply states (around 12:30 to 12:40) that "Calling [the] IEnumerator [function] returns an IEnumerator which is basically a pointer to the start of the function" - he doesn't explicitly state that this behavior is Unity-specific, so I'm now assuming this is a .Net thing?

Sorry if I'm simply adding confusion here. I'm very curious about this myself. Please correct me if I'm misinterpreting your statement.

Edit: Listened again. 11:04 in makes it a bit more clear (I glossed over it because I was making an attempt to find a specific part of the speech). So the C# compiler is responsible for this, as you've said! So it is a .Net thing... very interesting.

Edit 2: How did I miss this in the .Net specification???

"Technical Implementation

Although you write an iterator as a method, the compiler translates it into a nested class that is, in effect, a state machine. This class keeps track of the position of the iterator as long the For Each...Next or foreach loop in the client code continues.

To see what the compiler does, you can use the Ildasm.exe tool to view the Microsoft intermediate language code that is generated for an iterator method.

When you create an iterator for a class or struct, you do not have to implement the whole IEnumerator interface. When the compiler detects the iterator, it automatically generates the Current, MoveNext, and Dispose methods of the IEnumerator or IEnumerator<T>interface."

 

@Stoven - this is a mono/.net thing, not a unity thing, I'll confirm that for you.

And yes, it works pretty much how Stoven described it. The method gets turned into its own anonymous class in which all the local variables of the method actually are fields of this class. That is how their states are saved, and this model isn't perfect. If you mix all sorts of anonymous syntax sugar together, things like this happen. You're mixing anonymous/lambda functions with iterator methods, and it doesn't handle it the way you expect.

If you have to have these lamdas inside the coroutine. Pull that bit of code out into its own method. The compiler instead of creating state fields for all those local variables will instead just call the method, and it will work as expected.

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections.Generic;
4.  
5. public class CoroutineTestScript : MonoBehaviour {
6.  
7.     private List<System.Action> _actions = new List<System.Action>();
8.  
9.     System.Collections.IEnumerator Start ()
10.     {
11.         yield return null;
12.  
13.         FillUpActions();
14.  
15.         Init();
16.     }
17.  
18.     private void FillUpActions()
19.     {
20.         for (int i = 0; i < 3; i++)
21.         {
22.             int j = i;
23.             Debug.Log("Adding DoThing() for index " + i.ToString());
24.             _actions.Add(() =>
25.             {
26.                 DoThing(j);
27.             });
28.         }
29.     }
30.  
31.     private void DoThing(int i)
32.     {
33.         Debug.Log("Doing the thing " + i.ToString() + "!");
34.     }
35.  
36.     private void Init()
37.     {
38.         for(int i =0; i < _actions.Count; i++)
39.         {
40.             _actions[i]();
41.         }
42.     }
43.  
44. }
45.  

 

Thanks everyone for the helpful links and explanations! Putting the for-loop in its own method indeed solves my issue, as does just placing the lambda in its own method only (which is more suitable for my scenario). But that got me thinking: Why not just keep going with this, and scope all the things! 0/ ? (Or in non-meme-speak: I'd rather keep this functionality local; without creating a specific top-level function for it.)

Code (csharp):

1.  
2. using UnityEngine;
3. using System;
4. using System.Collections;
5. using System.Collections.Generic;
6.  
7. public class CoroutineTestScript : MonoBehaviour {
8.     List<System.Action> actions = new List<System.Action>();
9.  
10.     IEnumerator Start (){
11.         Action<int> AddAction = new Action<int>((int index) => {
12.             Debug.Log("Adding DoThing() for index " + index);
13.             actions.Add(() => DoThing(index));
14.         }); //local method with its own scope!
15.  
16.         yield return null;
17.    
18.         for (int i = 0; i < 3; i++){
19.             AddAction(i);
20.             //or if you want to get *really* conservative:
21.             new Action<int>((int index) => {
22.                 Debug.Log("Adding DoThing() for index " + index);
23.                 actions.Add(() => DoThing(index));
24.             })(i);
25.         }
26.  
27.         Init();
28.     }
29.  
30.     private void DoThing(int i){
31.         Debug.Log("Doing the thing " + i.ToString() + "!");
32.     }
33.  
34.     private void Init(){
35.         for(int i=0; i<actions.Count; i++){
36.             actions[i]();
37.         }
38.     }
39. }
40.  

So yeah, problem solved at least! I am still curious about the results of a Microsoft compiler compiled DLL, but reading this I would expect it to yield (pun not intended) the same results.

Anyway, thanks again to everyone! This was a fun thing to explore and I learned a bunch! =)