ExecutionEngineException on iOS only ??

Discussion in 'iOS Development' started by Alchemyst, Nov 30, 2011.

  1. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    I'm running some custom networking code (not sure if that's related or not), but said networking code makes use of the C# Interlocked functionality. The following exception is only thrown when running the app on iOS hardware (IPad 2, 5.0)

    ... it works flawlessly on Windows and on Mac.

    Any idea why this is blowing up on iOS? :confused:

    ExecutionEngineException: Attempting to JIT compile method '(wrapper managed-to-native) System.Threading.Interlocked:CompareExchange (Shared.LogMessageDelegate,Shared.LogMessageDelegate,Shared.LogMessageDelegate)' while running with --aot-only.
     
    Last edited: Dec 2, 2011
  2. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    It looks like this is actually happening anytime I try to hook up a .net style event handler in an external C# assembly...i.e.

    Logger.Log += new LogMessageDelegate(Method);

    Is this not supported?
     
  3. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    As iOS does not allow dynamic code generation, all code for iOS is AOT compiled which has various restrictions especially along the line of reflection and dynamic evaluation of the real content.

    Also don't forget that iOS does not have the full .NET subset, so just cause it works on win or osx has no meaning for iOS development (not even that it works on android has that, as Android is a JIT platform, not an AOT)
     
  4. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    Got it, but I'm still having trouble understanding how hooking up an event handler (located in a standard DLL assembly) via the += syntax causes a JIT compilation. I thought JIT compilations are only a danger in certain instances concerning Generics and Generic collections.

    No? I was under the impression that events are supported on iOS.
     
  5. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    Events normally work and I've never seen it fail either.

    Your error though also points towards threading which you can't use with unityengine.object extending classes for example (but that would not result in an exception on assignement, it would crash upon calling)
     
  6. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    Looke like the problem is related to DLLs created in Visual Studio 2010.
    ------------
    Full-AOTing a VS2008 built .NET assembly and using an event within said
    assembly works fine. However if the assembly is built in VS2010, all other
    things being equal, it does JIT code during event subscriptions,
    Specifically within 'Interlocked.CompareExchange<EventHandler>'.

    I used reflector on two basic .NET assemblies, one built with VS2008, and
    the other with VS2010, and it turns out the default generated add-remove
    event methods have changed:

    ##### 2008 #####

    Code (csharp):
    1. public event EventHandler SomethingHappened;
    2. [MethodImpl(MethodImplOptions.Synchronized)]
    3. public void add_SomethingHappened(EventHandler value)
    4. {
    5.     this.SomethingHappened = (EventHandler)
    6. Delegate.Combine(this.SomethingHappened, value);
    7. }
    8. [MethodImpl(MethodImplOptions.Synchronized)]
    9. public void remove_SomethingHappened(EventHandler value)
    10. {
    11.     this.SomethingHappened = (EventHandler)
    12. Delegate.Remove(this.SomethingHappened, value);
    13. }
    ##### 2010 #####

    Code (csharp):
    1. public event EventHandler SomethingHappened
    2. {
    3.     add
    4.     {
    5.         EventHandler handler2;
    6.         EventHandler somethingHappened = this.SomethingHappened;
    7.         do
    8.         {
    9.             handler2 = somethingHappened;
    10.             EventHandler handler3 = (EventHandler)
    11. Delegate.Combine(handler2, value);
    12.             somethingHappened =
    13. Interlocked.CompareExchange<EventHandler>(ref this.SomethingHappened,
    14. handler3, handler2);
    15.         }
    16.         while (somethingHappened != handler2);
    17.     }
    18.     remove
    19.     {
    20.         EventHandler handler2;
    21.         EventHandler somethingHappened = this.SomethingHappened;
    22.         do
    23.         {
    24.             handler2 = somethingHappened;
    25.             EventHandler handler3 = (EventHandler) Delegate.Remove(handler2,
    26.             somethingHappened =
    27. Interlocked.CompareExchange<EventHandler>(ref this.SomethingHappened,
    28. handler3, handler2);
    29.         }
    30.         while (somethingHappened != handler2);
    31.     }
    32. }
    33.  
     
  7. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    Further research indicates (elsewhere on the internet) that calling the offending BCL method yourself somewhere in your code sould force the AOT compiler to precompile the method. I've attempted to do so with the above referenced method:

    System.Threading.Interlocked:CompareExchange (Shared.LogMessageDelegate,Shared.LogMessageDeleg ate,Shared.LogMessageDelegate)

    This didn't work and the iOS build still blew up when hooking up an event. I will install VS2008 and will attempt to rebuild the libraries there. I'll see if that's what it comes down to.
     
  8. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    You could go the save path and not compile the DLL with VS at all and use MonoDevelop instead (just use the same sln), then its as compatible as it can be.
    That it should work like this on VS 2010 vs 2008 sounds strange, unless you built it for .NET 4 where it would make sense due to the auto parallelizing compiler, because both use the same compiler normally which is part of the .net installation.
    But .NET 4 DLLs shouldn't even run in Unity, only .NET 2 DLLs (I only use VS 2010 for WP7 dev, the rest is done through VS 2008 Pro, so I can't speak from experience here)
     
  9. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    Okay. I've spent another day trying to track this down. My conclusion is that it is not possible to hook up a C# event handler on iOS if that event resides in a 3rd party C# DLL. At least not since Unity 3.0 when they upgraded to the newer Mono version (not sure about before then).

    Apparently, the .NET runtime switched from using "lock" when appending event delegates (when you use += to attach a new event handler) to using a lockless version that calls Interlocked.CompareExchange, which for some reason isn't AOT'd during the AOT compile pass and so iOS blows up on it every time you try to attach an event handler. It works just fine on Windows/Mac. Specifically, this is the error:

    Code (csharp):
    1.  
    2. ExecutionEngineException: Attempting to JIT compile method '(wrapper managed-to-native) System.Threading.Interlocked:CompareExchange (iOSTest.MyEventDelegate,iOSTest.MyEventDelegate,iOSTest.MyEventDelegate)' while running with --aot-only.
    3.  
    4.   at iOSTest.EventTest.add_TestEvent (iOSTest.MyEventDelegate value) [0x00000] in <filename unknown>:0
    5.   at TestScript.Awake () [0x00000] in <filename unknown>:0
    6.  
    - I have tried compiling the code on Windows with VS and on Mac with MonoDevelop
    - I've tried changing the .NET/Mono runtime from 3.5 to 3.0 to 2.0. Still no dice.

    I have created a test project that illustrates this error. Attached.

    This is quite bad, for me, because I've invested a great deal of time and money developing a client library for Unity, which relies significantly on events. It will cost a great deal more to convert it to an eventless system. I hope there is a magic compiler switch or something to that effect that would allow me to solve this problem. I'll submit a bug report, I suppose.
     

    Attached Files:

    Last edited: Dec 2, 2011
  10. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    I have found a (rather annoying, but doable) workaround. Essentially, you have to define your own _add / _remove handler for the delegate invocation list to prevent the compiler from generating the code that for some reason isn't included in the AOT pass.

    Code that doesn't work on iOS looked like this (from the previously attached package/zip):
    Code (csharp):
    1.  
    2. using System;
    3. namespace iOSTest
    4. {
    5.     // Event handler method signature
    6.     public delegate void MyEventDelegate(string msg);
    7.    
    8.     public class EventTest
    9.     {
    10.         // Delegate / Handler method signature
    11.         public event MyEventDelegate TestEvent;
    12.                        
    13.         public void FireEvent()
    14.         {
    15.             // Fire the event if a delegate has been hooked up.
    16.             if(TestEvent != null)
    17.             {
    18.                 TestEvent("Event fired successfully.");
    19.             }
    20.         }              
    21.     }
    22. }
    23.  

    Fixed code looks like this:
    Code (csharp):
    1.  
    2. using System;
    3. namespace iOSTest
    4. {
    5.     // Event handler method signature
    6.     public delegate void MyEventDelegate(string msg);
    7.    
    8.     public class EventTest
    9.     {
    10.         // Multi-cast function pointer.
    11.         MyEventDelegate TestEventInvoker;
    12.        
    13.         // Delegate / Handler method signature
    14.         public event MyEventDelegate TestEvent
    15.         {
    16.             add
    17.             {
    18.                 TestEventInvoker += value;
    19.             }
    20.             remove
    21.             {
    22.                 TestEventInvoker -= value;
    23.             }
    24.         }
    25.            
    26.         public void FireEvent()
    27.         {
    28.             // Fire the event if a delegate has been hooked up.
    29.             if(TestEventInvoker != null)
    30.             {
    31.                 TestEventInvoker("!!!Test Message Successful.!!!");
    32.             }
    33.         }
    34.        
    35.        
    36.     }
    37. }
    Why does everything in iOS have to be SO painful? ;(
     
    Last edited: Dec 2, 2011
  11. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    has nothing to do with iOS. Affects all AOT platform (so all consoles too for example).

    But its interesting if you are really right that 2010 is causing it, would be another reason on the list on why I am on 2008 and will stay there ;) (it might have to do with the C# changes along .net 4 that the generated cil differs. As 2008 is up to C# 3.5 its in sync with unity, as .NET 4 / C# 4 etc is not supported at all)
     
  12. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    Yes, you're absolutely right - it's not specific to iOS - rather it's an issue with all AOT platforms. Although I'm guessing that means Android should be exempt from this bug since it's a JIT platform.

    Also, I've come to the conclusion that it has nothing to do with the IDE, but rather with the .NET runtime version that the MSIL (DLL) is being AOT compiled against - which in the case of Unity3 is the version that generates the Interlocked.CompareExchange code.

    All in all, it looks like when the AOT compiler emits code there are some instances where it doesn't AOT compile the code that it emits - at least in the case of events.

    After looking into it some more, this may be a mono bug that could have been resolved in a later version than what Unity is using.
    https://bugzilla.novell.com/show_bug.cgi?id=444218#c1

    So... in conclusion to anyone with the same problem reading this - Since there is a strong indication that this is a Mono runtime bug, I doubt this will be getting fixed any time soon (unless there is some magical way of building the package which resolves this issue).

    Use the workaround of coding your own _add/_remove methods that's posted above.
     
    Last edited: Dec 2, 2011
  13. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    Unity does not use MS .NET at all, its mono 2.6 which is .NET 3.5 and lower + an own custom AOT compiler from Unity Technologies potentially (they wrote their own for unity iphone 1.x and have an own custom mono 2.6 version)
     
  14. Alchemyst

    Alchemyst

    New Member

    Joined:
    Jun 1, 2010
    Messages:
    51
    This is quite trivial, but might help someone who's running into this... just a quick C# console script to generate the necessary event signatures (the resulting signatures are thread safe, for whatever that's worth). I've tested this on iOS, OSX and Windows.

    Just switch out "delType" and "eventName" variables and run. Resulting code is automatically copied to clip board. Just paste it into your CS file.

    Code (csharp):
    1.  
    2. using System;
    3. using System.Text;
    4. using System.Windows.Forms;
    5.  
    6. namespace EventHandlerGenerator
    7. {
    8.     class Program
    9.     {
    10.         static StringBuilder sb = new StringBuilder();
    11.         static void w(string msg)
    12.         {
    13.             System.Diagnostics.Debug.WriteLine(msg);
    14.             sb.AppendLine(msg);
    15.         }
    16.        
    17.         [STAThread]
    18.         static void Main(string[] args)
    19.         {
    20.             string delType = "EventHandlerDelegateType";
    21.             string eventName = "EventName";
    22.  
    23.              w(string.Format("#region {0} Event", eventName));
    24.              w(string.Format("private {0} {1}Invoker;", delType, eventName));
    25.             w("");
    26.             w("/// <summary>");
    27.             w("/// Your comment here");
    28.             w("/// </summary>");
    29.             w(string.Format("public event {0} {1}", delType, eventName));
    30.             w("{");
    31.             w("add");
    32.             w("{");
    33.             w(string.Format("AddHandler_{0}(value);", eventName));
    34.             w("}");
    35.             w("remove");
    36.             w("{");
    37.             w(string.Format("RemoveHandler_{0}(value);", eventName));
    38.             w("}");
    39.             w("}");
    40.             w("");
    41.             w("[MethodImpl(MethodImplOptions.Synchronized)]");
    42.             w(string.Format("private void AddHandler_{0}({1} value)", eventName, delType));
    43.             w("{");
    44.             w(string.Format("{0}Invoker = ({1})Delegate.Combine({2}Invoker, value);", eventName, delType, eventName));
    45.             w("}");
    46.             w("");
    47.             w("[MethodImpl(MethodImplOptions.Synchronized)]");
    48.             w(string.Format("private void RemoveHandler_{0}({1} value)", eventName, delType));
    49.             w("{");
    50.             w(string.Format("{0}Invoker = ({1})Delegate.Remove({2}Invoker, value);", eventName, delType, eventName));
    51.             w("}");
    52.             w("");
    53.             w(string.Format("private void Fire{0}()", eventName));
    54.             w("{");
    55.             w(string.Format("if ({0}Invoker != null)", eventName));
    56.             w("{");
    57.             w(string.Format("{0}Invoker();", eventName));
    58.             w("}");
    59.             w("}");
    60.             w("#endregion");
    61.  
    62.  
    63.             Clipboard.SetText(sb.ToString());
    64.  
    65.         }
    66.     }
    67. }
    68.  
    69.  
     
    Last edited: Dec 2, 2011
  15. arunr

    arunr

    New Member

    Joined:
    Jan 17, 2012
    Messages:
    2
    Hi Guys,
    I'm having the exact same issue - have a dll built from VS2010 and I get the AOT compile error when I run my game on iOS (on the IPAD2 device). I'm going to try the alternative that Alchemist suggested (with add/remove ) methods. Thanks for the solution though, I was almost beginning to think I should get rid of events from my code.
    arun
     
  16. KvanteTore

    KvanteTore

    New Member

    Joined:
    Feb 16, 2010
    Messages:
    87
    Just want to say thank you to @Alchemyst. You've just saved me hours upon hours of painful AOT debugging!
     
  17. sunkas85

    sunkas85

    New Member

    Joined:
    Aug 16, 2012
    Messages:
    14
    Hi

    Are experiencing the same problem trying to use the XMPP framework to work with iOS using Unity/Monotouch/C#.

    This is the error I get:

    Code (csharp):
    1. ExecutionEngineException: Attempting to JIT compile method '(wrapper managed-to-native)
    2. System.Threading.Interlocked:CompareExchange  
    3. (System.EventHandler`1<Matrix.EventArgs>,System.EventHandler`1<Matrix.EventArgs>,
    4. System.Eve ntHandler`1<Matrix.EventArgs>)' while running with --aot-only.
    5.  
    6.   at Matrix.Net.BaseSocket.add_OnConnect (System.EventHandler`1 value) [0x00000]
    7. in <filename unknown>:0
    8.   at Matrix.XmppStream..ctor (StreamType type) [0x00000] in <filename unknown>:0
    9.   at Matrix.Xmpp.Client.XmppClient..ctor () [0x00000] in <filename unknown>:0
    10.   at TestFacebook.setup () [0x00000] in <filename unknown>:0
    11.   at TestFacebook.Start () [0x00000] in <filename unknown>:0
    12.  
    13. (Filename:  Line: -1)
    Most likely from this line:

    Code (csharp):
    1. xmppClient.OnBeforeSasl += this.xmppClient_OnBeforeSasl;
    where xmppClient_OnBeforeSasl is my own custom method:

    Code (csharp):
    1. public void xmppClient_OnBeforeSasl(object sender, Matrix.Xmpp.Sasl.SaslEventArgs e)
    2.     { ... }
    How to apply your workaround on my example? Can't see any += in your case.

    Best Regards
    Jonas Andersson

    Edit: Looking at your workaround again, it looks like you are changing the code of the library? What if it is in a dll library?
     
    Last edited: Aug 21, 2012
  18. gnauck

    gnauck

    New Member

    Joined:
    Aug 22, 2012
    Messages:
    1
    Above there was posted that those problems don't exist when the code is compiled with VS2008. Is this true? Our code does not depend on 4.0 and could be copiled also with VS2008. Is this a solution?

    Currently we compile with default settings in Mono Develop under windows which causes the same problems. Are there any compiler settings for Monodevelop to fix this?

    Edit: The same code works on MonoTouch without any problems and changes. So what is MonoTouch doing different?

    Thanks,
    Alex
     
    Last edited: Aug 22, 2012
  19. MadDave

    MadDave

    New Member

    Joined:
    Jul 9, 2012
    Messages:
    6
    Thanks @Alchemyst, saved us a lot of trouble and guessing!
     
  20. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,589
    Mono touch has a different mono and aot compiler than Unity. Both use their own fork of Mono for this purpose.
     
  21. JaredThirsk

    JaredThirsk

    Member

    Joined:
    Jan 17, 2010
    Messages:
    32
    What a nightmare! I use events everywhere. I don't think this workaround is anywhere near feasible. This is disappointing, as I thought it may be reasonable to get my C# code working on AOT for iOS. I have already been over perhaps 1000-2000 hurdles in my pre-existing codebase, trying to make the code AOT ready by avoiding generic methods and other restrictions, and creating back ports from .NET 4. To get this far and discover that there is a mono bug that was fixed years ago, but Unity is lagging behind the times (by over 4 years?!!) is really disappointing.

    Can UT get the fix from Mono and patch this problem?

    Perhaps another workaround is to combine all my DLLs into one. I may give that a try.
     
  22. JaredThirsk

    JaredThirsk

    Member

    Joined:
    Jan 17, 2010
    Messages:
    32
    I came up with a workaround. I have been waiting to get an excuse to decompile and mangle DLLs using Mono.Cecil, and this is it. This little program will replace add_/remove_ methods with versions that resemble the simpler versions without it. I did not put on the Synchronized attribute, and this is probably not thread safe at all. Use at your own risk!

    https://gist.github.com/4358405
     
  23. mbolt

    mbolt

    New Member

    Joined:
    Oct 10, 2012
    Messages:
    20
    I have still not been able to resolve this issue, even after using @JaredThirsk s DLL processor. It seems that there are a lot of cases where events are used in my DLL, but for some reason, it only occurs on a few specific ones. I've tried multiple work-arounds without success. It's extremely aggravating. Does anyone have any other solutions?
     
  24. chsu

    chsu

    New Member

    Joined:
    Sep 26, 2012
    Messages:
    4
    Many thanks to Alchemyst for diving into this problem. This would have been super difficult otherwise.

    I hope UT can provide a fix for their cross compiler, since it doesn't appear to be a problem on monotouch.
     
  25. JaredThirsk

    JaredThirsk

    Member

    Joined:
    Jan 17, 2010
    Messages:
    32
    I have updated my compatibility tool and created a project on github: https://github.com/jaredthirsk/AOT-Compatlyzer

    One new feature is I allow keeping my old generic method invocations with compile time type checking, if my tool can detect an alternative non-generic method to use as a replacement.
     
  26. giusepe

    giusepe

    Member

    Joined:
    Oct 30, 2009
    Messages:
    4
    Hello Everyone,

    After some time trying to solve this same issue, a friend of mine showed me the solution. And while this may be not the solution to everyone, seem to be for the most part of this thread:
    Just compile your DLLs with unity compiler and not VS or XS.

    To do this, just take a look on:
    Code (csharp):
    1.  
    2. /Applications/Unity/Unity.app/Contents/Frameworks/Mono/bin/mono /Applications/Unity/Unity.app/Contents/Frameworks/Mono/lib/mono/2.0/gmcs.exe
    3.  
    Other little tweak that may help sometimes, is just increase the number or trampolines.
    Take a look here: http://monotouch.2284126.n4.nabble.com/Understanding-the-impact-of-trampolines-td4495086.html

    Hope that helps.
     
  27. coeing

    coeing

    Member

    Joined:
    Mar 17, 2011
    Messages:
    22
    Thanks a lot for the tool JaredThirsk. It's really a shame that the bug is still not fixed by Unity after all those years.
     
  28. guavaman

    guavaman

    Member

    Joined:
    Nov 20, 2009
    Messages:
    787
    End of 2014 and this evil, evil issue is still alive and well. Thanks so much @Alchemyst for digging so deeply into this and explaining the cause in detail. This really needs to be added to the iOS troubleshooting page in the docs to save us devs a ton of wasted time.
     
  29. flintmech

    flintmech

    Member

    Joined:
    Sep 29, 2011
    Messages:
    7
    I also want to chime in as having this issue. Compiling a DLL with VS2012 or Unity 4.6's built-in MonoDevelop results in the exact same problem. Thank you @Alchemyst for the investigation, reporting, and workaround.
     
  30. guavaman

    guavaman

    Member

    Joined:
    Nov 20, 2009
    Messages:
    787
    I didn't realize it happened with MonoDevelop also. That's very good to know. Thanks!
     
  31. Smilediver

    Smilediver

    Unity Technologies

    Joined:
    May 5, 2011
    Messages:
    65
    With AOT issues like these, the issue is that compiler doesn't know (or fails to detect) that it needs to AOT some method. In most cases this happens with generic classes/methods. Usual workaround is to "give a hint" for compiler to force AOTing. So for instance with error like this:

    Code (CSharp):
    1. ExecutionEngineException: Attempting to JIT compile method '(wrapper managed-to-native) System.Threading.Interlocked:CompareExchange(iOSTest.MyEventDelegate,iOSTest.MyEventDelegate,iOSTest.MyEventDelegate)' while running with --aot-only.
    you can see that System.Threading.Interlocked:CompareExchange generic method with those specific types wasn't AOTed. You can try to declare a dummy class, and explicitly invoke this method. Something like:

    Code (CSharp):
    1. public class ForceAOT {
    2.     public void Dummy() {
    3.         iOSTest.MyEventDelegate d1 = null;
    4.         iOSTest.MyEventDelegate d2 = null;
    5.         System.Threading.Interlocked.CompareExchange(ref d1, d2, d2);
    6.     }
    7. }
    I don't know if this will work in this particular case, but worth a try.