Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[Official] The collected il2cpp forum topic.

Discussion in 'General Discussion' started by RalphH, May 20, 2014.

  1. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    In following of our blogpost about the future of scripting and il2cpp, we wanted to centralize discussion about it here, next to the comments. We're very excited to be talking about this. We'll all be watching this thread and answering your questions; the blogpost is explanatory about the topic, so we'll pick up what ever comes up in questions here !
     
    shkar-noori and Rodolfo-Rubens like this.
  2. Carpe-Denius

    Carpe-Denius

    Joined:
    May 17, 2013
    Posts:
    842
    Do I read that right, that the first platform will be WebGL, the second iOS? Do I miss "PC, Mac Standalone" or is that intended for a future release?
     
    Fab4 likes this.
  3. [RV]CWolf

    [RV]CWolf

    Joined:
    Jan 24, 2013
    Posts:
    33
    Thanks for a great blog post there. IL2CPP sounds very exciting but, I'm with Carpe Denius here, I'm a little worried that 'PC, Mac, Linux' standalone doesn't seem to get a huge amount of attention.
     
    shkar-noori likes this.
  4. TylerPerry

    TylerPerry

    Joined:
    May 29, 2011
    Posts:
    5,577
    I'd assume that's because IOS is already AOT.
     
  5. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    The plan is definitely to move more platforms over to il2cpp. AOT platforms like iOS are an ideal first goal, though, as

    1. they are AOT already, so you can expect a more seamless transition, and

    2. because they are AOT, they cannot use mono according to the LGPL, which is currently preventing us from updating mono. So once we have AOT platforms on IL2CPP, there is nothing stopping us from using the latest mono in the editor and other platforms.
     
    landon912, ArnoC, Cynicat and 4 others like this.
  6. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,137
    Seriously?!? I mean... seriously?!?
     
  7. Carpe-Denius

    Carpe-Denius

    Joined:
    May 17, 2013
    Posts:
    842
    Can you specify that question?
     
    4fthawaiian likes this.
  8. TylerPerry

    TylerPerry

    Joined:
    May 29, 2011
    Posts:
    5,577
    Do consoles already run on a different base or are they Mono? Probs will be a big wait if all the consoles need to be ported before the normal platforms.
     
  9. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,649
    I don't believe the consoles have restrictions that prevent LGPL code from being used, so I imagine they're using Mono like everything else.
     
  10. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,137
    - VM machines for every platform which means binding people on those VM for several months
    - Reimplementing the entire CLI background (not the entire toolchain... at least, they didn't do that...)
    - Going from JIT to AOT (AOT is platform dependent, so you ideally need a compiler and SDK for that platform)
    - WebGL as first platform (WHO is going to use that if you have to pay this... or are you going to drop the webplayer?)
    - Going from C#/UnityScript/Boo(?) to IL to CPP to machine code adds an additional layer of possible "S*** that can go wrong"
    - Still in research...
    - ... and thus could be not viable on specific platforms... which means that the difference between the platforms can and will be increased as we cannot be sure anymore that the code on one platform runs smooth on another one.

    Knowing Unity and their history of long turn around times, we can expect the scripting features go downhill for a while before climbing back up the hill again... has been like that with any other feature Unity has provided up to now... will not change in the future (unless you prove me wrong)...
     
  11. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    Internally, most development is done using Standalone players as they are easy to build, debug, and test on. So, they are receiving more direct attention than most.

    However, the existing Standalone player is not currently AOT and exposes the most .Net functionality of any supported player. That means

    1. An IL2CPP of the Standalone player will have less functionality than current Standalone as it restricts features to AOT compatible ones
    2. The chances of a project working 'out of the box' in Standalone is less as existing projects will be using some things that don't work in AOT mode
    3. Mono is most robust and stable on x86/amd64, so IL2CPP is not as in demand on those platforms

    tl;dr; Standalone players work internally with IL2CPP as well as any other platform, but they are not the focus for IL2CPP in the near term. I'd guess that we may ship them as an option alongside mono Standalones for you to test/debug with.
     
  12. tteneder

    tteneder

    Unity Technologies

    Joined:
    Feb 22, 2011
    Posts:
    174
    il2cpp sounds very exciting!

    I'm using Unity views as a part in a bigger, native iOS/Android app quite often. The communication is from CPP to C# is via MonoBindings.

    In a future with IL2CPP, will it be possible to hook up custom C++ code? In other words, will we have access to the unity generated code BEFORE final compilation? Or is the only way to do so by loading runtime libraries?

    thanks!
     
  13. Dantus

    Dantus

    Joined:
    Oct 21, 2009
    Posts:
    5,667
    Thanks a lot for sharing that information! This is really appreciated!
     
  14. AnomalusUndrdog

    AnomalusUndrdog

    Joined:
    Jul 3, 2009
    Posts:
    1,551
    I forgot about that. So stuff like reflection won't work anymore with IL2CPP?
     
  15. shkar-noori

    shkar-noori

    Joined:
    Jun 10, 2013
    Posts:
    833
    @RalpH so standalone are less stable in IL2CPP?
     
    Last edited: May 20, 2014
  16. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    We already have this with Mono, in a much harder to maintain situation

    Yes, we are not trying to reinvent the wheel.

    Yes, the choice was to leverage all the work being done by the C++ compiler vendors rather than writing and maintaining all the code generation, optimization, etc ourselves for each architecture in mono.

    I won't go into pricing, but dropping the web player is not our choice as mentioned many other places. At least 2 top browsers are going to be disabling plugins which is out of our control.

    Almost all this applies to Mono, or any JIT/compiler

    I'd say things have been going downhill and this is our best attempt at resolving that.
     
  17. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    I'm not completely surprised by this announcement, having been at the GDC WebGL talk. They hinted at other possible uses for the technology, but it's definitely a bold direction to move in. While I actually do find this to be very cool (I love C#), it sort of begs the question... why not just allow side-stepping to c++? I mean, it's not my favorite language, but it would not be the end of the world for me, and it would enable those who want the performance to get it.
     
  18. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    Reflection will work. Stuff like System.Reflection.Emit will not. No dynamic code generation at runtime.
     
  19. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    Most of reflection will work, just not dynamic code generation.
     
  20. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    You can already write C++ when you need it via plugins. You can't script directly in C++, but you can easily move performance intensive logic there if you need it.

    The problem with exposing C++ as a 'scripting' language is that you still need to deal with all the rules of the .Net ecosystem (garbage collection for example). We have found internally that writing C++ which plays by the rules is extremely hard.
     
  21. jemast

    jemast

    Joined:
    Dec 7, 2011
    Posts:
    141
    First thanks for sharing the insight on this future tech!

    In my opinion this is great news. Seems like win-win to me, you're regaining control of key technology and provide what looks to be a much improved solution (both on the users side and on your end). Sure it seems overkill to have to reimplement your own Mono VM and AOT but seeing the potential benefits it seems like it's the right thing to do for the future.

    Congrats on the hard work!
     
  22. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    I'll be happy if 4.5 has WebGL beta il2cpp in it. I have some easy tests.
     
  23. darkhog

    darkhog

    Joined:
    Dec 4, 2012
    Posts:
    2,218
    Yeah, ILCPP is slick, but why concentrate on iOS first? While I understand why it first should be for WebGL (new target + optimization for web games weren't always good from my experience, excluding simple demos), choice of iOS as second platform baffles me. If you want to go about improving performance of mobiles, go for Android (most of Android device aren't necessarily as powerful as iDevices). Not only you'd target platform with biggest market share around the world, but also performance improvement will be visible, especially on lower-end devices.

    But I'd prefer next platform that will support IL2CPP to be desktops, as even with simple objects that have physics enabled and some game logic attached (tried to do game like Tiny Big), it has big performance impact - and I know it is because scripts, because when I remove them from objects in question, it makes game run smoothly, even though they still have physics.
     
  24. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    WebGL won't be available in 4.5. It will be released as early access in 5.0.
     
  25. genail

    genail

    Joined:
    Jul 2, 2012
    Posts:
    1,045
    How do you plan to compile the code for a target platform? What I mean is that now you can build a Mac binary on Windows and vice versa, but if you're planing to use target platform compiler wouldn't it be a little more constrained?
     
  26. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    Absolutely. And frankly we still have a lot of work to do there. But as Jonathan said earlier, our main development and debugging on il2cpp is already done on the standalones today; cross compiling is not something highest on the priority list, and it is likely we'll have (an upgraded) mono runtime on standalones for some time, along side an option to use IL2CPP.
     
  27. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    Fair enough. My question was more to the second point, rather than the plugin approach. It's just that to an outsider, this method seems to be so much more difficult in terms of UT's effort, though obviously it's basically transparent to the end user.

    EDIT: I think this is definitely cool tech, if you can pull it off on a wide scale. I'm personally cautiously optimistic, though already people are showing skepticism.
     
    Last edited: May 20, 2014
  28. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,137
    Thanks for your replies :)

    It's one things to adapt an existing codebase (i.e. Mono for different platforms) than to rewrite and entire VM/CLI from scratch. So, the number of people you have to bind on those tasks is much higher than improving existing stuff. Doing this platform by platform will require a team of specialists for each platform who cannot be put on other tasks, because of bugfixing/enhancements/improvements/...

    Well... reimplementing CLI and VMs sounds like reinventing the wheel to me.

    So, will you deliver all compilers needed? Do we get all SDKs from you? Or do we have to side-install those (as we already have to do for Android/iOS)? Which compiler on Windows? VS, gcc, Intel?

    Well... then do it the flash way... drop it...

    With the difference that there is an additional CPP layer which doesn't exist now. Plus the AOT compiling... Also, those problems have been solved on the different platforms (using Mono, etc), so it's not really a "research" topic anymore...


    As you've said yourself, the AOT will not be feature complete for the stand alone version... thus you're trying to solve the problem by worsening the situation and then catching up again...

    Don't get me wrong... I see the benefits of rolling it this way... But this new "feature" is currently not what people have expected of Unity. There are too many other things to fix. How in hell can we be sure that a bug is related to the new scripting or to the feature itself?
     
  29. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    To be fair, this does address a number of major issues and complaints people have had for a long time (GC, outdated .NET).
     
  30. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    I think it's the opposite. For mono, we did need this team of specialists for every platform we added. If it's not a platform mono supports out of the box, it can be a huge amount of work, as you practically have to write your own JIT backend for that platform from scratch. I've seen some platform ports we did were the mono porting effort was 3x all the other work of the port. With IL2CPP, we don't have to write a new JIT backend, we can just hook into an existing C++ compiler, which is much simpler.
     
  31. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    Fair enough; and skepticism is fine; we'll have to work to prove this, and we haven't really layed out all pieces of the roadmap in this initial post (we have said what we do know, not what we maybe know, or don't know) There is work to be done there, but at the same time, if we don't talk about this we wouldn't be transparent about what are plans are, right ? And we do have 2 demos in webgl running, for which I can safely state that DeadTrigger 2 is exercising a lot of moving parts in IL2CPP very well.
     
  32. jemast

    jemast

    Joined:
    Dec 7, 2011
    Posts:
    141
    What? I've seen a lot of requests for Mono upgrade (Unity 5 announcement, How can we serve you better topic, ...). To me it seems there weren't a lot of available options since Unity editor and runtime seem to be tied quite deeply to Mono/.NET: either throw an unknown but probably insane amount of money at Xamarin or rewrite the key parts themselves.
     
  33. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    Exactly. That has been standing for a long time and we have been working on this for quite some time too; we didn't want to be to early to talk more about our plans either, nor did we want to wait longer in telling you at least of what we know we will be doing.
     
  34. SixEcho

    SixEcho

    Joined:
    May 14, 2014
    Posts:
    4
    Thanks for the update, this sort of information is very interesting and useful for planning. I appreciate this is an early view and it's a long way out most likely.

    Am I correct in thinking the process would be:
    I write C# which is
    ... compiled to .NET assemblies by (upgraded!) Mono
    ... compiled by Unity's new il2cpp tool to standard C++
    ... compiled by an existing platform compiler to native code
    ... which uses a Unity "VM" (more of a HAL?) to provide a consistent view of target platforms

    Is libgc an improvement over the GC in the current Mono Unity uses?

    FWIW, a C++ intermediate layer like this is the same approach as some other frameworks I've used (Monkey and OpenFL-based Haxe), where it's worked well and given decent native performance.
     
  35. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    OK by me. That makes my easy tests easier.
     
  36. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    I've worked on Mono for 9 years and I have to respectfully disagree. Historically porting mono to a new platform takes almost a year for an *expert* in both Mono and the platform in question. That's assuming you can find an expert in Mono and the target platform. And can keep them around for every feature requiring platform specific implementations.

    Also, the platform specialists are *already* fighting Mono. It's hard to convey how excited the platform teams are for this. They are getting to trade generating, debugging, and maintaining assembly code for C++. I think it was mentioned in the blog post, but ports are happening in terms of days not months or years.


    The only current support platform (WebGL) will come with all compilers/SDKs/etc installed. Any future platform would be on case by case basis.



    I think we are going in the direction that many others are. We started this work before Microsoft announced Native .Net, but it validates the strategy IMO. By restricting the runtime environment a bit (no dynamic code generation), you get a lot of benefits.

    And as I said, no one is making you use this for Standalone. Standalones will get the same updated Mono as the editor once IL2CPP is stable enough on AOT platforms. IL2CPP's main purpose is to serve platforms where AOT makes sense.

    I understand you concern, but again the same situation exists with Mono today.
     
  37. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    Cross-compiling would be nice. One could do all work on PC and avoid upgrading macs every 3 years and when you do upgrade it's be for 1/4th the cost. Apple will love it.
     
  38. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    Oh, I absolutely agree on the point regarding transparency. I think it's a perfectly sound approach to release this information at this stage, especially since there are live demos of the tech in action, even if WebGL isn't inherently a good platform to demonstrate that, given the performance hurdles unrelated to IL2CPP.

    I think, seeing the tech working already is where the "optimistic" part comes in. Getting this working across all the major platforms any time soon... that's where the "cautiously" part comes in.
     
    shkar-noori likes this.
  39. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    Yes, that is basically the workflow.

    It's not radically better, but it's ~8 years newer. It's been cleaned up in that time. It's more portable, more configurable, and more stable. That alone makes it worthwhile. Additionally, I am seeing some performance improvements, even more when you start trying generational/incremental support and parallel marking. We'll continue to investigate and publish results when we have something meaningful.

    Yes, it's a tradeoff but it solves many issues for us.
     
  40. DallonF

    DallonF

    Joined:
    Nov 12, 2009
    Posts:
    620
    Out of curiosity, what kind of features would this limit? Is dynamic code generation used by LINQ? Are there any popular Asset Store addons that use dynamic code generation? It seems a reasonable limitation given the nature of IL2CPP but I'm curious about the impact.
     
  41. pragmascript

    pragmascript

    Joined:
    Dec 31, 2010
    Posts:
    107
    It seems that the technology is very similar to that of .NET Native (http://blogs.msdn.com/b/dotnet/archive/2014/04/02/announcing-net-native-preview.aspx)

    Since .NET Native seems to be further along in development and aims to support the whole and newest .NET runtime wouldn't it be an option to just licence that technology instead of making the huge investment of building something very similar on your own?

    Also even if building it on your own I agree that the far best option would be to open source IL2CPP so that it can benefit from user submitted patches, I assume that the interest in IL2CPP would be rather big even outside the game-development community.
     
    Last edited: May 20, 2014
  42. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    You'll find System.Reflection.Emit in the weirdest places and dealing with this is definitely an ongoing task. Anything on the asset store that does use System.Reflection.Emit at runtime (aka, outside of editor usage), will not work; I don't think this will exclude many things, but some, for sure.

    We have been trying IL2CPP on some of the most popular asset store packages and for those we can confirm they work, but we are going further down that path in a later stage.
     
  43. raybarrera

    raybarrera

    Joined:
    Nov 20, 2010
    Posts:
    207
    iOS, already uses AOT, so you can kind of use that as a baseline for what will and won't work. I think things like sorting using .ThenBy() in LINQ won't work, but most things do work.
     
  44. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Yay Unity adopts C++ as it's programming language! :D
     
  45. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    276
    The goal is to have similar restrictions as Mono AOT has on platforms like iOS.

    The most common case I see for dynamic code generation is related to serializers. Many serialization libraries generate serialization code for types at runtime; but many provide options for doing this at compile time or a slower non-generated path at runtime. I think between .Net Native, Xamarin iOS, and Unity's IL2CPP we'll see more and more 3rd parties avoiding dynamic code generation.
     
  46. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    It is very similar, but it doesn't include the 2 CPP part (maybe that could be added on). IL2CPP has been long underway and while I think it's fair to say that we are happy to say that .NET Native at least validates our AOT approach, for now we will continue to work on IL2CPP, as it also allows us to do fairly Unity specific optimizations and leverage our existing work to output C++, which heavily reduces platform porting cost.
     
  47. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    Funny ;-) But to be absolutely sure everyone reading that snippet understands; that is not what this is about; the name might put some emphasis on the CPP part, but that's because it's a very...descriptive name.
     
  48. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,137
    Ok... I get that... basically, you split the cake in two: instead of writing a JIT compiler, you write a IL/Cpp converter which takes into account the specifics of the target platform (compiler specifics, target platform capabilities, ...). Then a 3rd party provider delivers a compiler (whoever that is) and you/we rely on that compiler to do the right optimization for the code.

    Wouldn't you still have to make sure that the compiler doesn't "overoptimize" and rearrage API calls? How can we determine which layer is responsible for a bug? (i.e. is it due to a bug in the VM/HAL, is it a bug in the translation IL2CPP, is it a general API bug/misunderstanding of the API)
     
  49. Metron

    Metron

    Joined:
    Aug 24, 2009
    Posts:
    1,137
    Well... updating Mono (espacially the GC part as requested by many) and making some alternative, non-memory-allocating calls, or rewrite key parts are two different shoes. Finally it comes back to the estimation: How much does it cost me to do it myself (estimated # of man-months x salary x security value) or buy it from a 3rd party provider.

    As it is now, there has been requests to improve Mono, but seeing this announcement and the responses to the questions tells me that a complete switch from current Mono/JIT to AOT compiling and IL2CPP is still several years away (complete means switch all platforms; feature complete). And this most likely means that there will be no changes to the current Mono/GC issues.
     
    shkar-noori likes this.
  50. RalphH

    RalphH

    Administrator

    Joined:
    Dec 22, 2011
    Posts:
    592
    I'm not sure if we didn't answer your question already in the blogpost; we *are* looking at GC as a whole. We *are* going to upgrade mono. But equally, some platforms won't see that upgrade, but rather go IL2CPP.
     
    shkar-noori likes this.