Memory-leak galore

Discussion in 'Unity Support' started by Boondock_Saint, Nov 10, 2011.

  1. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Below is a cross post from: http://answers.unity3d.com/questions/184680/memory-leak-gallore.html

    At our company we have several projects relying heavily on Unity. We used all the tricks out there (http://answers.unity3d.com/questions/129364/memory-management-tips.html), but somehow when we close our webplayer, one title still has 150 MB and another has 200 MB of unreleased memory (both in Firefox and Chrome). What is worse, is that when you start a new game, without killing the previous plugin container, it simply adds to this memory residue, so it's not a question of caching. The editor too, appears to have this problem.

    Now, of course, you can accuse us of bad programming. Sure. We're the worst programmers ever. However, it cannot be that the virtual machine does not release the memory claimed by our games, anno 2011. So my question is twofold:

    * Do other developers recognize any of this?
    * When is Unity3D going to upgrade its version of Mono? (and don't say it's too ingrained in Unity, that's what design patterns are for)

    "Too many heap sections" ---> http://tirania.org/blog/archive/2009/Nov-09.html

    Sorry for the tone, but something needs to be done, since 3.5, at the moment, doesn't hint at any improvement in this area.

    ps: Starting the game again and closing it adds 150 or 200 mb respectively, so the residue increases 150, 300, 450 etc. And remember that users have more than 1 tab open, so machines start swapping rather fast.
    pps: Another thing we've looked at http://answers.unity3d.com/questions/153050/memory-management-and-list.html
  2. tatoforever

    tatoforever

    Member

    Joined:
    Apr 16, 2009
    Messages:
    2,300
    File a bug with a repro case. Unity staff will surely fix it.
  3. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Good point. However, I've found numerous posts on this issue. I refuse to believe that they are not aware of this problem.

    Furthermore, there is a danger in using too many tricks to dance with Unity's GC, because some of them actually contain performance penalties, if Unity decides to upgrade its GC....
  4. larvantholos

    larvantholos

    Member

    Joined:
    Oct 29, 2009
    Messages:
    668
    A lot of time people complain first, and report as a last resort, reporting first would probably get a lot of these bugs fixed ;)
  5. tatoforever

    tatoforever

    Member

    Joined:
    Apr 16, 2009
    Messages:
    2,300
    I'm on the Unity beta list, if you could provide me with something that repro those issues i can make a thread on the beta list and your bug will get some direct attention (though isn't necessary but I'll make sure they will hear me first). ;)
  6. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Yes, but Unity developers also read and post on the forum and the Answers section.

    But we are working on a bug report, simply to get that response out of the way ;)
  7. tatoforever

    tatoforever

    Member

    Joined:
    Apr 16, 2009
    Messages:
    2,300
    Awesome,
    Let me know if/when i can help you.
  8. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    That's very kind of you, tatoforever! Maybe we will :)

    In the meantime: Have you never ever encountered the "Too many heap sections" error message?
  9. larvantholos

    larvantholos

    Member

    Joined:
    Oct 29, 2009
    Messages:
    668
    When they told me the all you can eat buffet was not actually....

    Oh programmer stuffs :)
  10. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
  11. Julien.Lynge

    Julien.Lynge

    Member

    Joined:
    Nov 5, 2010
    Messages:
    87
    Thanks for the poll link. Just went and cast my vote there as well.

    I have seen numerous issues with the garbage collector in the games / visualizations we've been working. I was getting a huge (100s of MB) leak like Boondock_Saint when trying to use List<>s to read in a binary file and use it to build multiple meshes. Every time I would Clear() or otherwise try to reset the List<> for the next run of the loop it would simply add to the existing memory rather than replacing it, and this problem existed even after closing Unity, both in the editor and standalone app.

    Additionally, the GC often doesn't play nice at all with the community ocean model, causing large garbage spikes that freeze the game, though I haven't poked through it enough to be certain it isn't just a bug with the code.
  12. Demostenes

    Demostenes

    New Member

    Joined:
    Sep 10, 2010
    Messages:
    1,107
    I reported similar memory leak more than 1 year ago (it was there since unity 3.0 beta). According to change log it was fixed in 3.4.1 (Fixed texture importer rescan memory leak, which caused out of memory crashes.), but according to my testing it is still there (I was able to crash unity by browsing textures in large project). So my answer is no. Until there is 50 people on the forum, who are complaining each day, no effort to fix this. There are more important tasks then fixing bug, adding lots of fancy features and xbox and ps3 support, this is what makes money!

    Anyway there must be filled bug report, or the chance of fixing is ZERO.
    Last edited: Nov 13, 2011
  13. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Yes, we also suspect the textures to be part of it.

    Concerning the bug report: We are one of the bigger customers of Unity, so we're going via the direct route. I don't know exactly what that will look like yet (I'm not the contact between Unity and the company), but I'll keep you posted.
  14. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Ok, it's been communicated to Unity. Now it's hoping for a reply :)
  15. goat

    goat

    Member

    Joined:
    Aug 24, 2009
    Messages:
    2,946
    In my experience, since you are running this in a browser, I would suggest you publish it to run stand alone and compare the metrics with the browser version. I bet the problem goes away, especial since you are using Firefox and it's plugin container. Try running it in other versions of other browser and older versions of Firefox.
  16. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    No, Chrome has the same problem. Also, it is a browser game, so we don't have much choice in the matter. Even if a standalone doesn't have this problem, that still means the problem has to be solved for the webplayer, and even if Internet Explorer doesn't have this problem, we cannot ignore the users that are using Firefox and Chrome :)
  17. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    It just means that you must use a smarter trick to force Chrome and Firefox to really close the plugin.

    It seems at the time they play 'smart and optimized' by not releasing the plugin as such the old one is not released as it relies on plugin termination to do it. Out of head the only way that I could think of as a working way is a full intermediate page without it so it can't auto optimize the plugin handling 'to keep it alive' as it would do with a normal page refresh (a full force refresh should reload the whole container too)
  18. goat

    goat

    Member

    Joined:
    Aug 24, 2009
    Messages:
    2,946
    Is this chrome plugin a java container? Firefox used to be nothing more than a java program wrapped in a jvm exe. I bet Chrome is too. Although it's a browser game I would still compile it for standalone compare. If standalone has the same problem you can bet Unity will be all over it. I'm not really a windows guy but if you run it in osX attach whatever osX's equivalent to 'truss' is to the browser game instance.
  19. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Ok, I'm not exactly sure what you mean. Loading a new page in the same tab should destroy the plugin container, but actually closing the tab (as most users will do) doesn't?

    Also, as mentioned in the GP, there is no caching there. Refreshing the page (so starting the game new), only adds to the memory residue.

    For the record: Of course we put all kind of GC calls into Main.OnApplicationQuit(), but this brought nothing, except that it took a bit longer to close the tab (indicating that the calls were executed).
  20. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    The plugin is the plugin downloaded from unity3d.com. If it's wrapped in Java, then it's done by UnityTech. However, for the sake of argument, I'll build a standalone.

    Also: Firefox a Java program?!?! Do you have a source for that? (It's written in C++)
  21. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Ok, I'm still hacking around to build a stand alone, it's a bit complicated because only our client is build in Unity and all our assets are streamed.

    Goat, what should I be looking for with the standalone? The webplayer is a process in a process (the browser), a standalone gets its memory directly from the OS. Of course the memory gets released, because Windows' memory management is much more rigorous than a browser's.
  22. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Wow, it runs much smoother in standalone ;)

    Having said that, of course no memory resides, because there is no browserplugin. To my knowledge there is no way of terminating the plugin container from Main.OnApplicationQuit().
    Last edited: Nov 28, 2011
  23. goat

    goat

    Member

    Joined:
    Aug 24, 2009
    Messages:
    2,946
    Hi Boondock,

    My source was a co-worker at Fort Knox so if you say it's in C++ I'll take your word on it. I thought it odd but then at the time Firefox was so slow so it sounded feasible. I never checked.

    So know you know it is the plugin / browser and not your app. To be sure can your app be left running overnight standalone and you check it's status?

    As far as your game being so much faster stand-alone while we'd like to blame everything or Flash or Java let's just say fishing, trojan's and all the other junk browsers have to watch out for has made them slow.

    Unity and all browser plugins must all be running in some type virtual machine (I guess it's browser dependent). I would guess by the time your game is running there are severals layers of VMs wrapped around it.

    Run it in your browser and attach to the plugin. The plugin should be a vm or some sort you can inspect. Maybe it has it's own CLI you can attach to...ask Unity support about such type interfaces...

    Maybe this browser plugin problem is a know problem at Unity and so...you have the upcoming publish to Flash.
  24. goat

    goat

    Member

    Joined:
    Aug 24, 2009
    Messages:
    2,946
    At the end of your game put a message, 'Please close this browser window.' and maybe run your game in a pop-up window to encourage that.The when the Flash publish comes out publish to that instead.
  25. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    Thats not going to change it likely, cause flash is a standalone plugin too so if the browser does not do his job right it likely means it affects all.
    If it affects only unity report it with an example that shows it in an extreme form so they can add a 'startup handle' that force kills the previous memory.
  26. Boondock_Saint

    Boondock_Saint

    New Member

    Joined:
    Nov 10, 2011
    Messages:
    15
    Thank you for your lengthy reply. Yes, plugins run in a virtual machine. For Unity this VM is (based on) the Mono VM.

    The point is this: If a program running in the VM is terminated, the VM should return ALL of the memory used by that program. Currently it doesn't because this old version of Mono suffers from heap fragmentation (see link in OP).
    Therefore, I and other users, would like to know when Unity is going to upgrade its version of Mono :)
  27. protatoe

    protatoe

    New Member

    Joined:
    Mar 12, 2012
    Messages:
    3
    I have serious memory issues with 3.5 that I did not have in 3.4 as well. I get the feeling it was rushed to be ready for GDC. This is on a empty project with standard assets imported, nothing custom.
  28. zumwalt

    zumwalt

    New Member

    Joined:
    Apr 18, 2007
    Messages:
    2,186
    Just a thought, don't rely on OnApplicationQuit with the browser, you can only possibly control this leak internal to the game per say, since all game objects are in essance (in a stack), you need to simply use a "quit" button that redirects the user to a new "thank you for playing" level, which in turn dumps out the prior stack as long as you do not allow the "keep alive" between levels. (few hints here), now, the new level of "thank you for playing" should have triggered Unity to dump the prior stack, in total your game, if left in memory with the browser closed, will take up no more than say 5 meg, although still possibly progressive.

    Since you are a larger company with direct access to internal resources, I would imagine that they will contact you directly to resolve with a new plugin beta, either way, if you danced the OnApplicationQuit, got all "gameobjects" in scene and called destroy, that would move the object collection to the internal Unity garbage collector for destruction without worry.

    Whey you close the browser window, all game objects are still in "active" mode without a destroy going through them then a thread terminate call, that is bypassed by the window close event thus causing the memory leak. Unity tries to destroy all game objects as it shuts down the program, as far as I can tell anyway, after that is cleaned up then it terminates all active threads internal to it self. There is no idle self destruct built into the browser in the event that the browser has been closed, otherwise end task but that is not an option for players.