Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Out of memory

Discussion in 'Editor & General Support' started by Demostenes, Jan 11, 2011.

  1. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    I am working with big unity packages and lots of textures and I see "out of memory" very often. During package import, during browsing textures, during changing settings to normal map...
    Just example:

    http://img824.imageshack.us/img824/1724/crashz.jpg

    This out of memory appeared when unity had cca 269MB in task manager.

    My studio is planning huge project in Unity pro and we are starting to be scared. These are the most basic functions, which everybody uses and they are not stable. What other problems are there?
     
    Last edited: Jan 11, 2011
  2. Shawn[QS]

    Shawn[QS]

    Joined:
    Dec 8, 2009
    Posts:
    46
    Demostenes,

    We have been using Unity in some of our development for over 2 years now and I have never seen this message. Are you certain there is not something else running as well that is consuming your system resources? What are the specs of the development rig where you are running Unity? What OS and Unity version are you running?
     
  3. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Latest unity pro, 4GB ram, winxp sp3 32-bit, athlon x2, lots of free memory (no junk running on background). The normal map was big (4096x4096 - we prefere to use large textures and downscale them in unity to be more flexible for future), but it shouldnt crash unity even if I had 1GB of physical ram.
    I receive out of memory bug quite often, especialy when browsing textures.
     
  4. robert

    robert

    Moderator

    Joined:
    Dec 21, 2008
    Posts:
    265
    We are aware of the issue. It's related to how the memory is allocated during texture import, so you might get bumpy experience when you're importing more big textures at the same time.

    Even though your system reports you still have available memory, the memory is too fragmented and large chunks cannot be allocated, which is what the texture importer needs.

    We are working on some major improvements to memory allocation patterns in the Editor and are hoping to release them not long from now (not with the release that's just around the corner, but with the next one that comes shortly after).
     
  5. Don-Gray

    Don-Gray

    Joined:
    Mar 18, 2009
    Posts:
    2,278
    Good to hear, thanks Robert!
     
  6. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Perfect, thanks.
     
  7. Shawn[QS]

    Shawn[QS]

    Joined:
    Dec 8, 2009
    Posts:
    46
    Haven't even seen the problem and a solution is already in the works! Thanks for all the hard work robert!
     
  8. novo

    novo

    Joined:
    Oct 23, 2010
    Posts:
    6
    Hallelujah. This problem has been plaguing me for months and I have not found a workaround other than to run my texture importer script multiple times on small sets of images. This would be the single biggest help for me right now.
     
  9. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Out of memory import story continues (3.2 pro), when importing unity package (cca 30MB), i always crash on one texture:

    http://dropmocks.com/mRPpM

    So still not fixed in unity 3.2. When is next release?

    I am not able to start my project again, it always crashes. How can i fix it? What should i delete, to be able to properly start?
     
  10. Frank Oz

    Frank Oz

    Joined:
    Oct 13, 2010
    Posts:
    1,560
    Try the texture you said it always crashes on when importing, move it to another folder outside the project until you're done importing then bring it back, see how it goes then?
     
  11. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Texture is in unity package. I will try to delete whole unity package.
     
  12. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    4 months later and still nothing? We have still problems with crashes during building cache when opening large projects for the first time (Unity 3.3).
     
  13. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    Any news about this issue?

    Demostenes, Did you managed to go over this problem some how?

    I'm really frustrated becuase deadline is coming and I'm getting this "out of memory" stuff quite easily by just compiling script while editor is in "play" -mode and example when editor in play mode selecting from gameobject which contains several textures "debug"-view. And this happens in Mac's and Windows machines...

    It's almost impossible to work with Unity3d tool in project which contains huge amount of textures....
     
  14. Spectre9000

    Spectre9000

    Joined:
    Aug 30, 2010
    Posts:
    170
    Judging from what's been said, I'd suggest smaller textures and increasing your memory if possible. The problem seems to be with allocating chunks of memory large enough to handle huge textures.

    Realistically, your textures should be as small as possible and reused as often as possible, but perhaps you need that gigantic texture and perhaps you need lots of them for whatever reason. Well, in this case you just have to treat it like you're developing on a console and begin optimizing your game towards the memory barrier. Memory is an issue that's rarely touched on with PC's due to PC's having vastly more memory than consoles and handheld devices, but it is still there and you have to decide how best to handle the situation. Even with Unity using a more optimized method for handling large textures, you're still going to have a memory issue you'll have to face if your game is large enough. Also, you need to keep in mind what your consumers will be using, which is probably a lower end machine than what you're working on and you need to plan for that as well. Looks like you'll have to learn some memory management and how to incorporate it into Unity.
     
  15. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Moving to a 64 bit windows 7 os may help some of these issues, if only that there is more clear memory for unity to use. Unity might only be able to address half your ram, but at least your os and other tasks can sit in the other half.
     
  16. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Nonsense. Out of memory is Unity BUG. Bug must be repaired, not decreasing visual quality of game! I really hate this pasive atitude. Sollution is kick unity teams ass, not destroy your game.
    Memory leaks in general are bugs. Once they are there, they will sooner or later happen, no matter how intensively I use application.

    Texture memory issue can be solved by Diegos VT system. This is technology exactly for this. Or it can be solved by some custom resource manager, to load/upload assets according to need. Every bigger game MUST have something like this and it is shame, that this is not integrated in unity core. Decreasing texture quality is probably the most stupid idea, it solves nothing.
     
    Last edited: Jul 17, 2011
  17. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Agree, 64bit OS with plenty memory helps a lot. It is not sollution, but at least you can work.
     
  18. Spectre9000

    Spectre9000

    Joined:
    Aug 30, 2010
    Posts:
    170
    Out of memory is a Hardware problem (Assuming the algorithm isn't grossly inefficient, which I'm giving UT the benefit of the doubt it isn't. Even still it'd be a combination of both at best). Even the most efficient algorithms can't run if there's not enough actual memory. Sometimes you have to do some actual optimization on your stuff. A game engine and plugins/extensions aren't magical and still have limitations such as memory size.

    The issue isn't a memory leak, but an inefficient allocation algorithm which has hardware limitations even if it was efficient.

    Minimalizing texture size is exactly what AAA titles do constantly. Giant textures do not equal good graphics. Ask any artist out there. A 4k x 4k texture is insanely large. Most character models are 2046 x 2046 and look superb at this level. Some look great at 512x512. Furthermore, you're not even using them, as you said yourself, you're downsizing them anyway, which decreases the need for such a large texture.

    While Unity should strive for more efficient algorithms for memory allocation, they can not give you unlimited memory, and you need to do your own optimization for your game (which you should do regardless of whether you have enough resources or not).
     
  19. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    Unity has 3 cases of out of memory problem:

    1. Out of memory on baking PVS: Thats known and also confirmed as bug, Umbra and UT are working on it. Additionally its affected by the problem on 2.
    2. Out of memory on Beast, especially since 3.3: This problem is partly due to the 32bit nature, but since 3.3 additionally because the beast process no longer seems to be compiled with /LARGEMEMORYAWARE flag which would allow a 32bit process to use up to 4GB RAM on a 64bit OS with enough RAM. Its not exactly a bug, but as this affects the editor, umbra and beast, I've supported the 64bit editor since its creation on the feedback page and adopted the avatar and will keep it until this is resolved. As a user of a 18GB Core i7 920 system, I definitely hate to see this case of out of memory.
    3. Out of memory generally in scenes: not a bug, but more due to the lack of streaming. Unity will load the whole scene oyu are in, point. This limitation, and all its consequences to the editor and runtime are the reason why I personally keep the PoV that Unity does not support any streaming at all, asset bundles are no streaming, they are just async loading mechanisms thats considered standard for some years now. This problem here is basically unsolvable until unity gets proper range based streaming which is nothing that will happen from one day to the other - chances that we see a 64bit editor prior that point is significantly higher and it would postphone the required solution by some years thanks to 12GB ram costing $200 and less


    The case of memory leaks causing out of memory is naturally a bug, but I personally haven't seen any such yet on the editor, not even with our 8GB projects running on its 2M++ verts scene with the editor not being closed for days at all (at which point I would suspect any kind of leak to rear its head)
     
  20. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    OMG, are you kidding, dont you? You should learn something about memory management in OS, garbage collectors and programming in general. I am really sorry, but this is bunch of nonsenses.




    AAA games are lowering res of textures and this is reason, why everybody is implementing some kind of VT system :)
    512 textures makes nowdays sense only on very small objects, or tiled. Go and play some AAA game, you would be surprised with resolution of textures. Resolution of texture is increasing quality tremendously, personaly I am using 4k textures on all big objects and I have no performance or memory problem. Now we are switching to Diegos VT system, to be able to use even bigger and of course drastical lowering of memory budget and texture streaming is perfect too, it solves part of streaming problem. And yes, our game is looking great, I was really surprised, how far it can get, if you use all tools properly.
     
    Last edited: Jul 18, 2011
  21. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Exactly my words, lack of proper streaming is probably biggest Unity problem now, because it is impossible to make big game without writing such system. And still you are limited by one stupid Unity thread. I am quite curious about promised multicore skinning and multicore renderer, it should help these issues too (main thread will have more space for loading etc..).
    Anyway I am not sure, if there is chance to write some resource manager (for loading/unloading textures, meshes, etc..) which will run on separate thread, so it can preload these items on backround with low priority and not lag game by any loading in main thread.

    I had never problem with "out of memory" during loading/playing scene, even on my large project (now 16GB of assets and still growing). I had problems only during import textures (which is confirmed bug sooner in this thread Posted: 11:13 AM 01-12-2011), or during browsing project. Switching to 64bit OS helped A LOT, memory is probably less fragmented since I have 16GB of memory.

    64bit editor does not solves much, it will of course make work more comfortable, but since you cant built game for 64bit, it does not solve lack of resource manager. Other problem is, that the day, when every gamer has 64bit OS is quite far in future, it will IMHO take at least 5 years, until 32bit OS vanish.
     
    Last edited: Jul 18, 2011
  22. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    There is a problem and at least unity have acknowledged its an issue and being worked on. Meantime we can look at ways to enable us to keep working. That could mean using dummy textures that are a quarter of the size until the issue is resolved (it could be it's fixed around autumn if 3.5 is the target).

    Also if you're constantly using massive textures, a texture paging scheme may suit your game better regardless (or megatextures). Might want to peek at http://forum.unity3d.com/threads/96...turing-for-Unity-Pro-RELEASED!-Starting-at-99 ... Not saying unity is off the hook, but it may actually be a better solution for your game overall anyway.
     
  23. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    The lag does not come from the loading, that part is resolvable with AssetBundles load async function. Thats already running in an own thread since 2.6.

    The lag comes from the texture being seen by the camera first at which point the driver has to upload it to VRAM, thats a blocking operation and the only way to overcome that would be unity uploading the texture in blocks over multiple frames as far as I'm aware instead of waiting till they are needed and then bomb the whole thing up (this is the reason behind the 10-15 initial stutter many games are suffering from), but this logic would again base on some streaming prediction mechanism, an engine can not arbitarily decide to start to stream in texture blocks.
    At the time the only way to minimize this halt is indeed smaller textures and/or higher compression, ie the less bytes need to be sent to the VRAM, the less time it takes, the shorter the halt.

    You can't overcome this with own code without going for something that replaces whole textures and reduces the amount of them (the linked Amplify basically) which overcomes the memory limitation and serious slowdown on some machines too, because you can't control the upload nor do it yourself with sub image regions as unity does not expose the low level access to blit only areas, apply always updates the whole texture

    For static scenes its easy to overcome it with loading screen and "load enforce cam", but when streaming you can't do anything about it.
     
  24. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Well, I think there is other problem. We were trying our predictive resouce loader with Amplify, so textures were loaded separately. And loading started MUCH sooner before real rendering happened, our aim is to have everything in memory before it is seen by player. I think there was some problem with mesh loading and unpacking in memory, this operationg caused noticable lag for bigger meshes (or groups).
     
  25. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    if you load from resources, then yeah, you are doomed. With asset bundles meshes can actually stream.
    The worst thing hit wise when loading from resources through are animated meshes. When you expect a large world, NEVER pack animations into the meshes, use the @AnimationName based system instead to reduce the size of distinct meshes you need as animations take a small eternity to load due to the sheer size of data they bring along (assuming you didn't use animation compression or couldn't cause it was too destructive)
     
  26. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Thanks for advice, this will be usefull for sure. Recently I was thinking, which animation system we will use, so this solves it. I am really pissed, that such information are not in documentation.
     
  27. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    I don't have HUGE world with enormous amount of textures ( only huge amount for iphone game ) and meshes in game view.

    I have 3 texture atlases sis 1024*1024, 1 tranparency 256*256, not any big meshes. In scene I have maximum of 4 skinnedmeshes ( optimised ) and texture size for each is 256*256. If I check my whole project size, it's right now 1.7Gb. I don't anyhow think that this is too massive project to handle for Unity or even my HW. Anyway this game is going to iPhone, so everything is done in low-poly, animation rig, meshes and everything is in such a smal size as possible.

    And ofcourse there is some elements in UI shown. But game is anyway running 30fps without problems on target HW. So I don't think that I have HUGE amount of textures comparing to PC game. That's why it is so weird that this editor doesn't want to stay in non-crashing mode :D.

    I ported this project back to 1.7 vesion of unity ( I started development in there ) , this whole package is running nicely in editor without ( almost any ) crashes. You can pause and edit scripts and it acts like real development tool.
     
    Last edited: Jul 18, 2011
  28. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    Ok, I got answer from Q&A team ( They might be little annoyed to amount of fatal errors in error db.. ). I don't have any other windows active while Unity3d crashes because it stucks whole machine, so I can use that spare time to fill out error reports...

    They cannot reproduce this ( or at least it's quite hard )without me to send my current project to them, so fix won't be anytime soon available.

    This experience with issue definately affect to my opinions about current gameengine in use. I just cannot trust that if I end up to have again such a problem it just disappears..It's so frustrating to have deadline and each crash is taking few minutes away. I have now adapted this mantra :"Stop Play before doing anything to scripts". It ain't that easy to remember when used to edit script at any time. Well, I just got to live with this situation.

    Here is some stats rendered on screen when last crash happened: DrawCalls:11, batched: 13, Tris: 1.5K, Verts 1.9k, UsedTextures: 12-16.7MB, VRAM 1.7 to 20....It ain't that huge amount of stuff on screen or anywhere else, that it should crash...And it crashes even if I have just rebooted the machine and no other windows active. Blame definately is somehow in my project files, because other development machines we have experience same situation...
     
  29. Chub

    Chub

    Joined:
    Mar 14, 2010
    Posts:
    247
    There is your problem right there 32 bit operating systems only support up to 3.5gb of ram, so you have got 4gb but only 3.5 can be used.
     
  30. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    According activity monitor last crash details: Just before crash Unity uses ( unity is in play mode ) ( Real.mem )255.4Mb and when crashed it shows 504Mb...Definately something is leaking memory inside Unity...And all I did between these memory checks is small edit to script, save it and selecting Unity icon. I really hope that this issue is in my project and I can easily solve it when find, but problem is that I don't have any idea which might cause this...
     
  31. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    This cant be your fault. Once you are using unity from "inside", all leaks are fault of Unity. This is typical problem with fragmented memory, there is some memory peak, garbage collector reacts too slow a and Unity crashes, becase it does not wait.
     
  32. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    Yeah, this definately is problem in engine. Unity MUST not crash because of this sort of issues, or at least give some more indication what is problem. "Out of memory" without any indication which might be the root cause is not helping to avoid this scenario anyhow. And even avoiding with success this problem would ease up my development at this phase huge amount. I mean that now I have seen this problem about 3 times / hour and each crash is stealing my development time 3 minutes -> 9minutes from 60 minutes are going to surviving from this "out of memory" -error. If there is anything in side my project, which I could quickly modify and this problem disappears, it would be a HUGE help to keep deadline ( and nerves, my computer is @ risk to fly out from window ).
     
  33. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    Right now Unity is impossible to use. It crashes to "out of memory" when scene is playing and clicking my precalculated pathfinding node array container. Other way to reproduce crash always is to put any of objects into Debug mode and ofcourse simple way to crash is just to edit scripts.

    Well, most of the errors I have reported to Unity has been transfered to closed state, so fix definately will be available soon...\o/...hah..
     
  34. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    I am not surprised. They are adding lots of new features and reparing only obvious bugs. This kind of bugs is hard to track and until there is more people, nobody cares....
     
  35. ooppari

    ooppari

    Joined:
    Jan 29, 2009
    Posts:
    68
    Some update to my problem. I might found root cause from my project which is causing it. Now I'm able to edit scripts without error while in "Play" - -Mode, I can put Object even to Debug mode. Cause is really interesting. Nothing related to textures, or texture sizes. Not even near. It's because of some C# script files.

    Here are key issues, which caused my puzzle:
    [System.Serializable]
    List<>

    Not a good combination together. Leaks..memory...heavily :D. Luckily I didn't buy new HW, or decreased size of my textures.

    Changed to use ArrayLists, now my project works fine without crashes. Wasn't easiest task to find, but at least I have my nerves back...
     
    Last edited: Jul 20, 2011
  36. Demostenes

    Demostenes

    Joined:
    Sep 10, 2010
    Posts:
    1,106
    Thanks for warning.
     
  37. BigBulle

    BigBulle

    Joined:
    Oct 16, 2010
    Posts:
    17
    This is crazy :/
    I'm going to try your solution oopari
     
  38. BigBulle

    BigBulle

    Joined:
    Oct 16, 2010
    Posts:
    17
    @Demostenes:
    Is it not posible for you to divide your scene in several scenes and use Application.LoadLevelAdditiveAsync ?
    I know that you have to decide when to load scenes and when to delete them but it seems to be not too complex.
    I've also seen that Amplify hasn't solved all your problem.
    Is it because the memory is mainly used for meshes and not textures? What amount of memory could save thanks to Amplify?
     
    Last edited: Jul 26, 2011