Unity 4.5.2f1 Crash on Standalone Build [Out of Memory]

I just upgraded my project from unity 4.2.2 (or whatever the last 4.2 version was) and I am now getting a crash when trying to build the game. I have made no other changes to the project.

Every time, the crash is due to being out of memory. I know this is a unity issue, as I have 32 GB of ram and unity is crashing when it hits ~3.6 GB of usage. I also notice there are typically 5-7 UnityShaderCompiler threads running at the time of crash.

I have not been able to locate anything in my project that would cause the crash. The build process typically crashes on level 1, and have asset references to 95% of all my assets in level 0.

I have submitted a bug report, Case #00158630 and Case #623288 (once I emailed and the other I submitted my editor log).

I have not been able to find anyone else with this issue, so I was wondering if anyone has run into this and what I can do to fix it. It's hard to develop a game when you can't create builds.

 

Unity is a 32-bit application, so will only be able to access less than 4G of memory. When Unity asks for memory from the OS and gets no memory, Unity will crash with a window showing how much memory each of the various memory subsystems are using. Typically the easy way to move forward is to reduce the size of your textures by 50%, since textures often use the most memory.

Also, 158630 isn't a bug report. It's an email into the support team.

 

Thanks for the response Graham.

I don't think it's a texture issue, as I was able to create builds perfectly fine in unity 4.2. The only changes I made when going from unity 4.2 to 4.5 was editing a little bit of code that broke with the new version of mono.

Also, my total number of textures are way less than 4 GB. I believe I have ~1 GB of raw texture data, which is about 300 MB in the runtime. My build sizes for the whole project in Unity 4.2 were ~1.2 GB.

Did something change with how unity creates it builds? I also see that unity 5 will have a 64 bit editor. Any new on when I can expect that? I can always downgrade back to unity 4.2 and wait for unity 5, though I don't typically like being the first to update.

 

I didn't say it was a texture issue, just that's the most common cause that customers have when Unity runs out of memory. (And shrinking textures is relatively easy, and means running a smaller app can be done without a lot of pain.)

4.5 will have some new features, so it's likely that the runtime is a bit larger. If your 4.2 app was close the the memory limit, it could be that with 4.5 you just hit that limit with no change to your code. :-(

 

Is the only fix to reduce the quality of the game (reducing texture size) or wait for a 64bit editor?

 

Well, if you know that every texture is needed at the full mipmap level, then reducing texture size will impact the visual look of your game. Since you've said you only have 300M of textures in the run-time, it makes me think that you're running out of memory due to other reasons. I have no insight into what they might be. If you get a log file written with the various memory system memory usage then share that here and I can help you come up with some other experiments to make. Waiting for the 64-bit editor doesn't make a lot of sense since no-one knows when that will ship. Finding parts of the game where you can save memory makes the most sense to me.

(But, if it was me, I'd find the 10 largest textures, shrink them by 50% in x and y, and make a build and see if that still crashes. The editor log records the size of various assets as it makes a build, so you could compare before and after and see what difference it makes. Should take 15 minutes. If the game obviously looks quality reduced following this experiment, then keeping those 10 large textures might be mission critical.)

 

I have attached my editor log, which is a fresh restart and instantly hitting build.

It is showing it used ~242.3 MB of memory building level 0, which is my load screen level that I load up 95% of my assets in. The crash happens at the main menu level, Level 1, which only has some music as additional assets.

 

So I tried your suggestion, and I was only able to get a build to work, without unity crashing, by doing the following steps.

1. Reduce ALL textures by 50%
2. Turn all mesh compression to high.
3. Remove every level besides the load screen
4. Delete half the games assets.

After all that, unity will produce a build, but it's close. It uses ~3.2 GB of memory do make the build.

There is no way this is normal, or that I was just barely ok before. Is there a possible fix?

 

I was able to get a build with just my load screen, but all game assets present.

It seems the issue is the new way that Unity 4.5 handles shaders. Before, the shaders would be compiled and cached before runtime. Now, all shaders are being compiled as part of the build process. This dramatically increases the amount of memory required for larger projects.

I was able to get the build by making smaller builds, allowing the build process to build up the shader cache, and then adding assets and repeating. By breaking my assets into 3 groups (and doing 3 builds) I was able to get a 'full' build without crashing. Once I add in another level though, I get the same crash.

 

Is there an approved way of reducing the required memory needed to make a build. Would Asset bundles fix this?

 

Exporting assets as asset bundles and removing the assets from the build is a solid, reliable way of reducing the memory consumption used whilst the build is happening.

 

I don't understand how this is a solution. People are claiming they were able to build properly in older versions of Unity and now the solution is nerf the quality of something that WAS working.

 

I've been encountering this same issue for several months now. As far as I can tell at this point, there appears to be an internal memory leak in Unity 4.5 and 4.6 when making builds. I'm working on a rather large game (~30 GB in Assets, each scene uses ~2-2.5 GB when playing). We've been developing in 4.3 for many months and never had any problems with builds. We attempted to update to 4.5 and when build the project with no changes Unity runs out of memory and crashes.

When building each new scene, the memory usage spikes. In 4.3 this spike goes back down before the next scene is processed. In 4.5 the memory only goes back down part way. The memory continues to increase until eventually the Editor crashes.

This is still happening in 4.6 as well. We are unable to move past 4.3 and gain the benefits of newer versions of Unity because of this issue.

I created a ticket on FogBuz in regard to this on 10/11/14 but it hasn't been looked at: 637027_hmc18cjittlbjvgi

Graham, is there any chance you could comment on this or raise awareness of the ticket in question? If this is indeed an internal bug, it's an extremely problematic issue.

 

I think UT may have rushed a bit with the release of 4.6... I updated to 4.6 not long ago and remade all of my main menu's with the new 4.6 GUI just to realize that 4.6 has a weird error with freezing my editor after I edit a script in Monodevelop. Haven't tried any other IDE's... Simply because I love Mono.. I don't want to switch and shouldn't have to lol. Anyways, I hope all goes well on your end. I'm sure an update will come soon that will help you. As for me, I'm done with Unity until someone comes along bearing some actual GOOD news about it.

 

My guess is it's the shader system. I was able to track my issue to the changes in the shader compiling system in 4.3 and I know there were more updates to the shaders in 4.5

It seems the only fix is to wait until unity 5 comes or partition everything out into asset bundles and manually control your build process and asset loading.

 

Yeah, I refuse the asset bundle method lol. Plus, since I'm not a Pro user, I don't have the Asset Bundle option.

 

The bug was submitted using 4.5 but the editor.log and editor_prev.log that was included come from your 4.3 version. When the 4.5 crash happens, often you'll have a window appear that shows the various internals Unity systems and how much memory they consume. Can you screenshot that and reply to your bug number and include it? Alternatively, see if the editor.log that gets written does have the same textual information in it. These things will help the QA team work out what's going wrong.

 

Yeah, those 21 beta releases we made for 4.6 does show a rushing to get a release out the door. ;-)

 

Thanks for your response. I've replied to the bug confirmation email I received and attached new logs.

I'm not entirely sure which window you refer to. The only one I've seen in regard to this issue is one that prints memory usage broken into a few categories like so:



Is this the one you mean? If so, that's always printed to the log as well it seems.

 

Lol yeah i see what you're saying.. But the are still quite a few bugs, and a lot were still present with RC3.

 

I believe I've narrowed down the cause. Shader compilation. Starting with 4.5 shaders are compiled during the build process instead of at import time. This compilation is using exorbitant amounts of memory.

Short Description
Compiling shaders adds several gigabytes of memory usage to the build process resulting in memory crashes. I copied all of the shaders in our game to a 4.6 test project. Building without the shaders uses ~400 MB. Building with the shaders uses ~1400 MB. Also the memory never goes back down after the build.

Long Description
Deleting all shaders from our existing project instantly fixed the build crashes. With the shaders removed, Unity's memory usage during the build process looks exactly like it does in 4.3. Memory bounces between 1000 MB and 1500 MB while building levels and doesn't steadily increase. With the shaders added back, Unity creates a UnityShaderCompilation.exe process for each CPU core. At the end of building each scene, these processes spin up and presumably compile the needed shaders. Each time this happens there is a massive spike in memory usage. On the order of 500-1000 MB. This memory usage never goes back down. Repeat over several scenes and eventually the Editor runs out of memory and crashes. Building in 4.3 requires ~1500 MB of memory. Building in 4.6 crashes ~75% of the way through at ~3600 MB.

This leads to new questions

• Is this normal or intended behavior?
• Can this behavior be fixed or avoided?
• Is this a problem with a specific shader or a general issue with the new system?

I've replied again to the bug case email and I included a demonstration project that just has a bunch of shaders and nothing else. Building this project shows these weird memory behaviors.

 

Or help us understand the problem you have? Share the project, tell us how you make a build, let our QA team investigate?

 

I've told QA about this thread, and the bug case number.

 

Thanks for your help.

 

(Don't think I did anything, but yeah, that screenshot was what I meant.)

 

I am having this problem since 4.3 too, it crashes every time I try to compile... my game is only 125 MB so it is ridiculous that it should crash...

Could you please add a unity update which just add's an option for compiling shaders the new way or the old way? its not like this is a one off case this is all over your forums it really needs to be addressed. one guy could get that done in a day, all that needs to be done is a single button added.... Even I could do that in one day.
Meanwhile tons of people are defecting to ue4 which is fully functional despite being in an earlier stage of development.

 

another thing, the box that comes up to send error reports always crashes on my computer so I cant send them that way...

 

If you were crashing in 4.3, it's a different problem. The shader system changed in 4.5.

Further, that's almost certainly not a task that can be done in a single day. You're talking about reintegrating an old, inefficient system and maintaining 2 completely different methods of compiling shaders. Further, that means twice as many places for bugs, twice as much maintenance work, etc. That would be a terrible solution.

The fix for the shader compilation memory problem has been found and is queued up for a patch release. If that really is your problem, it will be gone soon enough.