Building Asset Bundles "Compiling Scripts..." step repeats for each bundle?

Hello,

I'm trying to convert a very large project (100gb project folder, 250+ scenes) to use the latest Asset Bundle system, in hopes of saving build time for builds with small changes between them.

Currently we use the old asset bundle system for a few types of assets, but the scenes are all built with the standard Unity build process. I am trying to convert these scenes (and their required assets) to asset bundles. I've ended up with a process that I think is very similar to how Unity builds scenes by default, with each scene in it's own file (now asset bundle), and shared assets files to go with each scene (separate asset bundle from the scene.)

Now, building these all the first time seems to take 4-5 hours (a regular build for us is about 2.5 hours.) An incremental asset bundle rebuild with no, or very small changes, goes quickly (minutes.) However, during the initial build, and any other build with a decent number of changes, I've noticed while it is building the bundles, it seems to repeat a "Compiling Scripts..." step for every bundle (or at least every scene bundle.) This step in our project takes about 40-60 seconds unfortunately, and because we have so many scenes, that really adds up. The other parts of the asset bundle build steps seem to go very quickly, but then it get stucks on "Compiling Scripts..." again, etc. So in effect, it looks as if the build process is compiling the scripts ~250+ times when we do a full asset bundle rebuild. Is this really necessary? Is it actually compiling scripts in this step, or doing something else? I don't see why they would need to be recompiled for every asset bundle.

We are on 5.3.3p3. I have tried various asset bundle build options (such as DisableWriteTypeTree and IgnoreTypeTreeChanges) and they all seem to behave the same in this respect.

Has anyone else noticed this, and found a way to avoid it, or is there perhaps something Unity could do to improve this?

Thanks

 

I double checked, and it is indeed only asset bundles with scenes inside them that cause this "Compiling Scripts...." step. So, I can speed things up a bit by putting more scenes per asset bundle, but for dependency and other reasons, I would much prefer to have one scene per asset bundle.

 

I've also confirmed that this happens in a sample project, such as the Asset Bundle Manager sample from Unity. Just add extra scenes, and put them in their own bundle, and build asset bundles. You can see the Compiling Scripts step flash by... it goes quick in this case because script compilation is fast, but in a project like ours with 40+ second compile times... it is a real pain.

 

I submitted a bug report for this, 780886

 

Great, glad you guys are aware now! A fix for this would really help out!

 

The bug for this is now public on the issue tracker, in case anyone else has run into this slowing down builds. https://issuetracker.unity3d.com/is...nes-tries-to-compile-scripts-for-every-bundle

 

I don't know if it's a separate issue, but not only does it recompile all the scripts for each of the bundled scenes, it also rebuilds all the associated assets and that takes a huge amount of time. Any minor change to one asset, and it will rebuild all of the scenes (in our case it takes about 20 mins). It even does so for empty scenes.

 

It should be a separate issue, seems asset bundle incremental build totally breaks for you. Could you please open a bug for it with your project? Then we can have a look.

 

Yeah, you're absolutely right. After further investigation, we have found out that it happens somewhat randomly and is not limited to scenes. It won't rebuild the bundles when run immediately after the previous build, but sometimes just 15 minutes later Unity decides a bunch of bundles need to be rebuilt, often even empty scenes and small single object bundles that reference only a model, texture, material and a shader. Looking at Editor.log we can see the list of files included in each bundle and we're sure none of them changed in any way (git also conifrms this).

I would file a bug, but our assets folder is currently 24 Gb, so I think attaching it is going to be problematic. We've tested on a clean project and everything worked fine, so the problem seems to be with our code. Now, is there a way to debug how Unity decides if the assets were changed? Even if there may not be an API for this, maybe we could reimplement the check ourselves (calculating hashes?) to try to catch the moment when this happens and for which files to further detect what causes this.

 

me too

 

Same here, this was ok a while ago but now we get assetbundles forcing a complete script recompile per assetbundle..very slow.

 

Has anyone found a workaround for this? Incremental building for asset bundles seemed to be working for me in 5.3.3p2 but after upgrading to 5.3.4p6 (and then 5.3.5) Unity is re-building untouched asset bundles. I can't seem to find any rhyme or reason to it.

 

Any chance you are using the unity cache server? In some recent patches, I have noticed that having the cache server enabled during the build process causes Unity to reimport many things that it doesn't need to. If you are using it, try disabling it.

The specific issue this thread is about is when a scene does actually need updating, and it needs to build a bundle with a scene in it, it re-compiles scripts for each such bundle.

 

Not using the Unity cache server.

Seems to be something I'm doing on my end. I wanted to detect asset bundle changes (when rebuilding) and then automatically increment an asset bundle version number. The approach I'm taking:

- Compare the main platform manifest (in this case, iOS.manifest) checksum before and after calling BuildPipeline.BuildAssetBundles.
- If the checksum changes, increment the version number that's stored in an xml file (that's also part of an "xml" bundle) and then call BuildPipeline.BuildAssetBundles again.

This seems to work well enough at first. If I rebuild the asset bundles again Unity reports no changes, as expected. However, if I switch to another app and then back to Unity (I'm running OSX) and then rebuild bundles, it rebuilds the main iOS bundle and the CRC value in iOS.manifest changes. I'm guessing that some sort of asset refresh occurs when the app is brought back into the foreground.

I've tried forcing a refresh of the two bundles/manifests (iOS, xml) before second call to BuildPipeline.BuildAssetBundles, but that doesn't seem to help. I'm at a loss and honestly a bit frustrated that we can't just easily make use of the seemingly unused "ManifestFileVersion" value. I'm sure there's another/easier way to go about this so if anyone has suggestions I'm all ears.

 

Any status updates from Unity available on this issue? It still hasn't been fixed yet unfortunately https://issuetracker.unity3d.com/is...nes-tries-to-compile-scripts-for-every-bundle

 

Alright, thanks very much for the status update!

 

Looks like the issue is marked as resolved for a future release, great! Any idea if the fix will make it to a 5.3 patch, or only 5.4? Thanks!

 

Alright, thanks for the info again. Luckily our regular build times have come down quite a lot with a recent Unity patch that reduced our build times by 2 hours (we suspect shader compilation fixes during scene building?) and also another 2 hours when we turned off "Optimize Mesh Data" as a build option (wish we found that sooner...) So we've gone from roughly 6 hour builds to 2 hour builds, making the transition to Asset Bundles less of a priority (though still something we want to do when this compilation fix makes it in!)

 

I just wanted to get an update on this bug ? I'm running 5.5p3 and still getting this recompilation occurring each asset bundle build. Very annoying little bug as i have many small asset bundles each taking on this extra overhead. Any time frames for when this will be completed ?

 

@SteenLund
Hello, in our project we have the similar issue that occurs when we iteratively in one method call build a lot of small bundles (prefabs). After each bundle in task manager you can see that separate mono.exe process starts and hangs on for a 10+ seconds, after which terminates. What does it stand for? With older unity versions 5.2.x it took 30 minutes, but now more than half and hour =(