Compile Time Optimizer (Editor Extension)

Hi!

I'm working on a editor extension that will help you optimize your project compile times by a lot!

If you're a Unity3D user then most probably you're taking advantage of what Asset Store have to offer. Asset Store is just great, isn't it? Still it can make your project to compile slower and slower and you know what I mean if you're a programmer.

What I did is to make advantage of how Unity compiles scripts. You can split up your scripts between scripts that are frequently modified (your personal scripts) and 3rd-party scripts that are rarely modified.

Here's a presentation of a tool that I am working on.



I've created script compile timer (1) to measure how long my test script (on the left) compiles. Since I've imported PlayMaker, Photon Unity Networking, iTween and NGUI (2) the compilation time is over 9 seconds.

Then I am opening my optimize tool wizard:



I am selecting all 3-rd party assets directories and clicking on the Optimize button below.



Optimization takes its time. It does not modify any of source files content. It just moves it in the right place.


And when it's finished...


First import took about 6 seconds. That's because all the scripts needed to be compiled again.
At this point I am modifying test.cs script by adding one space character to trigger the compilation:



And again:



OK, what happened here?! Let me explain. My optimization script is automatically moving all 3rd-party source files into Sandard Assets directory preserving the hierarchy. Only scripts are moved there, all other resources are kept in place.



Moving scripts there will make Unity compile it only once, since Standard Assets compilation is done only if any script in Standard Assets is modified. Then smaller compilation time (4 secs vs 3 secs) is because of disk cache, so compiling in short intervals is in most cases a little faster.

OK, so if this is so simple, why I am writing a tool for this? It is simple if you're doing it once for a single 3rd-party asset, but it is a pain in the ass if you need to do it for more of them. What's more asset updates will make you to do it again and again. I want my tool to have a "revert" option to make your 3rd-party script to be reverted to previous state, then you can apply the update and do the optimization again. In that way your project will be kept in clean state and there's a smaller chance that something will break.

That's for the presentation. If you're interested in testing my script before the release, please write here! First 10 testers (with any feedback) will receive a voucher for my product. I think this is fair, because you will help me to make it stable and safe before the release, and you will receive lifetime license for upgrades

Thanks!
Piotr

 

Sign me up as a tester! Can't wait!

 

Sounds interesting. Any limitations or warnings? (e.g. don't use on dlls)
Happy to try it out (after v. careful backup! )

 

Is there a possibility to use the Plugins folder, instead of the Standard Assets folder?

 

@Speedrun Labs @Arkade please see your conversation inbox.
@Dantus I can make it an option. Also I'd like to ask why Plugins instead of Standard Assets?

 

Standard Assets is usually used by Unity for their samples and their content, while the Plugins folder is used by many developers in the Asset Store (including myself) for their scripts, such that they can be used with C# and UnityScript.
Because of that, it is my subjective impression not to place my stuff into the Standard Assets folder, but into the Plugins folder instead.

 

Based on this it seems like Plugins folder is meant to contain native dlls, but still scripts included in there are compiled first. Since not Plugins nor Standard Assets are meant for this kind of optimization, I think I will allow user to choose where he want to move his scripts

 

I am aware of that page and it seems they changed it in Unity 5. Unfortunately there is no enforced standard folder for that and I am not sure whether there is a consensus about that within Unity. However, let's not go too much off topic here.

Great asset!

 

I'm interested to be a beta tester.

 

Great! Just send me your e-mail address in a private conversation. I will send you the new (updated version) tomorrow!

Cheers!
Piotr

 

Hi! Here's the update!

I was working on revert possibility, removing empty directories and plugins/standard assets target directory selection. Visuals has been changed, optimizing is a lot faster, and some other things that I'm too lazy to mention

​

So here's how the menu looks like. Now everything is in one place.

​

This is the Optimize Window. Now you cannot optimize anything.

• Nothing To Optimize means that in this directory are no scripts that can be optimized
• Forbidden is my doing. You are forbidden to optimize Compile Time Optimizer, Standard Assets and Plugins directories
• Optimized means that this directory has been already optimized.

​

You can also choose between Standard Assets and Plugins as a target directory. Both are doing the same from practical point of view, but you may have your own preference.

​

Here's the Revert window. Here you can undo optimization of directories. It can also remove empty directories from Plugins/Standard Assets if these would be left behind.

​

After reverting Unity editor needs to be restarted. This is because a bug that I was trying to resolve, but with no luck. Moving script files from Standard Assets or Plugins into any first-level directory results in Internal Compiler Error that is breaking Unity ability to compile scripts. Forced restart is a workaround.

​

After restart you will be informed about removed empty directories if you've decided to do so.

​

And as proof of concept, here's Compile Timer window! First 3 entries are compile times when Mad Level Manager, Energy Bar Toolkit, Bird Flock and NGUI were imported into the project. Last 3 entries are when the optimization has been done. The middle entry is the entry generated by optimization process, so it doesn't count

All of you who've sent me your email address will receive the updated unitypackage in a moment!

Cheers!
Piotr

 

Looks great!

Although I signed up as beta tester, I just wanted to apologise that I haven't managed to try it yet -- want to get my current changes all committed and backed-up before your tool helps (...or runs amok ;-) ). I'm site you understand.

Thanks again.

 

Some feedback:

• After using "Revert", the "Optimize" windows mark all the folders as Optimized. I need to Reboot Unity (a second time) to view it correctly.

I'm using Unity 5.

Great work.

 

@Arkade Don't worry about it! Feel free to try it anytime!
@javock I cannot reproduce it but I think I know why this might happening. Instead of rebooting unity try to close and reopen Optimize window.

P.S. I've just found out that Unity 5 most probably does not require the editor to be restarted after the revert! I need to take a closer look at this.

 

genail This looks like something I could really do with as every time I make a change to my scripts the compile time is now 4 minutes .. yes I said 4 minutes

How will this cope with scripts that access other scripts?

I have lots of assets from the store, and LOTS of my own scripts, I'm organised as I have named dir to keep track of my scripts, I mostly code in Javascript and also use some C# so have scripts accessing the C# from either the Editor or Plugin dir, how will it cope with this or will I have to sort those compiling order out?

Also I've sent you a PM because if you think I can use you plugin I'd really like to try it.

Thanks.

 

Hi Griffo!
Wow 4 minutes is a lot! I will send you the unitypackage in a moment. Seems that your scenario may be a good testing field. Please backup your project before trying anything.

My script is pretty simple and it does not search for references between scripts. I would recommend it for optimizing compilation of 3rd party scripts only. I will clarify what will work:

- A -> B dependency if only B is optimized and the language doesn't matter
- A -> B dependency if A and B are both optimized (or not) but only if A and B are written in the same language

So if your code is in javascript, then you shouldn't try to optimize it. Just the 3rd party libraries.

Please let me know how it will go!

Cheers!
Piotr

 

@genail Thanks for that, I'll let you know how I get on ..

 

More feedback:

• If I turn off the optimization option, the "Standard Assets" folders are created anyway.

Working on Unity3d 5 and Windows.

 

Thanks!

 

Just started trying this out (in a medium sized carefully backed-up project before trying on the biggie!).

First comment is one that should have been obvious to me in advance: it makes things be different assemblies ... which affects accessibility of internal members.

What I hadn't expected was that namespace accessibility doesn't seem to be as enforced as I thought it was within an assembly. Meaning I had some code accessing internal members that weren't in the same namespace.

Not sure this is something you could even do something about but it might be something to put in a FAQ doc. If you were to try something... ?

Continuing!

 

Two little observations (not very serious).

1. The "Optimize" window doesn't refresh after optimizing until the mouse enters. Might you stick a redraw on it after its work is finished just for tidiness?

2. Might the "Compile Time" window, while lovely be better in the "Optimize" window? One wants that info when one does the first compilation... when you tend to have that window open. However, unless I missed something, it doesn't have the data on first opening after optimization so you don't get to see what you really want. Not explained well but perhaps you get my meaning? Ideally, one would want an obvious use flow:
E.g. 1. open "Optimize" window (perhaps with it already showing the last compile time? -- is that available/calculable from the Editor.log?), 2. choose assets & click "Optimize", 3. wait a while, 4. window refreshes with indication of how much time saved!.

Just my thoughts. HTH.

Otherwise, seems pretty solid so far. Good job. About to try it on ... The Big Project! ;-)

 

Hello Arkade,
Thank you for the feedback!

I'm not sure if I understand what you mean. Did something broke after the optimization procedure? If yes, how can I reproduce it?

I will write about it. I am working on the documentation right now

Yep, I will fix that!

I think that would be awesome!
The compile time window currently needs to be opened to measure the time. I think of extending the idea (I cannot rely on compilation time of things that I don't know about) and making a compile time benchmark (accessible from optimize window) that will trigger the compilation of main assembly a few times.

Thank you for testing and ideas!

 

From memory, I had the following (I can check if this proves inaccurate/insufficient):

• Asset A with namespace N1 had a class C1 with an internal method M1.
• My code with no namespace had a class accessing C1Instance.M1
• Before optimization, this worked. (though personally I'd not have expected it to)
• After optimization, my code was told C1.M1 was inaccessible due to protection level.
• Neatly, Asset A is somewhat EOL so I felt safe modifying it.

Does that clarify?

 

Hi Arkade,
Yes, I understand now! Thanks

Is there a reason why using internal methods in Unity scripts? I'm asking because personally I wasn't able to find it useful if you don't have a full control over project assemblies.

There's one more known to me scenario where optimization may cause compilation errors. This is a circular dependency. If asset A uses asset B and asset B uses asset A, then optimizing only one of them will cause in compilation errors. I will write about this in the documentation as well

Cheers!
Piotr

 

A perfectly thought-out API requires none. The API I'm interfacing with isn't. Perfect design, eh! Don't make 'em like they ... never did ;-)

Hmm, personally I wouldn't want to trigger compilation a few times. This really wants to be a feel-good asset -- install, use, feel good. Not install, wait, use, wait, feel bored. (After using for a bit, I withdraw my previous feedback -- I quite like the Compile Timer window but still think it'd probably deserve to be in the main Optimize window.)
All said, your primary customers are likely to be those with a large codebase (with much time invested) so I suspect many will feel a bit like I did -- namely *REALLY* paranoid that it's gonna go horribly wrong! (they'll do many... maaaaany backups while cautiously trying it incrementally). After all that, I can report that, to the shock of experience that has taught me to be paranoid and pessimistic about all software, your tool...


... functioned perfectly. Wow!

Compile time reduced from 11-18 seconds before to 5 seconds now. For the curious, I applied it to:

• Build Report Tool
• Master Audio
• Dynamic Bone
• FPSGraph
• Game Analytics
• LitJson
• Ouya SDK
• ProCore QuickBrush
• Snowify
• SwiftShadows
• Toolkit 2D
• Fracturing
• Unity Test Tools
• iTween (including Editor and Hinting)

(total line count for all C# source under Assets: 198526 lines. Who'd'a thought snowball fighting could be so complicated?! ... and that's not including this internal-bypass-requiring voxel library! So hope this is gonna be worth it! </digression> )

A couple more motes of feedback:

1. Why does it offer to optimize a top-level "Editor" directory? (I know it's possible to have Standard Assets Editor stuff but am I right thinking this only benefits compilation time of the non-SA Editor code?) Maybe something to mention in the FAQ list?

2. I'm not sure whether possible but I really handy addition would be something that helps when we try to update an asset from the Asset Store that is optimized. Even preventing and reminding of the need to Revert might be sufficient. (yeah, we probably delete things beforehand but all the same... .)

HTH.

p.s. I think you can consider me a Day 0 customer. Thanks!

 

p.s. Think I'll expand on this last post in a blog post. Would you prefer I wait for your final release?
(Um, be reassured(?) the blog I'll post to has *zero* readers -- it's a brand new Wordpress one that I'm soon to migrate my current website to! Ya gotta love indies ;-) )

 

Hi Arkade,
Thank you for testing it on your project! To be honest I am also surprised that it went without any issues On my project there was one, although small issue but based on my experience what works for me, does not for others

I won't include benchmarks in 1.0. I've decided to freeze this version on the beta stage (no more features added) because I need to be sure to release a product that is well tested for now. I will try to do something with information about compile times in 1.1 or later

Replying to your questions:
1. Of course if you're experienced Unity programmer then you may also want to write your own editor code. I have plenty of these in my game projects, so it is also important for me that CSharp-Editor assembly is compiling as fast as possible. If you're not familar with extending Unity editor, I strongly advise you to take a look about what can you do

2. Yes, I thought about that too. Unfortunately there is no (or I don't know about any) way of knowing that user is about to upgrade asset from the asset store. Yet there's a way to find out that asset has been just upgraded. What I can do now is to yield a big message that something has been upgraded without revering the asset beforehand. I know that this is calling the fireman when the house is already in ashes, but this is better than nothing

Don't forget that I promised that 10 first testers will get a voucher when my tool will be released! This means that you're going to receive one too!

Can you send me a URL to your blog? I'd love to see it!
I will be sending Mad Compile Time Optimizer today to the Asset Store so it will be available most probably on Friday or next week.

Cheers!
Piotr

 

Hi everyone!

Mad Compile Time Optimizer is now available in the Asset Store!
https://www.assetstore.unity3d.com/#!/content/34012

To each one who've sent me the e-mail address: please check your inboxes. You will find there vouchers for the package as my thanks for attending the beta stage

If I forgot about someone, please let me know!

Cheers!
Piotr

 

New thread in Asset Store forum has been opened! Please write there from now on
http://forum.unity3d.com/threads/mad-compile-time-optimizer-speed-up-your-development.319216/