Garbage Collection, Allocations, and Third Party Assets in the Asset Store

I like the idea as a possible secondary overload ( passing lists being #1 preference ). Sync presumably rather than async?

 

Found another one:

Code (csharp):

1. RaycastHit[] Rigidbody.SweepTestAll()

 

So, long story short, we're screwed by a decrepit GC, should write bad, boilerplate filled, maintenance nightmare code to deal with it, including pushing tons of extra data around on the stack to avoid short lived instantiations, doing object pooling which has actually been a COUNTER-productive anti-pattern in managed environments with a proper generational GC for nearly a decade, and avoiding all of the clean, uber-productive, uber-maintainable goodness of enumerables and LINQ?

And why don't all of these array returning calls return IEnumerable<>s anyway? Who the hell is designing these APIs and do they know anything about programming? Is this all handholding to make non-coders who aren't doing the bulk of the programming feel comfortable slopping out a couple lines of UnityScript?

(But, hey, maybe in Unity 5 they'll get around to updating the core foundations that affect everything and everyone... and maybe they'll get around to addressing the unspeakable shame of OnGui(), or the "for our own good" broken prefab nesting, or the half-assed physics implementation / API... Of course, like anyone sufficiently competent and motivated, I don't use OnGui(), have written my own input binding system, and have gotten used to the suckage of non-nesting prefabs (thank God I'm not an asset dude or that one would outright kill me), but I don't have the time or means to try to integrate Bullet, and nobody can get around the GC.)

 

IEnumerables are slower than allocating an array usually. But of course, doing it a lot causes memory problems.
The fastest way I have found is either 1. Give the method an array to fill and return a larger array if the passed array was too small, or 2. Use delegates.

But I agree that the GC currently sucks. I have to write tons of pooling code not to run in to GC collection spikes in the FPS.

 

TLDR but from just reading the start i see a lot of very wrong performance advice, like the advice around for loops moving the count check outside the loop, that's what's very wrong with syntetic checks using tons of iteration, what is the last time you actually had to do what pretty much amounts to a no op, in a loop, over a million times per frame? Never, so here you're timing just that insignificant part, it's exactly the kind of code you should NEVER write, if you profile it in anything meaningfull and non syntetic it will be less than a 0.0001% speedup, at the expense of code clarity, use a foreach there, that's what it's here for. Applies to a lot of other points in here, good code is writing clear and maintainable code "only", and then optimizing that code based on profiling that code, not some syntetic striped down version run a ton of times.

 

Actually, there are quite a few cases where I have to do that. Like assigning some variable on a large number of nodes in my pathfinding package.
Not every frame, but those loops do count.

 

@OP

Because of you, I became a paranoid GC hunter.

 

No actually i really doubt you do, if you check the results you're actually wasting 4milliseconds "per million" of iteration in for vs for with external variable, now do you actually update 1million nodes in your package? If it's just 10K nodes we're at 0.04milliseconds for exemple, and that's not even per frame.

For foreach i don't know how it's implemented, if it's checking for IList as an early optimisation (like many linq methods do) then it may just be as fast as the first for version even pretty much.

 

I'd prefer IEnumerable<>s with properly optimized implementations (which generally only need either an index var or a pointer) over arrays any day. Even with a good GC. Delegate passing would work too, but if you passed a delegate and need state then you need to either wrap state in a closure (not a problem with a semi-decent GC) or write more boilerplate, and the callee would internally be doing the exact same logic to enumerate anyway. And you lose the ability to use query semantics. (Of course, you *could* push query logic into your delegate, but I'd rather just use the clean, productive goodness of LINQ.)


And, yeah, pooling is a bitch. In my previous life writing server side apps, we used to pool tons of crap way back in Java 1.3. Later, when the multi-generational GC came out we removed all of our pooling code (yay!) and saw performance increase.

After discovering this thread I made some simple changes like making my personal algebraic classes like Option<> (and its IEnumerator) structs instead of objects and immediately saw a noticeable affect on GC. Then I implemented object pooling on my short lived classes and saw a huge benefit.

Now, I haven't put bounds vars outside of loops or done other micro-optimizations aimed at potentially shaving a couple processor cycles, but I think the basic advice to pretend it's 1999 and avoid GC like the plague stands up pretty well. In fact, I'm tempted to give up writing my game and instead rewrite Linq.Enumerable and the entire Collections.Generic package to use struct IEnumerators and sell that code on the Asset store for 1K a pop. Such is the utter ridiculousness of the current GC.

 

The problem is that the API, at least in my opinion, should also be suited for people who need the last bit of performance from it. Something which using IEnumerable would hinder.

 

@Ronan Your point about writing maintainable and readable code is one with which I'm sure we all agree. Certainly with for loops, it's unlikely the performance gain will be noticeable unless you are dealing with millions of nodes ( we use Aron/TowerOfBricks A*Pathfinding Project and have a million nodes ). Since starting this thread, I've moved to writing loops as shown below because I find this the best compromise. Of course, every developer will have their own preference.

Code (csharp):

1.  
2. int count = whatever.Length;
3. for ( int i=0; i<count; i++ )
4. {
5.    ...
6. }
7.  

The purpose of this thread is to highlight some of the issues less experienced programmers should be thinking about if they wish to avoid issues later in development, and some of the facts ( such as minuscule performance gains ) need to be taken in context of a developer's own project to forecast any performance gains. However, using foreach loops will cause allocation and thus feed the garbage collector leading to spikes, and I therefore personally consider that bad advice.

As an update to this thread, Aurore the Community Manager informed us that the folks at UT have been reviewing the API calls and garbage collection ( of which Enumerators are compounding the problem! ) following another thread. We are now waiting to see what they come up with.

 

PS: To avoid extra assignment and an extra line, use this

Code (csharp):

1. for (int i=whatever.length-1; i>=0;i--) {
2.     ...
3. }

Or just because it looks cool

Code (csharp):

1. int i = whatever.length
2. while (i --> 0) {
3.    ...
4. }

 

Found this interesting article...
http://www.codeproject.com/Articles/146797/Fast-and-Less-Fast-Loops-in-C

Think I'll replicate these tests once more when I get a spot of free time.

 

Is my understand of that thread correct, that a Unity dev had the balls to imply that latest mono didn't show much GC improvment? That the latest multi-threaded, multi-generational, non-blocking GC isn't much different than the ancient, non-threaded, non-generational PoS we currently have? Oh, wait, he didn't actually specify which GC he was using, but that their "internal prototype" didn't show much improvement... I think anyone who's been around the software development industry knows that any time someone claims stuff about an "internal prototype" that you can safely take that comment with a pound and a half of rock salt.

I'm hoping these investigations go better than when they "investigated" Bullet, but decided to stick with what they had... that worked out pretty well. Ugh.

 

It's actually quite the opposite of what I do.

1. I don't care about short-living instances. I write the code whatever way feels natural and just throw away all the stuff I don't longer need. Dead objects don't increase garbage collection time much (if increase it at all).

2. I keep away from using pools. I tried several times, but then gave up. The more objects you there are on the heap, the more work there is for the GC.

3. I use LINQ all the time.

Since I don't keep the useless data on the heap, the GC does its job very quickly. The only way I can notice it is by watching how GC.CollectionCount(0) increments from time to time. It's worth to mention is that almost everything (except textures) in my project is procedurally generated and dynamic.



I'm not sure this approach works for mobiles, but for normal PCs it works fine.

 

I'm not on mobile, and it's good to know that I can stick with LINQ. But in my (informal) tests, I noticed way more stuttering before I pooled what I'm currently pooling. Which I should note is state data for rewind / replay (a whole 'nother world of pain with the half-assed physics with a half-baked API), and that the pool only ever has a very small number of objects available, with nearly all of the objects in use at any particular time. So I avoid a lot of allocation / deallocation / fragmentation for the price of an insignificant number of extra object references.

Edit: Given the current GC is non-generational, the lifetime of objects is irrelevant in terms of their GC cost, is it not? Deallocating a short lived object has the same cost as deallocating a long lived one, and both cost the same to reference check. Your approach seems to be to keep the heap small, which is solid advice, but keeping it small AND minimizing GC pressure should be better, if not necessarily *required* depending on your memory usage profile and goals.

 

My understanding is that it's unlikely that we'll see an upgrade to the Mono version primarily because of backwards compatibility ( significant problem for the web player ). Not having a new gc might not be a deal breaker as we've been able to eradicate most allocations in our own code and have encouraged asset store publishers to do the same. However, the sealed API classes use enumeration and return new arrays a lot ( all contributing towards the gc crunch ), so a lot can be done to improve the situation if UT rewrite/extend the API. I believe that is now where the focus lies, we just have to wait now until they have news.

 

Well, doing such a big game for so many months, I decided to feed the GC animal sometimes. That's just tough luck and I don't see why I (as one man) should break my life over a trivial issue that a better garbage collector should deal with. It's the year 2013, when we have robots on mars and a space station sorted out, so I'm not exactly being demanding here.

No disrespect to the OP or this thread, I agree we need to bear it in mind but for long term improvements, we need a better garbage collector. Avoiding garbage is tricky, time consuming and directly loses me money spent burning time on avoiding it.

That approach would destroy mobile performance unfortunately. GC on desktops is trivial.

 

The thing is, while you've been able to handle all that on your own, this is a huge layer of mess for people (especially those with the Indie version, lacking a profiler) who haven't been able to eliminate or troubleshoot these problems. One of the big reasons I grabbed the Pro trial a while back was to use the profiler to help manage my GC problems. An improved GC system like with SGen would be a huge deal for a huge amount of users.

Exactly. My current target platform is the Ouya and having to keep GC in mind all the time is nerve-wracking and time consuming.

 

@alexzzzz I certainly can understand the counter-argument to object pooling in that it's a lot of extra references that the garbage collector doesn't need to check for null each time it kicks in. However, that's how we first started out on Folk Tale and we were experiencing frequent and noticeable spikes as the gc locked all the threads to perform cleanup. We then tried forcing gc, but the regular spikes were still way too noticeable. So then we moved to object pooling and the interval between gc's improved to around 60 seconds with ultra-smooth gameplay inbetween. The downside is that each gc cycle takes significantly longer because of all the extra variables gc needs to check. However, we've done a few smart things and now it's not too noticeable. As for a best approach, I think it really varies by project and what the developer considers players will consider as acceptable.

 

After months of campaigning, reality sank in that its highly improbable that we'll get a Mono update and the faster gc. I accepted that reality, and instead have focused my lobbying on more practical suggestions that UT can act upon, and that would be sufficient for us to finish Folk Tale. Hence, the focus has shifted from banging a drum about a new version of mono, and on to those sealed API classes and all the unnecessary gc feeding it does.

 

I'm interested in keeping the pressure on unity regarding GC. I don't see it being all that acceptable when you have an ancient bit of kit like blitzmax with a GC that's 100x faster than what mono uses. It's ridiculous. I do feel improvements can be done. And should be done. It is what people are paid for.

Regardless, as the competition are now all getting into mono we'll see unity shift once cryengine and unreal engine have better automatic GC. Unless Unity will pioneer here

 

Is it me, or are there only a handful of us stupid enough to be developing games for standalone? I just looked at the list of "made with unity" games, and several of them are still in development and haven't been released yet, leaving a very limited and exclusive club. I might fire an email over to the Guns Of Icarus guys to see what approach they took to sidestep the gc issues.

 

Well foreach doesn't need to cause "any" allocation by design, neither does any IEnumerable for that matter, however i don't know how the underlying types are implemented in mono so i can't speak there (i come from .net).

To me the expected allocation inherent to foreach is strictly 0

 

I would love for unity (which now has DX11 support and mecanim) to take a stance saying "next version, 100% of our devs are taken off from new features and just on migrating to latest mono + fixing all known bugs, 0 new features introduced"

This would rock

 

I agree, and on top of being tricky and time consumming, it's outright stupid, having a language that has garbage collection as one of it's core features, and making it unusable is just a really bad deal, i don't get why it's not the #1 priority today for unity

If i was getting latest .net 4 version of mono + 64 bit editor, i'd be happy with the state of things for quite a while, i don't care so much for new features, the asset store has most everything covered there!

 

Exactly, GetEnumerator<>() can return a struct. No allocation, and in most cases just a reference to the collection and int / uint / whatever as an index var. I changed my Option<> class from a object to a struct, and from a "yield return" style enumerator to another struct with a byte index and saw immediate GC improvments.

Of course, by design the APIs most likely don't return structs because, well, structs are generally a really, really bad idea except for specific use cases under a horrible, horrible GC.

If the Unity team is unwilling to move on the Mono version (yet), they should have a couple people implement an alternative Linq.Enumerable and Collections package with struct IEnumerable<>s so we can actually be productive and write clean, maintainable code rather than spend time, money, and effort on shoddy workarounds. Not that this whole idea isn't just a shoddy workaround for a SOLVED ISSUE if we had a proper GC.

(Of course, if they did write an alternative collections package, they'd probably make it non-generic to be less "scary" for artists doing a couple lines of script, lols).

 

Well there's allocation and allocation, if the issue is just having a non struct enumerator, that's unlikely to cause memory pressure and thus GC spikes, the enumerator implementation probably only has an int as it's single state element, so at 1 foreach per frame you're looking at 1 int of allocation per frame, hardly what i'd call memory pressure.

 

While the size of the objects in this case are small (though to be pedantic, a typical enumerator would need an int and a reference to the collection), it's still stuff you're sticking on the heap that needs to be reference checked and causes fragmentation. Even if they build up slowly, you still have to pay the GC piper eventually. And let's say you have a multi-level Linq query nested inside another loop firing on numerous MBs every FixedUpdate... now the number of (completely unnecessary) objects are adding up quick.

 

Nono, i agree with you there, i'm just saying that the numbers in question aren't large enough to justify the actual frequent GC spikes issues (with screenshots with like 8KB per frame allocation), of course if the GC sucks and you're doing deep linq then yea you're going to end up with nested yield auto generated enumerators with a few bytes each and it's going to add up (not so sure about fragmentation there, no clue how the mono GC works but i assume outside of the LoH it compacts at the same time as it unallocates anyway so unless you're doing many i doubt you'll suffer from fragmentation first?). I'm more of wondering about how people manage to allocate so much per frame and pointing that it's unlikely that using foreach co are the root cause, as that would trigger a GC very infrequently (and not every few frames).

 

Btw (this doesn't fix the linq issue but does the foreach), why not create a SingleEnumeratorList<T> class? should be a few lines long, just inherit from list, store the enumerator , call getenumerator in the constructor and store it, and implement GetEnumerator as such :

cachedEnumerator.Reset();
return cachedEnumerator;

No allocation there, can enumerate as much as you want.

 

I would like to play with async/await, so I would need .Net 4.5/C# 5/Mono 2.11. I'm just dreaming.

 

It's not you... the number of PC/Mac standalone developers really seems to be low. You can almost count the PC/Mac games on one hand (at least looking at the "made with Unity" list). To me it seems that Unity has evolved mostly for the mobile market. While it's true that it has gotten the DX11 update and quite some other stuff, it looks more like a "side-effect" of trying to push into the AAA market (a lot of us complained about the lack of AAA capable support last year including me).

And I highly doubt that the complaints about the GC are really taken seriously. I think the Unity Editor-response-time-problem several of us have with very high object counts in a single scene is a side effect of this. It literally takes up to 20 seconds in a scene with >20.000 GOs to delete one of them (well, it took so long last time I checked if this "error" was fixed when 4.0 came out).

 

+1, I really hate that. It really needs to be fixed asap.

 

Well, the feedback I got on my error report was that they have "fixed" it, but didn't have an ETA when it will be included in the release... but this was long before 4.0 was released... and it's still not in...

Edit: Checked my mails... got a "fixed" statement in march 2012...

 

8KB per frame * 60FPS = 0.47MB per second allocation. Over 60 seconds that grows to 28MB. Now as you point out that shouldn't be an issue on modern computers with 4GB of RAM as a minimum, but gc still kicks in when it doesn't need to.

Let's consider the options available to UT:

• A new gc / Mono update ( * )
• Improve control over when gc happens in the current version of Mono
• Eradicate allocations from the API

* This would have a business impact, so I can imagine that in a meeting to discuss the future of gc, migrating to Mono 2.8+ was ruled out in the short term ( 1-12 months ) and possibly even medium term ( 1-5 years ).

Which leaves the last two items where actionable decisions could be made. As much as we all want a better gc, commercial reasoning suggests that it is improbable in the short term.

 

I'm not sure if this has been covered before, but accessing a transform's .parent adds a couple of bytes to the GC.

 

Games Foundry, could you or the author of the pathfinding system make those millions of nodes to be structs? A million of class instances on the heap and one array instance that contains a million of structs are two huge differences.

Strictly speaking, the size of the heap doesn't matter, the number of references matters.

 

I stand corrected then (Not that it changes anything to the fact that it doesn't seem to be really "fixed" by now).

 

Probably yes. I have noticed that scene changes tend to take a huge time in the editor, but run really smoothly when deploying the game (that can be a factor of several seconds to no lag at all).

 

Any update from UT on removing allocations in UnityEngine.dll?

 

Bump. Aurora, any news? Not looking for details of what is coming, just when something might be coming. UT have had several months now to work on a solution.

 

Sorry to bump again folks. Getting that 'sweep it under a rug in the hope it goes away' feeling.

TL;DR: UT have said they had something in the works to address the issues of unnecessary allocations in the API, but couldn't reveal details because it was commercially sensitive. So I've asked not what the solution will be, but approximately when we might expect something.

 

Hi Joachim,

I'm impressed by the new proposals to limit the amount of CPU time GC can take. Will we be able to adjust the ms figure, say for mobile or desktop?

 

Wouldn't it be ideal even to be able to override it? If the system to run the gc for x ms is available then it would be cool to be able to say
Unity.GC.IsAutoGCEnabled = false;
And then in update be able to call it as you wish (pseudocode)
if(onmenuscreen || (gc.MemoryPressure >5))
{
gc.LimitedRun(1)// in ms
}