Unity 5.3 Performance issues

Let me repeat the core issue here: performance on Desktop or each release for mobile in particular is poor.

Exactly..... the QA team should have a complex test environment for each area of the engine features

Physics, Rendering, etc (and I mean a complex ), also the QA need to understand that these performance issues is not something that users can just submit a report about, as they lie deep in the engine core.

As I had said before I don't even kno whats causing the bad performance...

 

I can't agree more with that. Personally, I've got very high expectations for 5.4 (in terms of general stability and performance), for me it's the test to see if what Unity QA said during the 5.3 beta was true or bullshit.
On the roadmap, the growing list of new features in 5.4 looks to me as a bad sign but as bug fixing is on the top of the list there is still hope. For now my agenda is wait and see.

 

Hi everybody.

Firstly, thank you very much for your feedback and input. We take this very seriously. We have fixed a mountain of bugs/issues and increased our velocity substantially this year. For the past couple of releases there is a sense that overall the product has had fewer issues, but the issues it has were more serious or hit more users harder, resulting in a net loss of perceived quality. That sucks for everybody.

We have spent a great deal of time looking at this internally and we'll make some changes to the way we wrap up, test and roll out releases to address these concerns. We are still working on the exact set of changes to we'll be making, so I can't post those in detail here now, but it will be a combination of getting more beta testing by users with real world projects, being more aggressive about bumping stuff that is risky or coming in too late to the following release, and enhancing the way we benchmark performance to ensure we don't regress it among other things.

Stay tuned and we'll have more updates in the new year.

Cheers,
Brett

 

Sounds Great

 

Hi Brett, thank you for the reply, finally something

Can you please tell me and confirm the below:

1. Which unity team are you part of?
2. Are the changes mentioned specific to QA processes?
3. More beta testing by users, why again does this sound like you using customers as QA in a way, nothing wrong with this but it does not give confidence in internal QA teams work.
4. How are you going to tackle performance issues?

In a way your reply is much welcomed but really thinking about it, you not told us anything, except changes are coming in new year.

 

I can confirm that my Android builds became playable again on 5.3.1f1.

 

@MrEsquire

I'm the VP Engineering at Unity. Your problems are ultimately my fault/responsibility and I apologize to all who feel we're not serving them well. I will work hard to fix this.

We do a lot of QA testing. We have lots of automated tests that run on each release, manual testing, exploratory testing, testing by alpha/beta customers and internal users. With a complex product, many times the issues come from combinations or use-cases we didn't test for. When this happens we try to adapt to those in the future, add tests, and so forth. But the reality is we just don't get widespread use by customers until a release and given there are more than 300,000 projects actively using Unity in any given month, there are so many issues that come up only after a release.

As for performance, we have a performance regression test we run all the time that checks to be sure performance didn't get worse. But again, the tests must account for all features, or combinations of features customers are using. We keep expanding that coverage, but it's a daunting task.

Up to now, beta use has been not only voluntary, but most users must discover it and actively seek it out. We'll start emailing users directly for the next release and there will hopefully be an increased number of users that will try out the betas and report issues. This is, and will generally be, the single best way to ensure real world coverage. We're not shucking our responsibility, but the more varied the testing of our product the better it will be.

 

Thanks for the reply and answering some of the questions I had asked.
I do not think its fair to blame yourself, its a combination of things.
I think the community can now rest and feel more confident that things will start changing and improving for the better.
Do not worry, we not give up hope or faith, just needed some reassurance.

I guess we can now move on and to more positive releases and feedback

Cheers

 

Thanks for your words of kindness and encouragement, it means a lot to all of us. We exchange thousands of messages internally on topics like this; Rest assured, we will continue to get better and better.

Happy holidays everybody

 

I think you guys have a really hard job and think your doing great considering the challenges. VR/AR must mean for some pretty big changes under the hood, not to mention other large features needed for future versions of Unity.

I came to Unity after making this game in the Garage Games engine:

.
If you watch the video you'll notice I fade to white at the end of each round. That's so I can set the rendering backbuffer to white and rebuild the rendering engine behind the scenes, due to all the memory leaks. (render bins were created when an object moved and never deleted). I left GG because I was spending most of my time dealing with engine bugs. While I expect some engine bugs especially being that I'm working on a new platform, its still frightening as in the past six months the programmers on my team have spent a large percentage of our time working around and discovering engine bugs.

I worry your business model must make it hard for engineers, you need to add new flashy features to get people to upgrade (you need to eat) and also have to support existing features. I really hope your team is considering adding a % charge to released games that make over a set amount of money, or that you will consider moving to a pure subscription model for pros. If a change would allow your team to focus on quality of life improvements for existing owners without having to worry so much about attracting users to upgrade for new features, it would be worth paying a bit more.

Also I hope your team is considering doing some in house game development. (full releases of AAA products not just demos). Unity has a strange habit of releasing new features while missing some of the basic core functionality and skipping huge quality of life improvements that someone who uses the tool would implement first. Bringing large products to ship internally would be a great way to tighten that feedback loop. E.g. As a former professional FX artist I'd ask for non-uniform scale on particles long before most of the higher end features the system offers. Having programed tools in the past I know how hard it is to hit all the edge cases, especially if your not working directly with a user working on a real product that needs that feature.

Lastly most of that time spent on engine bugs has actually been in discovering the bug is an engine bug and not a bug in our code. Usually the engine bug ends up being a known bug. It's hard to search for a bug as you don't know what to search for. As such I'd love to see bugs linked in the manual sections. That way when my team is implementing a new feature they could quickly check the associated manual section to get all the information they needed. Often knowing the bug exists is 90% of the battle.

Unity is still light-years better then anything I used previously (I still like it more then Unreal). I really appreciate the work you all are doing. I hope you have a good holiday and I'm excited to see Unity's progress in the next year.

 

@hoesterey

Thanks for sharing your experiences and suggestions.

We have discussed whether we should make games or not, and just doesn't seem like a good fit. What we have done though is a few things... We have in house teams making demo/tutorial content and we learn a lot from that so we do know the value it provides. We sublet space to game developers sometimes (currently Camoflaj in Seattle) and that's something we might do more of in the future. We also have hired more artists, and will continue to hire more in 2016. We also have a weekly game jam where Unity staff make games together.

We just don't believe in the royalty model. We believe in a services model where we can both share, such as ads, analytics, cloud build and more to come. We believe that if our customers are successful we will be successful without imposing a royalty on the games they make.

In 2016 we're more than tripling the size of the doc team and you'll see a marked improvement in the quality of docs, including more deep, useful info than ever before. Hopefully this will help developers use Unity better and avoid pitfalls.

 

I do agree that Unity is Light years ahead..... with improved performance it will be unstoppable...

check out Night Crisis below

 

I just want one solid release I can make money at. Clients do not care about software bugs. I contract with them for services and they pay me when i meet the contract. I cannot collect final payment on my current contract that was done 10 or so days ago except for my trying to overcome issues I spoke to..a Mecanim IK bug, light baking, frame rates are low with terrain, scattered trees, very little detail and all sliders set as low as possible and one directional light with low resolution shadows and one motorcycle sim. Christmas looks bleak here with resources depleted to zilch. Please make 5.4.0 solid as a rock for iOS, Android and desktop. You want me to upgrade to pro?? Help me make that money and stop wasting my time. I had pro since 2.6.1 but have not gotten ahead of that curve to purchase since the 5.x builds due to time lost fighting it.

 

According to the roadmap, 5.4's release is set to March 16th 2016.
I, too, would be very interested in a stable release that might hopefully be available much sooner than that. (please?)
Just like most of the people here, I, too, have deadlines and I will certainly miss those if the next stable version arrives in about 3 months from now. If 5.4 really is the next best stable thing happening, I guess I will have to do quite some nasty Unity version (and thus, project-) downgrading work to do.
Just to add some extra words about "my story": I was having some serious bugs in the last versions and after reading about those getting fixed, I was hoping for 5.3 getting rid of the problems. In fact, I've also seen Unity IAP as a new feature introduced in 5.3, which got me excited, because I am on a budget, which makes me think twice about $70 Asset Store purchases for IAP implementation. So yeah, a stable 5.3.X release would be really nice.

 

Agreed.... maybe they release another 5.3 version before 5.4

 

I have only been using Unity as a hobby although I have put quite a substantial amount of money in the asset store.

Some months ago my company was trying to decide on an engine for mobile apps and I was pushing Unity as a good solution. We build a couple of games but eventually had to abandon them because there were just too many problems with Unity. Once we found workarounds new Unity versions broke other things. Instead we went for a PIXI / Cordova solution, granted, with lower performance, but at least it works.

People here have been suggesting that once you start developing you version lock Unity and just work with what it has to offer. That is pretty much unacceptable to me. Image the outrage if Photoshop / Flash etc. users had to do the same? Even having to fear that an upgrade breaks / deletes files? (Yes, I use GIT, but still).

A Unity employee stated here that new features had priority over making a stable product because (important?) clients expected certain features to come with certain Unity versions. But what good is a car with new features if suddenly the wheels are missing?

With the state Unity is currently in I can relate to the developers being dependant on Unity and feel their pain. I'm glad I'm not one of them.

Personally I will still continue with my own project, albeit slowly while waiting for the major bugs to be sorted out. I can afford that because I have no deadline. But Unity eventually lost sales to my company and who knows how many more?

 

sounds great

 

Aurore, while I don't doubt you're being entirely honest and true, you do realise that the situation is so grave, and the users so irate, that there is a decided risk that nobody believes you? It's not the first time a discussion such as this arises, and it's not the first time we as Pro users have been told to shut up and take the discussion elsewhere.

Unity staff is usually very, very quiet on these topics, which of course is to be expected. You can't bite the hand that feeds you. But if Unity really are serious in wanting to create and sustain a dialog about what is perceived as deep systemic problems, you need to promote open and honest discussion. Take the initiative. Set up a discussion thread. Admit fault. Pose questions to your users. Take your users seriously and respect our experience. Take us with you on your development journey; be open about your problems and aims.

The VP of Engineering is right at the centre of things, right along with the CTO; thus Bibbinator's recent entries are an excellent start. Many thanks, it's just the thing that's been missing and is a very encouraging sign. But please let the discussion continue.

 

Just to get some positive tone here:

Situation for Unity users:


So Unity developers should

 

Catch with a net full of wholes lol

 

Well. I ain't gonna be broke for Christmas. I got another contract start payment just now but I sure as heck ain't doing it in the 5.x series till that gets ironed out. Back to the latest of the 4.6.x series which was fairly rock solid last time I messed with it months back. So yeah..positive vibes and Merry Christmas to my fellow Uniteers!

 

I see the issue with Unity's community outreach as "too much talking, not enough listening," as I hear over and over about the new services and features of Unity, and hardly a word about the things I care about - making the tools I use today better.

While I don't want to see Unity spending their budget on all the stuff required to actually make polished, releasable games, I also think you guys need to get yourselves closer to commercial gamedev, get into the trenches, and see what we see.

Maybe you should open an indie incubator for teams building real-world products in Unity, instead of "sometimes" subletting? Opening up the Betas to more people will help too, so that's good step saying you will proactively try to onboard new Beta users.

This is one of the best things about Unity! Never change please

That's great to hear!!! Please also try to figure out how to get more new users to RTFM (good luck )

 

hang in there buddy =)

 

The problem is that similiar things were promised on this forum before, months ago. I am pretty sure it was after the 5.0 release, somewhere in early summer, when a lot of bugs popped up, and Unity representatives promised on this very forum that internal processies would be optimised so that future releases would come in a more solid state. 5.0 had it's problems. 5.1 had it's problems. Didn't I read on this very forum that Unity is in full "all hands on deck mode" when it comes to fixing bugs for 5.3? And yes, the list of bug fixes in the 5.3 release notes is impressive. Next 5.3 comes out, I see a problem like this and I am just shaking my head.

But, in all fairness, I also so a lot of good in 5.3 and over the past few days I seriously considered starting a thread where I list my biggest positive observations with 5.3.

 

the QA team should have a complex test environment for each area of the engine features

Physics, Rendering, etc (and I mean a complex ) so that simple asset bundle error would have been found if someone was there feeding in the unity versions into the test environments(not just fixing user bug reports)

That's why if you noticed bugs will be fixed, but then there will still be obvious bugs showing up in releases. Which is from not doing alot of in house testing on top of user reports...

 

I'm using Unity 5.3.0f4 64-bit. Does everybody know why is this happened after i baked the lighting? First, it appeared normal, but when i move around and stop, in specific places the lighting is become over bright like in the picture. Anyone can help me?

 

Think you should download 5.3.1patch1 and retry.

 

================================================
READ ME!!!!!
================================================

In case you didn't see, 5.3.1p1 is live!
http://unity3d.com/unity/qa/patch-releases

"Remember, I'm pullin for ya. We're all in this together" - Red Green

 

Anyone test to see if this patch has improved any of the issues yet?

 

The patch has fixed several issues for me - including the render texture issues, and *most* of the speed issues. I had seen a FPS drop to completely unusable 1.5 frames per second. With the new patch, the FPS has increased to 48fps, which is significantly better, but still 30% short of the 75fps I was seeing under OpenGL 2.1.

In short: A significant improvement, hoping further patches will continue the trend.

 

That is great feedback! I'll keep passing it on and @Aras has been watching the forums pretty heavily lately

 

I'll create a bug report with the project that is showing the slowdown. Strangely enough the slowdown seems to affect both OpenGLCore and Metal, but not OpenGL 2.1.

 

The issue is that Unity had stunning performance until v5.1.1f1 and now gets random big spikes and bottlenecks in a big number of cases.

It was perfect as it was in v5.1.1f1, all my effects run fine until v5.3, the performance was getting steadilly better until that. I am sure the amazing Unity team will figure it out eventually, maybe a roll back to previous versions for particles and threading would solve the issues.

A lot of issues could have also been easilly grabbed by allowing all Unity store developers in the beta as well, for example my particle system would have easilly picked the particle issues as it uses most of the particle system functionality and has options to push the system to the limits.

 

I think the slow down issues, which is the major problem with 5.3 release could have been predicted easilly. In fact it would take a single minute with playing with my Particle Dynamic Magic v2.0 demo which was super smooth in v5.1.1f1 and almost a slideshow in v5.3.

This is not a minor thing that could pass by quality testing, i am still at a loss how this passed through, but as you say everything can happen, there is a line where some things are not expected with a great probability though. Without drawing such line how would one leave v1.0 of any software ?

 

You are correct - it was loading the OpenGLCore for the editor.

I've sent the bug with the project attached -

(Case 756802) 40% drop in speed using the OpenGLCore API under OSX

 

I will test with the new version and if the issue persists i will file a bug report. My system uses internal particle workings, so i am sure the particles as standalone (wthout scripting) may have been faster, but the combination of using the scripting may be the cause of the slow downs.

The whole system worked super smooth and flawlessly though until v5.1.1f1, so this should be recoverable, i am confident will happen soon and i will be doing constant testing and reports on my side to help the cause .

I use standard Windows PC with DX9 and Webplayer for reference.

This is the demo i am talking about, it has been created with v51.1.f1 of Unity
https://dl.dropboxusercontent.com/u/79230236/Particle Dynamic Magic 2/PDM 2.0 DEMO.html

It is also included in my asset, so the team can download and test with it directly, it had major slowdowns and hickups in v5.3.

 

I trust your assessments and you are using the same dev platform as me. Does baking one directional light in the latest give any more fps?

 

No kidding. I have tested 5.3.0, 5.3.1 and 5.3.1p1 now with two Arch Viz projects. I don't even bother with gaming projects right now, mostly for #754512.

Those recurring performance drops / Profiler spikes that a lot of people had (propably coming from particle systems) were gone with 5.3.1. Performance however with OpenGL Core is horrible since 5.3.0. Using Metal helped, but for whatever reason 5.3.1 made that one perform worse. From the patch notes I would assume the opposite. But now you have the problem that performance in the Editor (OpenGL Core) is drastically worse and therefore no preview of Build performance (Metal). 5.3.1 did fix the render texture issues (#753677). 5.3.1p1 then improved OpenGL Core a bit. The time spend in Overhead is now maybe 12ms. In 5.2.x this entry never came up (<1ms). With OpenGL 2 everything runs smooth as butter.

My testing platfrom is OS X El Cap exclusively right now. Over the holidays I will test my project on different Mac and Win platforms. At it happens I will have access to a handful of machines over the next few days.

I do however appreciate the Unity team releasing 5.3.1p1 before the holidays. Props to everybody involved.

 

Yes I also do ,appreciate the Unity team releasing before the holidays hopefully the next month things are even better

 

Is there a way to submit a bug without submitting the whole project ? Or maybe a report way with just the description and a way to reproduce it.

I will try to file a bug if v5.3.1 still has an issue, though submitting the whole project to report a bug is not very easy. Maybe there should be an option to just describe the issue and the way to reproduce it, for example in my case it would be as simple as downloading my asset and running its main demo, uploading to me could take a day with my connection, while downloading the asset to test will take nothing on the testing side.

 

And as we in the Performance Thread, this is one thing that Unity QA simply have to improve on.
QA don't read bug reports or reports without a repo, they have said this in the past. Even when a repo is not needed, please add dummy data so least there system can pick it up as a valid bug report. (will be happy if they speak up and prove me wrong)

Most users projects are in the gigs of size, no one wants to send all there hard work to Unity when not needed (oh for some of us this maybe hours due to internet connection speeds)reoccurring theme through out the threads is Time Wastage, so one has to weigh up, amount of time needed submitting bug report VS how useful will this bug fix be for one self, its customers decision I guess.

The bug reporter in 5x is better than the 4x series, but simply the processes again are failing some customers, I seen in the forums many times users find really valid bugs and simply due to the "Repo Scenario" cannot really get there findings properly send to developers.

I know Brett has already spoken in this thread and we trying move to more positive vibes, but as we hitting it hard now and new year is a fresh slate.

Good luck nasos_333 with the bug report..

 

Sigh... Case 751697 (Blindingly bright materials on Mac after 5.3 upgrade) is definitely not fixed in 5.3.1p1.

So, again I find myself forced to remain on 5.2.4f1.

Happy holidays, everyone.

 

I stumbled across a bug similar to this and fixed it by removing anti aliasing in the quality settings. It was a few weeks ago - and it might be totally different - but worth a try though

 

Thanks for the tip, but anti-aliasing isn't used at all. The above bug was submitted on the 8th and has been confirmed. Let's hope someone is working on it. For me it just means, "5.3 has still not yet been released for the Mac".

 

I've seen two kinds of issues in this. Graphic (Culprit: Particle system.Sort) and physics (unknown culprit).
Screenies:

 

You seem confident that a bug report will be processed and it will be resolved most of the time... well i don't agree, i've submitted a bug a few months ago for something that did not work anymore on Unity 5.2 (...and a framerate drop issue as well).

My issue was related to the Write Default setting in Animation states, which worked well with 5.1.1 and after upgrading to 5.2 it was broken for my project. I had to roll back and i have uploaded the project to the staff with all details in bug report..
you know what? i've got no response...

So there is something wrong happening since 5.2 which is not related to performance only but broken features as well which are on top of previous issues since 5.0.
They're too overwhelmed by bugs at this time that they don't have time to handle all of them.

I think for the next game i should stick to 5.1.3 i don't know.. what do you suggest?
I don't want to switch to Unreal Engine since i would throw away all the time i have invested learning unity

p.s. i was about to buy Pro license but.. is it worth paying 4500$ bucks at this conditions?