Collaboration slows unity to a halt?

Well, I have the Windows firewall, that should be it. My router has a firewall as well, but both should use UPNP.

 

I can share the project with you, I will begin opening it, so it is ready.. maybe.. when you PM me.

I dont use any proxies.

 

I also just noticed that there is a sort of loading bar on the Unity icon. It has a green color moving across it. I just timed it again, 4 minutes opening Unity, then a bit over 7 minutes with the Collab part. Well, close to 8 minutes.

 

I mixed up some numbers. The build of the project is 1.5 gb, the project itself (with cache files) is 16 gb. Without cache files and meta data it is just below 10 gb.

 

Ok,

I think I might have found something.

I fully support the description of having our project held "hostage" to Collaborate at times, since I have also had really long freezes on load for projects using Collaborate for the most of the duration of our five month project. It has been worse with some projects, notably those with a large number of assets.

The last couple of days I noticed that during freeze, the disk load is reported to be at 100% on my computer, even though transfer rates are in the kilobytes. This happens exactly when I load a project that is linked to Collaborate.

The load is not from Unity, however, but from the Windows Anti-malware executable.

When temporarily disabling "real time protection" in Windows Defender, the freeze goes away and the project loads and the editor becomes responsive almost instantly.
https://support.microsoft.com/sv-se/help/4027187/windows-turn-off-windows-defender-antivirus

I'm not happy leaving the default real time protection off so I don't think this is a solution but if someone can verify it works for them too, it might be useful as a workaround for the time being, and also give some leads to what is happening ie the Unity editor working with files in a way that triggers a real time scan that creates some kind of lockup, possibly looking for a newly created file that is not yet possible to read or something like that.

FWIW, this was tested using Unity 5.5.2f1 and Windows 10 Creators Update x64. But I've seen the exact same behaviour Before getting the Creator's update and also tried using 5.6.2 which was just as bad.

 

I think I already tried that ixdeveloper, but I tried it again just in case. Even with Windows Defender turned off, my loading time is still about 10 minutes. I am trying to move the project to a MacBook Air now since it should have less loading time.

 

Okay, so I think I got to the bottom of this. I copied the whole project to my SSD and opened it from it. First it was slow as usual, but I checked my active processes and noticed the anti-malware thing mentioned, so I deactivated that and the whole thing loaded within a minute, rather than the usual 10 minutes. But I also noticed that it was reading 300 MB per second for a few seconds while opening. Seems it actually read through EVERYTHING in the whole project. For some reason Unity seems to be capped at 25% of the disks possible read speed, but for the SSD that is a bit over 300mb per second. For my hard disk its about 40-50mb per second.

Pretty sure that will shorten the lifespan of the SSD a bit if it does this every time, and at any rate, it should really not need to do this every time. I am pretty sure Unity already has faster ways of checking the project for changes.

I hope this helps get to the bottom of what the problem is. Maybe make it standard that Unity does not expect changes to have happened while it was closed?

 

Don't worry about that. It takes about a petabyte of writes on a 256GB SSD before it's likely to fail completely

It could either be that Unity uses access time rather than actual change time for the project directory to trigger a rescan (which would be wrong), or that the protection in the system changes last modified timestamps, technically making files "new".

If Unity uses mere access time (reads) to decide when files need to be refreshed in version control and such, I consider that a very serious bug. It'll at least cost unnecessary upload/reimport time.

 

A Petabyte gets much closer if at least once each day it gets more than 16gb data read just because you open the project. This is all on top of Windows booting, caching and loading other programs. Not that its likely to become a problem, especially if its mainly reads. I usually do try to use my hard disks for at least 5 years though.

Either way though, it does not need to be done for no reason.

 

Hi @ixdeveloper! Have you tried leaving the realtime protection on but adding Unity Editor to Windows Defender's list of apps that it will let through to see if that still removes the freeze?

 

I tested this. Adding the Unity Editor itself is not enough, you also have to add the project folder as part of the exception. First I tried with no exceptions and Windows Defender on. It took about 3-4 minutes to open my project, the reason it opens this fast is that I now put it on my SSD instead, but Windows Defender reads files about 40-70 mb per second while my project opens, especially when it checks for files changed due to using Collab, while Unity also uses pretty much the same amount of disk access.

Then I tried with Unity and the Unity folder excluded but it still took a few minutes to open, same problem. So I tried disabling Windows Defender real time protection. Now it opens in about 1-2 minutes total. With Windows Defender active, the 3-4 minutes was mainly the part after Unity began with the file changes part, so it was probably 5 minutes total. So I excluded my whole project folder, and it now took the same 1-2 minutes to open the full project with Windows Defender active. But while checking for changes that one minute, Unity goes up to 200-350 mb per second of disk activity. So it seems to check every time you open the project for changes in every single file.

 

Any news on this? Seems we must have gotten closer to figuring out the source of this problem?

 

This is definitely a known problem. I've ran the collaboration feature on three separate projects until just moving back to Perforce. I am super excited that you guys are even trying to do this feature. I think it's functional but with large projects it's just not worth the hangs. My project hangs for 5-10 minutes every launch. This happens for everyone on the cloud project. It is pretty annoying to hear a Unity rep respond with "Do you have a firewall? Is your internet working"? No, I don't have my firewall enabled, yes, my internet is working and I have the same problem these people are having. I know you need to rule out the simple things but I can assure you that there is a problem with this feature.

 

Any news on this? I have tried and the results are exactly as @kblood stated.

 

Hi @legend768 and @ixdeveloper! I'm sorry that you're both running into this issue. We have a fix in 2017.2 that should help to reduce some of the scenarios where a temporary drop in network stability would cause the editor to freeze. We'll be looking into a backport for 2017.1 but many of our devs are in Austin for Unite so it'll be around a week until we can make more progress on this issue. Thank you all so much for your patience!

 

I had a constant and periodical freezes in Editor since I've installed 2017.2.1 after 5.6.3 (clear installed, not updated). So it seems, guys, your Editor sends some data upwards and can't be switched off. Only if I'm working offline, there's no freezes.

 

Hi @DmitriSkif! That is super frustrating! Would you mind going into a little more detail about the issue? Does it freeze when you try to publish or update using Collab, when starting the Editor or when just idling? Does it ever unfreeze after waiting a long time?

Can you use our Network Reporting Tool and DM me the generated report on your Desktop? Then could you try running Unity with the -enableCollabEventLogs command line argument, then wait for a freeze to happen again and send me your Editor logs? You can find your Editor logs here.

 

Hi Ryan, I'm taking freezes constantly, not depending which action I'm doing in Editor, the only one significant factor is : is my Internet connection active or not. There's no freezes in Offline mode. As I have Personal edition, I have no ability to disable Editor statistics. Ok, I'll try tools you've mentioned and send you a report.

 

It's happening to me since I updated to 2018.1.0f2 :/ In beta it was working perfectly...

 

This started happening to me out of nowhere since today, on two different locations (different windows computers on different networks), I'm using 2017.4.1f1.

I have to turn off collaborate so I'm able to work.

 

Happens to me too since today!
http://prntscr.com/jdpafu
Unity 2018.1.0f2 and Unity 2018.2.b/2/3

Wireless connection + decent gaming notebook.
I am using latest Unity Hub. Version 0.16.
It was fine yesterday. I am unable to work with unity until I disable Collab.

 

Same, have to disable collab.

 

Thing is Collab tries to scan the the whole project, all the files, even the cache it seems, every time you open the project. It seems to expect that you changed something between opening and closing the project.

Should not be a big problem if its a pretty small project, but if you get a project that takes up several gigabyte... well, it better be on an SSD and have a good Internet connection I guess.

 

I just enabled Collaborate again and it seems, I hope, that the issue is gone.

 

Hey everyone! We added a post about the issues causing the Editor to lag last week. Check it out here and if you're still encountering issues then please let us know.

 

I'm still encountering the collab-freezing problem in Unity 2018.1.0f2 (64-bit) and Unity 2018.2.0f1 (64-bit).
The editor.log is too large(7MB) to upload so maybe I could e-mail you it as an attachment.
Right now my solution is to close the collaborate service till next time I need to commit a major version.

 

Hey Null-Fun did you find a fix

 

I'm suffering from unity freeze, too. Unity 2018.2.0f1

 

Sorry, not yet.
According to the post of kblood, I think it's because of my snail-speed uploading network, I filed a bug report to the Unity team and haven't got a feedback yet. Let's wait for their response first.

 

How can it takes MONTHS to fix this kind of problem? Every single update I have to wait 5-10 minutes to be even able to use UNITY. I think I go back to SVN, collab is complete joke.

 

Tell me about it im paying for a bunch of people to use something that dont work and no one at unity can answer

 

Hi everyone! I'm sorry for the delayed response to this issue. We definitely want to resolve this issue for everyone as quickly as possible.

I'd like to double check if any of you are developing out of China or Russia. Collab requires a reliable connection to certain services, such as AWS. Unfortunately, due to region-specific firewall settings, many Amazon-specific services are interrupted in those regions, which negatively affects Collab in the process. We are investigating ways to work around this issue and allow our services to be hosted in those regions but in the meantime there are users that have had some success with continuing to use Collab in those regions using VPNs.

If you are experiencing this issue and are not developing out of China or Russia then please get in touch with our Support Team here so that we can get back to you quickly.

 

*1843 is publishing test.
*2413 is the afterward test.

 

One of the biggest problems with Collab I think, is that it currently also decides to synchronize on its own, and you cannot disable it. So when you open a project, it will assume something might have changed and check ALL files in the project for changes and I think it also does this now and then while using the project, maybe not all files, but some of them.

Especially since Unity projects might become several gigabytes in size and they might not be running from SSD disks it seems like it should be something the user was given more control over, and hopefully we could even eventually get a more GIT or SVN like commit feature where we get to see and choose what changes to commit and push to Collab. Right now its pretty much a black box doing magic.

 

Yes I figured out the problem. I'm in China and thanks to the China's GFW that makes the Collab bothering me so LONG.
I solved it with a little free software called Proxifier that proxy Unity's internet connection to ShadowSocks which is a program breaks the GFW. And everything in Unity goes smooth then. I can see Unity try to connect Google Analytic service in Proxifier and it makes me confirm that the problem doesn't come from Unity or Collab or something else, it comes from CHINA.

 

I'm so disappointed that the problem still exists today in Unity 2018.3, it has been 3 years since the post come out

 

Yeah, I have stopped using it, unless its a really small project less than 500mb.

It does help a lot to have the project on an SSD btw.

At least for me the issue was read and write speeds of the hard disk, maybe along with other programs and services running in Windows. But I suspect the biggest problem is that it decides to check all the files for changes when you start the project, and then does this again now and then while using the project... rather than doing something like regular version management, tracking the changes and then pushing them to the repository.

So even when you just open the project again, it expects there might have been changes to the files since you closed it. Which might be the case, but should be something the users starts manually I think.

So maybe just look into alternatives. To have version control on a Unity project and not just its code, Git and SVN is not recommended, but there are other solutions that could work, or you could use a mix. Like using Git for scenes, scripts and all these small files and use an FTP or something similar for the larger files where you then just backup the larger files without version control, which does get really big really fast anyway.

If you upload a video to a git repository you have to consider it will stay there forever. You might delete it, but it will be part of the git history. On top of this git usually does not give you a very high speed to work with, so it would take a very long time to sync the changes of such files. But there are complete solutions as well for projects with large files and getting version history, but then they cost money... like this feature does if you want to use it for large projects.

 

It has been almost two years. Which is still way too long, I agree. However, we have been promised the end of 'Checking for changes' for end of March.


This year. Before you ask

 

The 2.0 package is now out, which will directly addres the "Checking for changes..." message! Check out the release notes / known issues post for more information. Thank you guys for your patience while we worked on this fix.

 

That is great news

 

Would be nice if I can use a program like source tree to see the changes, because I don't like using only a website to see the files. It slower on a site.

 

Try disable Windows Defender / Exclude your Unity Project folder

I got hangs with Defender sitting on 1.5gb, disk usage near 0, cpu hovering 10% possible just the Unity Editor rendering

Collaborate hung possibly calling back from Defender blocking file access?
dunno, but check Windows Defender maybe?

I wish the Unity workflow for setting up Diffs was better
What's the best Diff Tool?
How come I have to edit bash scripts or something to get the Diff tool to actually Diff the files (totoiseDiff installed, linked but it opens a blank diff where I have to select Original and Modified files, really confusing don't know how to fix)

Is the Collaborate supposed to work for merging Scenes?
e.g. if two people change a scene, different entities, will it be able to merge the result?
so far we're thinking of resorting to separate scenes to edit in because the Diffing is broken and we keep losing someone's changes on Scene updates

 

Hi @shodude, we recommend the following diff tools for use with Collaborate:

OS X:

• Apple File Merge

Windows:

• WinMerge

OS X and Windows:

• Beyond Compare 4
• SourceGear DiffMerge
• TkDiff

As for merging Scenes, Collaborate is able to use the Unity Smart Merge tool to handle merging of simple differences between scenes. However, I'd strongly recommend adopting the practice of keeping one master version of a Scene that is used for builds while making copies of those Scenes for each team member that is making changes. When a team member is done making changes then they can have their changes copied into the master Scene one at a time to ensure that no important changes are lost from an improper merge. This method has worked well for many teams, myself included, on previous projects.

 

Cool, this workflow sounds logical enough. I use git and set Unity to use text serialization but some scenes are still stored as binary files, is that a bug or caused by incorrect settings of my team?

 

Hi @zhuchun, your Scenes should be serialized as text if you specified in the Project Settings->Editor menu that their serialization is set to Forced Text (Collaborate does this automatically). Double check that those settings are correct and if they are but you're still seeing binary Scene files then please submit a bug. Also, while Scenes should be serialized to text, some assets, like Terrains, are always serialized as binary regardless of the Editor settings. I wanted to give you a heads up about that.