Unet fails to generate proper netId's until Library is deleted and rebuilt [Separate projects]

So scene from one project (Server) is built into standalone and client is connected from another project via editor => it works fine.

However, when I change something in the scene (Either in server or client project), it messes up netId order.

Note that objects and their enable/disable state is the same. SyncVars/Lists/Events/Cmd/Rpc are identical. AssetId's are the same.

And in those cases only full Library rebuild helps (Delete -> Rebuild).

Does anyone know what can be the reason of it?
Happened to me in 5.5, 5.6, and still present in 2017.1.

First I thought it was Assemblies and UnetWeaver somehow messing up classes. However, deleting and rebuilding only assemblies does nothing. It's still the same issue.

My wild guess is that scene is somehow serialized differently into separate projects, or writes wrong id's to the Library. Idk.

This is bogging me quite some time, and now it's literally happens all the time I do changes.