4.5 - Serialization Depth

Right best go back and edit.

 

I suspect that it is, but I don't think that means it's broken such that it needs to be "fixed"...

While I haven't done it for quite some time, I used to be able to edit and reload scripts at runtime in Unity. As far as I understand, that means the scripting runtime needs to be able to grab all the data out of the running objects before the reload and poke it back in afterwards. It wouldn't be much use if it couldn't do that to all of your fields, otherwise the internals of every object would get borked every time it happened (which is what typically happens to me now when I try, but I use a lot of stuff that I know isn't serialisation friendly anyway).

So, I think it's a requirement of being able to recompile code while your game is running in the Editor, and I suspect that's a deliberate feature since otherwise I'd expect recompiles to be delayed until you exit play mode.

 

But the whole reason we're supposed to put "Serializable" on private fields is because the private fields weren't supposed to be serialized. If they are being serialized anyway, then why make us add the redundant attribute in order to keep the state?

 

Good question.

My guess - and that's all it is - is that there's internally a difference between short term persistence (within an Editor session, where the intention is hot-loading of compiled scripts) and long term persistence (via disk, where the intention is to re-load data we explicitly wanted saved to disk).

Perhaps it's not documented because it's meant to be something the Editor does transparently for us?

In particular...

1. There's a difference between "weren't supposed to" and "aren't". As I already mentioned, there are actually good reasons that Unity might internally need to store that data.
2. We're supposed to put Serializable on private fields based on use case, not based on implementation. In other words, we're marking stuff we want saved permanently, we're not telling the Editor how to do its job.

Basically what I'm saying is that I think there are actually two jobs going on here that happen to have a lot of overlap in implementation. I don't think it's a bug that a bunch of other data also gets stored, I think it's deliberately done to facilitate other functionality, but this new error message is exposing it and confusing matters because of the functional overlap with things we do on our side of the fence.


To be completely honest, while I'm not aware of why they've chosen this particular approach and there could be great reasons I simply haven't thought of, I think this error is a bit... silly... for other reasons anyway. Partly because eight layers isn't very deep, and partly because I don't see this as an effective solution against cycles - depth alone isn't a good indicator, and I'd have thought it's better dealt with at write time rather than at read time... I mean, if I serialise something and the data is junk because it's got dodgy cycles or because cycles have been flattened, then it's far more useful to know about that at the time instead of later on when I go to re-load it.

 

For the record, the AudioController class in the Audio Toolkit by Clockstone ( http://unity.clockstone.com/assetStore-audioToolkit.html ) definitely exhibits this behavior. I've contacted them about it. Will keep everyone posted.

UPDATE - So the previous message was sent after I'd updated to the latest version of the plugin (6.7) and the plugin still exhibited this behavior. HOWEVER, I determined it was this by disabling the AudioController class instance which caused the errors to no longer display.

Once I enabled the instance of the AudioController class again, the errors no longer showed.

Not sure if that helps anyone, but that's what's happened on this end.

 

The sudden inclusion of this new warning error with the explanation that a class referencing itself is a "huge performance hit" despite no one else ever experiencing that in the last 4 and a half versions of Unity, with no followup answers from Unity, seems like maybe the dev who included this new warning is not on the same page as the rest of the dev team, and that Unity themselves aren't totally sure how their serialization works right now. My guess is they're going to leave this and almost all other Mono issues unresolved with the hope that they can get il2cpp out the door, which will likely be changing serialization anyway.

 

Where does the idea that this is in any way related to IL2CPP or versions of Mono come from? I also don't understand the implication that they "aren't totally sure how their serialization works" comes from.

Unity's serialisation is... odd, I'll give you that, but since I've got little experience creating serialisation systems I can't really pass meaningful judgement.

 

I mean since they're going to have a huge change to how Mono works plus an upgrade to a new Mono once they implement IL2CPP, they're probably not going to invest a whole lot of time trying to fix this version. My statement that they aren't sure how it works comes from the conflicting explanations in this and the "All about serialization" thread.

 

They use their serialization system to marshal everything between the Mono runtime and the underlying C++ engine. With il2cpp getting rid of the runtime, they will either skip serialization there or maybe have some new weird version to go between the il2cpp "sort of a runtime" and the engine. But yes, this is speculation. Still, I think if they were ever going to answer any of the questions in this thread they probably would have replied by now.

 

In this thread: http://forum.unity3d.com/threads/serialization-best-practices-megapost.155352/

They say when you press play, this happens:

• Pull all the serializable data out of managed land, creating an internal representation of the data on the C++ side of Unity.
• Destroy all memory / information associated with the managed side of Unity, and reload the assemblies.
• Reserialize the data that was saved in C++ back into managed land.

And I meant IL2CPP is getting rid of the Mono runtime specifically; i did mention they will have their own sort-of runtime.

 

So if I have a class as mentioned like so

Code (CSharp):

1. <code>
2. [Serialize]
3. public class Node
4. {
5.    public Node Child;
6. }
7. </code>

I don't necessarily care that Child serializes and then its child serializes and so on, if for instance I have something like:

Code (CSharp):

1. [Serialize]
2. public class NodeList
3. {
4.    public List<Node> Nodes;
5.    public Node RootNode;
6. }

So long as each Node in Nodes gets serialized (but their Node field isn't serialized recursively), BUT I need Node.Child to maintain the reference to the Node referred to in Node.Child.

I guess what I am saying is we could use a new attribute:

Code (CSharp):

1. <code>
2. [Serialize]
3. public class Node
4. {
5.    [SerialzieAsReference]
6.    public Node Child;
7. }
8. </code>

So the reference is maintained, but the Value of Child is assumed to be serialized elsewhere.

Otherwise I'm going to have to re-write everything to use some Node.Id and use those everywhere and find actual Nodes from a master list.

Is this the case?

 

If Node is a MonoBehaviour or ScriptableObject then serialisation by reference works fine, so that's another option (though there are other drawbacks).

 

I am surffering for this problem, too.

I love unity engine.
But I can't understand why serialization system was implemented this way.

I made my own serialize system in unity environment.
This support null, polymorphism (without scriptableobject), and circular reference.
There is no performance penalty.

Until now, I implemented serialization three times.
So, I think I know it is difficult to implement useful serialization system.
But, I really don't think it is not reachable goal.

Is there things i missed?
Or this function cause too massive change to currently unity implementation?

(I'm not native english speaker. sorry for my poor writing)

 

There probably is, but for your use cases it probably doesn't matter. Unity uses serialisation for a huge number of things and, while I'm not experienced enough in the area to be able to judge whether or not it's a great solution, it makes sense and lines up with what other game developers working on similar systems have told me in the past.

 

Okay, I can handle adding System.NonSerialized attributes to my private fields for whatever reason it's needed (despite the documentation saying the opposite). Could Unity at least give a backtrace of the fields when this error occurs to make it easier to find the "cause" of the problem?

 

I remember this thread from forever ago and we never really did get an answer: does anyone at UT know if the serialization of private variables that are not marked serializable is a bug or a feature? And if it's a feature, what is the point of marking things Serializable, and will there ever be a change to the documentation since it is all incorrect?

 

Does this emulate the same choices of serializing private variables that Unity itself does (which no-one here seems to understand nor can we seem to get a straight answer other than "sometimes Unity does it")?

 

Thanks, I'm glad we finally got an answer. I'd agree that private variables should not be serialized, even if it breaks some editor windows, because currently the documentation on how to deal with serialization in editor windows is all wrong anyway, and people shouldn't be relying on an undocumented, sort of accidental preservation of their variables. It also makes much more sense to ask people to put [SerializeField] on private variables they want to keep, as the docs say to, then changing the docs now to ask everyone to put [Nonserialized] on all the private variables they don't want to keep; something that I'm pretty sure nobody at all did until this thread appeared.

 

Actually, that whole thing might benefit from a fresh look. [SerializeField]'s use cases would change dramatically if we had to use it to preserve private data across domain reloads, because currently it's used among other things for displaying private data in the Inspector, ie: separating the designer interface from the coder interface. There already exists [HideInInspector] to make something that gets serialized but does not get displayed, so we're functionally covered, but is that a well designed workflow? I think if this stuff has to change anyway we might be better off if the designers consider the use cases rather than what's happening under the hood - [ExposeInInspector], [PreserveForDomainReload] and [SerializeWithScene] or similar.

Having said all of that, as long as we know when and why privates are serialized, what's the problem? It makes sense to me to preserve the entire state for domain reloads, because as a user I expect that when I play and unplay everything is exactly as I left it, including internal state.

 

To me, it's the opposite; I expect that private variables that are not settable from the inspector have their default values whenever entering or exiting play mode. Especially since that's what the documentation more or less says, and since there are instructions on how to preserve data within the editor for various reasons (ScriptableObjects, SerializeField, etc). I can't think of any situations where I would particularly want the current behavior, where some private variables are preserved in certain odd situations like exiting play mode but not in others like recompiling or opening/closing an editor window or clicking on a different inspector. Especially since this current, mostly unwanted behavior apparently leads to "huge performance hits" as mentioned earlier.

 

I'd probably agree if I'd run into the "certain odd situations" you mention. To be clear, though, I'm talking about stuff in the scene, not editor windows and such. In any case, if/where there are inconsistencies they should certainly be cleared up.

 

I got this error when I used an auto-implemented property, even though it shouldn't be serialized. I moved some code around and it seems to be fine now.

BEFORE (throws error):

Code (CSharp):

1.  
2.   [System.Serializable]
3.   public class AudioInfo {
4.   public List<AudioClipGroup> m_audioClipGroups = new List<AudioClipGroup>();
5.   //...
6.   }
7.  
8.   [System.Serializable]
9.   public class AudioClipGroup {
10.   public List<AudioClip> m_audioClips = new List<AudioClip>();
11.   public AudioInfo info { get; set; }
12.   //...
13.   }
14.  

AFTER (no error):

Code (CSharp):

1.  
2.   [System.Serializable]
3.   public class AudioInfo {
4.   public List<AudioClipGroup> m_audioClipGroups = new List<AudioClipGroup>();
5.   //...
6.   }
7.  
8.   [System.Serializable]
9.   public class AudioClipGroup {
10.   public List<AudioClip> m_audioClips = new List<AudioClip>();
11.   [System.NonSerialized]
12.   private AudioInfo m_info;
13.   public AudioInfo info { get { return m_info; } set { m_info = value; } }
14.   //...
15.   }
16.  

 

Weird, report a bug.

 

I started getting this error after I upgraded to 4.6.2 yesterday... :/

I am not clear on how to fix.

Nvm, I finally understood the error and found the cycle in my code and fixed it. Dunno why I coded it like that in the first place

 

If it helps anyone, I just had a recursion depth issue, that was a bit trickier to solve, as the problematic class was never actually identified.

The issue was caused by an API class that I had on my GameSaveData class. So it was structured like so:

Code (CSharp):

1.  
2. [Serializable]
3. GameSaveData {
4.    public GameSaveApi api;
5.  
6.    public GameSaveData(){
7.           api = new GameSaveAPI(this);
8.    }
9. }
10.  
11. [Serializable]
12. GameSaveApi {
13.       //Internal references to parent instances
14.       private GameSaveData data;
15.       private LevelProgress progress;
16.  
17.       public GameSaveApi(GameSaveData data){
18.             this.data = data;
19.             progress = data.progress;
20.       }
21.       //various api calls...
22.       public void SetLevelProgress(){}
23.       public void CollectStar(){}
24.       public void CollectItem(){}
25.       //etc
26.  
27. }

Having a child object, with a private reference to a parent object, causes this type of recursion.

I should never have been serializing GameSaveApi anyways, since it really has no state, simply removing [Serializable] fixed all issues.

It would have been nice though, if the error message would've been able to more accurately point to the issue. The error never once mentioned GameSaveAPI the source of the recursion, instead complaining about GameSaveData, LevelProgress as having recursion issues, when both of those classes were actually fine and exhibited no obvious signs of recursion.

It probably would have been more obvious, had I been aware that private fields were actually serialized behind the scenes, but since all of GameSaveApi's fields were private, I just assumed it was not an issue.

 

Is this something that will be worked on in Unity 5?
Since the code is working I would like to know if we should spend a week optimizing this or if unity 5 will bring some changes to this which will improve the performance.

 

There haven't been any major serialization changes in the Unity 5 beta so far, so I don't think they have any short term plans to fix any of this.

 

Just updated to 4.6.3 and this happened to me. I have a tree structure, which obviously has a recursive reference. I serialize it using the BinaryFormatter, so not having it serialized is just not an option for me. The editor doesn't even touch this class ever.

This is getting pretty annoying, why are all the bug reports on this marked as resolved, when it is clearly not resolved?

 

Well, from Unity's POV this is working as intended. :-/

I'm in the same boat as you. Developers prior to me used Serializable/SerializeField to tag non-scene data objects for serialization in a tree type structure, and now that I've updated from 4.5.4 to 4.6.3 (despite it NOT complaining during 4.5, which seems weird after reading this thread) I get all the errors.

So, currently my plan is to switch over to using actual XML tags. It's unclear to me why Serializable/SerializeField actually work with XMLSerializer. It seems like a bit of a "trap".

( http://wiki.unity3d.com/index.php?title=Saving_and_Loading_Data:_XmlSerializer )

edit: on further testing, it seems the developers before me put the SerializeField tags in for no reason whatsoever, and that the XMLSerializer was just doing its own analysis.

 

So, I understand the useful nature of this message for people just beginning to create new projects to be aware of the recursive limitations in Unity's serialization system, particularly if a large portion of performance based reports stem from this limitation. This is a good thing.

What I have a hard time understanding is the service failure. There are thousands of projects prior to this release that use recursive systems quite cautiously, so rendering them near useless because they could possibly be taking a performance hit and not allowing them to accept responsibility is just not logical.

I read somewhere in the past that custom inspector scripts are ignored in the compilation build of the end project. Is this true? If it is, is there a reason possible performance hits matter in custom inspectors? They shouldn't impact the game at all, ever.

Lucas wrote a blog about this in the summer of 2014. In that blog he states:
We actually messed up the warning implementation in such a way that it gives you so many warnings, you have no other option but to fix them right away. We’ll soon ship a fix for this in a patch release, the warning is not gone, but you will only get one per “entering playmode”, so you don’t get spammed crazy.

I just now updated my project to the very latest version of unity 4. Can we get an ETA on this patch release? I can't imagine it legitimately took a year to implement or figure out, so was it just forgotten?

Edit - 4.6.1 Does not throw the error message.
Edit II - For the record, "fixing" this is relatively easy. It's just not always necessary. My only gripe is just the 'my way' approach.

Regards,

 

One option that might be nice to see is allowing a build setting like "Don't ever serialize my private variables". As I understand it, the main reason Unity does it is so that you can live-edit code while a scene is running in play mode and keep going, but that feature has never actually worked for me in any build of Unity. In this case, is there a need to serialize private variables or references that will disappear when exiting play mode?

 

How can i track which class / field is causing this error? I'd gladly put the [NonSerialized] there, but the thing is i have no clue... the error log gives me nothing but the deepest class name, which is some random not serializable class. Is there any way how to track the root of this serialization tree? Where do i put the attribute? It's driving me and the whole team nuts, we had to update unity and this bug is slowing us down

 

Serialization depth limit exceeded at 'GameDraw:art'. There may be an object composition cycle in one or more of your serialized classes.
UnityEditor.HostView:OnGUI()
I get hundreds of this error too after each build and just confuse about this.

 

Well I'd love cyclic weak references for members that aren't serialized as well. I'm not sure how to do it without setting unity off on one.

But regarding it never working for you to update live, this is because you have to AFAIK repair all references, ie all scripts need to setup again using OnEnable or such. Been a few years since I last bothered.

 

We recently updated to 5.1 from 4.6 and are now getting the Serialization depth limit error even though our code is the same. The class that it is reporting does not have composition cycles - our project is very data driven and our data hierarchy legitimately goes 6 or 7 levels deep in the serialization with no recursive references.

It is a hassle to see a few hundred errors for this (we don't seem to be losing any data from it either). What I would like to see is an editor option to define an acceptable serialization depth (that is assuming you can't add real cycle detection) - this hard coded magic number approach is sloppy at best.

 

We hate to necro this thread but since this seem to be the first thread that pops up when doing a search for "Unity serialization depth limit exceeded", we figured we would post our solution here to give back to the community that we have leeched off of for so long

Quick disclaimers:

• Thanks to superpig for his suggestions which are the basis of this solution.
  
• This is probably one of many ways to solve this problem
• This code is lightly tested, but will be testing and updating this code, as we put it to use in our current game
• We're totally open for suggesting on how to make this code better
• We're using Full Inspector since its was the easiest way to test this code. There a couple bits of code that should be able to be remove/replace, as noted, if Full Inspector not part of your pipeline.
• We hope this isn't something super obvious and is helpful for the community.

Without further ado here is our solution.

First of all we've created a nestable interface that you can implement in your own classes:

INestable.cs

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3.  
4. public interface INestable : IEnumerable
5. {
6. // Make sure this is [NonSerialized] in the implementation
7. INestable parent
8. {
9. get;
10. set;
11. }
12.  
13. // Make sure this is [NonSerialized] in the implementation
14. List<INestable> children
15. {
16. get;
17. set;
18. }
19.  
20. List<int> indices
21. {
22. get;
23. set;
24. }
25.  
26. void Initialize();
27. }
28.  

Here is an example of an implementation that you can use as a base class if that works for your data structure or copy and paste this into your own class:

NestedNode.cs

Code (CSharp):

1. using FullInspector;
2. using System;
3. using System.Collections;
4. using System.Collections.Generic;
5. using UnityEngine;
6.  
7. [Serializable]
8. public class NestedNode : INestable
9. {
10.     [NonSerialized]
11.     private INestable _parent;
12.     public INestable parent
13.     {
14.         get
15.         {
16.             return _parent;
17.         }
18.         set
19.         {
20.             _parent = value;
21.         }
22.     }
23.  
24.     [NonSerialized]
25.     private List<INestable> _children = new List<INestable>();
26.     [ShowInInspector] // NOTE: Using Full Inspector to show list
27.     public List<INestable> children
28.     {
29.         get
30.         {
31.             return _children;
32.         }
33.         set
34.         {
35.             _children = value;
36.         }
37.     }
38.  
39.     // The indices are the only things serialized on this class and they are hidden from the user
40.     [HideInInspector]
41.     public List<int> indices
42.     {
43.         get;
44.         set;
45.     }
46.  
47.     public void Initialize()
48.     {
49.         if ( children == null )
50.         {
51.             children = new List<INestable>();
52.         }
53.  
54.         if ( indices == null )
55.         {
56.             indices = new List<int>();
57.         }
58.     }
59.  
60.     public IEnumerator GetEnumerator()
61.     {
62.         return children.GetEnumerator();
63.     }
64. }
65.  

And here is the root level structure that does all the book-keeping:

NestedList.py

Code (CSharp):

1. using System;
2. using System.Collections;
3. using System.Collections.Generic;
4. using UnityEngine;
5.  
6. [Serializable]
7. public class NestedList<T> : ISerializationCallbackReceiver where T : INestable
8. {
9.     [SerializeField]
10.     public List<T> children = new List<T>();
11.  
12.     [SerializeField]
13.     public List<T> hierarchy = new List<T>();
14.  
15.     public void Initialize()
16.     {
17.         if ( children == null )
18.         {
19.             children = new List<T>();
20.         }
21.  
22.         if ( hierarchy == null )
23.         {
24.             hierarchy = new List<T>();
25.         }
26.     }
27.  
28.     private int SerializeHierarchy( T node )
29.     {
30.         if ( node == null )
31.         {
32.             return -1;
33.         }
34.  
35.         node.Initialize();
36.  
37.         hierarchy.Add( node );
38.  
39.         node.indices.Clear();
40.  
41.         if ( node.children != null )
42.         {
43.             foreach ( T _child in node.children )
44.             {
45.  
46.                 node.indices.Add( SerializeHierarchy( _child ) );
47.             }
48.         }
49.  
50.         return hierarchy.IndexOf( node );
51.     }
52.  
53.     public void OnBeforeSerialize()
54.     {
55.         Initialize();
56.  
57.         hierarchy.Clear();
58.  
59.         foreach ( T _child in children )
60.         {
61.             SerializeHierarchy( _child );
62.         }
63.     }
64.  
65.     private void DeserializeHierarchy( T node )
66.     {
67.         node.Initialize();
68.  
69.         node.children.Clear();
70.  
71.         foreach ( var _index in node.indices )
72.         {
73.             if ( _index == -1 )
74.             {
75.                 node.children.Add( null );
76.             }
77.             else
78.             {
79.                 var _child = hierarchy[ _index ];
80.                 _child.parent = node;
81.  
82.                 node.children.Add( _child );
83.             }
84.         }
85.     }
86.  
87.     public void OnAfterDeserialize()
88.     {
89.         Initialize();
90.  
91.         foreach ( var _child in children )
92.         {
93.             DeserializeHierarchy( _child );
94.         }
95.     }
96.  
97.     public IEnumerator GetEnumerator()
98.     {
99.         return children.GetEnumerator();
100.     }
101. }
102.  

And finally here is an example usage:

Foo.cs

Code (CSharp):

1. using FullInspector;
2.  
3. // NOTE: Replace with Monobehaviour if not using Full Inspector
4. public class Foo : BaseBehavior
5. {
6.    // We are using our base class NestedNode here with the NestedList
7.    // but you can use your own sub-class or your own implementation of the INestable
8.     public NestedList<NestedNode> nodes = new NestedList<NestedNode>();
9. }
10.  

We hope this is helpful for anyone running into this problem. Please let us know what you think.

Cheers

 

Here is how i got around the Error message, quite simple.

This codes triggers the error:

Code (CSharp):

1. class MyComponent : MonoBehaviour
2. {
3.      private MyTree myTree;
4. }
5.  
6. [Serializable]
7. class MyTree
8. {
9.      public List<MyTree> children;
10. }
11.  

And this code doesnt:

Code (CSharp):

1. class MyComponent : MonoBehaviour
2. {
3.      private MyTreeContainer myTree;
4. }
5.  
6. class MyTreeContainer
7. {
8.      public MyTree MyTree;
9. }
10.  
11. [Serializable]
12. class MyTree
13. {
14.      public List<MyTree> Children;
15. }
16.  

So just add a class that is not marked [Serializeable] as a container for it.
Hope it helps someone.

 

Sorry to return to an old thread but why in the earth do I need to do this? If the list of children is empty, why do I get this warning? Isn't it up to me to make sure that the depth just doesn't go too deep? What is the reasoning for generating the warning in this case? Default list is empty, and empty list stops the hierarchy.

I find this child index book-keeping extremely unintuitive. It makes authoring hierarchical datastructures even more tedious. Speaking on which... is there a easy (asset store?) way of editing hierarchical things in same way that scene hierarchy or ReorderableList work? That'd be nice.

 

Thanks for taking time to answer this. It makes sense now, to be honest I was kind of assuming it's probably doing something like that in the background.

The issue is that I've come across the same limitation more than once, and every time it feels like I'm need to jump some unnecessary hoops to get the data structure to work the way I want -- at least in the users perspective (editor side).

 

It takes one extra minute just to print these errors in the console... is there seriously no way how to at least find the root / path of what exactly has the "composition cycle"? Because i am clueless.. i am now adding [NonSerialized] to any code i can find, but still get 30k of these errors and it's really slowing down the production of entire team!

 

I get 'Serialization depth limit exceeded' on simple non-monobehaviour classes (though with nested children of same type) I create which I have no intention of having serialized, so have to add nonserialized to the affected variables just to get them to compile.
This seems a bit silly but is it just a better alternative to having to add serialize to all classes you Want to serialize? (this is in js BTW (I know...) ). Can you not mark a whole class to serialize or not?

 

My problem is that i don't know which class is doing this and i have no way how to tell