The Architecture of a Saved Game (what's best?)

Or, perhaps a Game Saving system.

Without (at this point?) going deeply into code, I'm starting to think of how to write a saved game system for an RPG style game.

I am assuming, at this point, that I'm also going to be using multi-scene editing to help load and unload scenes, so I'll probably have a "persistent" scene and then load and unload more specific scenes as I move through areas or levels.

Doing some basic research, most suggestions seem to be like these:
http://gamedevelopment.tutsplus.com...oad-your-players-progress-in-unity--cms-20934
http://answers.unity3d.com/questions/8480/how-to-scrip-a-saveload-game-option.html

TLDR; use System.Runtime.Serialization.Formatters.Binary; and System.IO; to create a saved game file.

Now, as an aside, I'm assuming that even tho' Unity now has JSON support (http://docs.unity3d.com/Manual/JSONSerialization.html) that this isn't that much more wise than using PlayerPrefs (http://docs.unity3d.com/ScriptReference/PlayerPrefs.html), because even though there is much more control in using json; json is very human readable and editable.

So, with these assumptions:

• RPG style game with several levels
• Using Binary data files

What is the best way to set up this system?

When loading a scene / level, I'm assuming I'm going to need to save the states of all interactable objects. My first thought would be to put a "savable" component on each item that could change or have a state we need to know about. The save game system could look for this component (or really - all of these components) and save the data for each "savable" item.

This leads me to think I'm going to need some sort of spawner that, after the base scene is loaded, will spawn all of the saveable items in the scene and set their saved state. I would assume that this spawner might* be able to handle the spawning of updating of the player as well...

I'm more interested in a high level discussion about approach, rather than a too many details about code, at this point. I'm sure I'll be able to take out the machete and hack my way thru this jungle... but if someone's been here before and has some advice, I'd really like to discuss it.

What do you feel is the best approach?

 

That's how the Dialogue System's save system works. It provides a serialization framework and invokes two common methods on all GameObjects in the scene: a "save your data now" method when saving games, and a "load your saved data and apply it" when loading games.

It might be a bit more elegant for your "savable" components to register with a saved game manager. When it's time to save a game, the manager could directly tell the registered components to save themselves, rather than looking through the entire scene for savable components. Since the Dialogue System is middleware, I omitted this to make it simpler for developers to write their own savable components.

Side note: In some cases, when loading games, your savable components may need to load and apply saved data after a frame or two, since the GameObject may need the first frame of Start() to set itself up first before the savable component can correctly change it.

Also consider saving in different levels of detail. This will help cut down the size of your saved game files, which will also speed up saving and loading. It's important that objects close to the player save their state as accurately as possible. However, farther objects, and certainly objects in other levels, don't need to be as accurate. The player probably won't notice if the orcs in a faraway dungeon reset themselves to their original positions.

Finally, consider decoupling the back-end serializer. (Serialization is the easy part anyway.) During development, you might want to serialize your games to JSON so you can read them to help you debug issues. Then switch to binary serialization for testing and release.

 

Saving is actually a pretty easy problem to tackle. Loading that saved data back is the hard part, because it involves wrestling with scene initialization. Unity does not give us very many useful options for controlling the order of loading. If you've done things The Unity Way and utilized Awakes and Starts to control important game flow, you're going to curse yourself. Saving and Loading game state is something that should be considered early in the design of the application. It can be very troublesome to add it in as an afterthought.

The first step is to identify the data that is important to save. Objects which contain important data need to be marked, and the important fields need to be extracted to a serializable format. Bear in mind how you plan to load this data back in, however. MonoBehaviours can be serialized, but they can't be deserialized by us plebes. So I suggest considering a surrogate class to store the data from monobehaviours. If you care about separation of concerns, you can create a Saver, who knows how to write out the important infor for each class which contains valuable information, if not, just teach each class how to write itself to its surrogate.

You're going to need a way to uniquely identify every object that is referenced in a scene. These unique ids need to be consistent across play sessions. The same component that is used to identify an object as needing to save can probably be entrusted with the responsibility to store its unique identifier.

One interesting challenge with deserializing game state is handling references to other objects. You'll either need to recognize object references and convert them to unique ids at serialization time, or you can just change your game to store unique ids instead of object references (the latter is what I do). Object references are a hard, but solvable problem with normal C# classes. Referencing MonoBehaviours directly, adds additional considerations.

I was a developer on a large-ish singleplayer RPG made with Unity. I wrote the saveload system. Separate your object initialization from Awake, OnLevelWasLoaded, Start or any other method that you don't directly control. Make sure that either all of your objects can be reinitialized when it is convenient for you, or that you absolutely control their first and only initialization.

 

To take @kru's suggestion even further, I've seen some devs advocate treating the initial state of the game as a save file. That way there is only ever one path for object initialisation.

 

So, @TonyLi - What I'm hearing from you is that each savable GameObject keeps track of itself. Is this correct? So, rather than having a SaveManager that controls the state of a scene, the SaveManager simply instructs the GameObjects to keep track of themselves? I could see this making sense for, say, a chest - where the chest on Start finds and loads it's own state. But what about destroyed objects? If the player can remove the gold idol from the stand and replace it with, say, a bag of sand - how would the destroyed object know not to instantiate itself? Or does it instantiate itself, check its own state and then delete itself?

The self registration of the "savable" component does make sense.

Your advice feels absolutely sound. I'm trying to wrap my head around how to actually implement all of this. I know I did say I was interested in a high level discussion, but in this case I might need some more detail.

Putting my mind to it, saving has been worrying me and I have this gut feeling it should be thought about first rather than last. Before I make my "things" or elements in the game, I feel like I need to know how they'll be saved as part of their design.

How to restore elements or "load a saved game" was bending my head enough that I started this thread. Some part of me was imagining needing the SaveLoad system to control the instantiation and the state of every loaded element. BUT - if I leave every element to save itself and then reconfigure its own state on load? I'm not sure how best do do this.

So, if I give each item a "savable" component and I use it to hold a unique ID (Can I get this from Unity? Don't all objects have a unique ID? Or does this change every time the project is run?) - how would I be using that ID to recreate a reference? Am I asking the SaveLoad system to use the ID to find me the associate GameObject and then I get my components from there?

Very interesting thought. And it makes sense. Everything is "saved" in its initial state and loaded.

In any case, I'll keep thinking about this...

 

Yup.

That's what it does. (Deletes itself.) Similarly, you can also have empty GameObjects with savable components that respawn new objects that were spawned during play. One of the reasons for this design is that the Dialogue System is middleware. It needs to integrate easily into existing scenes and frameworks. It can't mandate that users change the way UFPS, Adventure Creator, or other products work, which almost universally initialize themselves in Awake, Start, etc. If you have complete control over the code in the project, however, you could do what @kru suggested.

Steer clear of Object.GetInstanceID(). It's unique within your scene, but not unique from run to run. You could use System.Guid.

 

That makes sense. I'll have to wrap my head around an empty parent for each saveable object.

Yeah, I had a feeling that was true (not being unique between sessions). I'll look into system.guid.

Maybe it's time to start a micro prototype.

 

I can describe what I did in some detail. I'm not suggesting that my method is best, but its what I did and it worked for our project. I thought about writing a blog post about my adventures with saveload. This'll be more like a stream of consciousness. And before you charge ahead with anything that I write, see the limitations that I'll write about at the bottom.

Unique IDs

Every object that is important is given a component which stores a System.Guid. The guid is assigned to each object in the editor when that object is placed in a level by designers. Uniqueness is assured by some editor scripts that know how to scan unloaded scenes and gather a list of Guids in those scenes. This was accomplished by reading the .unity3d scene file in the editor. This editor feature was a time sink, not only to implement but to maintain. It took about a week to implement and then whenever unity changed their scene's yaml format, the scene parsing broke, which was another several hours of a developer's time.

In a personal project, I'm considering using scriptable object assets in the project to store the unique ids. It seems promising, but I haven't gone far enough to feel confident that it is a solution. The gist is that every time a UniqueID component requests a new unique guid, an asset is created and tucked away in a project folder to store that guid. My unknown concern is the runtime overhead of loading and storing a scriptable object that is nothing more than a guid. But this does make cross-scene uniqueness easier to track. Any time I need to generate a new id, I can compare it against all of the assets in the tucked-away project folder before validating it. At runtime, I load them all in to memory as part of the application initialization, so that unique ids created for runtime objects can check against the list. This is probably overkill, but it ensures uniqueness.

Depending on how you store your saved data, this may not be necessary. If you store your saved data by scene, then the scene becomes part of the unique id for any object that doesn't cross scene boundaries. So guid collisions (which are exceedingly rare, btw) won't matter.

Initialization

I took Awakes and Starts out of the equation entirely. I have a SceneLoad component which exists in each scene (and it must exist in the scene, not as a global). This component is always at the top of the Script Execution Order list. Its sole job is so that, on Awake(), it creates a gameobject, disables that game object, then reparents every root object in the newly-loaded scene to that disabled gameobject. Thus we prevent any Unity methods from being called on scene objects, except for OnLevelWasLoaded(), which I just don't use at all. One important detail is that the SceneLoad component has to filter out some objects - the UI camera, for instance, since we don't want the camera that is displaying the Loading image to suddenly get disabled!

Alternatively, you could structure your scenes to have a single Scene Root gameobject, and then disable that object on SceneLoad's awake. I created the disabled gameobject programmatically because it was easier than convincing our area designers to make a change to a 100 or so scenes.

Now I have a loaded scene, which I can poke at and manipulate, without any Awakes or Starts or OnEnables having been run, because everything is disabled. At this point, you can take your saved data surrogates, write their fields to the monobehaviours of your loaded scene at your leisure. As a bonus, you can take as many frames to do this as you need, so you won't run afoul of console frame time limits! Huzzah!

For your example with the golden idol and the bag of sand, you could disable the idol and enable the sand at this point.

Once the saved data is loaded, I enable that root game object, which causes the scene to initialize just as it would have normally.

Saved Data Format

We used Json.NET. Unity's JSON implementation wasn't available at the time. Json.NET's converters were very useful. I wrote a GameObjectConverter that wrote out each of its monobehaviors surrogates to a list, and serialized that list. Although, for a first-pass attempt at saving, you could forgo the use of a full serializer. Write your own surrogate classes and teach your custom monobehaviors how to write their data to their surrogates.

My scene saving routine went like this
1) Get all the objects in the scene with a UniqueID component, including inactive objects.
2) Ask each object to serialize its monobehaviors' data.
3) Write that data to disk.

#1 involved some trickery. I get a list of all objects with UniqueID using Resources.FindObjectsOfTypeAll<>(), then I filter the list to remove prefabs which are resident in memory (explained below).

Limitations

The limitations that I imposed were that GameObjects were saved independently. I didn't save parent-child relationships. If a child GameObject had a component that needed to be saved, then that child had its own UniqueID component, and needed to somehow know to reparent itself appropriately on initialization.

The other big limitation is that all prefabs had to saved enabled. A disabled prefab was verboten. The reason for the second limitation is due to the way that I went about detecting prefabs in memory at runtime in builds. In the editor, prefabs are easy to detect. At runtime, they're trickier. Basically, given an object, if its transform.root.gameObject is activeSelf, but not activeInHierarchy, then its a prefab. However, if the transform.root.gameObject is not activeSelf and not activeInHierarchy, I don't know anything. Hence, let's just force all prefabs to be activeSelf - removing any ambiguity.

 

Ultimately, though, it was a pile of hacks on top of hacks. The reason it was such is because we came at this mid-way through the project, and we don't have a clean way of building scenes from stored data the way that Unity does.

If we had a way to build a scene the way unity does, or at least build game objects, I'd have been a much nicer person during those weeks.

The suggestion to change your dynamic scenes so that you build them yourself is a great one. I'd definitely recommend it. You do lose much of the benefit of Unity as an editor. However, if you're early enough in the project where separating static scenes from dynamic scenes is feasible, it is worth a strong consideration.

 

Interesting... The basic concept of building a scene from a saved game file makes sense.

I'd also vote for scriptable objects for persistent data that's read only at run time.

I have used json for .net for another project there's a unit implementation of it on the assets store.

In the end, I'm not sure you'd want to serialize an entire game object, as it's a lot of data, so just saving the important data may be the best bet anyway.

 

@kru just checking: I think I know what you are getting at, but what do you mean by surrogate class?

 

A surrogate is a plain old C# class that has all of the important fields of a monobehavior. You can't create a MonoBehaviour by calling new MonoBehaviour(), but you can create a C# object in such a way. Most serialization libraries make use of surrogates behind the scenes.

this (https://msdn.microsoft.com/en-us/library/ms733064(v=vs.110).aspx) might help provide some context

 

Heh. Yeah. I went through the headache of porting the public json.net repository to our Unity project, only to later realize there was a $20 asset store purchase that did all that work for me. UGH!!!

 

Why waste overhead on System.Guid when a static int counter would do the same job (or a long, if you're really worried about running out of IDs).

I suppose it's micro-optimization but "3" vs "4" is just as unique as "{FB26B684-09E8-4C0E-8FD0-8E98EF8D6524}" vs "{5E453BE5-F808-417A-88E8-5EE5714DC8E1}"...

 

I’m slowly starting to work on a save system now, but why do I need to keep track of the id of the gameobject?

However if the object is based on a prefab, how do i store the reference to the prefab in json?

 

This is how I'm attempting to solve this problem in a networked co-op RPG with shared campaigns. It's a work in progress, but helps me to write it out (and get feedback!) plus I think my needs/solution haven't been covered here yet. I think it's pretty elegant, but also, most things are before you test them

Constraints:

• I must use NetworkBehaviours to store most of my data in-game order to have it synchronized to players through Mirror (no plain classes allowed)
• I would like data to be stored in the cloud so that it can be accessed by whichever player in the group would like to host the game this session on any given day
• I would like my SaveBlobs to be tolerant to change, i.e. schema migrations

Solution

• Each Saveable NetworkBehaviour has a ToScriptableObject method which simply copies all fields including collections to a ScriptableObject with an identical collections.
  • Some conversion is done at this point away from play-specific data structures, e.g. "SyncList" -> "List"
  • I'm using code generation here to make the maintenance trivial, a FileWatcher looks for [Saveable] attributes in my workspace and builds the ScriptableObject definitions and transformation code for me
• I'm (mostly) not dealing with assigning identifiers to things automatically. Most of my data is hierarchical: Campaigns have PlayedCharacters, NPCs, Stages, Flags, PlayedCharacters have CharacterAppearance and Items, etc.
  • When loading a campaign, SceneIndependent objects (Players, NetworkManager, etc) are built first, then a stage (scene) is loaded and GameObjects within the scene choose to build themselves differently based on the StageStateManager which load a StageStateScriptableObject.
    • e.g. if `Stage0State.MannyWasKilledByPlayers==true` then MannyCharacterPiece destroys itself on Awake and MannysWifeCharacterPiece.pissed=true is set on awake.
  • I assume that the SaveManager holds an up-to-date root ScriptableObject and everything else just has getters which know how to access the ScriptableObjects which concern them and uses it to initialize itself on Awake.
    • When I write these getters, they allow for a local override. So if a SaveScriptableObject is attached, it uses that, otherwise it looks for what it needs in the hierarchy. Hopefully this will allow me to test different game states easily by making different possible states on the fly and plugging them in within RuntimeTests.
• In cases where I have non-tree-like saved graph (NPCs might be an example, since they could appear in different stages) the getters simply pull their initial state from a higher level of the save hierarchy.
  • Code (CSharp):

    1. public MannyState curMannyState => (overrideMannyState != null) ? overrideMannyState : SaveManager.campagin.npcs["Manny"];
    2.  
    3. private void Awake(){
    4.    InitializeFromScriptableObject(curMannyState);
    5. }
• To a point made earlier, starting a new game simply uses a prebuilt SaveBlob with everything's initial state. So starting a new game is really just loading a special "NewGame" save blob from my perspective.
• When the schema changes (i.e. fields are removed, added, changed to my network behaviours)
  • I iterate an integer "schemaVersion" on the top of my SaveBlob hierarchy
  • Adding fields is always fine, renaming can be done with [FormerlySerializedAs] I think
  • Fields are never removed, only marked [Obsolete]
    
  • Any more complex changes will have to be manually specified in a OnAfterLoad method of the SaveBlob. If the contents of a List need to be placed into a Dict or something, then I'll just have to do something like `if (oldList != null) {DoTransformToDict(); oldList=null;}`
    
    • It would be important here that any transformation and initialization are performed in the correct order and intra-object referential integrity is essential. A global OnAfterLoad method would be better ensure that any more complicated initialization and transformation is performed in sequential order.
  • Generated tests to make sure that all previous save versions will load without error in the current version of the app
  • Two potential strategies to avoid complexity:
    • Transform data in the cloud just scripting with Mongo or something (feels prone to error, but would better support rollbacks and eliminate the need to support old versions of save blobs)
    • When a release goes out, automatically load the game for each campaign on a dedicated server to make sure we don't have to support very old versions.
  • Hopefully schema migration is done as little as possible after the initial release, but it's important for me to be able to safely iterate on the schema during Beta.

 

May I observe that is a SPECTACULAR engineering lift. I don't think any game in history has done that level of granularity of saves, especially in a peer-to-peer way, except perhaps games that run entirely on a back-end such as WoW and such, and they get it for free simply because the actual game is running against a persistent cached server of some kind, probably a deep hierarchy of RAM caches, disk caches, and ultimately a journaled database of some kind that gets backed up and audited.

Usually one just saves positions and rotations of key actors, as well as some global notion of game state, such as which quests have been done, perhaps just a Dictionary<string,int>() telling how far along each unique quest is. Even just getting THAT working in the general sense is a huge lift.

At least you have identified issues going forward as your dev lifecycle changes, but to support every version of save back to 1.0 is going to add a STUPENDOUS engineering cost to your process, or else you will get bugs. It's common for games to wipe (fully or partially) when a major update comes along for this reason, such as keeping ONLY the player's stats and possessions, nothing else.

Finally don't forget security. People will hack your save game data. If I hack to say I have 100 gold and in reality I don't according to your save on your system, who do you trust?

Anyway, good luck to you. Here is my standard blurb:

Load/Save steps:

https://forum.unity.com/threads/save-system-questions.930366/#post-6087384

Don't use the binary formatter/serializer: it is insecure, it cannot be made secure, and it makes debugging very difficult, plus it actually will NOT prevent people from modifying your save data on their computers.

https://docs.microsoft.com/en-us/dotnet/standard/serialization/binaryformatter-security-guide

When loading, you can never re-create a MonoBehaviour or ScriptableObject instance directly from JSON. The reason is they are hybrid C# and native engine objects, and when the JSON package calls

new

to make one, it cannot make the native engine portion of the object.

Instead you must first create the MonoBehaviour using AddComponent<T>() on a GameObject instance, or use ScriptableObject.CreateInstance<T>() to make your SO, then use the appropriate JSON "populate object" call to fill in its public fields.

 

Thank you so much for reading the whole thing and providing feedback! I can't even tell you how much I appreciate your added wisdom, especially with regard to how much of a lift this is. I'm hoping that I can avoid supporting old versions of save files by doing some kind of forced update on-server.

>When loading, you can never re-create a MonoBehaviour or ScriptableObject instance directly from JSON

The way I'm handling this is by spawning prefabs whose "datamodel" components (any which have persisted fields) initialize themselves on wake from the scriptable objects that have been deserialized from JSON.


>use the appropriate JSON "populate object"
I'm surprised to hear about this limitation and I'm curious what problems you've run into. In my tests so far, I've been using JSON .NET `JsonConvert.SerializeObject` and `JsonConvert.DeserializeObject<>` on ScriptableObjects and it's been working flawlessly. Of special note here, my SaveableScriptableObjects are generated classes with data only. Anything which contains a reference to an asset (textures, prefabs, 3dmodels, etc) is reduced to an identifying string during the generated `ToScriptableObject` method


> don't forget security
A great point-- mine is a server authoritative game, and it's small-group co-op, so I'm not too worried about cheating for the most part. I could see a PVP mode being fun down the line but it would not have save files, just player stats I'd store on PlayFab. I'll probably use PlayFab's in-game economy system for any DLC proof-of-ownership if that's a thing (luxury problem, if I have it )