Bro, do you even save?

Yesterday I made a post in the Asset Store section about a tool I've been working on that helps you make saved games:


http://forum.unity3d.com/threads/recorder-serialization-seeking-testers.314568/

(No, keep reading. This post isn't an ad, really.)

It does something I consider useful: makes it easy to save and and load data that's stored in your MonoBehaviour-based scripts. It basically lets you treat MonoBehaviours as if they were Serializable, which lets you store your game data in a more natural way.

So, the post was bumped out of the first page within a few hours, as sometimes happens. While thinking about why (to see if it could be improved), I figured I might've overestimated the number of people who actually care about adding a Save/Load feature to their games.

So that's what this post is about. How many of you actually care about adding Saving/Loading to your games? And if you do, is there an existing approach that does the job so well you don't really worry about it?

Much obliged for any info. I'm curious about how you people do things.
- Orion

 

Player Pref's are great for prototypes and smaller games, I generally use them in prototyping. Until the save / load system becomes an actual problem I wouldn't consider building one or replacing it. So there might not be an immediate need.

Stick it out, you only posted it yesterday. Come back in a week and see if it has caught any traction..

 

"Generic statement and praise about JSON" -People

Any details about how the data is stored?

@ShadowK I like to roll a new saving structure and parsing system for every project I have that involves saving

Working with strings is fun!

 

Your data is still written in JSON, Xml, Binary, etc. You JSON soldiers- keep doing your thing.

The tool isn't a new serialization format. Instead, well...I can't say it any better than the copy text I wrote earlier :

- - -

Basically, Recorder Serialization works by inheriting your script from a 'Recorder' class. This lets you add 'Read' and 'Write' methods to your scripts. Here's an example:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. using RecorderSerialization;
4.  
5. public class ExampleLight : Recorder
6. {
7.     private bool lightIsOn = false;
8.  
9.     // toggle the light on/off
10.     public void ToggleLight()
11.     {
12.         lightIsOn = !lightIsOn;
13.         SetDirty("LightToggled");
14.     }
15.  
16.     // called when this ExampleLight's data is saved, if "LightToggled" is dirty
17.     [Serializer("LightToggled")]
18.     void OnSaveLight()
19.     {
20.         // save the light's 'on' state
21.         SaveValue("lightIsOn", lightIsOn);
22.     }
23.  
24.     // called when this ExampleLight's data is saved, if "LightToggled" is dirty
25.     [Deserializer("LightToggled")]
26.     void OnLoadLight(int version)
27.     {
28.         // load the light's 'on' state
29.         if(!TryLoadValue<bool>("lightIsOn", out lightIsOn))
30.         {
31.             Debug.LogError("lightIsOn was not loaded.");
32.         }
33.     }
34. }

Using these, you can save any data you'd like from inside those scripts, plus data from built in Unity Components like Transforms, RigidBodies, Animators, etc. The actual values are stored as key/value pairs. These Read/Write methods can also be called conditionally, so you're not saving every field/property of your objects every time you make a save- only what's changed.

There are some frilly extras as well, like the ability to save and load Object references, and the option to reinstantiate prefabs on load. Also save-versioning, for backwards compatibility with older saves.

When it's time to save, you call a 'SaveData' property on your object. This will bundle everything that needs to be saved into a surrogate object, which you can serialize. When you deserialize that object later, it will automatically find the object that created it in your game, and use it to load back it's saved data.

 

Ok, so it does a pretty significant thing...

... and it fell off the front page of a busy forum quickly:

My guess is that not enough people are making decisions about how to handle non-trivial game saves for large projects on an hourly basis. I wouldn't jump to the conclusion that nobody's interested based on just a few hours.

If I need this I design it in up-front and make sure I've got a clear delineation between data the needs to be saved and data that doesn't. Once that's done I don't think I'd feel the need for some fancy generic system to handle it for me. Having said that, I'm a professional software developer with nearly a decade of experience - probably not your target market.

 

I will need a saving game solution. I've kind of been glancing at a few of them for over a year now. And I'm not a professional developer, just a hobbyist. That being said, I keep my eye out for things, but only purchase assets or systems when I absolutely need them, or I'm at that stage in my game where I'm ready to implement a given system.

Your system might be great for me, I don't know. I'm not at that stage in my game yet though -- but I will need a "save game" system in the near future. And by near I mean between now and 3 years from now lol.

 

@angrypenguin Thanks. You make a good point there- a lot of developers who will actually take a project past the prototype stage are used to rolling their own serialization features. And I'm like that too- it would take a lot for me to actually adopt someone else's code for a large part of my game- I just like writing those systems myself.

So, just out of curiosity, let's say you're at the start of a new project, and you haven't designed anything for serialization yet. What would it take to make you consider using something like this? What would you be worried about/turned off by?

You're right, I'm not bummed that my thread got bumped. I see how me bringing up in the OP could have sounded like whining. So to set the record straight, that wasn't intentional. Thanks for the feedback.

 

@Velo222 Thanks for the info. Let me know if you'd ever like to take this solution for a spin. And since you've been shopping around for a while, I'd be curious to hear what features you'd like to see in a saving solution/have seen in the ones you've considered.

 

I went with a somewhat unconventional method of saving/loading on my current game, which is using a SQLite database. So far it's working great. I use an asset called SimpleSQL that lets you do LINQ-like queries. It's all rather straightforward and civilized. There are down sides (like any solution to anything), but I dig it.

It'd be overkill for a lot of games, but for a game with RPG elements, like for storing many inventory items that can be owned by any number of entities, it's a pretty sweet solution.

I initially tried a free generic serialization solution on the Asset Store, but could never get it to work. I could see a more bullet-proof asset being rather useful for most games, though I imagine there are already a ton of those?

 

In the interest of helping your market research, here's why I would not be your target audience: While I'm fine with trusting someone else's code for quite a few things, there is no way I would trust someone else's code for saving important game data. You screw up player save data and they will be breaking out the torches and pitchforks and heading to your house. No way am I going to risk saving on anything but code I know down to the last detail.

Okay. So maybe I know more about saving client data than saving game data, but the principle is the same. Because I've had to make adjustments to pretty much every piece of Asset Store code I have bought, I just don't see myself ever trusting any Asset Store code for mission critical things like saving. (This is not a knock on the Asset Store. It is just that an Asset Store save system won't have the testing and engineering behind it as, say, one of the major database systems. Thousands of people use those every day, so I feel I can trust them within their known limits.)

 

That is an excellent set of arguments, Socrates, an excellent set of arguments...

Tell you what- I care much more about sharing this approach than making any money off of this asset. I'm promoting this because I think it's a smart way to work, and I'd love to contribute something that might being a good solution for someone.

I'm considering putting it on the Asset Store, mainly so there would be an official version that could be tracked and updated. But you're right- there would be a specific type of stank associated with that, even (especially?) if it was free. Trouble is, posting it online for free somewhere else might have an amateurish connotation that would turn some folks away. And it's slightly too complex to post on a repo like Github- and at least have people easily make sense of it.

I'm going to spend some time thinking about the best way to do this. Thanks again.

 

It'd have to fit in with that stage of design, not be an overkill/clunky/everything-including-the-kitchen-sink approach, and it'd have to be clear that the person who made it was more experienced in this area than I am (and at least as experienced in general).

The thing is, as long as you design it in up front I can't see what a fancy 3rd party solution can do for me that built in .NET functionality wouldn't. (That could of course be because I haven't done any mammoth custom saving of my own. And I'll admit, I have run into issues with the built in DataContractSerializer...)

 

The rhetoric I picked up from 4 years of college in both CS and IT courses is that you shouldn't reinvent the wheel. If your solution is easy to use and reasonably efficient, then people may opt to use that instead of write their own. I would suggest maybe having a video of using it for very complex project but also very basic projects so you reach out to a broad range of people.

Not everyone is going to love dealing with strings like some of us do

 

Go explicit or go home. Using string (or int, etc.) keys to name fields for serialization can save so much hassle is you ever need to add, remove, or change data. It adds a level of security and future-proofing, which is why every major serialization format supports it (Xml, JSON, soap, binary, protocol buffers, etc.).

It's only my opinion, but I'd rather spend a little time being explicit about how stuff gets saved, rather than hand my fate to a system that attempts to automatically serialize stuff in mysterious ways.

If I didn't know exactly how my game's serialization was happening, I would never feel comfortable asking players to use it.

 

@orionburcham if you offer a way to encrypt / secure this data and possibly easily make the serialization be sent to a web server or have a choice of local directory, you'll have a lot more to offer. Storing and parsing is pretty simple, but there are a lot of developers with the "play my way or go away" mindset (like the devs at diablo 3). If you can make the saved files unreadable outside of the game then I'm pretty sure more 'serious' developers will be interested.

I do my best to have an external, plain text document that my users can 'mod' and corrupt to their heart's content.

 

hmm...Aha! Ok, I think we're actually talking about different things.

When I said:

I meant this kind of thing (this example uses the ISerializable interface- unrelated to my product):

Code (CSharp):

1. [Serializable]
2. public class MyItemType : ISerializable
3. {
4.     // The value to serialize.
5.     private string myText;
6.  
7.     // Implement this method to serialize data. The method is called
8.     // on serialization.
9.     public void GetObjectData(SerializationInfo info, StreamingContext context)
10.     {
11.         // Use the AddValue method to specify serialized values.
12.         info.AddValue("myText", myText, typeof(string));
13.  
14.     }
15.  
16.     // The special constructor is used to deserialize values.
17.     public MyItemType(SerializationInfo info, StreamingContext context)
18.     {
19.         // Reset the property value using the GetValue method.
20.         myText = (string) info.GetValue("myText", typeof(string));
21.     }
22. }
23.  

Take a look at the last line of code inside of GetObjectData(). The name string property is being serialized as a key value pair with the string key "myText".

That's different from the serialization format, which determines how human-readable the end result file is. If the above example was serialized to a binary format, it wouldn't be human readable at all (and could be further encrypted however the developer might like). But in code, on the authoring side, the values would still be saved/loaded as string/value pairs.

The main benefit is that it avoids the serializer trying to load values based on the name of the field, or by the order the fields appears in the class, or by alphabetical order, etc. All of these can end up breaking save files if you end up adding or changing data, which can be painful to fix.

So, apologies for being pedantic there. That's what I was was saying in my last comment.

Here's an example of the above class done the same way using a DataContract model. Same concept- the field are being saving using explicit string keys:

Code (CSharp):

1.  
2. [DataContract]
3. public class MyItemType : ISerializable
4. {
5.     // The value to serialize.
6.     [DataMember(Name = "MyText")]
7.     private string myText;
8. }
9.  

 

There's high demand for reliable, effortless save systems.

The Dialogue System has a general-purpose save/load subsystem. Its core purpose is to save the Dialogue System's state (who you've talked to, what you've said, what quests are active, etc.), but you can also tie other data into it. Since the Dialogue System includes integration packages for frameworks like UFPS and ORK, it was easy enough to provide out of the box saving and loading for them. Saves and loads just use a big string that the developer has to save somewhere such as PlayerPrefs or a local disk file, so I wouldn't call it 100% comprehensive, although it does most of the hard work for you.

Even so, to my surprise some people buy the Dialogue System primarily for the save/load subsystem. I think this hints at a big need for a plugin that you can just drop into an existing framework like UFPS and get a complete save system without having to fiddle with any scripts.

 

They may, but it's competing with stuff already built into .NET as well. My question would be, as an experienced programmer who's (presumably) going to do this right from the start, what does this solution do for me that built-in .NET serialisation does not?

On that note, this...

... is a great point - not everyone designs every project from the ground up, so a drop-in solution that "just works" could absolutely have value there.

 

I buy game frameworks from the same author and their frameworks include gave save/restore functionality already.

Likewise, it depends on the developer/hobbiest - are they out to become experts at the current technical systems and their hobby is essentially learning more of those things and relearning the systems as they are obsoleted (I'm a developer and systems administrator and I'm used to doing that already) or, like me, will they try to minimize the learning as much of the technical things as possible and learn to create games as simply as possible.

Both have their merits but the 1st is liable to write or want to write their own game save/load system and the 2nd is liable to use as few assets and packages that get the job done. So a package like yours is strictly for those that truly have commercial aspirations and that don't want their staff writing such software for each game but want the same technique used every time.

Now, when & if I get to expanding the functionality of those frameworks I bought it may be better to expand the author's included system or if it gets to be too non-optimal, look at another system.

There is no shortage of game save systems already in the asset store though.

 

My [questionable] understanding of this topic was that some degree of serialization could be done for objects that could previously not be serialized with simple .Net calls.

Especially in the Unity community, haha.

Just goes to show the skill level of developers entering the game development community. Not necessarily a good or bad thing, but still interesting. Parsing and string manipulation was touched briefly in high school, CS100 and CS280. Are people getting into this field also this new to programming?

Could be quite the tutoring opportunity on the job forums here.

 

.NET calls can do in Unity/Mono more or less what they do in normal .NET, including serialisation. The issue with Unity stuff is that a lot of Unity things aren't .NET objects, so .NET serialisation doesn't apply. If you design up front to make sure all of your persistent data is implemented where persistence is already supported, though, that's not so much of an issue.

 

Maybe that's the target audience

 

That's it exactly (and thanks for describing this better than I did). Perhaps this package would be better described as a utility that sits in-between your data and your serialization code. The specific problem it solves: making it easy to serialize data that's stored in your MonoBehaviours.

Like angrypenguin said- since Unity Objects aren't serializable, normally you have to design a clear delineation into your projects between data that needs to be saved and data that doesn't. Non-saved data can live in your scripts, but any saved data must be stored in custom serializable objects with no other purpose (or at least must end up there before it's saved). This means your game's data is being separated and stored differently based on a technical restriction- not a logical grouping that makes sense for your game.

Recorder Serialization lets you treat your scripts like they're serializable. It prevents you from needing to use custom "save" objects just to store data that needs to be saved. You can go ahead and store your enemy's HP stat in your Enemy.cs script and not worry about how you'll save it.

When it comes time to save some data that's stored in a Component, you can call a "SaveData" property of that script. This will bundle up everything you've specified should be saved, and hand it back to you in an object that can be serialized. Once your have this object, Recorder Serialization makes no other demands of your workflow. You can serialize it anytime you want, using a .Net serializer and format of your choosing.

When you deserialize that object later, it will automatically seek out the Object that originally made it in your game, and use that to reload all of it's data.

- - -

Along the way, I ended up adding some extra features that support ways you might want to serialize stuff in Unity, like the ability to save some Object references, and the ability to automatically reinstantiate prefabs. there's also versioning support to maintain backwards compatibility with old saves, because any serialization tool you'd want to ship with should have that ability.

- - -

Long story short, I wrote this because I had a need for a general purpose serialization system. I needed something that would work across a wide variety of projects, but wouldn't sacrifice the low level of control that comes from project-specific save systems. I needed something that would let me control every aspect of what was being saved, in a way I could predict, without imposing any restrictions on what file formats or platforms I could use. I needed something performant, and that I could confidently ship in a real product.

I didn't have much luck with 'do it all' solutions I found out there, which don't seem to often 'do it very well'. There seems to a tendency there to target people who want it all taken care of without any thought. As a result, they try to replace your entire serialization pipeline, or require that adopt a new workflow. Some of them end up doing questionable things, like saving everything to playerprefs, or attempting to save every aspect of a GameObject and entirely reconstruct it on load. Things like this might work technically, but try incorporating them into your professional projects and a lot of unanswered questions tend to pop up. Certainly this is my personal take, and I would always love to hear reasons for decisions like these.

- - -

But anyhow, that's where this product is coming from. It's intended to be something smart- something you could actually ship with. I've been using and testing it for about a year, and it's saved me a good deal of headache. Just starting a project knowing that you can always save the data that's stored in your scripts can make things a lot simpler. It removes what can sometimes be a stressful barrier.

Whew that was long. Thanks for reading!

 

@orionburcham I'm sure with that clarification, users who don't want to learn things like getter / setter / init functions to do their own serialization will be very interested. Who knows, maybe your asset will be a way for new users to learn this important stuff and having saving be a super relevant example.

That's basically how I handled playmaker. I used it to learn how to design and organize a state machine properly, and now it collects dust

 

Err... I don't quite see the implications like that.

For starters, I don't care that Unity Objects can't be serialized, because I don't want to do that. Example: I don't want to save a MeshRenderer (the Unity object), I want to save whatever script component I had that controlled it (a Mono object).

Secondly, I agree that the "splitting" could be seen as primarily being based on a technical restriction, but it's still also a really good design approach because I want to explicitly decide (and design around) what is and isn't persistent data anyway. Persistent data isn't specific to an instance, so for data management purposes I don't think it should look or act or be treated like it is.

Edit: Again, not shooting it down. You asked about how I'd do things and, ideally, the above is my current approach. It could change as I gain experience, and if I'm working with 3rd party stuff I might not get to follow my own ideas all the time.

Edit 2: Actually, one of those wasn't quite right, so I removed it.

Edit 3: And a bunch of that still sounds really useful, and/or like it'd fit in with my approach anyway.

 

Ha! (About your considerate edits). I don't disagree with your approach at all, and it's great to know more about it. One point of clarification:

That's exactly what my tool does. It's for saving data in your own custom scripts (which I'm assuming inherit from MonoBehaviour- is that wrong?). MonoBehaviours derive from Unity.Object, so your scripts can't be directly serialized. Recorder Serialization lets you treat them as if you can, and lets you control exactly how they get saved and loaded by adding your own 'Read/Write' methods to those scripts.

I see this as totally legit. If someone like yourself decides to make persistence a design consideration when it comes to organizing data, that makes total sense to me. However, right now it's also not particularly optional. Toward whatever purpose, this tool makes it optional and opens up other possibilities.

Is persistent data never specific to an instance? For example- I recently wrote an open world level streaming system (a la Elder Scrolls). The world is broken up into different levels that get loaded and unloaded based on visibility, player proximity, etc. The system uses Recorder Serialization to save the state of those sections when they get unloaded, and to reapply that state when they're streamed back in (I also run some code to update things based on how much time has passed).

Nearly all the data that's being saved and loaded is specific to Objects that live in those scenes- the position of clock hands, a light being on or off, etc. There could be thousands of objects in that game that need to maintain a persistent state- it wouldn't make much sense to store all of it as global data- especially when the object that data applies to are frequently streamed out as part of their level. It's a real relief to be able to just store the data about that clock as fields in it's "Clock" script (where it would be useful for controlling the clock), and to then be able to rely on serializing it out safely anyway.

So, I would consider that data specific to those instances. Please let me know if I misunderstood what you were saying there- I figured I needed to write out that example to know either way.

Also, I really appreciate you guys' feedback on this- it's been very helpful to learn how to think about this tool.

Cheers-
Orion

 

Well, that's what makes it "persistent", right? Otherwise it's "instance data" or whatnot. The characteristic that makes data "persistent" is that it persists outside of the life cycle of the various objects that use it. Your example is actually classic:

An object (instance) is created. Some data about it is saved and the instance is destroyed. A new instance is later created, which loads the data, potentially changes it, saves it, and is destroyed again. The data isn't tied to any specific instance.

Edit: See here for clarity about exactly what I mean when I say "instance".

 

Many do, many don't.

 

Oh! *face palm* I get what you mean. Of course! I certainly appreciate the engineering principle. Though in the case of loading data associated with objects that are part of Unity scenes, things are a bit murkier. The data Unity uses to create those objects at runtime is persistent, by nature of being serialized with the scenes. So that's not a case of linking persistence data with instances. It's linking persistent data to other persistent data. At most, it's using the same approach Unity does: applying persistent data to a runtime instance of the Object.

When it comes to saving data about prefabs that are spawned at runtime- I think that would be a classic case of an instance object. It's not stored anywhere in a scene; most data about that object lived and died entirely at runtime. Recorder Serialization handles that by reinstantiating a new instance of that prefab, then applying any saved data to the new instance.

Again, lemme know if I'm reading you wrong here.

Also:

Hmm. In your original quote, you said you wanted to save data from script components:

I was under the impression Unity didn't support adding user-written Components to GameObjects that weren't derived from MonoBehaviour. Were you referring to custom non-Component objects?

Thanks again!

 

Sure... but either way, at runtime an instance of something is created, and data persisted from whatever mechanisms are at play is ultimately used by those instances. My understanding is that from there it's all discussion about levels of indirection. (I could be mistaken?)

You can indeed only attach a MonoBehaviour to a GameObject. The MonoBehaviour is just the hook into the scene/GameObject, though. It doesn't have to do all of the work/store all of the data directly.

Lets say I create some complex data for a character system. A lot of that doesn't have anything to do with a specific scene or GameObject or whatever, so I probably don't want it to be a MonoBehaviour. I might then have a CharacterController MonoBehaviour that uses that data inside a scene to do stuff. In that case I probably want to save any updates to the data, but don't care about any particular instance of the controller.

But you're right, what I wrote there isn't specifically correct, and if for ease of use I did indeed want to persist a MonoBehaviour in particular then what you're talking about seems useful.

 

@angrypenguin all that talk about monobehaviors and them being on gameobjects reminds me of a proof of concept nightmare I made with unity. 1 main gameobject, 1 monobehavior script. The main script made a bunch of other scripts that were not monobehaviors. Each one of those scripts created a 3D object to manipulate in the world and had functions that were run only by the main script's update loop. That's how I would do things in other, normal environments. Doesn't quite seem to be the unity way.

Would that design have more or less use for the tool in this thread? And that's a question of how much design can affect its utility.

 

If it could tie into an SQL or Excel solution, that would seriously peak my interest. The main issue with RPG's is tag data for items / levels / skills / money and general crafting. On top of that, storing vector addresses (load position) and having the solution externally active with blank components that can be loaded at start up (for modding).

Whilst not extensively difficult to do, I would go out of my way and pay someone to provide a drop in solution.

 

Haha, see @orionburcham? That's exactly what I said via pm

@ShadowK actually knows about this stuff and will pay for a reliable drop in solution, as would other knowledgeable devs looking for the same kind of standard / cots drop in solution.

 

My God Tomnnn! What happens in Private Messages stays in Private Messages!

 

the asset store forum is very active and all posts modified in one day typically span at least a few pages, so pretty much everything bumps off the first page every day, nothing weird there.

 

I'm not so sure about that, private messaging seems pretty ambiguous.

 

Yessir- this type of support is included. Working with databases is actually one of the target usage cases for this tool. My time at home has been tight lately, but later on this week I'll write up some code examples of how these things might be handled.

In the short term: One feature of Recorder Serialization is that, if you save data about a prefab that was instantiated during runtime, you can choose to automatically reinstantiate a new instance of that prefab later when loading the data. By default, the prefab path of the asset is saved with your file, then retrieved during a load, and used to instantiate the prefab using Resources.Load().

However, this isn't always the desired approach- the classic example being when you want to reinstantiate prefabs based on a prefab path that's stored in a database somewhere. Recorder Serialization makes it easy to override the default prefab reinstantiation behavior by assigning a static "reinstantiatePrefabFunc" delegate. Inside that delegate, you can reinstantiate the prefab however you'd like.

I've done this recently in association with an Xml database setup myself.

- - -

saving and loading something like the contents of an RPG inventory that relies of a database would be even easier. I'll be back when I can with some code examples.

-Cheers! And thanks for the interest.
Orion