JSON .NET for Unity

actually.. Thank You!... without your assistance, I was pretty much ready to throw a lot of exceptions

 

Is Async Serialization and De-Serialization supported?

On Windows Store (Mobile and PC) it seens JSON.NET is very slow.

Saving a 34kb BSON takes 0.6430900 seconds.

What I can check to try to improve this? On all other platforms the speed seens to be ok.

 

The speed problems seems to happens only with WSA8.1. On WSA10 it is so fast as on other platforms.

 

So... there was a change in JSON .NET v8. The Async methods in JSON .NET have been marked obsolete (in the official source), and the recommended method is to use TaskFactory.StartNew(() => JsonConvert.DeserializeObject<Type>(json));

But, the async implementation uses Tasks and is not compatible with Unity's version of Mono. It requires .NET 4.0 and above. It also does not work for Portable which means it isn't supported in the PCL that the UWP app is built on. However, you can use the above method to launch it async as a task yourself.

That being said, I'd love to see the JSON and the class structure of what you're serializing and deserializing. A simple repro project would be even better. Both UWP PC and Phone are using the same portable class library profile, so I'd love to step through and see if I can figure out where the slowness is coming from. My email address is on the Developer Contact section of the PDF documentation included with the asset.

 

Ahh... well that's interesting. WSA 8.1 actually uses the exact same DLL as WSA10 so this might be an issue with 8.1 itself.... Are you able to get any debugging / output / trace information?

 

Hi,

I wonder if anyone has ever asked, but when I serialize a plain class with UnityEngine.Color in it, the program spits an exception:

Is there any way to get it work with this?

 

Yes, by default only the VectorConverter is enabled (and the HashSet converter in AOT environments). If you're working with Color or Color32 you'll have to enable the ColorConverter. I don't enable all of them because I want to keep performance as tight as possible. The following will fix that call for you:

Code (csharp):

1.  
2. JsonConvert.DefaultSettings().Converters.Add(new ColorConverter());
3.  

You'll only have to ever make that call once. If you're passing in your own custom instance of SerializerSettings you'll have to add it to those as well. That converter (as well as converters for Matrix4x4, Quaternion, etc) are in the Newtonsoft.Json.Converters namespace.

 

Is there a way to default ignore all variables in a class except ones specifically tagged? That was my major hope with this instead of having to mark all fields I don't want to serialize. Similar to how protobuf works which is what I'm replacing since it's annoying to deal with for AoT devices.

 

Sure is. Use the JsonObjectAttribute on your class and set the serialization to OptIn. It's documented really well here:
http://www.newtonsoft.com/json/help/html/JsonObjectAttributeOptIn.htm

 

Amazing product and support, I'll give it a proper review after I play around more but in exactly 1 hour I converted what appears to be the entire game (RPG Inventory system, Item data, character xp/skills/rankings, equipped items, to name a few systems) and it just rolled over seamlessly. BSON works great and what a perfect feature to have the ability to ignore default values, that's going to save so much data over the network when used for simple objects.

Worth every penny thanks

 

Hi, just updated to 2.0.1 via the asset store and i'm getting The call is ambiguous between the following methods or properties: in Assets/JsonDotNet/Source/Linq/LinqExtensions.cs

Code (CSharp):

1.  
2. publicstaticIJEnumerable<JProperty>Properties(thisIEnumerable<JObject>source)
3. {
4. ValidationUtils.ArgumentNotNull(source,"source");
5.  
6. returnsource.SelectMany(d=>d.Properties()).AsJEnumerable();
7. }
8.  
9. publicstaticIJEnumerable<JToken>AsJEnumerable(thisIEnumerable<JToken>source)
10. {
11. returnsource.AsJEnumerable<JToken>();
12. }
13.  
14. publicstaticIJEnumerable<JProperty>Properties(thisIEnumerable<JObject>source)
15. {
16. ValidationUtils.ArgumentNotNull(source,"source");
17.  
18. returnsource.SelectMany(d=>d.Properties()).AsJEnumerable();
19. }
20.  
21.  
22. Assets/JsonDotNet/Source/Linq/LinqExtensions.cs(291,21): error CS0121: The call is ambiguous between the following methods or properties: `Newtonsoft.Json.Linq.Extensions.AsJEnumerable<Newtonsoft.Json.Linq.JToken>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JToken>)' and `Newtonsoft.Json.Linq.LinqExtensions.AsJEnumerable<Newtonsoft.Json.Linq.JToken>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JToken>)'

 

You have to delete the old asset first completely. The new 2.x version now includes the source in a zip file but it's precompiled.

 

Ah nice, deleting and reinstalling solved my issue!

 

Any idea ? ? ? ?

ExecutionEngineException: Attempting to call method 'System.Collections.Generic.List`1[[System.Single, mscorlib, Version=2.0.0.0, Culture=, PublicKeyToken=b77a5c561934e089]]::.cctor' for which no ahead of time (AOT) code was generated.

at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Utilities.LateBoundReflectionDelegateFactory+<>c__DisplayClass5_0`1[T].<CreateDefaultConstructor>b__1 () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonArrayContract.CreateTemporaryCollection () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonArrayContract contract, Boolean& createdFromNonDefaultCreator) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, Boolean checkAdditionalContent) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject (System.String value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject[T] (System.String value, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at PlayerController.LoadPlayerData () [0x00000] in <filename unknown>:0

at PlayerController.Start () [0x00000] in <filename unknown>:0

Rethrow as TypeInitializationException: The type initializer for 'System.Collections.Generic.List<System.Single>' threw an exception.

at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Utilities.LateBoundReflectionDelegateFactory+<>c__DisplayClass5_0`1[T].<CreateDefaultConstructor>b__1 () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonArrayContract.CreateTemporaryCollection () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonArrayContract contract, Boolean& createdFromNonDefaultCreator) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, Boolean checkAdditionalContent) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject (System.String value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject[T] (System.String value, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at PlayerController.LoadPlayerData () [0x00000] in <filename unknown>:0

at PlayerController.Start () [0x00000] in <filename unknown>:0

Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.

at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Utilities.LateBoundReflectionDelegateFactory+<>c__DisplayClass5_0`1[T].<CreateDefaultConstructor>b__1 () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonArrayContract.CreateTemporaryCollection () [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonArrayContract contract, Boolean& createdFromNonDefaultCreator) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, Boolean checkAdditionalContent) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject (System.String value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at Newtonsoft.Json.JsonConvert.DeserializeObject[T] (System.String value, Newtonsoft.Json.JsonSerializerSettings settings) [0x00000] in <filename unknown>:0

at PlayerController.LoadPlayerData () [0x00000] in <filename unknown>:0

at PlayerController.Start () [0x00000] in <filename unknown>:0

 

Whoah... what platform is this? Is it .NET or IL2CPP? This looks like Unity has stripped a constructor off of List<T>... I'd need more information:

1. Asset Version
2. Unity Version
3. Platform
4. Does it happen in the editor or only on device?
5. Could I get some repro? The JSON you're using... the code for the class you're deserializing.

 

Fixed changed float[] this to list<float>

 

using Unity5.3.6f1 latest asset iOS on Device not on Editor

 

public class PlayerPosition
{
public float[] pos;
public float[] rot;
}
changed to this and now its working like a charm
public class PlayerPosition
{
public List<float>pos;
public List<float>rot;
}

 

Thanks! So it's IL2CPP... looks like it's erroring when it tries to deserialize an array of float (float[]), but List<float> works. I think it's probably an issue with stripping... since a List is used internally to reconstruct the data dynamically, it was stripping the constructor. Since you started using List<float> in your code, the constructor was no longer being stripped. I'll dig into it a little deeper and see if I can reproduce. It's probably as simple as adding System.Collections.Generic namespace into your links.xml file to make sure they don't get stripped.

 

Thanks Josh. Also, you can use more than one link.xml file per project. Json .net has its own link.xml file for some types. And if you open it up and look you'll see specific types but also a commented out line that shows how to do a whole namespace.

 

Right now we have a game with a huge 13mb nested json file (as bson) which hangs unity for 7 seconds. The save will grow with time so is there anything async or known way to save and yield wait for the save to complete that we can work with? Not necessarily the file IO but pretty sure it's the json work causing the hang because Google protocol buffers didn't result in the same problem.

 

I would highly recommend finding a way to break up that file, especially if it's going to grow over time. Protobuf has the advantage of being deterministic, ordinal based so you lose the flexibility to change your data model later (or reorder properties) but it results in better speed. 7 seems like a long time, but 13 MB is a large amount of json.

That being said, you can multi-thread it. Json .Net used to have Async support built in but it's been deprecated in favor of using tasks... something like:

Code (csharp):

1.  
2. var task = Task.New(YourSaveMethodHere);
3. task.Start();
4.  

I can't recall of Task is available in Mono equivalent of .NET 3.5 as it's not availalbe in standard .NET 3.5 framework. You may have to use Threads and BackgroundWorker.

 

Hey,
I have a json file sitting at 6~ megabytes. On Android everything runs fine but I think on iOS the app closes after reading the json file. Still a bit skeptical considering you need 3 memory warnings on ios before it kills the app. But memory shoots up and then just crashes.

Right now I parse the json file and let it sit in memory. But I don't have a good estimate how a json from text to memory expands with all of the tokens and what not being created. Or do I have the wrong idea here?

I can definitely downsize the json by getting rid of empty values/arrays. In my database editor when creating a new item i create a block of json so all fields are present. But I can definitely tweak this.

What other way is there to to load a json without putting all the data in memory at once?

 

Well there are ways... One thing is definitely don't include empty properties. JSON .NET will handle this just fine and will ignore any missing properties so you won't have to worry about that. There are also settings on the Serializer so if you're Serializing from JSON .NET you can have it omit any default value properties (so anything that is null or a primitive set to it's default value such as an int that's set to 0).

Six megabytes is a pretty huge json file. If possible I would recommend breaking it up, but that's not always possible. One thing you can do is use JObject.Parse(jsonString) instead of doing a full JSON Deserialization right away. This might solve your immediate issues as you'll essentially get a JObject that's a key/value collection (and each value may in turn be an array and/or key/value collection). Then you can deserialize only the bits you need, so let's say your monolithic object has a property called User and it's of type UserInfo, you can do:

Code (csharp):

1.  
2. yourJObject["Person"].ToObject<UserInfo>();
3.  

That will deserialize the Person object for you but then you'd want to cache so you're not deserializing it every time you need to access it.

If that doesn't trim it down enough you can go deeper and use the JsonReader and JsonSerializer classes directly which will allow you to walk through and process your JSON as a stream. The drawback to doing this is that you lose some of the features such as JsonConverters if you process manually, however if you do use the serializer, you can use the Reader to walk through your JSON and find object boundaries, then you can create a JObject from the reader for the current object (or use the serializer) and deserialize just that object directly.

If you want, shoot me an email and we can work through some scenarios for dealing with large data blocks. Also, unzip the source and take a look at the Converters in the /Newtonsoft.Json/Converters folder. They make use of the reader and serializer as well as JObjects and will give you a feel of what it will take to use.

 

Thanks for the reply

As of right now we don't deserialize any object in the sense that we take a piece of data and directly convert it to an object. We use it as a data base on which we do Linq queries. So losing converters features isn't an issue. We never write to the data base once it's running. I also failed to convert data from json to objects. So in some parts I've been doing the manual labor by looping trough it.

Unfortunately the way it's setup right now is that values aren't set to null but rather to empty strings and empty arrays ([]). So we need to make some changes to our code so that it checks if the data actually exists. I can't split up the data either, I wouldn't know how and It wouldn't make much sense either for our sittuation i think.

Perhaps I should deserialize my data into objects ( last time i tried i failed ). We rather made some rash decisions to get our data up and running and we might have overlooked a few json features we can benefit from. Ill lookinto the JsonReader. I think the stream is going to be my best bet. Signing off right now, heading home. On monday ill have to look into it.

Edit:

I already do JObject.Parse though

 

Since there is no real way to check how much memory is consumed I used GC to sample two stamps of memory usage. I got around deserializing the data directly with JsonConvert. The difference in each stamp is around 25 Megabytes.

I then tried to use stream objects instead in the hopes that the json text file wouldn't be loaded into memory but the difference is still around 25 megabytes. I might be misinterpreting the data and it could also be completely meaningless in this case.

This the code I used:

Code (csharp):

1.  
2.  
3. private void DoDataBase()
4.     {
5.  
6.         ItemsBlock ib;
7.         Debug.Log(GC.GetTotalMemory(true));
8.         using (StreamReader reader = new StreamReader(Application.dataPath + "\\Resources\\db.json"))
9.         using (JsonTextReader jsonReader = new JsonTextReader(reader))
10.         {
11.             JsonSerializer ser = new JsonSerializer();
12.             ib = ser.Deserialize<ItemsBlock>(jsonReader);
13.             reader.Close();
14.             jsonReader.Close();
15.         }
16.         Debug.Log(GC.GetTotalMemory(true));
17.     }
18.  
19.  

Any advice?

 

The difference in total usage is probably negligible to be honest, with any extra being used by the type cache in json .net but that's going to be pretty small. As soon as you read that stream from the server the whole thing is in memory. A possibility would be to fetch the json from the server and immediately write it to device storage. Then use the File stream in System.IO to access the data on disk and feed it to the reader. Then it should process using that FileStream which should just read in the bytes it needs and not buffer the entire contents into memory.

 

Im not streaming the data from the server, it's in a Resources folder. It's a local hard cached database.

And I was mistaken in my test. I just tried again loading the json text file trough Resource.load and pass that in the deserializer. I got around 140 Megabytes of data. So using stream objects will be beneficial to our solution. I could further experiment with "streaming assets" and see how unity handles that.

140 megabyte to 23~ megabytes is more than a win in this case. Probably have to rewrite certain structures of our data handling.

 

So doing some further testing I realized the ~140 megabytes came from the fact that I was also parsing the json to a JObject. The JObject consistently returned a difference of ~110 megabytes. Whereas deserializing from a text loaded with Resource.load also gave a result around 20~25 megabytes. Which makes me question how both streamers and loading assets trough Resources.load affect the garbage collector.

For 1700 item entries I think 20 megabytes is ok, i guess.

 

I have a question regarding converters.

Currently a part of my data is serialized as:

Code (csharp):

1.  
2. public class ItemProperties
3. {
4.     public ItemStringPair[] Types { get; set; }
5.     public ItemStringPair[] Strings { get; set; }
6.     public ItemFloatPair[] Numbers { get; set; }
7.     public AssetItem[] Textures { get; set; }
8.     public AssetItem[] Materials { get; set; }
9.     public AssetItem[] Meshes { get; set; }
10.     public AssetItem[] Audio { get; set; }
11.     public StringsCollection[] StringCollection { get; set; }
12.     public NumberCollection[] NumbersCollection { get; set; }
13.     public AssetCollection[] TexturesCollection { get; set; }
14.     public AssetCollection[] MeshesCollection { get; set; }
15.     public AssetCollection[] MaterialsCollection { get; set; }
16.     public AssetCollection[] AudioCollection { get; set; }
17. }
18.  

This all works fine, however considering in our game we have specific set of data to fill. Finding their values is a bit cumbersome. Nothing wrong with using Linq, but it's easier to have the data sets directly to something like:

Code (csharp):

1.  
2.  
3. public class CarPart
4. {
5.  
6.     public float Health { get; set; }
7.     public float Damage { get; set; }
8.     public float Defense { get; set; }
9.     public float Speed { get; set; }
10.     public float Price { get; set; }
11.  
12.     public string Title { get; set; }
13.     public string Description { get; set; }
14.  
15.     public string Main { get; set; }
16.     public string Addon { get; set; }
17.     public string Extra { get; set; }
18.  
19.     public string[] MainCollection { get; set; }
20.     public string[] AddonCollection { get; set; }
21.  
22. }
23.  
24.  

The float values are defined as

Code (csharp):

1.  
2. "properties": {
3. "numbers": [
4.           {
5.             "name": "Health",
6.             "value": 99.9749984741211
7.           },
8.           {
9.             "name": "Damage",
10.             "value": 166.625
11.           },
12.           {
13.             "name": "Defense",
14.             "value": 33.325000762939453
15.           },
16.           {
17.             "name": "Speed",
18.             "value": 99.9749984741211
19.           },
20.           {
21.             "name": "Price",
22.             "value": 80
23.           }
24.         ],
25. }
26.  

In my converter I tried :

Code (csharp):

1.  
2.  
3. public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
4.     {
5.         CarPart c = new CarPart();
6.  
7.         JObject item = JObject.Load(reader);
8.  
9.         // Do stuff --
10.  
11.         return c;
12.     }
13.  
14.  

Q: How do I get the current " properties " of the item? JObject.Load(reader) returns the whole json.

Edit:

If I can get those values to a dictionary that would work too.

 

Dustin, I got a question regarding `JsonCreationConverter`. I have a model class that requires multiple objects with custom JsonCreationConverter's. Is there a way to do that? Any suggestions on a different approach to accomplish it?

Code (CSharp):

1. public class CustomObject1Converter :  JsonCreationConverter<CustomObject1>
2. {
3.     protected override CustomObject1 Create(TypeobjectType,JObjectjObject) { ... }
4. }
5. public class CustomObject2Converter :  JsonCreationConverter<CustomObject2>
6. {
7.     protected override CustomObject2 Create(TypeobjectType,JObjectjObject) { ... }
8. }
9. public class Model
10. {
11.     public CustomObject1 co1;
12.     public CustomObject2 co2;
13.     ...
14. }
15. ....
16.  
17. Model model = JsonConvert.DeserializeObject<Model>(modelString,???);

 

I'm pretty swamped with meetings today, but give me some time to digest what you have going on here and I'll help you come up wit ha solution. Dictionaries would be easy by the way (Unless I missed a detail), so let me look it over and get back to you.

 

I'm not sure I quite understand the question... Are you saying that the parameters passed into Create are also objects that you use a custom CreationConverter to instantiate?

 

No, I'm trying to deserialize a Model, but there are two objects in Model that requires custom JsonCreationConverter. If it's only one I would do the following and it works fine. However, I need another one.

Code (CSharp):

1. public class CustomObjectConverter :  JsonCreationConverter<CustomObject>
2. {
3.     protected override CustomObject Create(TypeobjectType,JObjectjObject) { ... }
4. }
5. public class Model
6. {
7.     public CustomObject co;
8.     ...
9. }
10. ....
11. Model model = JsonConvert.DeserializeObject<Model>(modelString,new CustomObjectConverter());

I think I found my answer. The signature on DeserializeObject for JsonConverter is params.

http://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_JsonConvert_DeserializeObject__1_1.htm

Code (CSharp):

1. public static T DeserializeObject<T>(
2.     string value,
3.     params JsonConverter[] converters
4. )

Therefore, I should be able to do the following for what i need.

Code (CSharp):

1. Model model = JsonConvert.DeserializeObject<Model>(modelString, new CustomObject1Converter(), new CustomObject2Converter());

Thanks! BTW... JSON.net is an awesome product. I don't know what I would do without it.

 

Before all this I created dictionaries manually. But I wasn't too sure how that works with allocating data. Arrays are more straight forward in memory. Considering the biggest bottleneck was the JObject.parse I'm willing to give dictionaries a try if the difference in memory usage is negligible.

Edit:

After digging around I finally got all my objects transformed to Dictionaries. Memory consumption seems to be higher but I think we can manage. I think I've figured out how the converter works now.

Edit 2:

I got my data transformed into the specific class I wanted. According to the memory stamps it's consistently around 12 megabytes. Which as a programmer I find extremely suspicious. But it if it's true then the bottleneck should be eliminated.

Q: The existingValue in ReadJson, is that the object that is deserialized and populated by the default converter if there are any matching fields? Just wondering for future reference, it would be possible to let the system auto populate some and catch those that need special handling?

Q2: Is there an Async deserializer?

 

I see, there are a couple of ways. For those classes you can add a JsonConverter attribute. So let's say we have ClassA and ClassB each which need a custom creation converter:

Code (csharp):

1.  
2.  
3. public class Model
4. {
5.    [JsonConverter(typeof(CustomObjectConverter))]
6.     public CustomObjcet co { get; set; }
7.  
8.     [JsonConverter(typeof(CustomObjectConverter))]
9.     public CustomObject co2 {get; set;}
10.  
11. }
12.  

That's the best way. The other way you can do it is to add those to the Converters collection:

Code (csharp):

1.  
2. var settings = new JsonSerializerSettings();
3. settings.Converters.Add(new CustomObjectConverter());
4.  
5. var model = JsonConvert.DeserializeObject<Model>(jsonString, settings);
6.  

Edit: If this is something you serialize and deserialize often you can also add that creation converter to the default settings, so you would call this one time somewhere in your application:

Code (csharp):

1.  
2. JsonConvert.DefaultSettings().Converters.Add(new CustomObjectConverter());
3.  
4.  

Now you can just do JsonConvert.DeserializeObject<Model>(jsonString); and that converter will be used.

Edit 2: Fixed the code, specified the wrong creation converter type in my example. It's fixed now.

 

You have a large amount of data so 12MB isn't surprising to me. There is additional overhead for having a Dictionary. JObject has overhead as well and it inherits KeyedCollection so it is much like a dictionary itself.

To answer your questions, existingValue is used in several scenarios:

A: If you're doing JsonConvert.PopulateObject instead of DeserializeObject, the existing value will be whatever is already existing on that property.

B: During deserialization, maybe you have a class that initializes a value in it's constructor, such as creating a new instance of an object for a property. Json .NET will use that existing object and populate it's properties rather than construct a new one.

B2*: With the above, if you peek at the documentation I've bundled, JsonSerializerSettings has some options that allow you to configure that behavior so it always replaces instead of uses existing if you want.

As to your second question, the Async stuff that used to be built into JSON .NET used .NET 4.X specific code so it wouldn't work for Unity, but Newtonsoft has also marked it as deprecated. Instead they suggest that you use Task.Run which is also .NET 4, so you'd have to use the .NET 3.5 equivalent and create a new Thread() and run it async yourself, but it does support being run async.

 

Probably should have asked this here first... but I threw a question up on StackOverflow regarding a new warming being thrown by the latest JSON.NET Library.

I'll summarize it here as well:

On the most recent version of JSON.NET for Unity3D I am receiving the following compiler warning message:

Code (csharp):

1.  
2. [CS0618]`Newtonsoft.Json.Serialization.DefaultContractResolver.DefaultMembersSearchFlags'is obsolete:` DefaultMembersSearchFlags is obsolete. To modify the members serialized inherit from DefaultContractResolver and override the GetSerializableMembers method instead.'
3.  

The signature of the GetSerializableMembers method is:

Code (csharp):

1.  
2. protected virtual List<MemberInfo> GetSerializableMembers (Type objectType)
3.  

I'm very confused how to override this method to accomplish the same logical equivalent we used to have when simply setting the DefaultMembersSearchFlags. Our previous usage we need to move into an override of GetSerializableMembers was:

Code (csharp):

1.  
2. contractResolver = new DefaultContractResolver() {
3.   DefaultMembersSearchFlags = BindingFlags.Public | BindingFlags.Instance | BindingFlags.NonPublic
4. };
5.  

Does an example exist of a working override for this method? Google is failing me.

Here's what I've tried so far by reverse-engineering the current usage of DefaultMembersSearchFlags in DefaultContractResolver.cs:

Code (csharp):

1.  
2. private class ContractResolver : DefaultContractResolver {
3.   override protected List<MemberInfo> GetSerializableMembers(Type objectType) {
4. BindingFlags DefaultMembersSearchFlags = BindingFlags.Public | BindingFlags.Instance | BindingFlags.NonPublic;
5. return ReflectionUtils.GetFieldsAndProperties(objectType, DefaultMembersSearchFlags).Where(m =>!ReflectionUtils.IsIndexedProperty(m)).ToList();
6.   }
7. }
8.  

Problem is... ReflectionUtils doesn't seem to be importable from the current JSON.NET package and rolling my own version of all that sounds like a rabbit trail when this should be 'easy'... right?

 

The old way will continue to work and you can suppress the warnings, but I'll put together an example for you on how to create and specify a custom contract resolver. I won't be around a PC until late so it'll be late to fight or tomorrow morning. You shouldn't need to rewrite anything in ReflectionUtils and in fact you don't want to touch that as there are some heavy per platform differences, especially with UWP. To get you started, you can either implement IContractResolver or create a new class that inherits from DefaultContractResolver and override the specific methods. Here is an example:

http://www.newtonsoft.com/json/help/html/contractresolver.htm

 

No worries, I'll wait for your example. This warning has been popping for a few weeks now and we've been ignoring it. Just figured I'd take a stab at silencing it the correct way finally and got bogged down in the fun with all that ReflectionUtils stuff.

 

Why does this happen?

i.e why is my result not of type T?

Code (CSharp):

1.  
2. // T is Dictionary<int,int> for example
3. // obj is Newtonsoft.Json.Linq.JObject
4. T result = obj.ToObject<T>();
5.  
6. // type of result is Newtonsoft.Json.Linq.JObject and not Dictionary<int,int>
7. print(result.GetType());
8.  

 

Hmm... that doesn't seem right. Which platform is this one? If you just return ToObject<Dictionary<int, int>>() do you get the same behavior? It's hard without seeing the entire context of the code.

 

Right I'll investigate further then.

 

My bad. Error in code meant T was always object.

But, can I ask about something?

If I'm deserialising some json text into a Dictionary<string,object> then will each kvp have a value type of either JObject, JArray, System.Int64, System.Double, System.String or System.Boolean?

i.e. there are no other types it could be?

 

For which? In order to produce proper json, the key has to be a primitive type, so int, bool, string, enum, etc. When you deserialize into <string, object>, the KVP will be a KeyValuePair<string, object>. You don't want to use "object" though, because the deserializer doesn't know what type to construct and therefor won't populate any of the properties.

 

Let's say I have json like this

Code (CSharp):

1. {
2.     "dic":{"a":1,"b":2},
3.     "list":[1,2,3],
4.     "int":123,
5. }

and I want to get the various items without creating a custom class to deserialise to.

So, I can deserialise to Dictionary<string,object> and then if I look at each object's type I see that "dic" is a Newtonsoft.Json.Linq.JObject, "list" is a Newtonsoft.Json.Linq.JArray, and "int" is a System.Int64.

If I want to create a Dictionary<string,int> from the "dic" object I have to get cast to JObject and call ToObject<Dictionary<string,int>> on it.

If I want to create a List<int> from the "list" object I have to get cast to JArray and call ToObject<List<int>> on it.

If I want to create an int for the "int" object I have to cast to System.Int64 and then cast to int.

 

Instead of casting you could also do this:

Code (csharp):

1.  
2. var list = JsonConvert.DeserializeObject<List<int>>(obj["list"].ToString());
3.  

That would avoid the casting. Calling .ToString() on a JObject or JArray property returns the json for that particular node, so you can traverse the nodes and then just deserialize the pieces you want. The same would work with the Dictionary, though I would recommend against using just "object". As I said before, when you use "object" as your type, the serializer has no way to know what type to cast to or construct, that's why you're ending up with a JObject back. JObject is already a KeyedCollection, so it's essentially already a Dictionary<string, object>, so unless you specify a type there's no sense in the serializer converting it.

 

I don't follow. Using just object where?

What do I deserialize the complete json text to if not Dictionary<string,object>?

 

By entire object you mean this?

Code (csharp):

1.  
2. {"a": 1, "b":2, "c": 3}
3.  

Then deserialize to Dictionary<string, int>. But if you're talking about the parent object that has "dic", "list", etc properties... then no you cannot deserialiez to <string, object>... because what "object" do you deserialize the values to? If they are primitives then I suppose it would work, but if it's a collection, or a complex object, then it can't work. Since the deserialier has no idea what kind of object to create, then it has no idea how to populate those values with the values from the json. If you want to deserialize you have to supply some kind of type, otherwise you have to stick with JObject and JArray. JObject itself implements IKeyedCollection and it's essentially a dictionary itself (or behaves like one).