JSON .NET for Unity

Unity. It works fine on 5.1.3. It conflicts on other platforms, not just windows phone

 

Ugh, I haven't even had a chance to look yet. I'll be installing 5.2 tonight. I was aware of the Windows Universal / Windows 10 platform issues. I know there are other conflicts as well with extensions I put in. Is it also conflicting on non-windows platforms?

 

Yes,.

 

Alright, I'll be working on this furiously all weekend. Guess I won't get much football watching in.

 

Some questions :

- I'm serializing some Vector2 and Vector3, I only need to save x,y,z but it serializes Normalized, Magnitude. Is possible to avoid all this extra serialization?
- Is it possible to use JsonConvert.SerializeObject(SaveData) on a Thread?

My save game routine is taking half a second (the Serialization Process and Saving it To Disk), any suggestion of how to improve it?

 

Yes, look in the /Extras folder and there is a Vector2Converter and Vector3Converter (and a couple of others). Use those converters when serializing your Vectors and it will only serialize the x, y and z properties respectively.

It is certainly possible to do your serialization on a thread but you'll have to be careful. If you're serializing in the background, the main thread could potentially change values in some of your classes. You'll need to make sure you take concurrency between threads into account. Also, if you have any properties (get methods) that access some Unity APIs that interact with the main thread, you won't be able to serialize those in a separate thread.

As for the performance, drop me an email with what the JSON looks like so I can see how big it is. We can take a look at how you're saving it and see if we can figure out where the bottleneck is. It'll take me a little time to get to it as I'm working on Universal app compatibility with Unity 5.2 right now. It goes beyond fixing up the new errors as a final release mode compile in Visual Studio uses .NET Native and is causing some issues.

 

You saved my day!

Using the CustomConverters now my Save File is 60% smaller and it was 50% faster to save!

Thanks for the support!

 

I just noticed in the latest 5.2.1 p1 there is a release note:-
(none) - Universal Windows Apps: Removed APIs from WinRTLegacy which were reintroduced by Microsoft in .NET Core 5.0. This should fix errors where same class is implemented both in WinRTLegacy.dll and System.*.dll .

Going to try this out and see if it fixes the issue.

 

That might explain why I'm not seeing all of the same issues everyone else is, because I'm running 5.2.1. It still won't fix the Native build issues but it might clear up some of the ambiguous reference stuff.

 

Notice I'm talking about the patch release that came out today. I haven't yet tried the general release 5.2.1. I had issues with Patch 5.2.0p.

 

Ok, I hadn't seen that patch yet. I'll install it this evening and see what has changed.

 

Is it working with 5.2.1? I need to generate a Windows Store Build and I didn't updated to Unity 5.2 because I'm afraid of it not working.

 

It still does not works!

Code (csharp):

1.  
2. Assets\JsonDotNet\Source\WinRT\Converters\RT_JsonValueConverter.cs(52,52): error CS0433: The type 'IJsonValue' exists in both 'Windows.Foundation.UniversalApiContract, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null, ContentType=WindowsRuntime' and 'Windows, Version=255.255.255.255, Culture=neutral, PublicKeyToken=null, ContentType=WindowsRuntime'
3.  

 

Yes I'm aware and working on it. I've actually switched gears and I'm working on a workaround for now. I'm tweaking the newest official JSON .NET 7.x release and getting it precompiled for Windows 10, then you'll need to pull it into your project and set that assembly to only work for Windows 10 Universal (I'll put together some instructions) and I'll make sure that none of my existing source will compile under Windows 10. That will provide a stopgap while I work on my rewrite. It seems to have been a pretty significant change.

 

Its all good man. Keep it up! You're a legend.

 

Thanks, sorry it's taking so long. It was a much bigger deviation than I expected for a dot release.

 

Hey, I notice ItemConverterType is not included, is there an alternative to it? Trying to serialize a list of enums. Thanks.

 

That is correct. I will have it in the future once Unity gets their Mono upgrade in place and I can fully update JSON .NET. For now, you can accomplish it using a custom JsonConverter. I wrote one which another of my customers modified slightly and sent me. I haven't tested it yet but I didn't see anything in the code that looked out of place.

Usage:

Code (csharp):

1.  
2. public class SomeClass
3. {
4.     [JsonConverter(typeof(EnumStringCollectionConverter<MyEnumType>))]
5.     public List<MyEnumType> ListOfEnums {get; set;}
6. }
7.  

Or if you're just serializing and deserializing the list you can pass the EnumStringCollectionConverter<MyEnumType> in the converters collection.

Converter is attached.

 

That's perfect, thanks very much for all the great work!

 

Do you have any estimate of when it will be ready the Unity 5.2 compatible version?

 

Not an exact one. I'm working on it. It is 5.2 compatible, it's just Windows 10 Universal apps that are having issues.

 

Just another quick update, I was having trouble getting to build at all due to this (among a couple of others) issue:

http://forum.unity3d.com/threads/wi...ersalapicontract-and-windows-conflict.357786/

I'm installing 5.2.1p2 right now and will continue working with it to try to get this fixed.

 

Are you having any luck with the Windows 10 universal issue?

 

Not yet, I'm still running into issues. If it's blocking you though, send me a PM and I'll get you a workaround. Essentially, I'll give you a version that's precompiled for the editor only and I'll get you a working Windows 10 precompiled assembly. You'll put both assemblies on your project and then use the platform settings in Unity to specify that the second assembly is for Windows 10 Universal and the first is for Editor. That should allow you to build out to Windows 10.

 

It isn't blocking me to the point where it would be worth putting your time towards a work around. I can wait while you focus your energy on solving this issue. Thanks for replying.

 

Hi Dustin Horne,
Does your Json.Net plugin working for Webgl also?
I just started with a Unity project that using WWW class with json to communicate with server. I've been trying with multiple Json plugins for Unity but they seem not to work well with webgl. On Webgl build, whenever I try to deserialize/serialize between json string and my object it somehow stops although it runs on editor.
List of library that I've tested:
SimpleJson
Newtonsoft.Json
MiniJson
Could you let me know why? If your plugins can work I think I will purchase it.

Thanks.

 

Yes it works with WebGL. I'm not sure why the other packages don't. Supposedly, JsonFX works with WebGL so if you're just needing something simple I would certainly try that first since it's free and it's very similar to how JSON .NET works so you wouldn't really be wasting any time if you needed to convert over. I've not tried JsonFX on WebGL but it's supposed to work with IL2CPP. You're more than welcome to purchase my asset of course, I just want to make sure it's something you really need.

 

Thanks for your reply, I will make another try with JsonFX.

 

You're welcome. JSON .NET does work in WebGL, it just sounded like you're maybe not doing anything super complex and if a free solution works it would be a better fit for you.

 

@Barkers Crest

The fix is pretty large scale because of API changes Microsoft made, but I do have a working workaround now. Please send me a PM with your email address and I can send it over to you for testing. It's also precompiled so should save you some time.

 

@Nerosam - Forgot to mention you in the above post as well. PM me with your email address so I can get you that update. I sent you a PM about it as well.

 

I am pulling my hair out trying to get Deserialization to work with this package in Unity.

I have a server which is outputting this string as UTF-8:

[{"id":"1","name":"Mikes_Demo"},{"id":"2","name":"Bos_Demo"},{"id":"3","name":"Some Other Demo"}]

I have run this output direct from the server to several online JSON Validators, it does validate!

Hard coded the string deserializes as well.

But when I try to deserialize it in my code, I get the same error every time:

JsonReaderException: After parsing a value an unexpected character was encountered: ?. Line 4, position 1.

Code (CSharp):

1. string json = w.text;
2. games = JsonConvert.DeserializeObject <List<gameData>> (json);

I have scoured the web for answers -- there are things about stripping the first three bytes from the response, which I've tried to no avail, and other dead ends. I am stumped! Can anyone help?

 

@forzabo -

This will happen if your server produces a UTF-8 string that has the BOM marker on it. You should be able to do a String.Trim() since Unity uses .NET 3.5, which *should* remove the BOM, though you may want to handle it differently to account for future upgrade of the .NET profile. Here is another solution from StackOverflow:

Code (csharp):

1.  
2. stringBOMMarkUtf8=Encoding.UTF8.GetString(Encoding.UTF8.GetPreamble());
3.  
4. if(json.StartsWith(BOMMarkUtf8))
5.      json = json.Remove(0,BOMMarkUtf8.Length);
6.  
7. json = json.Replace("\0","");
8.  

There are lots of ways you can do it, with stream readers, or on the server before you return it, but you get the idea. That BOM has to come off of there in order for it to be processed properly.

 

It's also possible that your server is inserting a null byte character somewhere in the string. Could you send me a PM with a URL that generates the JSON? I can inspect all bytes and see what's going on with it.

 

Thanks so much for your quick reply Dustin. I don't think the BOM marker is there, though?

When I implement this code:

Code (CSharp):

1.             print(w.text);
2.             string json = w.text;
3.  
4.             string BOMMarkUtf8=Encoding.UTF8.GetString(Encoding.UTF8.GetPreamble());
5.            
6.             if(json.StartsWith(BOMMarkUtf8))
7.             {
8.                 print ("bom");
9.                 json = json.Remove(0,BOMMarkUtf8.Length);  
10.             }
11.             json = json.Replace("\0","");
12.  
13.             print(json);  //what's it now?
14.  
15.             string test = "[{\"id\":\"1\",\"name\":\"Mikes_Demo\"},{\"id\":\"2\",\"name\":\"Bos_Demo\"},{\"id\":\"3\",\"name\":\"Some Other Demo\"}]";
16.  
17.             games = JsonConvert.DeserializeObject <List<gameData>> (json);

the output is:

{"id":"1","name":"Mikes_Demo"},{"id":"2","name":"Bos_Demo"},{"id":"3","name":"Some Other Demo"}] // missing leading bracket!

and the deserialization fails (obviously, as it is no longer correctly formatted)

 

hit this:

http://acmenerdgames.com/api/responder.php

 

Nevermind, it's working now. I'm pulling the data to see what's going on.

 

Issue solved, my server responder was tacking on garbage at the end of the response -- an extra ?> to be precise (that "?" should have been a clue to me in the error message)

Many thanks to Dustin for beyond the call of duty hand holding in PMs while I tracked down this issue.

 

Glad we have it working!

 

Running into an issue building inside Unity client itself.

/JsonDotNet/Source/Linq/LinqExtensions.cs(148,20)
/JsonDotNet/Source/Linq/LinqExtensions.cs(291,21):
/JsonDotNet/Source/Linq/LinqExtensions.cs(93,53): error CS0121: The call is ambiguous between the following methods or properties:

`Newtonsoft.Json.Linq.Extensions.AsJEnumerable<Newtonsoft.Json.Linq.JProperty>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JProperty>)'
and
`Newtonsoft.Json.Linq.LinqExtensions.AsJEnumerable<Newtonsoft.Json.Linq.JProperty>(this System.Collections.Generic.IEnumerable<Newtonsoft.Json.Linq.JProperty>)'

any help is appreciated.

 

You must have another asset that has the official Newtonsoft.Json.dll in it. Generally other assets will do this for editor scripts. Look through your other assets and find that dll and delete it.

 

cheers, worked, thanks!

 

Actually, i have another library that has a dependency on the Newtonsoft.Json.dll and while removing the DLL allows this JSON parser to work, it breaks the library dependency. Any ideas to get around this?

 

You'll have to recompile your other library. Is it a Google library? Sometimes people compile these libraries without setting Specific Version = false in the properties, so it then breaks because it's expecting a specific version of the assembly. If it's your own library, that makes it easier because you can load it into visual studio and recompile with that property set to false. If it's another publisher's library, you may have to have them recompile it for you.

 

The library is AVRO: http://www.us.apache.org/dist/avro/avro-1.7.7/
the csharp tar should give you what you need. Currently Im using the dll it provides. Though it does have all the source files. I was hoping there was a simple solution rather than trying to dump all the source into the project.

 

You're going to have more trouble than that. :/ I dug through and they only have the C / C++ source. No source for the C# stuff. Also, I have a feeling that DLL is ancient. The JSON .NET version they are tied to is 3.5.0 from 2008 which is really old.

 

If it's binary serialization you're after, you could look at dropping Avro and going straight with JSON .NET. Instead of JSON you can use the BSON conversion options so you're serializing and deserializing with binary instead of JSON.

 

Hi Dustin,

If I'm to use your solution, what would you recommend as the best way to store JSON locally on a device? PlayerPrefs?

 

Hey @Meltdown. It depends on your platform. PlayerPrefs would certainly be a fine place to store it. On some platforms (like Standalone or even mobile) you could also use Application.PersistentDataPath and store that way. In standalone you'd be able to use subfolders, but in mobile you'd be limited to root. In WebGL you're kind of stuck (mostly) using web services and storing that way, though you could interface with javascript and use localStorage to store the data in WebGL.

 

Just a heads up, I have been working on building out a unitypackage with precompiled assemblies instead of source. My next asset update will bundle JSON .NET for Unity as precompiled DLLs but I've found a Unity bug I need to report. When compiling the IL2CPP compatible DLLs with Visual Studio 2015, Roslyn produces IL that the IL2CPP compiler can't handle so it needs built with VS2013.

This shouldn't be anything you have to worry about unless you're precompiling yourself which is possible by putting the code into a class library project and setting the proper symbols.

I do have one other issue I have to figure out. The .unitypackage file I'm building has an assembly for Windows Core (Windows 10 Universal). When importing that assembly in Unity on a machine that doesn't have Windows 10 / SDK installed, it results in an exception and the DLL would have to be manually removed. I'm not sure that this will be acceptable so I may see if I can include it as a nested .unitypackage that can be imported as needed. Does anyone have any thoughts or preference on this?