Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

DarkRift - Fast and Flexible Cross Platform Networking

Discussion in 'Assets and Asset Store' started by Jamster, Apr 20, 2015.

  1. rstrunge

    rstrunge

    Joined:
    Mar 19, 2016
    Posts:
    1
    Hello Jamster

    I've got a problem with DarkRift:
    I'm syncing the players position and rotation (in 2d). The player is moving forward continuously and you can control the players rotation. It's working fine for the first 20 seconds but then it just stops sending data. I've tested it in the Player Demo where i change the Player script to continuously move forward and the same happens. I've tried to set a position and rotation syncing threshold but it only works when i set the threshold to a high value where the syncing lags.

    Thank you. ;)
     
  2. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hmm this seems to be a temperamental issue some people are having, could you PM me with a way of reproducing this please?

    Jamie
     
    rstrunge likes this.
  3. Obsurveyor

    Obsurveyor

    Joined:
    Nov 22, 2012
    Posts:
    277
    Is the documentation online somewhere? None in the first post, links in the old thread are dead and none on the website I could find. Just trying to learn more about this framework.
     
  4. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hey!

    It's available on the website here (left hand side) :)
     
  5. Obsurveyor

    Obsurveyor

    Joined:
    Nov 22, 2012
    Posts:
    277
    Hah, not sure how I kept missing that this weekend, thanks!
     
    DarkFlameX1 and Jamster like this.
  6. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Just so everyone knows, I'm moving webhost this week so expect emails to go unanswered while the domain changes over!
     
    Lisk likes this.
  7. Nexam

    Nexam

    Joined:
    May 14, 2016
    Posts:
    28
    @Jamster : I got this bad boy on client side theses day... i'm not sure why :

    Code (CSharp):
    1. VersionException: Exception of type 'DarkRift.Transmission.VersionException' was thrown.
    2.   at DarkRift.Transmission.TransmissionProtocol.DecodeMessageData (System.Byte[] bytes) [0x00000] in <filename unknown>:0
    3.   at DarkRift.Transmission.TransmissionProtocol.DecodeMessage (System.Byte[] bytes) [0x00000] in <filename unknown>:0
    4.   at DarkRift.DarkRiftConnection.ReadBodyCallback (System.Byte[] bytes) [0x00000] in <filename unknown>:0
    5. Rethrow as Exception: An Exception was raised in a background thread.
    6.   at DarkRift.DarkRiftConnection.Receive () [0x00000] in <filename unknown>:0
    7.   at BTVNetwork.Update () [0x00000] in <filename unknown>:0
    8.   at BTVNetworkManager.Update () [0x00000] in <filename unknown>:0
     
  8. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Version exception most commonly occur because DarkRift has become unsynced with the communication stream, as I said in private message it would be good if you can send a reproducing example!
     
  9. NightmarexGR

    NightmarexGR

    Joined:
    Jun 7, 2012
    Posts:
    217
    Can i use this asset commercially with less than 20 concurrent users right ?
     
  10. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
  11. actif

    actif

    Joined:
    May 24, 2017
    Posts:
    1
    The darkrift server is up on localhost? Or Darkrift can be on cloud ?
     
  12. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    DarkRift can be on the cloud yes. Wouldn't be much use only on localhost! :p
     
  13. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Hey Jamster, I'm not sure if this is me being an idiot or if this has already been addressed. But I've noticed in all of your demos you have HandleOnData() spread over multiple scripts and multiple gameObjects. Correct me if I'm wrong but doesn't that mean every time a packet is received, every single script with HandleOnData() in it is called and performs a check to see whether the packet is for it and or if the packet meets conditions. If this is scenario is true, then i quiz you whether having MasterDataHandler script which is the only script that has HandleOnData(), which takes the packet and finds which script(s) it was destined for, then using an array/hashtable setup at start/awake, it will send the packet to the script(s) it was destined for.
    The reason I'm putting this idea forward is because I believe that this could require much less processing on both server and client side.

    I hope this wasn't written too poorly, also thanks for your consideration.
    Bazzalisk.
     
  14. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hey,

    Yes, every single HandleOnData would be called. The reason I did it like that is that it's simpler for beginners and people trying to understand how it works, plus it demos that it's possible.

    Having a master data handler would only be beneficial if it was implemented well. For example, a dictionary/hashset has a high overhead generating the hash code so would probably damage performance, on the other hand, an array-based lookup table would be very fast and would give better performance quite quickly. However, you would get the optimal performance (and arguably the best readability) from using a switch-case statement since after a certain number of cases are added the .NET JIT automatically converts it from a series of comparisons to an array lookup table implemented at a low level.

    Hope that helps!

    Jamie
     
    Bazzalisk likes this.
  15. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Hey James,

    Thanks for your explanation regarding my issue. Just to clarify, do switch-case statements have the same time complexity as an array lookup table? Also would you be able to give me a brief example of what the switch-case statement would look like?

    Thanks again!
    Bazzalisk.
     
  16. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    It depends, if the number of cases is over 5 yes - and chances are they'll be faster. I can't find the MSDN blog about it but there's loads of really good discussions about this on stackoverflow like this one. You should be able to find some examples there :)

    It's really worth noting Knuth's ever-true quote: "premature optimization is the root of all evil"; it's great to know the theory but unless you're 100% sure that it's the problem area always do what's most readable!

    Jamie
     
    Bazzalisk likes this.
  17. connorshipway

    connorshipway

    Joined:
    Jun 29, 2017
    Posts:
    3
    Jamie[/QUOTE]
    Hello, Jamster--
    I've just started using DarkRift recently but already I'm very impressed with it! The hands-off approach toward implementing features like rooms that the user might not necessarily need for their game is very appreciated; I figure that if I need fancy features I can implement them myself, and the result is a very lightweight and simple-to-use API.

    Unfortunately I am having some problems with the server though, it works fine at first but after 20 or 30 seconds it seems to "stall out" and fails to receive or send messages. I noticed that Rumle123 had a similar problem a couple of months ago...

    Have you discovered any useful info on this matter? If needed, I can send you my code so you can reproduce the issue.
     
  18. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hmmm, are you using the embedded server? I fixed an issue with deadlocks that I thought was probably the cause in the latest release (I think, maybe I haven't released it yet :oops:) so double check you're on the latest release if that's the case.

    I can't reproduce it here so I can't be sure that fix got it!

    Jamie
     
  19. connorshipway

    connorshipway

    Joined:
    Jun 29, 2017
    Posts:
    3
    I'm not using the embedded server, the game has very simple physics so in this case the synchronization is just simply sending updates for players' position and rotation.

    From what I can tell I'm using version 1.4.4, which the version that is currently available from the Unity Asset Store. The Asset Store says that 1.4.4 was released on Jan. 20 of 2017?
     
  20. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Ah... Not sure about that then, best bet is probably to wait it out till DarkRift 2 because it's a lot more stable! :)

    Interesting, thanks. I definitely need to send an update out to the store in that case!
     
  21. connorshipway

    connorshipway

    Joined:
    Jun 29, 2017
    Posts:
    3
    Ok, thanks! I'll try to focus on developing non-networking features for my game in the meantime.
     
  22. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Hey Jamster,
    Should DarkRift be easily usable on mobile devices or is the capability of this external to networking assets like DarkRift?
     
  23. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Should be fine to connect from mobile devices! The only unsupported platform (that I know of) is WebGL as it requires web sockets which DarkRift doesn't have yet :(
     
    Bazzalisk likes this.
  24. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Hey again Jamster,
    In your heavy sync demo, it appears you use sendMessageToOthers() in order to sync the position of objects. HeavySyncDemo is described in the readMe as an authoritative scenario. So, because I believe this in't a demonstration of authoritative, I would like to ask you how to sync with the server. I have an idea of how to do it but it in my case, syncing is the responsibility of the server, not client : Server runs simulation of where the objects would be and sends updates at an interval. My issues with this is that I'm not sure how to implement the repetitive sleeping and calling (because plugins don't have mono behaviour), but a more important issue imo is that the server sends out the position of EVERY object in order to ensure things are accurately represented which seems expensive. Hopefully you can shed some light on this

    Thanks again!
    Bazzalisk.
     
    Last edited: Jul 1, 2017
  25. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Yes, you are right there. The reason the Heavy Sync Demo was added was to simulate stuff like an Real Time Strategy game where the player may have a few hundred entities that belong to him and he controls. It's nearly authoritative but in a true authoritative fashion I would actually have the client send their key presses/mouse clicks/button clicks etc. to the server and have the server validate that rather than the client's simulation of their units. It's also a bit of a stress test for the server :)

    If you want timed events you should look at the System.Threading.Timer class :)

    ADDED: In a situation like that you would usually have both the server and client simulate the world individually, that way the server only needs to send less frequent updates to the client to ensure it's an accurate representation of the server's world.
     
  26. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Thanks a lot for you help Jamster I really appreciate it, I guess your response led me to question whether what I'm doing is okay in terms of an authoritative server. I'm not sending mouse clicks to the server I'm sending a conversion of click in screen space to position world space then sending it to server, is this the same thing?
     
  27. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    That would probably be fine :) It really comes down to how strict you want to be and whether compromising on something would open up a vulnerability at all :)
     
  28. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Hey Jamster, I've got a project that is setup like a demo, it's a non-embedded authoritative server that is interactive. I was wondering if you like me to send it to you. It has a decent amount of commenting.
     
  29. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    I appreciate the very kind offer but since DarkRift 2 is coming out soon it doesn't seem like it would be put to very good use :oops:

    Thank you though!
     
  30. Bazzalisk

    Bazzalisk

    Joined:
    Mar 10, 2015
    Posts:
    10
    Ah okay, that makes sense! Very excited for DR2.
     
  31. R3D0

    R3D0

    Joined:
    Sep 24, 2014
    Posts:
    6
    Ah, good to know others are having the same issue with the server lagging out when a player moves in a straight line for too long.

    From (much too much) trial and error, I narrowed the issue down to the server handling a lot of very similar messages being sent out repeatedly. I started out sending updates every frame, and the issue popped up after about .5 seconds walking in a straight line. After changing it to every third frame, the issue shows up after 1.5~3 seconds, but only if there are multiple clients connected (only one of which is sending data at any time).
    It also varies with how many Debug.Log()s I've got in the system. The more I add, the more common the issue.
    Finally, I tried changing from the easy messaging to the custom serialization and didn't notice any difference.

    The issue could also depend on the processing power of the computer it is being run on. My computer is fairly new (Intel i5-6400 2.70GHz) but still has the issue regularly.

    Finally, when the server does "crash", it stops sending data to anyone and stops receiving, but it ticks off a "send" message that is a copy of one of the last few messages sent before the crash. It sends these messages around 2 or 3 times a second.

    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-0
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-0
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-0

    ---CRASH HAPPENS BETWEEN THESE ^v MESSAGES---
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-0
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-0
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1
    Data: Sender: 3 DistributionType: Others Tag-Subject: 1-1


    I can copy the project and trim out as much of the game code as possible to make a minimal version that replicates the issue if you'd like?

    (P.S. any ETA on DR2? I'm aiming for a beta release of my game in a little less than a month and want to know if I should finish with DR1 for now, or just wait and pick up the networking later)
    (P.P.S can we get a new invite link to the discord if it's still around? Original one seems to have expired)
     
  32. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    @R3D0

    Definitely interesting that it varies with the number of Debug.Log()s you have in the system.

    Can you experiment with whether or not it varies with how large each message is? (e.g. just try padding your data with some extra junk to make each message bigger).

    No official estimate on DR2 open beta release, but Jamie mentioned somewhere he's hoping for sometime in July, hopefully early July? I don't have the thread off the top of my head.

    And here's an invite to the Discord: https://discord.gg/cz2FQ6k hope to see you there!
     
  33. R3D0

    R3D0

    Joined:
    Sep 24, 2014
    Posts:
    6
    @Lisk

    Wow, I spent a lot longer on this test than I meant to :oops:

    Anyway, data from the test on message size:

    first, changed from

    Code (CSharp):
    1. DarkRiftAPI.SendMessageToOthers(NetworkTagIndex.PlayerUpdate, (ushort)NetworkTagIndex.PlayerUpdateSubjects.RigidBody, lastVel);

    (lastVel is a Vector2)
    to


    Code (CSharp):
    1. using (DarkRiftWriter writer = new DarkRiftWriter())
    2.             {
    3.                 writer.Write(lastVel.x);
    4.                 writer.Write(lastVel.y);
    5.  
    6.                 //INSERT POINT IS HERE
    7.  
    8.                 DarkRiftAPI.SendMessageToOthers(NetworkTagIndex.PlayerUpdate, (ushort)NetworkTagIndex.PlayerUpdateSubjects.RigidBody, writer);
    9.             }

    At "INSERT POINT IS HERE" I had either 0, 1, 10, 50, or 500 lines of "writer.Write(lastVel.y)", writing a float value.

    The first thing I noticed was
    "ObjectDisposedException: Cannot read from a closed BinaryReader."
    being thrown by my "moderator scene" (This scene only receives information, doesn't send anything. It runs in the editor). This did not show up on the Clients. The messages were still being processed properly though. This continued to be thrown for all subsequent tests regardless of lines added.

    The tests were done by walking a character in a straight line into another character (rigidbodies and colliders) thereby pushing the second character as they walked. Both clients sent messages of ONLY their respective character to alert the server of their change in position (messaging unaltered for test) and velocity (as shown above).
    "Time before crash" is measured from the moment the characters collide. No freezing was noticed before collision (tested up to 10 seconds walking in straight line, got bored)
    The results are averages of 3 runs each
    • 0 lines:
    -- 120~140 messages processed in final full second before the crash
    -- approximately .25 seconds before the crash.

    • 1 line:
    -- 120~140 messages processed in final full second before the crash
    -- approximately 1 seconds before crash.

    • 10 lines:
    -- one of these tests managed more than 10 seconds of collision before the crash. This was not reproduced by subsequent tests. Considered an outlier, not used in average time or messages processed.
    -- 120~140 messages processed in final full second before the crash
    -- approximately 1 second before crash.

    • 50 lines:
    -- 120~140 messages processed in final full second before the crash
    -- approximately .75 second before crash.

    • 500 lines:
    -- 120~140 messages processed in final full second before the crash
    -- approximately 1.25 seconds before crash.


    Aaaaand that's all I've got. Looks like there isn't a correlation of any kind between the message lengths and the time it takes to crash. Seeing as it's crashing with around 130 messages per second, that is probably a result of there being around 60 frames per second, and two clients sending ~1 message per frame (Sometimes 2, sometimes 0).

    Edit:
    Ah, forgot to mention: The ObjectDisposedException that I got, if searched in quotation marks, only turns up three results on google. One of which points to page 13 of this thread.

    Edit2:
    I tried disabling the logging to see if that would help any. It made no noticeable difference.

    (P.S. Thanks for the Discord invite!)
     
    Last edited: Jul 4, 2017
    Jamster likes this.
  34. R3D0

    R3D0

    Joined:
    Sep 24, 2014
    Posts:
    6
    I managed to localize the issue and made a project to reproduce the server crash. Still not sure what the problem is though :/
    To anyone who is going to take a look, everything needed is in the asset bundle. Import it, follow the instructions in the readme, watch as the server crashes. If it doesn't, then it might just be my computer -.-'
     

    Attached Files:

    Lisk likes this.
  35. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    I can't spare the time to debug your project, but I skimmed your post and 130 messages per second should be well within DarkRift's capabilities.

    I made this packet stresstest project a while back:
    https://www.dropbox.com/s/e7h3do022oqbe8s/Packet Stresstest - DarkRift 1.zip?dl=0

    You can try running it on your machine (Readme is inside) and see if you experience the same issues, just to help isolate the problem.
     
  36. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Well you've certainly been busy! Thank you for looking into this, I guess I had better get digging though I have no idea what could be causing it o_O

    With regards to the earlier question, I'll try and get DarkRift 2 posted out to the asset store within the next day or so and if you need it sooner than it get's moderated (which is probably about 1-2 weeks) then ping me a message :) I can't remember what I had left to do on it!
     
  37. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Well following the clues nicely laid out by @R3D0 and combined with some debugging trickery it's a deadlock between locking the list of clients to enumerate and locking the socket to send! Thank you for the reproducible example!

    ADDED: All fixed!
     
    Last edited: Jul 4, 2017
  38. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Version 1.4.5 has been submitted to the asset store!

    This includes fixes for a few deadlocks (including the one above!) and a couple of other bugs that were less noticeable :)
     
  39. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Very nice! If I can't get DarkRift 2 to work for whatever reason, those fixes will be very welcome fixes for me personally. I have a feeling I was experiencing the same issue, because my clients would occasionally experience issues where the game would freeze until one player disconnected. I suspect that was the deadlock based on enumerating the list of clients!
     
    Jamster likes this.
  40. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    What circumstances would cause me to receive a VersionException?

    DarkRift.Transmission.VersionException

    For some reason I have been receiving a VersionException on all other clients, when one client disconnects. This causes all clients to then disconnect.
     
  41. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    You generally get it when the TCP stream gets out of sync with DarkRift, e.g. DarkRift expects there to be a message header next but actually it's the body of the previous message still. You shouldn't get it franky so it's a bug in DarkRift!
     
    Lisk likes this.
  42. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    I've got a small, but important feature request:

    Can we get a DarkRiftConnection.ConnectOnce and DarkRiftConnection.ConnectOnceInBackground API methods?

    I think the current Connect/ConnectInBackground methods have some internal retry logic that is causing bugs in my code. I would rather handle my own retry logic.

    Technically I only need the ConnectInBackground one, but I figure for completeness there should be both of them.
     
  43. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Nope, they don't have any retry stuff at the minute!
     
  44. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Interesting - it must be something built into TCP then.

    I found another bug.

    1) I set up my desktop and laptop and connect them to a game server.
    2) Then I disable wifi on my laptop.
    3) The server freezes. I think it is trying to send packets to the disconnected laptop, but it didn't realize it was disconnected since I just pulled the plug (Rather than quitting the application). This causes the server simulation to freeze until either the client disconnects or reconnects.

    In DarkRift 2, the issue doesn't happen. (after disabling wifi, the server simulation continues as normal).

    However, I'm unable to use DarkRift 2 due to stability issues/random disconnects.
     
  45. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Ok, I'll try and have a look into it at some point for you :)
     
    Lisk likes this.
  46. tomgie

    tomgie

    Joined:
    Jun 6, 2015
    Posts:
    4
    Is the 20 concurrent users thing if we use your servers to host? Or is it 20 concurrent users on our servers?
     
  47. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Thanks Jamster!

    @tomgie it is 20 concurrent users, self-hosted. DarkRift does not provide any servers.
     
  48. tomgie

    tomgie

    Joined:
    Jun 6, 2015
    Posts:
    4
    That sucks, was looking forward to using this framework :/
     
  49. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Is it the 20 concurrent users, or that DarkRift doesn't offer hosting?

    If it's the 20 users, it seems a fair price to purchase the Pro or Extreme version once you need them. It's way cheaper than other networking solutions out there.

    If it's the hosting, you can host on a different service, like AWS or Azure. I personally host DarkRift servers on AWS through PlayFab.
     
  50. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Any update on this? It feels frustrating to have a server-client model, but having one client's lag causing others to freeze.