Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Hazel Networking - Open Source RUDP/TCP Library

Discussion in 'Multiplayer' started by Jamster, Jun 8, 2016.

  1. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hazel Networking is an open source, low level networking library for C# providing connection orientated, message based communication via TCP, UDP and RUDP. It's aim is to provide a standardised interface for web communication so that using and switching between protocols is incredibly simple.

    Hazel is going to be the basis of DarkRift 2 and I'm releasing it completely open source so that members of the community can make use of it, improve it and help find any bugs before DarkRift 2 is released.

    Features

    • TCP, UDP, Reliable UDP and (at some point) Web Sockets
    • Completely thread safe
    • All protocols are connection orientated (similar to TCP) and message based (similar to UDP)
    • Standardised interface so that all protocols can be used interchangeably with each other
    • IPv4 and IPv6 support
    • Continued support as it's improved and maintained for DarkRift 2
    Hazel is now maintained by ForteBass! You can find his fork here!

    Jamie (your friendly DarkRift person!)


    Updates
    Update 17/07/16 - Now built for .NET 3.5, remade IPv6 support to fit.
    Update 27/09/16 - TCP bug fixes.
    Update 14/11/16 - Security patch and added the ability to send data with connection message.
    Update 18/11/16 - Unity UDP bug fixes.
    Update 30/11/16 - Connect timeout
    Update 31/12/16 - Fragmented Messages
    Update 03/01/17 - Fixed incorrect timeout values
    Update 31/01/17 - UDP Disconnection fix
    Update 31/10/17 - Compilation fix and bug fixes
     
    Last edited: Apr 26, 2019
    Ruchir, Armynator, LostPanda and 3 others like this.
  2. ManHunterITA

    ManHunterITA

    Joined:
    Sep 3, 2013
    Posts:
    341
    Forked!
    I'll give it a shot in few days!
    Keep up the good work! :D
     
    Jamster likes this.
  3. jrojas28

    jrojas28

    Joined:
    Jun 6, 2016
    Posts:
    2
    This is perfect Jamie! Having source code to work on will really allow a lot of things to happen from here! I know it's just a beta, and I'm not expecting, say, exactly the easiness that the original DarkRift provides, but I can really see me working up with this and trying a few things.

    One thing I really intend to try now that it is possible to modify the server / client structure is what I mentioned previously (Bluetooth support). I may not be a master of this I guess, but this sure gives me something to work on rather than simple bland solutions and a socket / thread based enviroment :)

    Thank you for this! i'm really glad to see it!
     
    Jamster likes this.
  4. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Well, similarly to DarkRift it was never intended to be used peer-to-peer. Bluetooth should be possible though as there's no NAT to bypass; give it a try! :)

    I always said I'd do a peer-to-peer DarkRift, maybe I'll get around to that one day...
     
    ManHunterITA likes this.
  5. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Just updated with bug fixes, improvements and also the resend timeout for RUDP is now based of the average ping :)
     
    ManHunterITA likes this.
  6. Conquestor2

    Conquestor2

    Joined:
    Aug 17, 2015
    Posts:
    5
    Ahh, so Hazel is the core you're building to run darkrift 2 on top of? It's pretty cool that you shared it, We can do things like have pools of connections (with our own properties) to help out with GC handling etc with this :)
     
  7. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Yup! That's the idea!
    You can definitely do stuff like that, also check out the Recycle methods on the event argument classes, if you call that it'll help sort out GC spikes on high usage :)
     
  8. ManHunterITA

    ManHunterITA

    Joined:
    Sep 3, 2013
    Posts:
    341
    Yeah, it is cool :D
    I already provided a wrapper for incoming and outcoming messages. It is pooled and it reuses buffers thanks to Microsoft.IO.RecyclableMemoryStreamManager (take a look at it, if you're interested).

    Now it is time to pool connections and wrap them, so I can proceed with Server and Client classes creation :D

    Just a little advice: I noticed that Hazel targets .NET 4.5. You can't use it under Unity in this way. I refactored it to target .NET 3.5, but I think I have to rewrite DualMode logic (because Socket.DualMode isn't allowed in 3.5).
     
    Jamster likes this.
  9. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Nice!

    Ahh yeah I need to sort that, there's a socket option that you can set instead.
     
    ManHunterITA likes this.
  10. ManHunterITA

    ManHunterITA

    Joined:
    Sep 3, 2013
    Posts:
    341
    Yeah, you can do something like that:
    Code (CSharp):
    1. socket.SetSocketOption(SocketOptionLevel.IPv6, (SocketOptionName)27, false);
    Anyway, I think I will wait for DarkRift 2 instead of roll my own solution based on Hazel.
    I tried the APIs and I like it :D
     
    Last edited: Jun 16, 2016
    Jamster likes this.
  11. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Just come back to this for a bit after a long stretch of development on DarkRift 2, needed a break while I make some crucial design decisions!

    Moving to .NET 3.5 is nearly complete, just one Unit Test failing but I'll hopefully have sorted that by tonight (hopefully). There are a couple of breaking changes but as it's still technically a beta that's to be expected; some constructors have changed for the listeners and the IPModes are now IPv4 and IPv6 (as opposed to IPv4 and IPv4AndIPv6) since I've temporarily (hopefully) removed dual mode sockets for the downgrade.

    After playing with DarkRift for the past few weeks, this is beautifully simple! :)

    EDIT: Nope... Now it wont connect... o_O

    Jamie
     
    Last edited: Jul 15, 2016
    ManHunterITA likes this.
  12. kumar123k

    kumar123k

    Joined:
    Jul 5, 2016
    Posts:
    25
    looks interesting can you make a small demo ability to send udp packets also tcp
     
  13. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hey!

    There a demo on the quickstart section of the documentation here that talks you through each step and also provides full example code! To do it with UDP is almost exactly the same except you would use UdpClientConnection instead of TcpConnection and UdpConnectionsListener instead of TcpConnectionListener. The API's for each are the same :)
     
  14. kumar123k

    kumar123k

    Joined:
    Jul 5, 2016
    Posts:
    25
    i dont have experience in c# out side unity i will try to make it working
     
  15. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Update 0.1
    Hazel now supports .NET 3.5!

    The latest build is available from GitHub here!
     
    ManHunterITA and tobiass like this.
  16. VoidFletcher

    VoidFletcher

    Joined:
    Apr 15, 2015
    Posts:
    4
    Dropping a comment to stay up to date! Great work so far, Jam!
     
    Jamster likes this.
  17. iileychen

    iileychen

    Joined:
    Oct 13, 2015
    Posts:
    103
    Hi, Hazel looks cool, i'm using it for my new game now, seems Web Sockets is not supported yet?
     
    Jamster likes this.
  18. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Not yet sorry! I've been so busy working on DarkRift 2 I forgot about them!
     
  19. iileychen

    iileychen

    Joined:
    Oct 13, 2015
    Posts:
    103
    H Jamster, Hazel Networking is very easy to use, i like it. Thanks for share, is it still in maintain as a part of DarkRift 2?
     
  20. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    I have no plans to discontinue Hazel Networking :) It'll be maintained as long as I'm still maintaining DarkRift 2, possibly longer :)
     
  21. afrokick

    afrokick

    Joined:
    May 13, 2011
    Posts:
    6
    Hi Jamster! Nice work! Did you know library Lidgren? Can you please provide a short list of pros/cons compared with it? As I remember, lidgren uses only UDP+RUDP.
     
  22. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    I did know about Lidgren, I've never used it though!

    Pros for Hazel:
    • Supported by a commercial product so will be constantly maintained.
    • Designed for speed.
    • Supports TCP and UDP/RUDP and is easily extendible for other transport methods (Lidgren only supports UDP/RUDP.
    • Very easy to swap between transport methods.
    • Will support web sockets in the future (not possible with Lidgren).
    • Fully thread safe (not sure about Lidgren on this one).
    Pros for Lidgren:
    • Fragmented packets on RUDP (Hazel will probably have this at some point).
    • Ordered packets on RUDP (Hazel may have this at some point but it's unlikely).
    • Bigger community/more widely known/more established.
    If there are any other points to add to/mistakes in this list please post them and I'll add/change them :)

    Jamie
     
    ManHunterITA likes this.
  23. sebas77

    sebas77

    Joined:
    Nov 4, 2011
    Posts:
    1,641
    Hi,

    I may be interested in testing your library with our game, but I'd like to know more about it.

    We are interested only in the low level support, since we manage states synchronisation with our custom code, but I'd like to understand better the difference between Hazel and Darkrift.

    All that said, we currently don't need the fragment packets feature, all our packets are smaller than the MTU, but I am not sure how you can manage state synchronisation without the ordered packet feature. Do you design your game around this limitation?
     
    ManHunterITA likes this.
  24. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Brilliant!

    DarkRift is a fairly low level networking solution, akin to a halfway house between Unity's HLAPI and LLAPI (it doesn't use either of these however). Hazel on the other hand is a low level networking library that DarkRift 2 is being written upon, more akin to Lidgren. DarkRift has higher level concepts like clients etc. and distributes messages etc., Hazel only handles connection - you pass it data and it sends/receives it, no distribution, IDs etc.

    If it helps make things clearer Hazel is built on Sockets, DarkRift 2 will be built on Hazel :)

    Ordered packets are tricky because Hazel is so multithreaded, even if Hazel did order packets as soon as it invokes the message received events if they were in separate threads the packets could end up out of order... I might add something similar to ordered packets in the future but it wont be simple :(

    In most circumstances you would probably just send unreliable packets and display the most recently received state but if you absolutely need order you could append an order byte/ushort and receive reliable packets into a priority queue/heap or receive unreliable packets and simply display that state if the order value is greater than the current displayed, whichever is more suitable for your game.

    Jamie
     
  25. iileychen

    iileychen

    Joined:
    Oct 13, 2015
    Posts:
    103
    If I only use the Hazel TCPConnector, will that occur?

    I had transfered to UNET LLAPI last week, since i very need the WebGL support both at client and server side.
    But, it is a nightmare, hard to use and crash much.

    So finally i wrote a adapter, it can switch between UNET and Hazel. I have to say, hazel is my current choose, it's stable and fast, and I like the API design, simple and efficient. I really need the webgl support, maybe i could write for it when i have time.
     
  26. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Sadly a limitation of Hazel is that even with TCP it can't guarantee order :( It's designed for speed (hence it's use of multiple threads) rather than ordered packets, though with TCP it would be a lot easier to modify it to your needs.

    I will try and think how Hazel could handle ordered packets, it seems people want it! :p
     
  27. iileychen

    iileychen

    Joined:
    Oct 13, 2015
    Posts:
    103
    Yes Indeed, please make it can be ordered.
     
  28. hesa2020

    hesa2020

    Joined:
    Jul 27, 2011
    Posts:
    261
    Hello i have been using Darkrift for a while now, i was wondering if you have any plan on adding SSL support at some point?

    oh and by the way i was able to use nginx as a proxy server for Darkrift 1.
    It works very well and clients are balanced accross servers and data are shared using memcached server.
    And when ever i need to trigger an event to a client which is currently not connected on the server instance i send the trigger event to the master server which then send it to the appropriate server.
     
  29. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    I have no plans to implement SSL as I have only a fairly limited knowledge of encryption/web security and I'm not sure how much demand there is for it. For any secure communications I'd always recommend having it via a HTTPS server on the side...

    That's awesome! I'm glad that's working well for you! Hopefully DarkRift 2 will have its own dedicated proxy application that might give you a performance boost and be a lot easier to configure!

    Jamie
     
  30. invictvs1

    invictvs1

    Joined:
    Mar 7, 2014
    Posts:
    51
    Just when I thought I was going to have to write this myself! This library looks excellent. I'm just curious, how has the process been with writing this library as opposed to using Unity's own networking library or a solution like Photon? I'm assuming we can use our own server with this?
     
  31. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    I'm not sure I fully understand your question but I'll try and answer it!

    Hazel is much much more low level than Photon and UNET, they understand things like 'players' etc. Think of Hazel more as a pipe, you put a message in and they come out the other side, that's all it does and you don't need to think about how the pipe gets your messages there it just does!

    (I cant believe I just reduced Hazel to a pipe analogy...)
     
  32. invictvs1

    invictvs1

    Joined:
    Mar 7, 2014
    Posts:
    51
    You answered well, apologies if I wasn't clear. That's perfect, I was more curious about the actual process, as in did you hit any roadblocks or are there major issues that you came across with working at the socket level alongside Unity? That makes sense that it is just for sending messages back and forth across TCP and UDP, now I just have to worry about client prediction and lag compensation ;)

    I am currently working on an RTS mobile game where we didn't want to use any of the existing libraries (Unet, Photon etc) due to their pricing model and also the fact that we wanted to build everything in house and have control over our data flow. Will give Hazel Client a go tonight!
     
  33. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hazel was completely developed outside of Unity actually, my main goal was to make it cross platform (and so at some point I will probably introduce a Java version as well) hence I did it outside and then made sure it worked within Unity after. It wont work on the platforms that don't enable sockets (I've frankly lost track of which Unity licenses you need now...)

    I think the next step is to do websockets but I feel like that's going to be 2 different projects, one for within Unity built from calls down to Javascript then to websockets and then one for outside Unity written purely in C#.
     
    invictvs1 likes this.
  34. iileychen

    iileychen

    Joined:
    Oct 13, 2015
    Posts:
    103
    Hi folks, i have worked with Hazel for a time, in the meanwhile, i actually played with LLAPI of UNET as well. Also worked the pure Socket at the beginning.
    I would say, Hazel is excellent, i love the pattern it use (like pipes yes). After 2 month facing on the networking, i finally make Hazel work in my official project - a fps kind game, but with some modification to support WebGL, yes i implemented WebSocket based on Hazel and jonas' websocket class. Based on Hazel's good structure, it is easy to add a implementation.

    @Jamster, i have modified the TcpConnection.BodyReadCallback method to :
    Code (CSharp):
    1.         void BodyReadCallback(byte[] bytes, bool mask)
    2.         {
    3.             Statistics.LogReceive(bytes.Length, bytes.Length + 4);
    4.  
    5.             //Fire DataReceived event
    6.             InvokeDataReceived(bytes, SendOption.FragmentedReliable);
    7.  
    8.             //Begin receiving from the start
    9.             StartWaitingForHeader();
    10.         }
    It seems fixed the issue of
    Not tested much, please check am i right for the modification.
     
    Jamster likes this.
  35. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Brilliant!

    Your modification should work and guarantee order yes :) I will probably add something like that/a toggle that will allow that as that a much neater solution to what I was thinking!

    Jamie
     
  36. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Added some bug fixes to TCP for those interested :)
     
    ManHunterITA likes this.
  37. ManHunterITA

    ManHunterITA

    Joined:
    Sep 3, 2013
    Posts:
    341
    What did you add?
    Did you already pushed it on your repo? :D
     
  38. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    There were a couple of points where it wasn't handling disconnections properly, already pushed :)
     
    ManHunterITA likes this.
  39. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hazel has seen a couple of improvements,firstly I've added the ability to send a portion of bytes with the initial connection packet, this allows you to send details about your client and any additional data you may need to send before communication actually begins.

    I've also added a major security patch that was allowing people to crash TCP connections by sending malformed packets (funnily enough, exactly those that telnet sends if you connect with it!). Thanks to Terry for pointing that one out!

    Rewards may be available for significant bugs found in DarkRift/Hazel! ;)
     
    ManHunterITA likes this.
  40. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Is anyone having a problem using Hazel in Unity builds?

    Currently using Unity 5.5.2 and whenever I build to Windows standalone I get errors trying to connect and nothing trying to receive... just me?
     
  41. ManHunterITA

    ManHunterITA

    Joined:
    Sep 3, 2013
    Posts:
    341
    I'll let you know: I'm building my own system on top of Hazel, but I didn't try it yet on Unity.
     
    Jamster likes this.
  42. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Seems to just be that Unity's Mono hasn't implemented the SendToAsync method that Hazel used so they just threw an UnsupportedOperationException (thanks Mono!)

    Interesting that it worked in editor but not build... Pushed to GitHub!
     
    ManHunterITA likes this.
  43. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    ManHunterITA likes this.
  44. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hazel now supports timeouts on Connect!

    Also, in my infinite wisdom I messed up the versioning on NuGet even further so if you would like to update then please use for this release
    Code (csharp):
    1. Update-Package DarkRiftNetworking.Hazel -Version 0.0.0-beta
    to get the latest version... :oops:
     
  45. Fred_Flintstones

    Fred_Flintstones

    Joined:
    Oct 27, 2015
    Posts:
    3
    I can Start a server in Unity, and a Server/Client in windows console app ---They work fine, but if I try to start a Client in Unity it freezes on Connect()

    If I close the server, the client goes on its mary way...



    NetworkEndPoint endPoint = new NetworkEndPoint("127.0.0.1", 4296);
    connection = new TcpConnection(endPoint);
    connection.DataReceived += DataReceived;

    connection.Connect(); //<--------------------------------------------------------------FREEZES (in till server is shutdown)

    connection.SendBytes(new byte[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 });

    connection.Close();
     
  46. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Hey, are you using the latest update from GitHub/NuGet because I changed things in Connect and it may very well be that that's causing problems...!
     
  47. Fred_Flintstones

    Fred_Flintstones

    Joined:
    Oct 27, 2015
    Posts:
    3
    Yes, got it yesterday
    EDIT: so it was completely my fault.....I had just commented out code in Darkrift/embeddedcubedemo...and replaced it with hazel.....BUT in the main camera there was a script trying to connect with darkrift...
     
    Last edited: Dec 3, 2016
    Jamster likes this.
  48. Supergeek

    Supergeek

    Joined:
    Aug 13, 2010
    Posts:
    103
    Jamster, have you given any thought to .NET Core compatibility in Hazel? Core is the future; minimal and performant. The perfect match for Unity.
     
  49. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    As I mentioned on GitHub I'll probably do it at a later date so that DarkRift can work for .NET Core rather than Mono in the future :)
     
  50. Jamster

    Jamster

    Joined:
    Apr 28, 2012
    Posts:
    1,102
    Fragmented messages have been added!

    This will probably be the last minor release in beta, the next release will likely be production (aside from fixes). I've also started adding releases to the GitHub page, have updated the Nuget release and updated the documentation (it was a bit behind!).

    Happy New Year

    Jamie
     
    ManHunterITA likes this.