1. Help us improve the editor usability and artist workflows. Join our discussion to provide your feedback.
    Dismiss Notice
  2. We're looking for feedback on Unity Starter Kits! Let us know what you’d like.
    Dismiss Notice
  3. We’re giving 2017.1 beta testers a chance to win t-shirts and a Nintendo Switch. Read more on the blog.
    Dismiss Notice
  4. Unity 2017.1 beta is now available for download.
    Dismiss Notice
  5. Unity 5.6 is now released.
    Dismiss Notice
  6. Get further faster with the Unity Plus Accelerator Pack, free for new Unity Plus subscribers for a limited time. Click here for more details.
    Dismiss Notice
  7. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  8. Learn how you'll soon be able to publish your games to China in four simple steps with Xiaomi. Sign up now for early access.
    Dismiss Notice

Unity Multiplayer Relay Server Bandwidth Explanation

Discussion in 'Multiplayer Networking' started by JeremyUnity, Oct 28, 2015.

  1. JeremyUnity

    JeremyUnity

    Unity Technologies

    Joined:
    Mar 4, 2014
    Posts:
    147
    We've received a lot of questions regarding exactly how Relay server tracks bandwidth used per game, and in today's server update we've rewritten that logic to be as fair as possible to help address any disconnect issues people encounter due to bandwidth limits.

    First off, some background. The bandwidth limit is in place to protect the service and make sure everyone has an optimal play experience. It's also there to prevent unchecked bandwidth use, which would directly affect our ability to provide the Unity Multiplayer backend service if no protection were in place. Based on experience and research we arrived at a 4 kilobyte per user per second protection limit. It's important to know this is calculated to include IP and UDP headers, which can represent a sizable portion of the traffic if a lot of small packets are sent instead of larger, less frequent payloads.

    Note that we also intend this to be used for game data, so things like voice or video aren't currently supported through relay server for bandwidth reasons.

    Even before today's update, we did (and still do) several things to make it more fair than a simple per second metering would allow:
    • The bandwidth is only totaled for bandwidth leaving relay server to each connected game.
    • The bandwidth is and always has been calculated over the lifetime of each client connection. That means for every second a client stays connected they are given another 4k per second to use, and if they don't use it at that instant they will be able to at any point until they leave.
    • When a new client connects it's given two minutes of grace period bandwidth in case the connection handshake is very spiky, which is a pretty typical use pattern. That equals about 480k before the first second passes.
    With today's update we also have the following change:
    • Bandwidth is still handed out on a per client basis as described above, but we don't kick individual clients for overages anymore. Instead, we deduct bandwidth used from that group pool and wait until the entire match is over it's bandwidth limit, then kick everyone if that pool is exhausted. This logic much better supports the client/server pattern where updates can be very heavy for the server to clients for instance, but a more limited amount of data is sent from the clients to the server.
    This can be roughly expressed as the following:
    ((4k * S) * U) + (480k * U)
    Where S is seconds a User is connected and U is the number of users connected.

    In the future we're planning on also adding the ability to query the information on a match's bandwidth use from relay server, though i don't have anything i can announce today with regards to the specifics.
     
    scottlaforge, bboysil, -chris and 2 others like this.
  2. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    18,842
    Thanks for the clarification. I'm concerned about tracking traffic within Unet - it would be pretty awesome to be able to measure client and host bandwidth respectively without resorting to 3rd party tools... also a compression utility class might help too, for example if we know a float will only be in the range of 0..1 or -1 to 1 we can probably go with a compressed float.

    And for drop in support (a player joins mid game), I'm guessing this is also part of the global pool.
     
    gregorypierce likes this.
  3. JeremyUnity

    JeremyUnity

    Unity Technologies

    Joined:
    Mar 4, 2014
    Posts:
    147
    Thanks for the feedback. We do realize tracking is difficult right now and we're working to fix that, though i cant make any promises yet on when that will be available.

    If i understand the drop in support, yes that's what we're doing now. Players are free to come and go, and as they do the number of bytes per second given to the global match pool increases or deceases appropriately.

    So for instance if you have 2 players connected for 60 seconds, you'll be accruing 8k per second over those 60 seconds to use in that match (4k * 2 players) in addition to the initial grace pool given to each player.

    If a 3erd player joins at that point you'll have whatever bandwidth the match has accrued so far, plus that player will add in their own grace period to that pool. In addition the match will gain 12k per second going forward from that point.
     
    gregorypierce and hippocoder like this.
  4. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    18,842
    Thanks for explaining. I did have a thought today that the Host would probably be using a lot more bandwidth than everyone else. Since the host is the "true cap" this means 4k is probably too harsh unless some exception is given for the host, since in effect, everyone is capped by whatever amount the host uses.
     
    moco2k, gregorypierce and JeremyUnity like this.
  5. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    3
    It'd be great to have some kind of bandwidth histogram, e.g. In the profiler.

    I believe with the new pooled limit, typically asymmetric scenarios as well as bandwidth spikes ((e.g. Player entering a crowded area) are quite manageable.

    I guess higher bandwidth limits will cost $$$ at some point, which I consider very fair and not unexpected.
     
    moco2k, gregorypierce and JeremyUnity like this.
  6. tliawi

    tliawi

    Joined:
    Sep 15, 2015
    Posts:
    3
    I need more than 4k/sec per client. I'd be willing to pay, in moderation. I'm using networkServer.sendByChannelToAll using Channels.DefaultUnreliable, and I believe that uses UDP, but I can't find any Unity claim that it's doing multicasting, and here I'm limited to 4k per client, as if Unity were NOT doing multicasting. Which is it? If not mulitcasting, why not? If multitcasting, why the limit per client on a multicasted send?

    JeremyUnity, if the bandwidth demand is pooled over all clients, couldn't I generate a bunch of fake clients that didn't use anything so that I can get enough bandwidth to run my game on the real clients?
     
  7. JeremyUnity

    JeremyUnity

    Unity Technologies

    Joined:
    Mar 4, 2014
    Posts:
    147
    The 100 CCU @ 4k/second is just for our free Beta period. When we are done with that we'll have more to announce as far as pricing options, but we will have a way for you to opt into numbers that are more favorable to your own service needs.
     
    BGAdders and hippocoder like this.
  8. tliawi

    tliawi

    Joined:
    Sep 15, 2015
    Posts:
    3
    Thanks. In the meantime, would it be possible for us to download your relay server and put it up on, say, our aws server, without the 4k/sec limit, and register it with your matchmaker? Then we'd be paying our own way for the bandwidth, and yet could test and develop using your matchmaker.
     
  9. TEBZ_22

    TEBZ_22

    Joined:
    Jan 28, 2014
    Posts:
    37
    I'm wondering if Unity are counting just the payload or the internet size?
    -
    Then bandwidth used are not only the payload, but also some overhead, and there are a minimum packet size to consider.
     
  10. JeremyUnity

    JeremyUnity

    Unity Technologies

    Joined:
    Mar 4, 2014
    Posts:
    147
    @TEBZ_22 You are exactly correct. We calculate header size and overhead into bandwidth calculations
     
  11. OEGInc

    OEGInc

    Joined:
    Apr 29, 2016
    Posts:
    5
    I don't think this would be as much of an issue if you released source code to a relay server we could host ourselves, or at least provided packet formats & routines for encoding/decoding them, etc...

    We have our own datacenter with more than 4 Gb/sec of Internet connectivity, we could very easily host our own.
     
  12. moco2k

    moco2k

    Joined:
    Apr 29, 2015
    Posts:
    227
    We really need precise information in the official docs regarding bandwidth consumption of overheads and headers, and also on best practices regarding good network code designs (also see these threads #1 #2). And, as already noted above, a way to directly measure bandwidth consumption in addition to the current profiling options would be great too.

    I think you already have these points on your improvement list, but still I wanted to recap them here because they have high relevance in context of the relay server functionality.
     
    Last edited: May 26, 2016
    scottlaforge, rikey and JeremyUnity like this.
  13. pavan123

    pavan123

    Joined:
    Nov 5, 2016
    Posts:
    4
    Hi i was working on implementing voice communication in my multiplayer application.For this i used matchmaker concepts and integrated multiplayer services. But here my client connection keeps breakdown in less than a minute.I have searched so many links and come to conclusion that might be happening due to less relay server bandwidth which could be 4kb/s limit.could anyone point me how to extend that bandwidth.
     
  14. mojtaba64

    mojtaba64

    Joined:
    Aug 3, 2010
    Posts:
    45
    Hi,

    I'm working on racer game and it published for test...
    It's a 1vs1, 1 minute game, I calculated the traffic and for a minute it only use 20KB per each player.
    My question is why I should pay for 480KB in this formula for each player: ((4k * S) * U) + (480k * U)

    Thanks
     
  15. DBronson

    DBronson

    Unity Technologies

    Joined:
    Oct 1, 2016
    Posts:
    9
    @mojtaba64 - That calculation is just for the bandwidth available to you before the service starts to throttle. You only pay for what you game actually uses.
     
    mojtaba64 likes this.
  16. Rungsted93

    Rungsted93

    Joined:
    Jun 29, 2016
    Posts:
    19
    Is this the explanation why my RTS game suddenly kinda stops working/freezes, units don't die, can't build new ones etc. ? If so when/if i sign up for the multiplayer services are these limits removed since i'll pay for data usage? :)

    Also is there a higher limit to the data usage users have if it's build from a Plus or Pro account? Would be nice to test if it's 100% the reason before signing up for multiplayer services :)

    And at last is the CCU from plus or pro still included in the multiplayer services? so lets say 202 plays my game and i've got pro i only pay for the 2 of them. If not that's a shame! it would add a lot more value to upgrade from plus to pro to get 150 additional free of charge + unlimited data CCU, atleast when you're an indie dev :)