Relay Server Bandwidth Explanation

We've received a lot of questions regarding exactly how Relay server tracks bandwidth used per game, and in today's server update we've rewritten that logic to be as fair as possible to help address any disconnect issues people encounter due to bandwidth limits.

First off, some background. The bandwidth limit is in place to protect the service and make sure everyone has an optimal play experience. It's also there to prevent unchecked bandwidth use, which would directly affect our ability to provide the Unity Multiplayer backend service if no protection were in place. Based on experience and research we arrived at a 4 kilobyte per user per second protection limit. It's important to know this is calculated to include IP and UDP headers, which can represent a sizable portion of the traffic if a lot of small packets are sent instead of larger, less frequent payloads.

Note that we also intend this to be used for game data, so things like voice or video aren't currently supported through relay server for bandwidth reasons.

Even before today's update, we did (and still do) several things to make it more fair than a simple per second metering would allow:

• The bandwidth is only totaled for bandwidth leaving relay server to each connected game.
• The bandwidth is and always has been calculated over the lifetime of each client connection. That means for every second a client stays connected they are given another 4k per second to use, and if they don't use it at that instant they will be able to at any point until they leave.
• When a new client connects it's given two minutes of grace period bandwidth in case the connection handshake is very spiky, which is a pretty typical use pattern. That equals about 480k before the first second passes.

With today's update we also have the following change:

• Bandwidth is still handed out on a per client basis as described above, but we don't kick individual clients for overages anymore. Instead, we deduct bandwidth used from that group pool and wait until the entire match is over it's bandwidth limit, then kick everyone if that pool is exhausted. This logic much better supports the client/server pattern where updates can be very heavy for the server to clients for instance, but a more limited amount of data is sent from the clients to the server.

This can be roughly expressed as the following:
((4k * S) * U) + (480k * U)
Where S is seconds a User is connected and U is the number of users connected.

In the future we're planning on also adding the ability to query the information on a match's bandwidth use from relay server, though i don't have anything i can announce today with regards to the specifics.

 

Thanks for the clarification. I'm concerned about tracking traffic within Unet - it would be pretty awesome to be able to measure client and host bandwidth respectively without resorting to 3rd party tools... also a compression utility class might help too, for example if we know a float will only be in the range of 0..1 or -1 to 1 we can probably go with a compressed float.

And for drop in support (a player joins mid game), I'm guessing this is also part of the global pool.

 

Thanks for the feedback. We do realize tracking is difficult right now and we're working to fix that, though i cant make any promises yet on when that will be available.

If i understand the drop in support, yes that's what we're doing now. Players are free to come and go, and as they do the number of bytes per second given to the global match pool increases or deceases appropriately.

So for instance if you have 2 players connected for 60 seconds, you'll be accruing 8k per second over those 60 seconds to use in that match (4k * 2 players) in addition to the initial grace pool given to each player.

If a 3erd player joins at that point you'll have whatever bandwidth the match has accrued so far, plus that player will add in their own grace period to that pool. In addition the match will gain 12k per second going forward from that point.

 

Thanks for explaining. I did have a thought today that the Host would probably be using a lot more bandwidth than everyone else. Since the host is the "true cap" this means 4k is probably too harsh unless some exception is given for the host, since in effect, everyone is capped by whatever amount the host uses.

 

It'd be great to have some kind of bandwidth histogram, e.g. In the profiler.

I believe with the new pooled limit, typically asymmetric scenarios as well as bandwidth spikes ((e.g. Player entering a crowded area) are quite manageable.

I guess higher bandwidth limits will cost $$$ at some point, which I consider very fair and not unexpected.

 

I need more than 4k/sec per client. I'd be willing to pay, in moderation. I'm using networkServer.sendByChannelToAll using Channels.DefaultUnreliable, and I believe that uses UDP, but I can't find any Unity claim that it's doing multicasting, and here I'm limited to 4k per client, as if Unity were NOT doing multicasting. Which is it? If not mulitcasting, why not? If multitcasting, why the limit per client on a multicasted send?

JeremyUnity, if the bandwidth demand is pooled over all clients, couldn't I generate a bunch of fake clients that didn't use anything so that I can get enough bandwidth to run my game on the real clients?

 

The 100 CCU @ 4k/second is just for our free Beta period. When we are done with that we'll have more to announce as far as pricing options, but we will have a way for you to opt into numbers that are more favorable to your own service needs.

 

Thanks. In the meantime, would it be possible for us to download your relay server and put it up on, say, our aws server, without the 4k/sec limit, and register it with your matchmaker? Then we'd be paying our own way for the bandwidth, and yet could test and develop using your matchmaker.

 

I'm wondering if Unity are counting just the payload or the internet size?
-
Then bandwidth used are not only the payload, but also some overhead, and there are a minimum packet size to consider.

 

@TEBZ_22 You are exactly correct. We calculate header size and overhead into bandwidth calculations

 

I don't think this would be as much of an issue if you released source code to a relay server we could host ourselves, or at least provided packet formats & routines for encoding/decoding them, etc...

We have our own datacenter with more than 4 Gb/sec of Internet connectivity, we could very easily host our own.

 

We really need precise information in the official docs regarding bandwidth consumption of overheads and headers, and also on best practices regarding good network code designs (also see these threads #1 #2). And, as already noted above, a way to directly measure bandwidth consumption in addition to the current profiling options would be great too.

I think you already have these points on your improvement list, but still I wanted to recap them here because they have high relevance in context of the relay server functionality.

 

Hi i was working on implementing voice communication in my multiplayer application.For this i used matchmaker concepts and integrated multiplayer services. But here my client connection keeps breakdown in less than a minute.I have searched so many links and come to conclusion that might be happening due to less relay server bandwidth which could be 4kb/s limit.could anyone point me how to extend that bandwidth.

 

Hi,

I'm working on racer game and it published for test...
It's a 1vs1, 1 minute game, I calculated the traffic and for a minute it only use 20KB per each player.
My question is why I should pay for 480KB in this formula for each player: ((4k * S) * U) + (480k * U)

Thanks

 

Is this the explanation why my RTS game suddenly kinda stops working/freezes, units don't die, can't build new ones etc. ? If so when/if i sign up for the multiplayer services are these limits removed since i'll pay for data usage?

Also is there a higher limit to the data usage users have if it's build from a Plus or Pro account? Would be nice to test if it's 100% the reason before signing up for multiplayer services

And at last is the CCU from plus or pro still included in the multiplayer services? so lets say 202 plays my game and i've got pro i only pay for the 2 of them. If not that's a shame! it would add a lot more value to upgrade from plus to pro to get 150 additional free of charge + unlimited data CCU, atleast when you're an indie dev

 

Please can you clarify if the 4KB/s limit is fixed for all released games or just "free" bandwidth for games still in development?
Are there pricing tiers which permit higher bandwidth?

 

can anyone answer this question? this is my question too

 

When you go live you can increase max bandwidth per client.

 

Any news on this? Is there currently a way to get more bandwidth during development? 4k is hard to swing in VR with tracked head/left hand/right hand per player.

 

4kb per user per second per user is plenty for multiplayer. You should be finding ways to compress your data into steady streams of smaller chunks. On average 50 users on a fully developed game server I run, with tons of things going on and a streaming constantly updated map (diggable) uses a total network traffic of around 80kb, and that is including the webserver and while I am RDP connected to it. So really its only using an average of 1kb per second per user during normal load (not having everyone in one spot at one time). Even with them all together I usually see actuall PID usage for the server run 50 users around 120-150kbs. 4 per user is plenty.

Though reading the last post here I have no experience with VR. Would imagine slightly more constant sending with more things to track. Still narrow it down like a mocap would to a few points, then find ways to encrpyt that down to a couple bytes per packet in a managable buffer
that you can properly mark and continue on the deserial side