Tips for server-authoritative player movement

I recently spend quite a while on server-authoritative movement for my multiplayer game.
At first, I thought it wouldn't be that hard: client simply sends input to server, which simulates object state. Server periodically broadcasts object state and client checks to make sure it's close enough to the server state.
I soon realized this problem was much harder than I thought originally. Eventually I borrowed the code from the uLink snowball demo and used that as a starting point. After much experimentation I'd like to point out some gotchas that I encountered while working on it.

Movement code must be entirely deterministic
Given the same input, your movement code should have almost exactly the same output, every time.
In my case, my character physics script has all movement code (the entire update function, basically) wrapped in a public Simulate function. This function takes delta time as a parameter, which is very important as you want to make sure client and server call Simulate with the same delta (this is what makes it deterministic, aside from the occasional precision error)

Use FixedUpdate
This isn't a strict necessity, but I highly recommend you use FixedUpdate for your movement code, because FixedUpdate is a lot more reliable than Update. FixedUpdate runs a fixed number of times per second, while Update is entirely dependent on framerate. For one, I use FixedUpdate to gather input and send it to the server, and using Update means the bandwidth will depend on framerate (network usage being linked to render speed is pretty fugly IMHO). For another, using Update means I have to transmit delta to the server, which opens me up to speedhacking (clients can specify a sufficiently large enough delta to teleport), using FixedUpdate means client and server have the same timestep and the server can just use Time.fixedDeltaTime

Make sure update order is deterministic.
For a while, I simply let my physics code run, and periodically grabbed the input state and sent it to the server. This led to frequent desyncs, and I modified my code to manually step the simulation after getting user input state. Now, it always runs in this order:

1. Client gathers user input state
2. Client steps the physics simulation with input state
3. Client sends input to server (also stores input state so it can replay them later)
4. Server steps the physics simulation with input state

This might seem obvious, but if you have your physics in a separate script you can easily end up with indeterministic tick order. In my case, I disabled the update function in my physics script and manually stepped it from my network controller.

Overview of my code
As a bit of an overview, here's how my server-authoritative code works:

1. Client gathers user input state
2. Client steps the physics simulation with input state
3. Client gets result of physics step (in my case, position)
4. Client sends user input state and simulation result (position) to server
5. Server steps the physics simulation with input state
6. Server compares its own simulation result with result sent from client. If they differ too much, server sends correct state to client
7. If client receives state correction, it goes back through stored input states back to the point of the "correct state" and replays them (in my case, I timestamp each input state with network time, so I find the one with the closest timestamp and replay all inputs from that point). This involves resetting the state to the correct state sent by the server, then iterating over each stored input stepping the physics simulation. This is necessary because the state correction sent by the server is old (due to network latency, it was sent some time in the past) so we need to re-apply all of our inputs since that time in order to stay current.

Basically, a lot of what I'm saying boils down to this: if you get a lot of rubber banding and desyncs happening in your code, that means you need to go back and make sure your client and server are running the EXACT same code, with the EXACT same inputs, in EXACTLY the same order. If it looks like they are, take a step back, drink a cup of coffee, and come back to it after taking a break. Chances are you're missing something small somewhere that accounts for the difference (in my case it was usually code running in a different order between client and server)

 

http://forum.exitgames.com/viewtopic.php?f=5&t=3234&p=14430
Any help, please?

 

For my server-authoritative code, I simply just have the server tell the client exactly where it needs to be if the server finds an issue. This prevents me from having to write all of the "go back and check" code. It also just works from my experience.

 

Must be a very limited case. In my experience, if the client does ANY sort of prediction of it's own state, you'll end up with very frequent rubber banding if you aren't careful.

 

I'm just about to hit the point where my MMORPG will be dealing with movement being transferred between one person to another, obviously through the server, and I read your post and found it interesting but I do have a question.

Why not just arbitrarily declare some value r where r is the maximum value of displacement in 3D space given the difference between the last vector position and the new vector position? If we only accept update information at a fixed rate on the server so even if they constantly send some data, because speed hacking would cause the client to send the update faster and thus the point of measuring displacement per change is pointless, will that not work? Say every second or two we could compare the magnitude of the difference between the old and the new vector on the server and if it's larger than the given r we can just reset the player in some way?

I'm just brainstorming as I type, and before I go to sleep for classes tomorrow, so maybe that's not a great idea. I can't imagine anything pro grammatically challenging about that approach except for maybe determining where to reset the player if their new vector won't pass the arbitrary r. I should we could just read and if statement and determine it by elimination since we know they either need to be moved forward or backward in world space. That might work. Would this not prevent the client from cheating?

 

It could prevent the client from cheating too much, to an extent, but there's problems with that approach.
For one, it's more of a heuristic than anything else. That means you have to ensure it's relaxed enough to account for slight differences between what the server expects and what the client expects, but not so relaxed that they can still cheat the system up to the limits. That takes some tweaking to achieve.
For another, it will result in plenty of edge cases which require special case logic. For instance, how do you handle vertical motion? It's certainly possible (and in many cases likely) that a player could fall vertically much faster than they could move horizontally (for instance, they could fall at 10m/s while the player move speed is capped at 5m/s). Let's also say we add teleporters to our game, Quake style. Now your server has to handle the case when a player teleports with the aid of a teleporter rather than hacking. You could have the server handle teleporting, but that will result in a delay when the player enters the teleporter. Not that bad in an MMO, however.

However, just providing input to the server and predicting the same input locally solves all of the edge cases pretty much for free just as a result of how it works.

 

You could just calculate whatever the greatest potential r could be, which is essentially the radius of some non-existent sphere, and add some cushion to that and call it your r. It'll give the client a small sphere-like area in which their speed hack is effective up to atleast the maximum allowed.

The instance you gave me, well they're capped at 10m/s+/-cushion then but I suppose if that is the case you could individually measure components of of the vector to see if they exceed some maximum value. Otherwise we can constantly pass the data as verifiable and only check their position for some given time or if they preform an operation such as picking up a weapon or sword. I just am not convinced though that calculating everything on the server as if it were a player is scale-able though. Won't their be some performance issues that come with calculating more than just displacement vectors as opposed to a full physics simulation that mirrors Unity's?

As for the teleporter case, when they send a request to the server all we have to do is verify their location and if they've just achieved that location legitimately. I'd assume we'd have only a handful of teleport locations so we could store general position information of those on the server. That should prevent anyone from sending an illegal teleportation request. Where they end up will also be determined by the server so it should seal up holes there. I do very much worry about exploitation. I find it the most fun conversation/thought-experiments related to development though. If you're worried about delay we can just have the client assume they've teleported. If they haven't we can recorrect them. That should effect too many legitimate players.

I won't be able to respond till tomorrow if you post again.

 

In a small player-to-player game, definitely not.
However, in a massive MMO-type game, yes there are some processing costs involved. That said, there are MMOs that employ full-blown server side physics, Second Life comes to mind (that said I don't much care for Second Life's physics - there tends to be a LOT of rubber banding and such)

 

That game is a bit odd from what I can recall, I read somewhere that for every individual plot of land they have a dedicated server to preforming all of the calculations for anything contained within it. I guess they have/had a network of thousands of single servers and if when goes down the whole thing goes down. It was an interesting read, it seems very unscaleable though the way they described it.

 

Actually, the last I read, they had a single dedicated server per region (not to be confused with plots of land, there could be any number of plots inside a region although that concerns gameplay/economy more than server tech I suppose).
As far as servers going down, often when a region server goes down (which a friend of mine may or may not have been guilty of by crashing the physics engine with a whirling merry-go-round of death >.>) it log you out (well, sometimes. Other times you just flew off into the sky in slow motion, or worse.), and when you log back in it teleports you to the nearest region from where you were (it won't let you back in the crashed region). So regions were fairly modular.

 

hi @phobicGunner,

really nice post
I am having issues while using authoritative server... its not working well. I need to compute scores of both players and transmit to client.
can you please guide me to really good tutorials links ?
I am totally stuck.

 

Check my first reply to this thread: http://forum.unity3d.com/threads/237065-Unity-2D-Client-Side-Prediction

 

You have an authoritative server. Players should never get control over their own score, and therefore the server is responsible for keeping score.
Where is score kept?
If you wanted to take a page out of UE4's book, you might have a PlayerInfo object for each player. This can store information like name, score, kills, deaths, etc. It has a NetworkView attached to it, set to Reliable Delta Compressed (if no updates occur, no updates are sent), and serializes these values in OnSerializeNetworkView. This is the general behaviour in UE4 (if any values are flagged as changed, the server will send the changed values to connected clients)
Your server could instantiate these for each player as they connect (and destroy them when players leave) and then set the proper values. When these values change, the network view will detect this and serialize the new values to connected clients.
This is the approach I will take (I'm using uLink, but the same general concepts apply to Unity Networking).
The same approach can be applied to game state in general. For instance, some kind of GameInfo object which is network instantiated by the server and contains values like Red Team Score, Blue Team Score, etc.

 

I had an interesting thought. Someone earlier in this thread mentioned how strange it was that Unity didn't have built-in support for stepping the state of a physics object manually.
I would like to point out some interesting findings of mine which should explain why this isn't really an oversight, but rather a problem of computational complexity. Apologies if this has already been stated, I haven't been to the thread in a while

Let's say we have a car and a stack of boxes. We, the player, want to drive the car through the boxes. So let's say the car comes into contact with the boxes, and then the server decides we don't have the correct game state. So we rewind the car and then begin to replay it.
But as we're replaying the car, if the car is the *only* thing we are replaying, it could end up in a situation where it intersects the boxes, OR the car simply treats the boxes as if they are static. Neither of these cases is correct. So we could begin stepping the state of the boxes too. But in order to be truly accurate we would have to begin stepping them at the same time as the car (instead of waiting for a collision between the car and the boxes). How does the physics determine which objects it needs to step in order to accurately advance the state of one single object? Should it just advance the state of the entire physics simulation? In that case we would have to rewind every physics object in the scene and replay it, resulting in a much higher CPU load because now we are performing many orders of magnitude more physics updates than we normally would.

The best way to deal with this, IMHO, is to let the player have direct control over characters (which is necessary because characters typically allow for twitch-speed movement which will noticeably suffer under latency conditions), but to let the server have full control over everything else with no client-side prediction (such as cars, which have relatively slow acceleration/deceleration which is far less likely to be noticed in the presence of latency).

EDIT: I was wrong. It wasn't anybody in this thread, I had it confused with another thread about how Unity was not well-suited for FPS games, and this was one of the arguments presented.
Please excuse my mental malfunction

 

Thank you, it worked very well for me using a character controller and the move function (After you call it the object moves so you can resimulate from any point and do the things as you suggested).

It didn't worked at first. Finally it was because I was not playing all the states,
Also, to reduce bandwith, I am using RPC at a certain rate sending an Array of states (if 2 consecutives are the same, it is not stored in the array) packed with mspack.

Each State has an int of pressed buttons, position, look direction and timestamp. The states are saved each fixed update (20ms deltatime) and sent in the packed array (100ms realtime). Each player uses 2500 bytes/second when moving and 800 bytes/second when still.

RPC can send a byte array even if it is no documented.

 

Also want to point out that, ideally, you'd want to send input unreliably. That said, I don't think you can do that in Unity networking with RPCs (you can in Bolt, TNet, uLink, or custom socket solutions - perhaps one built on Lidgren for instance)

 

how did you accomplished manually stepping the physics?

i'm with you, all the things you said in the OP is the same i do except manually stepping the physics after receiving client input. .... don't know how long i have to wait for all clients to receive theire input so the next physicstep includes all necessary information from all clients to calculate correct values. the waiting would also result a lag for all players - Doom-like-netcode

do you use unitys built-in physics (rigidbody with velocity) as movement or do u use transform.Translate(...) to be deterministic?

currently i'm using transform.Translate(...) on Client and Server. the problem with this movement is that the client doesnt recognize collisions and can walk trough walls if he tries long enough - server corrects him but there is also a small room of tolerance to ignor small differences and save bandwidth.

i also tried to save input, simulate and wait for the next fixedupdate to be sure client uses physics, sadly it didnt helped.

how did you implement the sync movement with physics, i'm very interessted!

 

I'm using CharacterController. Since you move a character controller with a call to Move, you get precise control over when a character moves, and by how much.
You cannot do clientside prediction with rigidbodies, since you can't manually step a rigidbody.

 

please describe the when...

http://docs.unity3d.com/ScriptReference/CharacterController.Move.html
strange that they run this movement in Update()

overall it looks very similar to transform.Translate() the only thing is you dont need to check manually if character is on ground.

 

Floating point arithmetic isn't deterministic cross-platform and CharacterController uses floats. It shouldn't be a problem though because the server's supposed to simulate and send the 'real' positions so everyone'll still be in sync.