Latency Compensation for Physics

We've been getting started with multiplayer code for Volo Airsport, a fast paced wingsuit skydiving simulator. This is our first time doing serious netcode, and we're currently struggling with a very specific issue.

Key requirement: Being able to fly next to one another. E.g. If I look to my right and see you flying there, you need to be able to look to your left and see me there as well.

Given the inescapable reality of non-zero round trip times, we'll need some form of client side prediction to make it appear remote players are right next to you, because the message we receive from them are from slightly back in time.

Now, we've doing lots of reading and prototyping for specific prediction and dead reckoning techniques. It's a big can of worms, but we've found decent enough ways to do it.



Using a hermite spline extrapolation based on the last two received states works well, as does simply integrating received acceleration and velocity with the Euler method.

What's turning out to be a very interesting problem is deciding how much time to predict forward on the client when we're using Unity's rigidbodies.

We've got a simple project setup in which we try to predict a single player controlling a rigidbody, using an authoritative server model. Let me walk you through it:

The serverside object takes input from a client as an RPC, and each FixedUpdate it applies forces using that input. It also calculates its current acceleration, which we include in the state update message sent back to the client.

Code (csharp):

1.  
2. private void FixedUpdate() {
3.         Simulate(_clientInput);
4.  
5.         _lastAcceleration = (_rigidbody.velocity - _lastVelocity) / _gameClockFixed.DeltaTime;
6.         _lastVelocity = _rigidbody.velocity;
7.  
8.         // Force OnSerialize to be called
9.         SetDirtyBit(uint.MaxValue);
10.     }
11.  
12.     private void Simulate(Vector2 input) {
13.         Vector3 force = new Vector3(input.x, 0f, input.y) * 10f;
14.         _rigidbody.AddForce(force, ForceMode.Acceleration);
15.     }
16.  
17.     // Called whenever input is received from client
18.     public void OnCmdMove(Vector2 input) {
19.         _clientInput = input;
20.     }
21.  

OnSerialize is called at our specified GetNetworkSendInterval, say 10 times per second. This is on QoS.UnreliableSequenced, by the way.

Code (csharp):

1.  
2. // Serialize Rigidbody data on server
3. // Rotation data omitted for clarity
4. public override bool OnSerialize(NetworkWriter writer, bool initialState) {
5.         var message = new TimestampedMessage<RigidbodyState> {
6.             Value = new RigidbodyState() {
7.                 Position = _rigidbody.position,
8.                 Velocity = _rigidbody.velocity,
9.                 Acceleration = _lastAcceleration
10.             },
11.             FixedTime = _gameClockFixed.Time, // Clock object giving us current fixedTime
12.         };
13.  
14.         writer.Write(message.FixedTime);
15.         writer.Write(message.Value.Position);
16.         writer.Write(message.Value.Velocity);
17.         writer.Write(message.Value.Acceleration);
18.         return true;
19.     }
20.  

On the client we then receive server states in OnDeserialize and cache them, like so:

Code (csharp):

1.  
2. public override void OnDeserialize(NetworkReader reader, bool initialState) {
3.         var serverState = new TimestampedMessage<RigidbodyState> {
4.             FixedTime = reader.ReadSingle(),
5.             Value = new RigidbodyState {
6.                 Position = reader.ReadVector3(),
7.                 Velocity = reader.ReadVector3(),
8.                 Acceleration = reader.ReadVector3()
9.             }
10.         };
11.  
12.         // Cache for use in FixedUpdate
13.         _lastServerState = serverState;
14.         _lastServerMessageArrivalTime = _gameClock.Time;
15.     }
16.  

The client runs the simulation locally, using player input directly. It then takes the most recently received server state, and integrates its acceleration and velocity values by however much time the client thinks it is ahead of the server. This is the best estimate we have for where the client should actually be. It then compares its current local position with this new 'correct' position, and the larger the difference the more it interpolates from its local erroneous state towards this 'correct' state.

Code (csharp):

1.  
2. private void FixedUpdate() {
3.         Vector2 input = new Vector2(Input.GetAxis("Horizontal"), Input.GetAxis("Vertical"));
4.  
5.         _player.CmdMove(input); /* Send input to server (I know, should not be done in FixedUpdate, but it can't hurt right?) */
6.  
7.         Correct();
8.         Simulate(input);
9.     }
10.  
11.     private void Simulate(Vector2 input) {
12.         Vector3 force = new Vector3(input.x, 0f, input.y) * 10f;
13.         _rigidbody.AddForce(force, ForceMode.Acceleration);
14.     }
15.  
16.     private void Correct() {
17.         if (!_lastServerState) {
18.             return;
19.         }
20.  
21.         float timeSinceLastServerState = _gameClockFixed.Time - _lastServerMessageArrivalTime;
22.         float simulationTime = timeSinceLastServerState + RamjetClient.Latency;
23.  
24.         _correctedState = RigidbodyState.ResimulateFromPastState(serverStateNewest.Value, simulationTime);
25.  
26.         const float maxError = 2f;
27.         float error = Mathf.Clamp01(Vector3.Distance(_rigidbody.position, _correctedState.Position) / maxError);
28.  
29.         RigidbodyState currentState = RigidbodyState.ToImmutableState(_rigidbody);
30.         RigidbodyState newState = RigidbodyState.Lerp(currentState, _correctedState, error * _gameClock.DeltaTime *  _correctionSpeed);
31.  
32.         RigidbodyState.ApplyState(_rigidbody, newState);
33.     }
34.  

Testing this, we find that the corrected position exhibits considerable amounts of jitter, presumably due to inaccurate values for _simulationTime. It seems to either simulate too little or too much, and regularly jitters between those cases.

Specifically, for a player with constant velocity and zero acceleration, we see this:



These are server and client running in the same scene, with simulated latency. Blue cube is server, white cube is client. Darker blue cube and the blue dots are server positions received by the client.

Possible source for this jitter, we think:

• Inaccurate estimation of latency (fixed by using filtering/statistical analysis)
• Inaccurate estimation of acceleration (fixed by differentiating differently, for less numerical error, perhaps with hermite splines as well)
• Possible inaccuracy caused by varying numbers of FixedUpdates between serialization.

We do not experience this problem when we don't use PhysX rigidbodies and run our own little simulation in Update, leading us to believe this problem could have something to do with the variable amount of FixedUpdate calls in-between Serialize calls on the client and server. Depending on the duration of the last render frame, FixedUpdate will be called a variable number of times to keep physics time roughly in sync with realtime. So sometimes the server will have simulated, say, 4 FixedUpdates in between Serialize calls, and sometimes 6.

This leads me to believe simulationTime = timeSinceLastServerState + RamjetClient.Latency; is not enough, as it does not take into account this other source of potential jitter. But for the life of my I cannot come up with a reliable way to track/calculate this jitter.

Oh, that RamjetClient.latency value is updated continuously by the client sending Ping messages and measuring the local time until it receives a Pong message.

TL;DR: Has anyone else here done prediction/dead-reckoning for physics simulations? I'd love to hear how you did it! I'll throw in a free Steam key to anyone that contributes something useful.

Need more info? Ask! I wasn't sure what to include and what to leave out. I'm half-way hoping someone will go "Fool, you're doing it all wrong!" and point out an easy flaw. If not, I'll post the source for this demo project.

 

From memory, but i think the fixedUpdate is the issue. If i remember, folks had collected the number of fixedUpdates and sent those along, so the server/client could var the amount of force applied in cases where there were out of sync.