Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

objects owned by other players twitch when moving in the same direction as the player

Discussion in 'Multiplayer' started by Temaran, Feb 25, 2012.

  1. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    Hello everyone!

    I've been trying to solve a problem regarding to a multiplayer test I've been working on. The test is so far a variation of the unity sample networking project, but I've stripped it of almost everything. Each player connecting to the server instantiates their own prefab, which contains:
    A player controller script which hooks the Update event where it reads the input axes and updates its position with a simple:
    Code (csharp):
    1.  
    2. var xMovement = Input.GetAxis("Horizontal") * Vector3.right;
    3. var zMovement = Input.GetAxis("Vertical") * Vector3.forward;
    4. Velocity = Vector3.Normalize(xMovement + zMovement) * MovementSpeed;
    5. transform.position += Velocity * Time.deltaTime;
    6.  
    A gui script that renders a mouse pointer.
    The player network init script from the sample.
    A weapon script that listens to fire1.
    A character controller, but as there is nothing else in the scene, I'm guessing this is irrelevant.
    The NetworkInterpolatedTransform script.
    And a network view observing the NetworkInterpolatedTransform script.

    The only thing in the scene at start is the main camera with the following simple script:
    Code (csharp):
    1.  
    2. public class CameraController : MonoBehaviour
    3. {
    4.     public Transform Target;
    5.     public float RelativeHeight = 200.0f;
    6.     public float ZDistance = 5.0f;
    7.     public float DampSpeed = 0.005f;
    8.  
    9.     void LateUpdate()
    10.     {
    11.         if(Target == null)
    12.             return;
    13.  
    14.         var newPos = Target.position + new Vector3(0, RelativeHeight, -ZDistance);
    15.         transform.position = Vector3.Lerp(transform.position, newPos, DampSpeed);
    16.     }
    17.  
    18.     public void SetTarget (Transform t)
    19.     {
    20.         Target = t;
    21.     }
    22. }
    23.  


    My debugging flow is as follows:
    Player1 starts a server
    Player2 connects

    Now everything works just as expected except if the two players are moving in the same direction.
    In this case, the remote player's gameobject seems to be twitching back and forth in the same axis as the two objects are moving.

    I'm guessing it's not the network code or interpolation that is at fault since everything runs smoothly if the observing player is standing still or the two players are not moving in the same direction.
    It's also probably not the movement code as it's simple enough.

    The two theories I've come up with is that either that the update functions are not being executed in the same order each game pass. That would probably give rise to this kind of error. I have however tried to use most combinations of FixedUpdate, Update and LateUpdate to try and force them into the correct order, as well as setting the custom execution order of the scripts in the editor.

    My other theory is that the execution of the write part of the OnSerializeNetworkView event on the network script is not being executed in regular intervals.This might give rise to different result velocities on the receiving side as a side effect of the interpolation.

    I've spend quite a lot of time trying to figure this out on my own, but no luck yet.
    Does anyone recognize this, or has an idea of what might be wrong?
     
  2. appels

    appels

    Joined:
    Jun 25, 2010
    Posts:
    2,687
    Hard to tell, what do you mean with 'twitching back' ?
    It sounds like there might be a problem with the interpolation code.
    If you could put a webplayer online we could have a look.
     
  3. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    I did some additional tests, and replacing the entire interpolation code with an instruction to simply move the object linearly in some direction removes the twitch. With twitching I mean it moves back and forth several times a second, only a few pixels though, but enough to be noticable.

    The interpolation code is the original code from the sample:

    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. public class NetworkInterpolatedTransform : MonoBehaviour {
    6.    
    7.     public double interpolationBackTime = 0.1;
    8.    
    9.     internal struct  State
    10.     {
    11.         internal double timestamp;
    12.         internal Vector3 pos;
    13.         internal Quaternion rot;
    14.     }
    15.  
    16.     // We store twenty states with "playback" information
    17.     State[] m_BufferedState = new State[20];
    18.     // Keep track of what slots are used
    19.     int m_TimestampCount;
    20.    
    21.     void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info)
    22.     {
    23.         // Always send transform (depending on reliability of the network view)
    24.         if (stream.isWriting)
    25.         {
    26.             Vector3 pos = transform.localPosition;
    27.             Quaternion rot = transform.localRotation;
    28.             stream.Serialize(ref pos);
    29.             stream.Serialize(ref rot);
    30.         }
    31.         // When receiving, buffer the information
    32.         else
    33.         {
    34.             // Receive latest state information
    35.             Vector3 pos = Vector3.zero;
    36.             Quaternion rot = Quaternion.identity;
    37.             stream.Serialize(ref pos);
    38.             stream.Serialize(ref rot);
    39.            
    40.             // Shift buffer contents, oldest data erased, 18 becomes 19, ... , 0 becomes 1
    41.             for (int i=m_BufferedState.Length-1;i>=1;i--)
    42.             {
    43.                 m_BufferedState[i] = m_BufferedState[i-1];
    44.             }
    45.            
    46.             // Save currect received state as 0 in the buffer, safe to overwrite after shifting
    47.             State state;
    48.             state.timestamp = info.timestamp;
    49.             state.pos = pos;
    50.             state.rot = rot;
    51.             m_BufferedState[0] = state;
    52.            
    53.             // Increment state count but never exceed buffer size
    54.             m_TimestampCount = Mathf.Min(m_TimestampCount + 1, m_BufferedState.Length);
    55.  
    56.             // Check integrity, lowest numbered state in the buffer is newest and so on
    57.             for (int i=0;i<m_TimestampCount-1;i++)
    58.             {
    59.                 if (m_BufferedState[i].timestamp < m_BufferedState[i+1].timestamp)
    60.                     Debug.Log("State inconsistent");
    61.             }
    62.            
    63.             //Debug.Log("stamp: " + info.timestamp + "my time: " + Network.time + "delta: " + (Network.time - info.timestamp));
    64.         }
    65.     }
    66.    
    67.     // This only runs where the component is enabled, which is only on remote peers (server/clients)
    68.     void Update () {
    69.         double currentTime = Network.time;
    70.         double interpolationTime = currentTime - interpolationBackTime;
    71.         // We have a window of interpolationBackTime where we basically play
    72.         // By having interpolationBackTime the average ping, you will usually use interpolation.
    73.         // And only if no more data arrives we will use extrapolation
    74.        
    75.         // Use interpolation
    76.         // Check if latest state exceeds interpolation time, if this is the case then
    77.         // it is too old and extrapolation should be used
    78.         if (m_BufferedState[0].timestamp > interpolationTime)
    79.         {
    80.             for (int i=0;i<m_TimestampCount;i++)
    81.             {
    82.                 // Find the state which matches the interpolation time (time+0.1) or use last state
    83.                 if (m_BufferedState[i].timestamp <= interpolationTime || i == m_TimestampCount-1)
    84.                 {
    85.                     // The state one slot newer (<100ms) than the best playback state
    86.                     State rhs = m_BufferedState[Mathf.Max(i-1, 0)];
    87.                     // The best playback state (closest to 100 ms old (default time))
    88.                     State lhs = m_BufferedState[i];
    89.                    
    90.                     // Use the time between the two slots to determine if interpolation is necessary
    91.                     double length = rhs.timestamp - lhs.timestamp;
    92.                     float t = 0.0F;
    93.                     // As the time difference gets closer to 100 ms t gets closer to 1 in
    94.                     // which case rhs is only used
    95.                     if (length > 0.0001)
    96.                         t = (float)((interpolationTime - lhs.timestamp) / length);
    97.                    
    98.                     // if t=0 => lhs is used directly
    99.                     transform.localPosition = Vector3.Lerp(lhs.pos, rhs.pos, t);
    100.                     transform.localRotation = Quaternion.Slerp(lhs.rot, rhs.rot, t);
    101.                     return;
    102.                 }
    103.             }
    104.         }
    105.         // Use extrapolation. Here we do something really simple and just repeat the last
    106.         // received state. You can do clever stuff with predicting what should happen.
    107.         else
    108.         {
    109.             State latest = m_BufferedState[0];
    110.            
    111.             transform.localPosition = latest.pos;
    112.             transform.localRotation = latest.rot;
    113.         }
    114.     }
    115. }
    116.  
    I'll try to get a webplayer up tomorrow, but as the twitch only appears when more than one player in the same game is moving in the same direction, you might have to connect twice using two different browsers or such.
     
  4. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Twitching could be caused by packets arriving out of order and so on. Have you compensated for that?
     
  5. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    The place to look would be multiplayer vehicle racing games. I know I've read about the problem in those games since you are going the same direction as all the other cars. Unfortunately I can't find the article I read for how they solved it, but I'm sure if you research those you'll be able to find a discussion of how they solved it.

    However, as a guess, since you're only interpolating position there, you may want to interpolate velocity (both linear and angular) as well.
     
  6. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    I don't have any compensation for the out of order problem, but I do have a check that writes to the log if states with out-of-order timestamps were to be added to the buffer, and I haven't seen it trigger so far, so I do not think it is the out of order problem.

    That multiplayer vehicle racing article would be awesome! Right now I'm only synching the transform, not the rigidbody, and extrapolation doesn't really happen at the moment (checked with logging), so I don't think sending the velocities would help. I have tried adding rigidbodies and synching those with the default script too (the one for rigidbodies), but I still get twitching.

    I'll try to get a web player working later today, having it would help illustrate the problem better :)
     
  7. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    Extrapolation may be happening implicitly, by default. If you have a rigidbody attached to the local copy of the remote object, then during local updates it'll do the usual simulation converting acceleration to velocity, velocity to position. It'll also apply drag and gravity, if those are set.

    Of course, if you are careful to remove all that simulation on the local side, then that's definitely not your issue.

    Also you can check this one and tell me if you see the same twitch: http://www.tosos.com/Longshot/Longshot-AI-pretest.html

    Maybe I've just convinced myself it doesn't :)
     
    Last edited: Feb 26, 2012
  8. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    Very cool game duhprey!

    And no, I don't experience twitching like the one I'm talking about when I run your game.

    I finally took the time to host the test-build in a webplayer on:
    temaranland.no-ip.org

    Connect with two players and then move in the same direction. For me the twitching occurs maybe once every second, but it can be fairly random. This definately suggests some type of interpolation error, but when I do debug prints and inspect the velocity of the ships, they seem constant. So that's why I started investigating the camera as well.

    Please let me know if you have any ideas :)

    /Temaran
     
    Last edited: Feb 27, 2012
  9. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    Edit: Nope... now I'm seeing it all the time on mine :) I guess I really do need to find that article.

    Well if it helps, here's my network controller. Notice I'm interpolating the velocities and the accelerations (forward, strafe and rotate).

    Code (csharp):
    1. public class NetworkController : MonoBehaviour {
    2.    
    3.     public double m_InterpolationBackTime = 0.1;
    4.     public double m_ExtrapolationLimit = 0.5;
    5.    
    6.     internal struct  State
    7.     {
    8.         internal double timestamp;
    9.         internal Vector3 pos;
    10.         internal Vector3 velocity;
    11.         internal Quaternion rot;
    12.         internal Vector3 angularVelocity;
    13.         internal float forward;
    14.         internal float strafe;
    15.         internal float rotate;
    16.     }
    17.    
    18.     // We store twenty states with "playback" information
    19.     State[] m_BufferedState = new State[20];
    20.     // Keep track of what slots are used
    21.     int m_TimestampCount;
    22.    
    23.     void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info)
    24.     {
    25.         BasicMovement sm = GetComponent<BasicMovement>();  
    26.  
    27.         // Send data to server
    28.         if (stream.isWriting)
    29.         {
    30.             Vector3 pos = transform.position;
    31.             Quaternion rot = transform.rotation;
    32.             Vector3 velocity = rigidbody.velocity;
    33.             Vector3 angularVelocity = rigidbody.angularVelocity;
    34.             float forward = sm.forward;
    35.             float strafe = sm.strafe;
    36.             float rotate = sm.rotate;
    37.  
    38.             stream.Serialize(ref pos);
    39.             stream.Serialize(ref velocity);
    40.             stream.Serialize(ref rot);
    41.             stream.Serialize(ref angularVelocity);
    42.             stream.Serialize(ref forward);
    43.             stream.Serialize(ref strafe);
    44.             stream.Serialize(ref rotate);
    45.         }
    46.         // Read data from remote client
    47.         else
    48.         {
    49.             Vector3 pos = Vector3.zero;
    50.             Vector3 velocity = Vector3.zero;
    51.             Quaternion rot = Quaternion.identity;
    52.             Vector3 angularVelocity = Vector3.zero;
    53.             float forward = 0f;
    54.             float strafe = 0f;
    55.             float rotate = 0f;
    56.             stream.Serialize(ref pos);
    57.             stream.Serialize(ref velocity);
    58.             stream.Serialize(ref rot);
    59.             stream.Serialize(ref angularVelocity);
    60.             stream.Serialize(ref forward);
    61.             stream.Serialize(ref strafe);
    62.             stream.Serialize(ref rotate);
    63.            
    64.             // Shift the buffer sideways, deleting state 20
    65.             for (int i=m_BufferedState.Length-1;i>=1;i--)
    66.             {
    67.                 m_BufferedState[i] = m_BufferedState[i-1];
    68.             }
    69.            
    70.             // Record current state in slot 0
    71.             State state;
    72.             state.timestamp = info.timestamp;
    73.             state.pos = pos;
    74.             state.velocity = velocity;
    75.             state.rot = rot;
    76.             state.angularVelocity = angularVelocity;
    77.             state.forward = forward;
    78.             state.strafe = strafe;
    79.             state.rotate = rotate;
    80.             m_BufferedState[0] = state;
    81.            
    82.             // Update used slot count, however never exceed the buffer size
    83.             // Slots aren't actually freed so this just makes sure the buffer is
    84.             // filled up and that uninitalized slots aren't used.
    85.             m_TimestampCount = Mathf.Min(m_TimestampCount + 1, m_BufferedState.Length);
    86.  
    87.             // Check if states are in order, if it is inconsistent you could reshuffel or
    88.             // drop the out-of-order state. Nothing is done here
    89.             for (int i=0;i<m_TimestampCount-1;i++)
    90.             {
    91.                 if (m_BufferedState[i].timestamp < m_BufferedState[i+1].timestamp)
    92.                     Debug.Log("State inconsistent");
    93.             }  
    94.         }
    95.     }
    96.    
    97.     // We have a window of interpolationBackTime where we basically play
    98.     // By having interpolationBackTime the average ping, you will usually use interpolation.
    99.     // And only if no more data arrives we will use extra polation
    100.     void Update () {
    101.         // This is the target playback time of the rigid body
    102.         double interpolationTime = Network.time - m_InterpolationBackTime;
    103.        
    104.         // Use interpolation if the target playback time is present in the buffer
    105.         if (m_BufferedState[0].timestamp > interpolationTime)
    106.         {
    107.             // Go through buffer and find correct state to play back
    108.             for (int i=0;i<m_TimestampCount;i++)
    109.             {
    110.                 if (m_BufferedState[i].timestamp <= interpolationTime || i == m_TimestampCount-1)
    111.                 {
    112.                     // The state one slot newer (<100ms) than the best playback state
    113.                     State rhs = m_BufferedState[Mathf.Max(i-1, 0)];
    114.                     // The best playback state (closest to 100 ms old (default time))
    115.                     State lhs = m_BufferedState[i];
    116.  
    117.                     // Use the time between the two slots to determine if interpolation is necessary
    118.                     double length = rhs.timestamp - lhs.timestamp;
    119.                     float t = 0.0F;
    120.                     // As the time difference gets closer to 100 ms t gets closer to 1 in
    121.                     // which case rhs is only used
    122.                     // Example:
    123.                     // Time is 10.000, so sampleTime is 9.900
    124.                     // lhs.time is 9.910 rhs.time is 9.980 length is 0.070
    125.                     // t is 9.900 - 9.910 / 0.070 = 0.14. So it uses 14% of rhs, 86% of lhs
    126.                     if (length > 0.0001){
    127.                         t = (float)((interpolationTime - lhs.timestamp) / length);
    128.                     }
    129.                     //  Debug.Log(t);
    130.                     // if t=0 => lhs is used directly
    131.                     transform.localPosition = Vector3.Lerp(lhs.pos, rhs.pos, t);
    132.                     transform.localRotation = Quaternion.Slerp(lhs.rot, rhs.rot, t);
    133.                     rigidbody.velocity = Vector3.Lerp (lhs.velocity, rhs.velocity, t);
    134.                     rigidbody.angularVelocity = Vector3.Lerp (lhs.angularVelocity, rhs.angularVelocity, t);
    135.                     BasicMovement sm = GetComponent<BasicMovement>();  
    136.                     sm.forward = Mathf.Lerp (lhs.forward, rhs.forward, t);
    137.                     sm.strafe = Mathf.Lerp (lhs.strafe, rhs.strafe, t);
    138.                     sm.rotate = Mathf.Lerp (lhs.rotate, rhs.rotate, t);
    139.                     return;
    140.                 }
    141.             }
    142.         }
    143.         // Use extrapolation
    144.         else
    145.         {
    146.             State latest = m_BufferedState[0];
    147.            
    148.             float extrapolationLength = (float)(interpolationTime - latest.timestamp);
    149.             // Don't extrapolation for more than 500 ms, you would need to do that carefully
    150.             if (extrapolationLength < m_ExtrapolationLimit)
    151.             {
    152.                 float axisLength = extrapolationLength * latest.angularVelocity.magnitude * Mathf.Rad2Deg;
    153.                 Quaternion angularRotation = Quaternion.AngleAxis(axisLength, latest.angularVelocity);
    154.                
    155.                 transform.position = latest.pos + latest.velocity * extrapolationLength;
    156.                 transform.rotation = angularRotation * latest.rot;
    157.                 rigidbody.velocity = latest.velocity;
    158.                 rigidbody.angularVelocity = latest.angularVelocity;
    159.                 BasicMovement sm = GetComponent<BasicMovement>();  
    160.                 sm.forward = latest.forward;
    161.                 sm.strafe = latest.strafe;
    162.                 sm.rotate = latest.rotate;
    163.             }
    164.         }
    165.     }
    166. }
     
    Last edited: Feb 27, 2012
  10. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    Yes, but as I don't use a rigidbody for my objects right now, I don't have access to physics data such as velocity and the like. But then again, maybe that is the problem.

    What if the OnSerializeNetwork view doesn't get called on the same frequency as the other calls? If that were the case, then it would be entirely possible for two ticks of OnSerializeNetworkView to serialize the same position data, and that could give rise to the twitching. I've so far assumed that it ran on the same timer as the physics engine did, but there is of course no guarantee for that.

    I'll do some tests later today and see if that might be it. Does anyone happen to know the frequency of that event?
     
  11. appels

    appels

    Joined:
    Jun 25, 2010
    Posts:
    2,687
    I think that it gets triggerd by the networkview send rate, so the value you have in the editor. I don't recall the default, maybe 15ms.
     
  12. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    I did find this article: http://web.cs.wpi.edu/~claypool/courses/4513-B03/papers/games/wolf.pdf but it defines the problem more than the solution. The problem is network jitter with constant velocity. Appels is right about the sendrate, but then delays in the network make some packets come in sooner and some later. They all come in around the average ping time. So on serialize is caled somewhat randomly. Its update() though is called the same rate as everything else. But might be better if it were fixed update. At least for me....

    The solution should be to smooth over the jitter, but I can't figure out where that should be.
     
  13. George Foot

    George Foot

    Joined:
    Feb 22, 2012
    Posts:
    399
    Where does your player and camera update code live? If it's driven by rigid bodies, then it's just after FixedUpdate. But FixedUpdate doesn't run on every frame, and if you execute your network interpolation/extrapolation code every frame in Update then, on frames when FixedUpdate doesn't run, you'll be moving your foreign player objects but not moving your local player object and camera.

    Try setting a bool in FixedUpdate to control whether you do execute the network code in your Update function, or skip it for this frame. Just a hunch! It shouldn't matter if multiple FixedUpdates run, as your network code doesn't look like it needs to run once for each.
     
  14. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    Thanks, George. I think that is a big part of my problem... the other part of the problem is that I'm overwriting the local rigidbody results entirely. ... I should be lerping those together with those computed in my update loop (and switch the update loop to fixed rate)

    I think though Temaran's original problem is that linear interpolation is discontinuous at the interpolation points (that is traveling forward in the packet stream) I think when you are both traveling at constant velocity you need a higher order interpolator like cubic or catmull-rom to smooth out the crossover. I will do some experiments with my stuff tonight (8 hours from now for me) and post the code if it works. Of course, if someone else wants to post some code ill be happy to use it :)
     
  15. lockbox

    lockbox

    Joined:
    Feb 10, 2012
    Posts:
    519
    Very interesting issue. Have you guys ever tried plotting the raw data positions in 3D space with the time stamp and a sequence number - and done the same for the interpolated positions, and then visually compared the two data sets?
     
  16. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    lockbox, I just started that :) Here's an image of the delta's between the packets.

    $samples.png

    Here's the image of the induced velocity (delta position / delta timestamp between packets)

    $velocity.png

    I tried doing cubic interpolation and it's smoother in the shifting when moving but I'm guessing my problem is still that because those deltas aren't a regular sample the cubic is failing... so I need to use non-uniform sampling. Here's the code for the regular sampling:

    Code (csharp):
    1. public class NetworkController : MonoBehaviour {
    2.    
    3.     public double m_InterpolationBackTime = 0.1;
    4.     public double m_ExtrapolationLimit = 0.5;
    5.    
    6.     internal struct  State
    7.     {
    8.         internal double timestamp;
    9.         internal Vector3 pos;
    10.         internal Vector3 velocity;
    11.         internal Quaternion rot;
    12.         internal Vector3 angularVelocity;
    13.         internal float forward;
    14.         internal float strafe;
    15.         internal float rotate;
    16.     }
    17.    
    18.     // We store twenty states with "playback" information
    19.     State[] m_BufferedState = new State[20];
    20.     // Keep track of what slots are used
    21.     int m_TimestampCount;
    22.    
    23.     void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info)
    24.     {
    25.         BasicMovement sm = GetComponent<BasicMovement>();  
    26.  
    27.         // Send data to server
    28.         if (stream.isWriting)
    29.         {
    30.             Vector3 pos = transform.position;
    31.             Quaternion rot = transform.rotation;
    32.             Vector3 velocity = rigidbody.velocity;
    33.             Vector3 angularVelocity = rigidbody.angularVelocity;
    34.             float forward = sm.forward;
    35.             float strafe = sm.strafe;
    36.             float rotate = sm.rotate;
    37.  
    38.             stream.Serialize(ref pos);
    39.             stream.Serialize(ref velocity);
    40.             stream.Serialize(ref rot);
    41.             stream.Serialize(ref angularVelocity);
    42.             stream.Serialize(ref forward);
    43.             stream.Serialize(ref strafe);
    44.             stream.Serialize(ref rotate);
    45.         }
    46.         // Read data from remote client
    47.         else
    48.         {
    49.             Vector3 pos = Vector3.zero;
    50.             Vector3 velocity = Vector3.zero;
    51.             Quaternion rot = Quaternion.identity;
    52.             Vector3 angularVelocity = Vector3.zero;
    53.             float forward = 0f;
    54.             float strafe = 0f;
    55.             float rotate = 0f;
    56.             stream.Serialize(ref pos);
    57.             stream.Serialize(ref velocity);
    58.             stream.Serialize(ref rot);
    59.             stream.Serialize(ref angularVelocity);
    60.             stream.Serialize(ref forward);
    61.             stream.Serialize(ref strafe);
    62.             stream.Serialize(ref rotate);
    63.            
    64.             // Shift the buffer sideways, deleting state 20
    65.             for (int i=m_BufferedState.Length-1;i>=1;i--)
    66.             {
    67.                 m_BufferedState[i] = m_BufferedState[i-1];
    68.             }
    69.            
    70.             // Record current state in slot 0
    71.             State state;
    72.             state.timestamp = info.timestamp;
    73.             state.pos = pos;
    74.             state.velocity = velocity;
    75.             state.rot = rot;
    76.             state.angularVelocity = angularVelocity;
    77.             state.forward = forward;
    78.             state.strafe = strafe;
    79.             state.rotate = rotate;
    80.             m_BufferedState[0] = state;
    81.  
    82.             Debug.Log ("State " + info.timestamp + " " + pos + " " + velocity);
    83.            
    84.             // Update used slot count, however never exceed the buffer size
    85.             // Slots aren't actually freed so this just makes sure the buffer is
    86.             // filled up and that uninitalized slots aren't used.
    87.             m_TimestampCount = Mathf.Min(m_TimestampCount + 1, m_BufferedState.Length);
    88.  
    89.             // Check if states are in order, if it is inconsistent you could reshuffel or
    90.             // drop the out-of-order state. Nothing is done here
    91.             for (int i=0;i<m_TimestampCount-1;i++)
    92.             {
    93.                 if (m_BufferedState[i].timestamp < m_BufferedState[i+1].timestamp)
    94.                     Debug.Log("State inconsistent");
    95.             }  
    96.         }
    97.     }
    98.    
    99.     // We have a window of interpolationBackTime where we basically play
    100.     // By having interpolationBackTime the average ping, you will usually use interpolation.
    101.     // And only if no more data arrives we will use extra polation
    102.     void FixedUpdate () {
    103.         // This is the target playback time of the rigid body
    104.         double interpolationTime = Network.time - m_InterpolationBackTime;
    105.        
    106.         // Use interpolation if the target playback time is present in the buffer
    107.         if (m_BufferedState[0].timestamp > interpolationTime)
    108.         {
    109.             // Go through buffer and find correct state to play back
    110.             for (int i=1;i<m_TimestampCount;i++)
    111.             {
    112.                 if (m_BufferedState[i].timestamp <= interpolationTime || i == m_TimestampCount-1)
    113.                 {
    114.                     State rhs = m_BufferedState[i-1];
    115.                     // The best playback state (closest to 100 ms old (default time))
    116.                     State lhs = m_BufferedState[i];
    117.                    
    118.                     Vector3 y0;
    119.                     Vector3 y1 = lhs.pos;
    120.                     Vector3 y2 = rhs.pos;
    121.                     Vector3 y3;
    122.  
    123.                     Vector3 v0;
    124.                     Vector3 v1 = lhs.velocity;
    125.                     Vector3 v2 = rhs.velocity;
    126.                     Vector3 v3;
    127.  
    128.                     float length = (float)(rhs.timestamp - lhs.timestamp);
    129.                     if (i == m_TimestampCount-1) {
    130.                         Vector3 a = new Vector3 (lhs.strafe, 0, lhs.forward) / rigidbody.mass;
    131.                         v0 = lhs.velocity - a * length;
    132.                         y0 = lhs.pos - v0 * length;
    133.                     } else {
    134.                         v0 = m_BufferedState[i+1].velocity;
    135.                         y0 = m_BufferedState[i+1].pos;
    136.                     }
    137.  
    138.                     if (i == 1) {
    139.                         Vector3 a = new Vector3 (rhs.strafe, 0, rhs.forward) / rigidbody.mass;
    140.                         v3 = rhs.velocity + a * length;
    141.                         y3 = rhs.pos + rhs.velocity * length;
    142.                     } else {
    143.                         v3 = m_BufferedState[i-2].velocity;
    144.                         y3 = m_BufferedState[i-2].pos;
    145.                     }
    146.  
    147.                     // Use the time between the two slots to determine if interpolation is necessary
    148.                     float t = 0.0F;
    149.                     // As the time difference gets closer to 100 ms t gets closer to 1 in
    150.                     // which case rhs is only used
    151.                     // Example:
    152.                     // Time is 10.000, so sampleTime is 9.900
    153.                     // lhs.time is 9.910 rhs.time is 9.980 length is 0.070
    154.                     // t is 9.900 - 9.910 / 0.070 = 0.14. So it uses 14% of rhs, 86% of lhs
    155.                     if (length > 0.0001){
    156.                         t = (float)((interpolationTime - lhs.timestamp) / length);
    157.                     }
    158.                     //  Debug.Log(t);
    159.                     // if t=0 => lhs is used directly
    160.                     transform.localPosition = Vector3.Lerp (transform.localPosition,
    161.                                                 Vector3CubicInterpolate (y0, y1, y2, y3, t), 0.5f);
    162.                     transform.localRotation = Quaternion.Slerp (transform.localRotation,
    163.                                                 Quaternion.Slerp(lhs.rot, rhs.rot, t), 0.5f);
    164.                     rigidbody.velocity = Vector3.Lerp (rigidbody.velocity,
    165.                                                 Vector3CubicInterpolate (v0, v1, v2, v3, t), 0.5f);
    166.                     rigidbody.angularVelocity = Vector3.Lerp (rigidbody.angularVelocity,
    167.                                                 Vector3.Lerp (lhs.angularVelocity, rhs.angularVelocity, t), 0.5f);
    168.                     BasicMovement sm = GetComponent<BasicMovement>();  
    169.                     sm.forward = Mathf.Lerp (lhs.forward, rhs.forward, t);
    170.                     sm.strafe = Mathf.Lerp (lhs.strafe, rhs.strafe, t);
    171.                     sm.rotate = Mathf.Lerp (lhs.rotate, rhs.rotate, t);
    172.                     return;
    173.                 }
    174.             }
    175.         }
    176. /*  Don't extrapolate here, just use the physics and BasicMovement to do it
    177.         // Use extrapolation
    178.         else
    179.         {
    180.             State latest = m_BufferedState[0];
    181.            
    182.             float extrapolationLength = (float)(interpolationTime - latest.timestamp);
    183.             // Don't extrapolation for more than 500 ms, you would need to do that carefully
    184.             if (extrapolationLength < m_ExtrapolationLimit)
    185.             {
    186.                 float axisLength = extrapolationLength * latest.angularVelocity.magnitude * Mathf.Rad2Deg;
    187.                 Quaternion angularRotation = Quaternion.AngleAxis(axisLength, latest.angularVelocity);
    188.                
    189.                 transform.position = latest.pos + latest.velocity * extrapolationLength;
    190.                 transform.rotation = angularRotation * latest.rot;
    191.                 rigidbody.velocity = latest.velocity;
    192.                 rigidbody.angularVelocity = latest.angularVelocity;
    193.                 BasicMovement sm = GetComponent<BasicMovement>();  
    194.                 sm.forward = latest.forward;
    195.                 sm.strafe = latest.strafe;
    196.                 sm.rotate = latest.rotate;
    197.             }
    198.         }
    199. */
    200.     }
    201.  
    202.     Vector3 Vector3CubicInterpolate(
    203.         Vector3 y0,Vector3 y1,
    204.         Vector3 y2,Vector3 y3,
    205.         float mu)
    206.     {
    207.         Vector3 a0,a1,a2,a3;
    208.         float mu2;
    209.  
    210.         mu2 = mu*mu;
    211.         a0 = -0.5f*y0 + 1.5f*y1 - 1.5f*y2 + 0.5f*y3;
    212.         a1 = y0 - 2.5f*y1 + 2*y2 - 0.5f*y3;
    213.         a2 = -0.5f*y0 + 0.5f*y2;
    214.         a3 = y1;
    215.  
    216.         // a0 = y3 - y2 - y0 + y1;
    217.         // a1 = y0 - y1 - a0;
    218.         // a2 = y2 - y0;
    219.         // a3 = y1;
    220.  
    221.         return (a0*mu*mu2+a1*mu2+a2*mu+a3);
    222.     }
    223. }
    Onward, non-uniform sampling! :) Talk about taking the blue pill...
     
    Last edited: Feb 28, 2012
  17. lockbox

    lockbox

    Joined:
    Feb 10, 2012
    Posts:
    519
    Just thinking outloud here...

    - It's just demo code. It's not like this code has been optimized for high speed moving objects operating in competitive gaming environment.

    - The comments do suggest implementing prediction, which may ulitmately be the correct solution to this problem. IMO, it would be more important to make corrections based on a deviation threshold than to try and get it 99.9% correct with interpolation. But this is just a theory that I need to explore.

    - In order to properly debug this code, you need a replay system, so that the same data can be processed a million times if need be - and then scale it up in number of players to see if the problem goes away. It will be interesting to see what happens with 8-10 players.

    - I'm starting to not like this implementation. I've written protocol analyzers that process things on a pretty large ring buffer and this "shift the buffer sideways" is not so good. The other thing too, is that the calls to OnSerializeNetworkView and Update are not asynchronous.

    - What I would almost prefer here, is to number the packets and insert them directly into a large array (accounting for virtual buffering) instead of using the timestamp, and then restarting the numbering sequence based on data rate, flipping and checking a bit to ensure it is part of the current datastream. This would account for both out of sequence, delayed and dropped packets. Would it be smoother? Hard to say without testing. Prediction may be the lesser of two evils. lol
     
    Last edited: Feb 28, 2012
  18. lockbox

    lockbox

    Joined:
    Feb 10, 2012
    Posts:
    519
  19. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    Here's the solution that finally appears to remove all the jitter for me (turns out it is similar to the code lockbox posted last):


    Code (csharp):
    1.                     transform.localPosition = Vector3.Lerp (transform.localPosition,
    2.                                                             Vector3.Lerp (lhs.pos, rhs.pos, t),
    3.                                                             interpolationConstant);
    4.                     transform.localRotation = Quaternion.Slerp (transform.localRotation,
    5.                                                                 Quaternion.Slerp(lhs.rot, rhs.rot, t),
    6.                                                                 interpolationConstant);
    7.                     rigidbody.velocity = Vector3.Lerp (rigidbody.velocity,
    8.                                                        Vector3.Lerp (lhs.velocity, rhs.velocity, t),
    9.                                                        interpolationConstant);
    10.                     rigidbody.angularVelocity = Vector3.Lerp (rigidbody.angularVelocity,
    11.                                                               Vector3.Lerp (lhs.angularVelocity, rhs.angularVelocity, t),
    12.                                                               interpolationConstant);
    I tried bsplines, rolling averages, rolling weighted averages... nothing really appeared to squash the noise. But once I got my local proxy simulation to match the remote's simulation, I could get away with using an interpolationConstant of .1 in the above code. That has killed all the noise because the .1 squashes large swings in the incoming packets.

    It should be possible to do with yours, Temaran, just send an RPC with the keys pressed on the remote and then run the "Move" updates on the local object as well as the remote. Then just use the double Lerp like I have above (only for you just Position and Rotation, not the rigidbody parts) and that should squash out the noise. If you want to PM me with segments of your code, I can give you specifics of what I mean if you want to try it.
     
  20. Temaran

    Temaran

    Joined:
    Feb 25, 2012
    Posts:
    6
    Wow!

    I've been renovating and haven't been able to use my computer for a few days, so much new good stuff!

    Appels:
    I have tried tweaking the sendrate, it unfortunately does not affect the problem.

    George foot:
    I don't have rigidbodies in the code right now, so the camera is purely driven by the targets position, which is updated in the Update() pass, and the camera itself updates in the LateUpdate() pass. So at this moment nothing lives in FixedUpdate (I thought about the same thing), good eye though! :D

    Lockbox:
    A lot of cool insights here, I'll look at the article when I start testing again tomorrow.

    Duhprey!
    Awesome man, I'll try the fixed interpolation method, this definitely seems like a likely culprit, as the interpolation done in the network script is done in a different way compared to the camera update interpolation!

    I'll post the results of my next programming session as soon as I've done it.

    Thank you all so much :)

    /Temaran
     
  21. duhprey

    duhprey

    Joined:
    Nov 13, 2009
    Posts:
    166
    Thanks for checking it out Tomoprime. I'll see if I can get that error to repeat. Though feel free to PM me instead to keep this thread from going off topic.
     
  22. Tomo-Games

    Tomo-Games

    Joined:
    Sep 20, 2010
    Posts:
    223
    Duhprey. No worries. Consider it removed... ;)
     
  23. Quadgnim

    Quadgnim

    Joined:
    Sep 10, 2012
    Posts:
    132
    lockbox, or anyone else,

    does anyone have a c# version of this referenced wiki post? I tried to port it but I'm having issues with it behaving weird and I'm not quite sure what I did wrong. I also tried to use the javascript version but it was complaining about Mathf.abs (... double ...) not supported. I tried to typecast it but I don't think javascript handles typecast the same was as C/C++/C#?
     
  24. lockbox

    lockbox

    Joined:
    Feb 10, 2012
    Posts:
    519
    Network.time is a double. Converting it to a float is *probably* not a good idea, but Lerp only takes floats. info.timeStamp is a double too.

    I have not tested this, but you can compare it to your conversion and try it

    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. public class PlayerLocal : MonoBehaviour {
    6.  
    7.     private Vector3 p;
    8.     private Quaternion r;
    9.     private int m = 0; 
    10.    
    11.     // Use this for initialization
    12.     void Start () {
    13.         networkView.observed = this;
    14.     }
    15.    
    16.     void OnSerializeNetworkView(BitStream stream) {
    17.         p = rigidbody.position;
    18.         r = rigidbody.rotation;
    19.         m = 0;
    20.         stream.Serialize(ref p);
    21.         stream.Serialize(ref r);
    22.         stream.Serialize(ref m);
    23.     }
    24. }
    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4.  
    5. public class PlayerRemote : MonoBehaviour {
    6.  
    7.     bool simulatePhysics = true;
    8.     bool updatePosition  = true;
    9.     float physInterp = 0.1f;
    10.     float netInterp = 0.2f;
    11.     float ping ;
    12.     float jitter;
    13.     GameObject localPlayer;                 //The "Player" GameObject for which this game instance is authoritative. Used to determine if we should be calculating physics on the object this script is controlling, in case it could be colliding with this game instance's "player"
    14.     bool isResponding = false;              //Updated by the script for diagnostic feedback of the status of this NetworkView
    15.     string netCode = " (No Connection)";    //Updated by the script for diagnostic feedback of the status of this NetworkView
    16.     private float m;
    17.     private Vector3 p;
    18.     private Quaternion r;
    19.     private State[] states = new State[15];
    20.     private int stateCount;
    21.    
    22.     [System.Serializable]  
    23.     public class State  {
    24.         public Vector3 p;
    25.         public Quaternion r;
    26.         public float t;
    27.         public State(Vector3 p, Quaternion r, float t) {
    28.             this.p = p;
    29.             this.r = r;
    30.             this.t = t;
    31.         }
    32.     }      
    33.    
    34.     // Use this for initialization
    35.     void Start () {
    36.         networkView.observed = this;
    37.     }
    38.    
    39.     void FixedUpdate() {
    40.         if(!updatePosition || states[10] != null)
    41.             return;
    42.      
    43.         simulatePhysics = (localPlayer  Vector3.Distance(localPlayer.rigidbody.position, rigidbody.position) < 30);;
    44.         jitter = Mathf.Lerp(jitter, Mathf.Abs(ping - ((float)Network.time - states[0].t)), Time.deltaTime * 0.3f);
    45.         ping = Mathf.Lerp(ping, (float)Network.time - states[0].t, Time.deltaTime * 0.3f);
    46.      
    47.         rigidbody.isKinematic = !simulatePhysics;
    48.         rigidbody.interpolation = (simulatePhysics ? RigidbodyInterpolation.Interpolate : RigidbodyInterpolation.None);
    49.      
    50.         //Interpolation
    51.          float interpolationTime  = (float)Network.time - netInterp;
    52.         if (states[0].t > interpolationTime) {                                              // Target playback time should be present in the buffer
    53.             for (int i=0; i<stateCount; i++) {                                                  // Go through buffer and find correct state to play back
    54.                 if (states[i] != null  (states[i].t <= interpolationTime || i == stateCount-1)) {
    55.                     State rhs = states[Mathf.Max(i-1, 0)];                          // The state one slot newer than the best playback state
    56.                     State lhs = states[i];                                          // The best playback state (closest to .1 seconds old)
    57.                     float l = rhs.t - lhs.t;                                            // Use the time between the two slots to determine if interpolation is necessary
    58.                     float t = 0.0f;                                                 // As the time difference gets closer to 100 ms, t gets closer to 1 - in which case rhs is used
    59.                     if (l > 0.0001) t = ((interpolationTime - lhs.t) / l);                  // if t=0 => lhs is used directly
    60.                     if(simulatePhysics) {
    61.                         rigidbody.position = Vector3.Lerp(rigidbody.position, Vector3.Lerp(lhs.p, rhs.p, t), physInterp);
    62.                         rigidbody.rotation = Quaternion.Slerp(rigidbody.rotation, Quaternion.Slerp(lhs.r, rhs.r, t), physInterp);
    63.                         rigidbody.velocity = ((rhs.p - states[i + 1].p) / (rhs.t - states[i + 1].t));
    64.                     }
    65.                     else {
    66.                         rigidbody.position = Vector3.Lerp(lhs.p, rhs.p, t);
    67.                         rigidbody.rotation = Quaternion.Slerp(lhs.r, rhs.r, t);
    68.                     }
    69.                     isResponding = true;
    70.                     netCode = "";
    71.                     return;
    72.                 }
    73.             }
    74.         }
    75.      
    76.         //Extrapolation
    77.         else  {
    78.             float extrapolationLength = (interpolationTime - states[0].t);
    79.             if (extrapolationLength < 1  states[0] != null  states[1] != null) {
    80.                 if(!simulatePhysics) {
    81.                     rigidbody.position = states[0].p + (((states[0].p - states[1].p) / (states[0].t - states[1].t)) * extrapolationLength);
    82.                     rigidbody.rotation = states[0].r;
    83.                 }
    84.                 isResponding = true;
    85.                 if(extrapolationLength < .5) netCode = ">";
    86.                 else netCode = " (Delayed)";
    87.             }
    88.             else {
    89.                 netCode = " (Not Responding)";
    90.                 isResponding = false;
    91.             }
    92.         }
    93.         if (simulatePhysics  (states[0].t > states[2].t)) {
    94.             rigidbody.velocity = ((states[0].p - states[2].p) / (states[0].t - states[2].t));
    95.         }
    96.     }
    97.    
    98.     void OnSerializeNetworkView(BitStream stream, NetworkMessageInfo info) {
    99.         //We are the server, and have to keep track of relaying messages between connected clients
    100.         if(stream.isWriting) {
    101.             if(stateCount == 0)
    102.                 return;
    103.                
    104.             p = states[0].p;
    105.             r = states[0].r;
    106.             m = ((float)Network.time - states[0].t) * 1000.0f;  //m is the number of milliseconds that transpire between the packet's original send time and the time it is resent from the server to all the other clients
    107.             stream.Serialize(ref p);
    108.             stream.Serialize(ref r);
    109.             stream.Serialize(ref m);
    110.         }
    111.      
    112.         //New packet recieved - add it to the states array for interpolation!
    113.         else {
    114.             stream.Serialize(ref p);
    115.             stream.Serialize(ref r);
    116.             stream.Serialize(ref m);
    117.             State state = new State(p, r, (float)info.timestamp - (m > 0 ? (m / 1000.0f) : 0));
    118.             if(stateCount == 0) states[0] = state;
    119.             else if(state.t > states[0].t) {
    120.                 for (int k=states.Length-1;k>0;k--) states[k] = states[k-1];
    121.                 states[0] = state;
    122.             }
    123.             //else Debug.Log(gameObject.name + ": Out-of-order state received and ignored (" + ((states[0].t - state.t) * 1000) + ")" + states[0].t + "---" + state.t + "---" + m + "---" + states[0].p.x + "---" + state.p.x);
    124.             stateCount = Mathf.Min(stateCount + 1, states.Length);
    125.         }
    126.     }  
    127.    
    128. }
    129.  
    130.  
    131.  
     
    Last edited: Feb 13, 2013
  25. Quadgnim

    Quadgnim

    Joined:
    Sep 10, 2012
    Posts:
    132
    thanks lockbox. It's compiling, but now I need to debug. Getting null references at run time, but I'll figure it out.

    Maybe you can help me understand something. There are two different classes, one to attach to the player and one to attach to the avatar. Ok I get that in theory, but the code for the avatar has a writer piece. the avatar should never write, should it? Therefore, I either don't need that, or can't I modify the writer to be the same as the player only class then attach the same class to both? The only other thing would be to add a check of networkView.isMine in the FixedUpdate, and only perform if it's false. Obviously doing it with two classes is more efficient, but can be trickier in code to attach the right one dynamically, vs preattaching as part of a prefab.

    thoughts?
     
  26. lockbox

    lockbox

    Joined:
    Feb 10, 2012
    Posts:
    519
    I've been thinking of a good answer to give you, but I don't have one. It really depends on your game type, implementation and what type of issues you're having with your networking piece. There is no one right way, but many different ways that present different challenges based on your player's hardware and connectivity. This code may solve one problem, but create another.

    Another thing that I've been thinking about as well, is that perhaps it's not your net code but something else that is causing the jitter. Just something to consider. :)

    Check this out...uses similar State code - C#

    http://web.archive.org/web/20120713...pport/resources/files/MultiplayerTutorial.pdf