Applications of TCP in Real-time Games

Bloggers haven't been too kind to TCP for use in real-time video games because it's slow and adds a lot of overhead. I have heard people say to use what suits your game bests though. Of course, TCP could be used in a turn-based game with no problem but how about in a real-time game where, say, the client communicates with the server 10 times a second?

I've heard that Starcraft 2 uses TCP because every packet is crucial for every player (for deterministic simulation) and they still have decent responsiveness. Of course, this claim isn't substantiated but it's interesting to think about.

On a side thought, how does RUDP compare with TCP? TCP has a lot of overhead from buffering packets. I'm not very sure about how RUDP works but it seems like a better alternative to TCP for an RTS game because of the lower overhead.

 

I've used TCP in games before, a long time ago, for matchmaking, lobby communications, and critical real-time data as a separate stream to general position data which was sent unreliably through UDP - and I wouldn't do it again except where the job matches the protocol's design - i.e. for streaming large amounts of non-time-sensitive data, not for delivering individual packet elements efficiently. It also works best if one machine is doing most of the talking, or at least individual bits of conversation are very one-sided. It does not work well for ping-pong exchanges of small amounts of data.

There are ways to improve TCP's performance for this kind of data, but they are subtle and hard to get right, and I am still not convinced about its suitability for almost all game communications. The main exception is if you are streaming large amounts of data, e.g. a serialized world state, or a lobby session list, in which case it makes a lot of sense to use TCP.

It is almost impossible to perform NAT punchthrough with TCP. It has been done, but the method is not widespread and works with even fewer router combinations than UDP punchthrough. This is less important where a server is involved, of course.

Another factor is the complexity of the server code. Every TCP connection requires a separate socket, and monitoring lots of sockets is hard to do efficiently. On the other hand, with UDP you can funnel everybody into a single socket - then your server code can just pull a message off and deal with it; you can also multiplex across as many sockets as you like with very little overhead.

Finally, you need to deal with connection drops, which are frankly almost impossible to deal with well. TCP guarantees delivery except if the connection drops, at which point you have no idea how much data actually got through; and your client and server need to start the whole dance again before negotiating how much data each received from the other, so you can catch up on the rest. On the other hand, a temporary outage will cause UDP packets to drop, but you get to decide how sensitive your app is to this, how long to wait before carrying on, and you've already written all the logic to deal with missed packets in an efficient way.

Designing an efficient reliable protocol does take a lot of thought, and a lot of research has gone into TCP. It's important not to just dismiss that. However, most game traffic admits fairly simple resend mechanisms anyway, and you can often tolerate missed packets. Even for important data that must not be dropped you can often coalesce the packets yourself without much trouble, and with less overhead, because you understand the data.

fholm's UdpKit supports an interesting level of UDP reliability out of the box - it doesn't resend data, but it does tell you when data didn't arrive. Right now in my RTS library I just resend those packets if they're important, which is easy but inefficient; and sequencing them is done naturally because they all contain a tick count anyway saying which tick they should take effect on. However I have a more efficient design in mind which I mean to imprement at some point - I made a raw UDP (non-UdpKit) prototype a while back that was promising.

If you do decide to go with TCP, do research into other similar uses of the protocol. Disable Nagle's algorithm (TCP_NODELAY), and read about webserver architecture because your servers are going to need to monitor a *lot* of sockets, as they scale up.

Here are some references - I didn't cherry-pick these and haven't read them, they are just the three that stood out from a Google search, which looked useful at first sight:

https://1024monkeys.wordpress.com/2014/04/01/game-servers-udp-vs-tcp/
http://gamedev.stackexchange.com/qu...l-good-enough-for-real-time-multiplayer-games
http://www.iis.sinica.edu.tw/~swc/pub/tcp_in_games.html

 

TCP just doesn't fit what works best for high throughput, low latency systems. These systems are characterized by architectures that use unreliable messaging. And I'm not just talking about client/server messaging. Synchronizing state is the cost you pay for reliable messages, and it adds significant performance cost especially at any kind of scale. It's the same reason why you don't use transactions in a database unless you really have to.

So many of these systems are already designed to deal with failure and handle it in other ways, such as implementing your logic to be idempotent. TCP just doesn't really give you any benefit in this case for the bulk of what your system does.

My own approach to reliability at the network level is just don't do it. If you think about it,what really matters is if your message reaches it's destination and is processed correctly, and network level reliability doesn't guarantee that anything was actually processed correctly. Reliability at the network level is just the wrong layer. It works much better if you move the reliability logic closer to the layer that defined the need.

In practice for example this means I implement reliable messaging in the base class used for most of my game logic. There is a default implementation, but specific game logic can modify it to suit it's own needs. And the game logic also determines if and when it's even used. It's worth noting that my system is messaging based entirely, so my base classes already deal with messaging. In other systems you might not have it right in the base game logic class.

A lot of the approaches I use I took from the finance industry, the guys running stock exchanges and such. There is also a lot of good info on reliability at the messaging level from the web industry. That approach has largely been abandoned throughout most industries for some of the reasons I stated. It's not that how games do it is bad per say, it's just not the best way.

 

World of Warcraft is actually TCP .

 

I recently looked at this thread, and it mentions reliability of transmission using RPC in unity.....

Is this not what I think it is? I assumed that the unity networking code would assure that an RPC call arrives...

http://answers.unity3d.com/questions/31865/how-to-make-unreliable-rpc-calls-as-opposed-to-rel.html

 

Ap0C552, that's a totally different topic...

cranky, yes, they made it work for themselves. I don't know much about the game or any compromises that were required (or not).

jpthek9: I thought a bit more about TCP in RTS games, and I guess disconnect/reconnect handling is not too bad in an RTS game - a bit like interrupted HTTP transfers, you need a connection protocol for specifying how many ticks the client already received, so the server can carry on from there, but it's not too hard to set that kind of thing up. Efficiently implementing a scalable server for this kind of thing is still scary though.

On the topic in general, I guess one area that's likely to require use of TCP is WebGL. It sounds like WebRTC is an option for accessing UDP in the browser, but I don't know whether Unity will support it. I'm not on the 5.0 beta.

 

Actually it was pertinent to exactly what you mentioned in this topic. I will be more clear.

You mentioned that a third party api supports an "interesting level of UDP reliability out of the box". Does Unity's networking API not have a layer to provide a certain degree of UDP reliability out of the box?

Original poster mentions using TCP when things must arrive reliably. I thought that Unity does provide a certain degree or reliability. So why then even worry about TCP?

 

I'm using Forge
Just bouncing some ideas off you guys to see whether or not I should use Forge's TCP, RUDP, or plain UDP. All the hard stuff's already done for me. I think the best way is to just test it out and see how much performance, bandwidth, and latency overhead there is with TCP compared to UDP and RUDP compared to UDP then see if one is worth it. I'll try to get back to you guys on it.