SlideShare a Scribd company logo
Lecture 10
Transport Protocol
        for
Networked Games
     29 March 2010


                     1
TCP or UDP ?



               2
Why use TCP?

• TCP provides reliable, in-order delivery
• TCP goes through most firewalls, UDP
  does not
• TCP manages connection for us


                                             3
Why not to use TCP?

• TCP incurs higher latency
• Don’t always need reliability and in-order
  delivery
• High header overhead


                                               4
position = 10
position = 13          X
position = 15

   Updated position not delivered to
 application until (outdated) lost packet
               is received

                                            5
A’s position = 10
B’s position = 13        X
C’s position = 15


   Some messages need not be delivered in
                sequence.


                                            6
Gestures from someone far away need
      not be received reliably.
                                      7
46%
of ShenZhou Online bandwidth is
     occupied by TCP header




                                  8
enet.cubik.org
 A library that provides
  reliability, sequencing,
connection managements
        over UDP

                             9
Delivery can be
stream-oriented (like TCP) or
message-oriented (like UDP)



                                10
Supports partial reliability

enet_packet_create (“abc”,
4, ENET_PACKET_FLAG_RELIABLE)




                                  11
Retransmission triggered by
  timeout-based on RTT




                              12
Data in queue are bundled into
 one packet if there is space




                                 13
enet.cubik.org

Portable, easy to use, but
still, most firewalls block
        UDP traffic

                             14
• Uses TCP: WoW, Lineage I/II,
 Guild Wars, Ragnarok Online,
 Anarchy Online, Mabinogi

• Uses UDP: EverQuest, SW
 Galaxies, City of Heroes, Ultima
 Online, Asherons Call, FFXI


                                    15
Need to study the use
of TCP for networked
        games


                        16
How to provide
reliability over UDP?


                        17
How slow is TCP, really?

Which part of TCP is the
root of slowness?

Can we fix TCP?

                           18
A Quick Review of TCP



                        19
s=100
s=200
s=300
s=400                     a=300

                          a=500




        TCP Delayed ACK
                                  20
TCP Spec: max 500ms delay
Most implementation: 200ms



                             21
Why delay ACK?
• reduce num of ACKs
• in case receiver wants to send data
  within 200ms (in which case it can
  piggyback the ACK with data)
• give sender time to buffer more data
  for sending (avoid silly window
  syndrome)

                                         22
s=100
s=200
s=300                             a=200
                      X
s=400
s=500                             a=200
                                  a=200
                                  a=200




s=200

        TCP Fast Retransmission
                                          23
Definition of Dup ACKs
in 4.4BSD and Stevens:
“pure ACK with no data”


                          24
RTO
            X

                      X

2xRTO




   TCP Timeout + Exponential Back-off
                                        25
RTO




      Spurious Retransmission

                                26
RTO estimation

Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
RTO = max(Ei + 4Vi, 1s)

                              27
Linux’s RTO estimation

Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
Wi = min(Vi, 50ms)
RTO = max(200ms, Ei+Wi)
                              28
delayed ACK

increase RTT

increase RTO
               29
Congestion Control



                     30
Window Size
                     Packet Loss




                                   Time
       TCP Congestion Control

                                          31
Congestion window
resets to 2 after an idle
    period (> RTO)


                            32
What does real game
 traffic look like?


                      33
low packet rate
small packet size


                    34
“Thin Streams”



                 35
36
Max Application Delay




            Max RTT


250ms
                         Avg RTT



                                   37
38
About 4 packets / sec



                        39
Average Payload:
  100 Bytes



                   40
Loss Rate 1%



               41
But some experience 6
    retransmissions


                        42
ShenZhou Online




                  43
44
45
46
“Thin Streams”



                 47
Findings 1:
Fast retransmission
  rarely triggered


                      48
In ShenZhou Online traces, fail
 to trigger fast retransmission
             because
  insufficient dup ACK (50%)
   interrupted by data (50%)


                                  49
50
51
Findings 2:
Delay due mostly to
      timeout


                      52
53
Findings 3:
Congestion window
 reset is frequent


                     54
55
12% - 18% of packets
 faces window reset


                       56
think..
think..
think..
click (tank attack here)
click (missile launch there)
click (charge soldiers)

   The last command is delayed as congestion
                 window = 2

                                               57
How to make TCP (or,
transport protocol) go
 faster in these games?


                          58
1. Remove exponential
       backoff


                        59
TO
      X
           X

TO




      TCP Timeout
                    60
2. Make RTO Smaller



                      61
make sure minimum
  RTO is not 1s


                    62
spurious retransmission
   is not disastrous


                          63
3. Make Fast Retransmit
         Faster


                          64
s=100
s=200
s=300                           a=200
                     X
s=400
s=500                           a=200
                                a=200
                                a=200

s=200



  Retransmit after one duplicate ACK
                                        65
4. Retransmission
     Bundling


                    66
X




Retransmit all unacknowledge data in queue
                                             67
5. Redundant Data
     Bundling


                    68
X
                                    a=300

                                    a=500




Send any unacknowledged segment in queue
  as long as there is space. Lost data gets
 recovered in the next transmission before
               retransmission.
                                              69
6. Turn off or reduce
    Delayed ACKs


                        70
Packet interarrival time
 on average > 200ms
(can’t combine two ACKs into one)




                                    71
7. Combine Thin
Streams into Thicker
      Stream


                       72
Server




Proxy




         73
TCP for Games

• remove exponential backoff
• reduce RTO
• make fast retransmit faster
• retransmit agressively
• don’t delay ACK
• combine into thick streams
                                74
Beyond TCP and UDP?



                      75
Ideal Transport Protocol for
           Games

• optional reliability
• optional order-of-delivery
• flexibility in organizing messages into
  different classes with different
  requirements



                                           76
SCTP
Stream Control Transport Protocol




                                    77
Multi-streaming: multiple
 independent streams




                            78
A stream can be either reliable or
          non-reliable




                                     79
Data from multiple streams can be
    bundled into one packets




                                    80
Message-oriented (like UDP)




                              81
Message can be flagged for
   unordered delivery




                            82
Ideal Transport Protocol for
           Games

• optional reliability
• optional order-of-delivery
• flexibility in organizing messages into
  different classes with different
  requirements



                                           83
SCTP for Games?



                  84
States of SCTP



• rumored to be in Vista
• available in FreeBSD 7 and Solaris 10
• lksctp project for Linux


                                          85

More Related Content

PDF
OpenStack networking juno l3 h-a, dvr
ODP
Virtual net performance
PDF
Investigating the Use of Synchronized Clocks in TCP Congestion Control
PDF
LF_OVS_17_Ingress Scheduling
PDF
Cilium - API-aware Networking and Security for Containers based on BPF
PDF
Recent advance in netmap/VALE(mSwitch)
PDF
Intel DPDK Step by Step instructions
PDF
LinuxCon 2015 Linux Kernel Networking Walkthrough
OpenStack networking juno l3 h-a, dvr
Virtual net performance
Investigating the Use of Synchronized Clocks in TCP Congestion Control
LF_OVS_17_Ingress Scheduling
Cilium - API-aware Networking and Security for Containers based on BPF
Recent advance in netmap/VALE(mSwitch)
Intel DPDK Step by Step instructions
LinuxCon 2015 Linux Kernel Networking Walkthrough

What's hot (20)

PDF
mTCP使ってみた
PDF
BPF & Cilium - Turning Linux into a Microservices-aware Operating System
PDF
Userspace networking
PDF
CloudStackユーザ会〜仮想ルータの謎に迫る
PDF
DPDK Summit 2015 - HP - Al Sanders
PDF
Playing BBR with a userspace network stack
PDF
How to Speak Intel DPDK KNI for Web Services.
PDF
Network Stack in Userspace (NUSE)
PDF
XPDS13: On Paravirualizing TCP - Congestion Control on Xen VMs - Luwei Cheng,...
PPTX
The n00bs guide to ovs dpdk
ODP
Virtual Network Performance Challenge
PPTX
Enable DPDK and SR-IOV for containerized virtual network functions with zun
PDF
Linux Bridging: Teaching an old dog new tricks
PPTX
Accelerating Neutron with Intel DPDK
PDF
Lagopus presentation on 14th Annual ON*VECTOR International Photonics Workshop
PDF
Virtualized network with openvswitch
PDF
Direct Code Execution - LinuxCon Japan 2014
PDF
DevConf 2014 Kernel Networking Walkthrough
PDF
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
PDF
DPDK in Containers Hands-on Lab
mTCP使ってみた
BPF & Cilium - Turning Linux into a Microservices-aware Operating System
Userspace networking
CloudStackユーザ会〜仮想ルータの謎に迫る
DPDK Summit 2015 - HP - Al Sanders
Playing BBR with a userspace network stack
How to Speak Intel DPDK KNI for Web Services.
Network Stack in Userspace (NUSE)
XPDS13: On Paravirualizing TCP - Congestion Control on Xen VMs - Luwei Cheng,...
The n00bs guide to ovs dpdk
Virtual Network Performance Challenge
Enable DPDK and SR-IOV for containerized virtual network functions with zun
Linux Bridging: Teaching an old dog new tricks
Accelerating Neutron with Intel DPDK
Lagopus presentation on 14th Annual ON*VECTOR International Photonics Workshop
Virtualized network with openvswitch
Direct Code Execution - LinuxCon Japan 2014
DevConf 2014 Kernel Networking Walkthrough
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DPDK in Containers Hands-on Lab
Ad

Viewers also liked (8)

PPT
Lecture 5
PDF
TCP with delayed ack for wireless networks
PPT
Course on TCP Dynamic Performance
PPTX
Socket网络编程
PPT
TCP Model
PPT
Flow & Error Control
PPT
2 reliability-network
PPTX
4 transport-sharing
Lecture 5
TCP with delayed ack for wireless networks
Course on TCP Dynamic Performance
Socket网络编程
TCP Model
Flow & Error Control
2 reliability-network
4 transport-sharing
Ad

Similar to CS4344 09/10 Lecture 10: Transport Protocol for Networked Games (20)

PPT
An Empirical Evaluation of TCP Performance in Online Games
PPT
Transport protocols
PPT
Features of tcp (part 2) .68
PDF
Osi model
PPT
Unit III IPV6 UDP
PPTX
Transport layer in OSI Layers in detail.pptx
PPT
TCP Section5
PPTX
6610-l14.pptx
PDF
High perf-networking
PDF
Text book 2 computer networks_a systems approach_peterson solution manual
PPT
Computer network (Lecture 2)
PPT
tcp-wireless-tutorial.ppt
PPTX
Networking essentials lect3
PDF
CS4344 09/10 Lecture 9: Characteristics of Networked Game Traffic
PPT
Collections3b datatransferstreams
PPT
Troubleshooting TCP/IP
PPTX
Online TCP-IP Networking Assignment Help
PPT
Chapter 01 - Overview
PPT
05Transport.ppt
An Empirical Evaluation of TCP Performance in Online Games
Transport protocols
Features of tcp (part 2) .68
Osi model
Unit III IPV6 UDP
Transport layer in OSI Layers in detail.pptx
TCP Section5
6610-l14.pptx
High perf-networking
Text book 2 computer networks_a systems approach_peterson solution manual
Computer network (Lecture 2)
tcp-wireless-tutorial.ppt
Networking essentials lect3
CS4344 09/10 Lecture 9: Characteristics of Networked Game Traffic
Collections3b datatransferstreams
Troubleshooting TCP/IP
Online TCP-IP Networking Assignment Help
Chapter 01 - Overview
05Transport.ppt

More from Wei Tsang Ooi (20)

PDF
SIGMM Education Effort Presentation at Business Meeting
PDF
CS2106 Tutorial 2
PDF
The Computer Science behind YouTube
PDF
CS4344 09/10 Lecture 8: Hybrid Architecture
PDF
CS4344 09/10 Lecture 7: P2P Interest Management
PDF
CS4344 09/10 Lecture 6: P2P Synchronization
PDF
Cs4344 09/10 Lecture 5: Interest Management
PDF
CS4344 09/10 Lecture 3: Dead Reckoning and Local Perception Filter
PDF
CS4344 09/10 Lecture 2: Consistency
PDF
CS4344 09/10 Lecture 1: Introduction
PDF
CS5229 09/10 Lecture 11: DNS
PDF
CS5229 09/10 Lecture 10: Internet Routing
PDF
CS5229 09/10 Lecture 9: Internet Packet Dynamics
PDF
CS5229 09/10 Lecture 6: Simulation
PDF
Lecture 5: RED
PDF
Lecture 4: TCP and TFRC
PDF
Lecture 2: Congestion Control and Avoidance
PDF
Lecture 0: Introduction to CS5229
PDF
Lecture 1: Design Principles of the Internet
PDF
CS4344 Lecture 8: Hybrid Architecture
SIGMM Education Effort Presentation at Business Meeting
CS2106 Tutorial 2
The Computer Science behind YouTube
CS4344 09/10 Lecture 8: Hybrid Architecture
CS4344 09/10 Lecture 7: P2P Interest Management
CS4344 09/10 Lecture 6: P2P Synchronization
Cs4344 09/10 Lecture 5: Interest Management
CS4344 09/10 Lecture 3: Dead Reckoning and Local Perception Filter
CS4344 09/10 Lecture 2: Consistency
CS4344 09/10 Lecture 1: Introduction
CS5229 09/10 Lecture 11: DNS
CS5229 09/10 Lecture 10: Internet Routing
CS5229 09/10 Lecture 9: Internet Packet Dynamics
CS5229 09/10 Lecture 6: Simulation
Lecture 5: RED
Lecture 4: TCP and TFRC
Lecture 2: Congestion Control and Avoidance
Lecture 0: Introduction to CS5229
Lecture 1: Design Principles of the Internet
CS4344 Lecture 8: Hybrid Architecture

CS4344 09/10 Lecture 10: Transport Protocol for Networked Games