Networking Layer Basics - ChinaNetCloud Training
- 1. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetcloud Training Program ChinaNetCloud Training Networking Layers By ChinaNetCloud Pioneers in OaaS – Operations-as-a-Service October, 2013 www.ChinaNetCloud.com
- 2. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 2 Introduction ● Networking is core to what we do ● Everyone familiar but few really know it ● Especially how to troubleshoot
- 3. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 3 Layers ● Layer 1 – Physical wire, NIC ● Layer 2 – Ethernet, MAC, VLAN, Switches ● Layer 3 – IP, Routing ● Layer 4 – TCP/UDP, Connections, Ports ● Layer 7 – HTTP, ssh, Applications
- 4. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 4 Layer 1 – Cable & NIC ● Basic Ethernet ● 10Mbps, 100Mbps, 1000Mbps, 10G+ ● Speed is x 1000, not 1024 for Kbps, Mbps, Gbps ● Cables ● Category 5, 5e 6 – all okay ● Throw away if connector, cable damaged ● Patch Panels & Jacks
- 5. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 5 Layer 1 – Cable & NIC ● NIC Cards ● Buffers – Ring buffers, sizes ● IRQ – How the NIC talks to CPU – Can overload a single CPU (advanced) – IRQ Balance ● Managed by ethtool – Options vary by driver ● Intel good, Broadcom bmx okay ● Switch also physical layer device ● Wireless is also physical layer system
- 6. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 6 Layer 1 - Tools ● Ethtool – Most useful ● Careful of options ● Options and value depend on NIC driver ● Has good statistics (-s) ● Controls IRQ binding, ring buffers ● Switch – Use normal Cisco commands
- 7. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 7 Layer 1 – Speed & Duplex ● Speed – 10M, 100M, 1000M, 10G ● Duplex – Talk in one or two directions ● Half – Only talk one dir at one time ● Full – Talk both ways, all the time (much better) ● Auto – Normally auto selected ● See problems, next slide
- 8. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 8 Layer 1 - Problems ● 100M Auto Speed/Duplex not good ● Can get wrong speed (10Mbps!) or duplex ● If very slow, check this with ethtool ● 1Gbps system better, but still check ● Bad cable (even with link light/status) ● Can't always believe lights, change cable ● Buffer overruns / IRQ Overload - Advanced
- 9. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 9 Layer 2 – Switch, MAC, VLAN ● MAC Address Oriented ● All packets move via MAC, not IP ● ARP relates IP address to MAC ● Switch Level Networking ● Connect ports by MAC ● Plus broadcasts Switches remember where each MAC is ● Problem if you move servers/cables
- 10. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 10 Layer 2 – Switch, MAC, VLAN ● VLAN Level Networking ● Separate L2 networks ● Managed in and by switch ● Switches connected by Trunks ● Each trunk carries many VLANs ● Most hosts are on a single VLAN ● Xen servers sometimes on several – Interface name will match VLAN #, like eth0.25
- 11. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 11 Layer 2 – Switch, MAC, VLAN ● Xen hosts also have internal switches ● Called bridges (same as switch) ● Xen eth0.. and all VM eth0.. all connect to bridge ● Managed with brctl – brctl show
- 12. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 12 Layer 2 - Tools ● arp – Shows what MACs a server/switch knows ● Ifconfig – Show if interface up/down ● Show MAC address ● Switch – Cisco commands
- 13. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 13 Layer 2 - Problems ● NOT common ● MAC Change on VM restart in Xen ● Usually not an issue, but MAC changes ● Switch lockout ● Too many MACs per port – Security protection ● Switch remembers old MAC ● MAC move on HA failover ● Not common. Mostly IP move issue.
- 14. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 14 Layer 3 - IP ● IP Address Oriented ● IP, Mask, Aliases ● Watch the mask ● Especially on public Internet ● Use arp to get MAC from IP ● Broadcast, then remember MAC for a while ● Gateway – Only one per server ! ● Even if dual-line, etc. – Special Policy Routing for this, Advanced
- 15. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 15 Layer 3 - Tools ● Ip – A whole set of commands ● ip addr – Show addresses ● ip route ● ping ● Traceroute ● Older tools ● Ifconfig - older, may not show everything ● Does show MAC ● route – older, won't show policy routes
- 16. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 16 Layer 3 - Problems ● Duplicate IP ● Bad gateway ● Bad mask ● Bad VLAN
- 17. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 17 Layer 4 – TCP & UDP ● Connection Oriented (TCP) ● Uses Ports – On both sides ● Flow Control & Retry (TCP) ● Most common iptables entries at Layer 4 ● Important to understand
- 18. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 18 Layer 4 – TCP Flow ● Server listening on an IP & Port (or all IP, *:80) ● Three way connection handshake / create ● Client sends SYN to Server's IP & port ● Server Send ACK ● Client sends SYN/ACK ● Connection is ESTABLISHED, or open ● Data flows ● With TCP Windows, ACK, and Retries ● Either side will close (complex)
- 19. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 19 Layer 4 – TCP States - netstat ● Some very rare, short time, others common ● SYN_SENT – We are trying to open new connection ● Other side not reply yet. ● If seen on PHP system, this is a big problem ● SYN_RECV – We are queuing incoming requests ● ESTAB - Normal ● FIN_WAIT1 – Very unusual ● FIN_WAIT2 – Very unusual ● TIME_WAIT – Socket closed, waiting. Normal. ● CLOSING, CLOSE_WAIT, LAST_ACK, CLOSED - Rare
- 20. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 20 Layer 4 - Tools ● netstat – Show states for TCP ● tcpdump – Read data from network
- 21. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 21 Layer 4 - Problems ● Firewall blocks inbound ● Outbound timeout ● PHP to other sites, memcache ● Syn cookies sent ● Conntrack/Iptables queue full
- 22. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 22 Layer 7 – HTTP & Application ● App Layer Oriented ● HTTP, SSH, etc. on TCP ● Or custom protocol like VoIP, Video on UDP ● Every protocol different
- 23. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 23 Layer 7 - Tools ● Tcdump – Read the data from network ● WireShark – Analyze, display data, all layers
- 24. ChinaNetCloud Running All the World's Internet Servers Copyright 2011 ChinaNetCloud 24 Layer 7 - Problems ● Specific to protocols ● See protocol training ● HTTP ● FTP – Uses two connections
- 25. ChinaNetCloud Running All the World's Internet Servers Copyright 2015 ChinaNetCloud 25 About ChinaNetCloud
- 26. ChinaNetCloud Running All the World's Internet Servers ChinaNetCloud Sales@ChinaNetCloud.com Beijing Office: Lee World Business Building #305 57 Happiness Village Road, Chaoyang District Beijing, 100027 China Silicon Valley Office: California Avenue Palo Alto, 94123 USA Shanghai Headquarters: X2 Space 1-601, 1238 Xietu Lu Shanghai, 200032 China T: +86-21-6422-1946 F: +86-21-6422-4911