How to Leverage Go for Your Networking Needs
0 likes106 views
The document outlines Sneha's experiences as a software engineer at DigitalOcean, discussing the advantages of using Go for networking services and the implementation of various network protocols and services such as load balancers and DHCP. It details the architecture of layer 7 and layer 4 load balancers, as well as providing code examples for building a TCP port scanner and a DHCP server using Go. Additionally, it summarizes key features of Go, including concurrency and raw socket handling, alongside references to external sources and libraries.
![http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
n := rand.Intn(len(backends))
r.URL.Host = backends[n]
r.URL.Scheme = "https"
req, err := http.NewRequest(r.Method, r.URL.String(), r.Body)
...
res, err := client.Do(req)
...
}
2. Read and send request to
backend](https://image.slidesharecdn.com/networkingandgodeploy2020-201130153111/85/How-to-Leverage-Go-for-Your-Networking-Needs-72-320.jpg)
![func handleConn(clientConn net.Conn) {
n := rand.Intn(len(backends))
backendConn, err := net.Dial("tcp", backends[n])
if err != nil {
log.Printf("error %s: %v", backends[n], err)
return
}
...
}
2. Open and connect backend TCP
socket](https://image.slidesharecdn.com/networkingandgodeploy2020-201130153111/85/How-to-Leverage-Go-for-Your-Networking-Needs-80-320.jpg)
![func copy(from net.Conn, to net.Conn) error {
for {
readBytes := make([]byte, 1024)
n, err := from.Read(readBytes)
if err != nil {
...
}
if _, err = to.Write(readBytes[:n]); err != nil {
...
}
}
}
4. Read and copy](https://image.slidesharecdn.com/networkingandgodeploy2020-201130153111/85/How-to-Leverage-Go-for-Your-Networking-Needs-82-320.jpg)
![b := make([]byte, 1500)
for {
n, from, err := s.pc.ReadFrom(b)
if err != nil {
...
continue
}
buf := make([]byte, n)
copy(buf, b)
workC <- request{
buf: buf,
from: from,
}
}
2. Read data into buffer](https://image.slidesharecdn.com/networkingandgodeploy2020-201130153111/85/How-to-Leverage-Go-for-Your-Networking-Needs-102-320.jpg)
How to Leverage Go for Your Networking Needs
- 2. I’m Sneha. I’m a software engineer at DigitalOcean.
- 3. DigitalOcean cloud-hosting company 14 data centers 1.5 million droplets k8saas, dbaas, lbaas, storage, VPC
- 4. My Journey
- 10. GOROUTINES
- 11. SYNC
- 12. CHANNELS
- 19. DOCC
- 20. .
- 27. “What is Go?” k8s, Prometheus DHCP, Gateways
- 28. What is the OSI model?
- 29. What is a socket?
- 30. DHCP, ARP, NDP Protocols
- 31. “What is Go?” k8s, Prometheus DHCP, Gateways LB, port scanner PIPOD
- 34. The plan for today
- 35. ▣ ⬚ ⬚
- 36. Why use Go for networking services?
- 37. Concurrency Easy To Fuzz-Test Server-SIDE Great for CLI TOOLS EASY TO WRITE REST/RPC SERVICES
- 38. But when might Go NOT be the answer?
- 41. Why did we use Go?
- 42. DigitalOcean has a monorepo and shared packages.
- 43. hvflowd hvaddrd OvS br0 hvflowctl RNS AddFlows OpenFlow SetParameters bond0 ethX ethX addr0 bolt DHCPv4 NDP gRPC DHCPv6 tapX dropletX hvaddrd traffic hvaddrctl AddFlows internet traffic Hypervisor main Using gRPC and CLI tools
- 44. Performance was already good enough.
- 45. ▣ ▣ ⬚
- 48. TCP/IP Model
- 50. The Layers: Data Encapsulation
- 52. socket: internal endpoint for sending or receiving data on a computer network
- 53. How do we read a segment?
- 54. TCP (stream socket): provides a connection-oriented, sequenced flow of data
- 55. UDP (datagram socket): provides a connectionless flow of data
- 56. How do we read a packet? How do we read a frame?
- 57. raw socket: a socket that allows sending and receiving of data without protocol-specific transport layer formatting
- 58. raw socket: IP layer (3 socket(AF_INET,SOCK_RAW,protocol) packet socket: link layer (2 socket(AF_PACKET,SOCK_RAW,protocol)
- 59. bpf (berkeley packet filter) interface: raw link-layer interface on most unix-based systems
- 60. bpf filters also used with raw sockets or tcpdump
- 65. “The best way to learn is simply by doing.” Me
- 66. ▣ ▣ ▣
- 68. Loadbalancer LB A device distributing traffic across a number of backends. https://gbhackers.com/load-balancer-reverse-proxy/
- 69. Building a layer-7 LB 1. Use HTTP protocol. 2. Accept client-side connections. 3. Pass client-side request to one of the backends. 4. Return server-response back to the client.
- 70. HTTP HyperText Transfer Protocol An application layer (7 protocol: ● request-response based ● stateless ● Has different methods GET, PUT, etc) ● Uses TCP or UDP sockets under-the-hood network definition Source: https://www.ntu.edu.sg/home/ehchua/programming/webprogramming/HTTP_Basic s.html
- 71. http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { ... }) log.Fatal(http.ListenAndServe(":8080", nil)) 1. ListenAndServe
- 72. http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { n := rand.Intn(len(backends)) r.URL.Host = backends[n] r.URL.Scheme = "https" req, err := http.NewRequest(r.Method, r.URL.String(), r.Body) ... res, err := client.Do(req) ... } 2. Read and send request to backend
- 73. http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { ... w.WriteHeader(res.StatusCode) _, err = io.Copy(w, res.Body) if err != nil { log.Printf("error writing response to client: %v", err) } ... } 3. Return response to client
- 75. Building a layer-4 LB 1. Use TCP protocol (via a streaming socket). 2. Accept client-side connections. 3. Open backend connection. 4. Use one goroutine to shuttle packets from the client to the backend. 5. Use another goroutine to shuttle packets from the backend to the client.
- 76. TCP Transmission Control Protocol Connection-oriented protocol with 3 phases: ● Connection establishment TCP handshake) ● Data transfer ● Connection termination network definition https://www.researchgate.net/figure/Three-way-Handshake-in-TCP-Connection- Establishment-Process_fig7_313951935
- 77. GO FEATURE net
- 79. listener, err := net.Listen("tcp","127.0.0.1:9090") if err != nil { log.Fatal(err) } for { conn, err := listener.Accept() if err != nil { log.Printf("error accepting client conn: %v", err) continue } go handleConn(conn) } 1. Open and bind client TCP socket
- 80. func handleConn(clientConn net.Conn) { n := rand.Intn(len(backends)) backendConn, err := net.Dial("tcp", backends[n]) if err != nil { log.Printf("error %s: %v", backends[n], err) return } ... } 2. Open and connect backend TCP socket
- 81. var g run.Group { g.Add(func() error { return copy(clientConn, backendConn) }, func(error) { clientConn.Close() backendConn.Close() }) } { g.Add(func() error { return copy(backendConn, clientConn) }, ...) } err = g.Run() 3. Configure rungroups
- 82. func copy(from net.Conn, to net.Conn) error { for { readBytes := make([]byte, 1024) n, err := from.Read(readBytes) if err != nil { ... } if _, err = to.Write(readBytes[:n]); err != nil { ... } } } 4. Read and copy
- 83. Loadbalancer IRL ● Exact mechanism of handling connections and/or requests depends on the layer. ● Layer-4 loadbalancers should be more efficient: ○ Won’t necessarily create a new backend connection per client conn. ○ Use NAT and munging incoming packets. ● Better loadbalancing algorithms. ● Client stickiness.
- 84. Layer 4 Port Scanner
- 85. Nmap Linux utility that scans hosts and attempts to determine open UDP/TCP ports network definition
- 86. GO FEATURE sync
- 87. Building a TCP port-scanner 1. Select range of ports to scan. 2. Try to open and connect a TCP socket to a remote address (and port).. 3. Print results.
- 88. addr := fmt.Sprintf("%s:%d", hostname, port) conn, err := net.DialTimeout("tcp", addr, timeout) if err != nil { fmt.Printf("port %d closed: %vn", port, err) } else { fmt.Printf("port %d openn", port) conn.Close() } 1. Open and connect TCP socket
- 89. conSema := make(chan struct{}, 10) var wg sync.WaitGroup for i := 1; i <= 65535; i++ { wg.Add(1) conSema <- struct{}{} go func(port int) { ... wg.Done() <-conSema }(i) } wg.Wait() 2. Use waitgroup and channels
- 90. Nmap IRL ● Checks UDP and TCP ports on local or remote hosts. ● Host discovery ● OS detection ● Auditing security of a firewall
- 92. Layer 2 DHCP Server
- 93. DHCP is technically layer-7.
- 94. DHCP Protocol ● Dynamic Host Configuration Protocol is used by routers to allocate IP addresses to network interfaces ● DHCPv6 uses NDP and DHCPv4 uses ARP network definition https://study-ccna.com/dhcp-dns/
- 95. Need link layer MAC addresses.
- 96. Building a DHCP server: 1. Open and bind a raw socket to an interface. 2. Read data from socket into bytes buffer. 3. Unmarshal into DHCP message and retrieve sender hardware address. 4. Switch between handlers based on message type. 5. Validate DHCP request message and craft response. 6. Unicast response back to sender.
- 98. GO FEATURE filename := "eth-packet-socket" typ := unix.SOCK_RAW if cfg.LinuxSockDGRAM { filename = "packet-socket" typ = unix.SOCK_DGRAM } sock, err := unix.Socket(unix.AF_PACKET, typ, 0) ... f := os.NewFile(uintptr(sock), filename) sc, err := f.SyscallConn() ... n, addr, err = unix.Recvfrom(int(fd), p, flags) github.com/mdlayher/raw/raw_linux.go
- 99. GO FEATURE // Try to find an available BPF device for i := 0; i <= 10; i++ { bpfPath := fmt.Sprintf( "/dev/bpf%d", i) f, err = os.OpenFile(bpfPath, os.O_RDWR, 0666) if err == nil { break } if perr, ok := err.(*os.PathError); ok { if perr.Err.(syscall.Errno) == syscall.EBUSY { continue } } return nil, err } github.com/mdlayher/raw/raw_bsd.go
- 101. ifi, err := net.InterfaceByName(iface) if err != nil { return nil, err } pc, err := raw.ListenPacket(ifi, uint16(ethernet.EtherTypeIPv4), &raw.Config{ // Don't set any timeouts to avoid syscall busy // loops, since this server will run forever anyway. NoTimeouts: true, }) 1. Open and bind raw socket
- 102. b := make([]byte, 1500) for { n, from, err := s.pc.ReadFrom(b) if err != nil { ... continue } buf := make([]byte, n) copy(buf, b) workC <- request{ buf: buf, from: from, } } 2. Read data into buffer
- 103. var workWG sync.WaitGroup workWG.Add(Workers) workC := make(chan request, Workers) for i := 0; i < Workers; i++ { go func() { defer workWG.Done() for r := range workC { s.serve(r.buf, r.from) } }() } 3. Worker handles request
- 104. func (h *handler) serveDHCPv4(ctx context.Context, req *dhcp4.Packet, from *dhcp4conn.Addr) (*dhcp4.Packet, net.Addr) { ... if !from.EthernetSource.Equal(reqMAC) { ... return h.shouldNAK(req.Type, to, broadcast) } ... } 4. Validate request
- 105. func (h *handler) serveDHCPv4(ctx context.Context, req *dhcp4.Packet, from *dhcp4conn.Addr) (*dhcp4.Packet, net.Addr) { ... res, err := h.buildResponse(ctx, params, req, from) if err != nil { topError(span, err, "failed to build response") return h.shouldNAK(req.Type, to, broadcast) } res.Broadcast = broadcast return res, to ... } 5. Build and send response
- 106. DHCP Summary: 1. DHCP dynamic host configuration protocol used to dynamically assign IP address. 2. Use raw sockets. 3. Jump down to link-layer to do source MAC validation.
- 108. Go can be good for building networking primitives.
- 111. Layer 4: net package
- 117. Sources ● Port Scanner: https://github.com/si74/portscanner ● Layer 7 loadbalancer: https://github.com/si74/layer7lb ● Layer 4 loadbalancer: https://github.com/si74/tcpproxy ● Raw package: https://github.com/mdlayher/raw ● Godocs: https://godoc.org/ ● Golang: https://godoc.org/