Network & Filesystem: Doing less cross rings memory copy
0 likes106 views
The document discusses optimization techniques in the Linux kernel to reduce memory copying costs, specifically zero-copy approaches. It provides: 1. A brief history of context switches in systems and the costs associated with copying data between kernel and user space during system calls. 2. An overview of common Linux system calls like read() and write() that involve data copies between kernel and user spaces. 3. Details on zero-copy Linux kernel APIs like splice() and sendfile() that can move data between file descriptors without copying to/from kernel space, improving performance. 4. Benchmark results showing a zero-copy file upload approach achieving over 3Gbps compared to 940Mbps for a normal copy-based
Network & Filesystem: Doing less cross rings memory copy
- 1. 1
- 2. Network & Filesystem: Doing less cross rings memory copy Louis Solofrizzo System & Kernel, Storage Team 2
- 3. Optimization: Zero-Copy in the Linux Kernel Quick history of the context-switch The cost of a ring-copy Normal copies in the Linux Kernel and the Glibc Zero-copy API in the Linux Kernel Going further: Avoiding the context-switch 3
- 4. Abbreviations Context Switch: Storing the state of a process Syscall: System Call MMU: Memory Management Unit Kernel / User mode: System Privilege IRQ: InteRrupt Request 4
- 5. Quick history of the context-switch The User - Kernel Space system One API to rule them all Syscalls are born! (yay!) Multitasking CPU context-switch, with privileges 5
- 6. The cost of a ring copy Copying memory from kernel to user space do cost a lot The physical translation cache is flushed (TLB) + The cost of the context-switch itself 6
- 9. Copies in the Linux Kernel and libc ssize_t read(int, void *, size_t) ssize_t write(int, const void *, size_t); ssize_t ksys_write(unsigned int, const char *, size_t); ssize_t ksys_read(unsigned int, char *, size_t); 9
- 10. Copies in the Linux Kernel and libc ssize_t readv(int, const struct iovec *, int); ssize_t writev(int, const struct iovec *, int); long sys_preadv(unsigned long, const struct iovec *, unsigned long, unsigned long, unsigned long); long sys_pwritev(unsigned long, const struct iovec *, unsigned long, unsigned long, unsigned long); 10
- 11. Zero-copy API in the Linux Kernel ssize_t splice(int, loff_t *, int, loff_t *, size_t, unsigned int); splice() moves data between two file descriptors without copying between kernel address space and user address space. 11
- 14. Zero-Copy upload architecture example 14
- 15. Benchmark Upload Normal: ~940Mbps Upload Zero-copy: ~3Gbps 15
- 16. Other Zero-copy API ssize_t sendfile(int, int, off_t *, size_t); ssize_t copy_file_range(int, loff_t *, int, loff_t *, size_t, unsigned int); sendfile() copies data between one file descriptor and another. The copy_file_range() system call performs an in-kernel copy between two file descriptors without the additional cost of transferring data from the kernel to user space and then back into the kernel. 16
- 17. Zero Copy in Linux common applications Nginx sendfile() HAProxy DIRECT_IO (ZFS, ext4) 17
- 18. Going further: Avoiding the context-switch UniKernels MicroKernels 18
- 19. Thanks! Follow our news, tutorials and cloud informations on Twitter et LinkedIn @Scaleway 19
- 20. 20