Userspace Linux I/O
Download as PPTX, PDF4 likes1,593 views
This document discusses userspace storage systems as an alternative to kernel-based storage for petascale workloads. It outlines several userspace filesystems, block storage systems, and object storage systems used in practice. Common languages used include C, Python, Java, and Golang. Interfaces to the kernel include FUSE, UIO, DPDK and libvma. Challenges include balancing performance, scalability, and complexity across unified, self-managing systems. Specific examples covered are NFS-Ganesha, GlusterFS, HDFS, NBD, tgt, and caching systems like Tachyon and Redis.
1 of 13
Downloaded 39 times
More Related Content
What's hot (20)
Ad
Viewers also liked (20)
Ad
Similar to Userspace Linux I/O (20)
Userspace Linux I/O
- 1. Userspace Linux I/O towards Petascale Storage Anand Babu (AB) Periasamy @abperiasamy
- 2. What is this talk not about?
- 3. Userspace is not SLOW! ❏GlusterFS demonstrates 2 GB/s per node ❏DPDK shows 35.2 Mpps on Xeon E5465 ❏Linux Kernel at 12.2 Mpps on Xeon E5465 ❏DPDK at 80 Mpps on Xeon E2600
- 4. User Space Storage Systems ❏Filesystems ❏GlusterFS, Hadoop HDFS, NFS-Ganesha, uNFS ❏ Block Storage ❏ NBD, LIO/TCM, scst, tgt, DRBD ❏Object Storage
- 5. Languages ❏C - GlusterFS ❏Python - OpenStack Swift ❏Java - Hadoop HDFS ❏Golang - Minio
- 6. Interface to Kernel ❏FUSE, CUSE, UIO, libusb ❏Intel DPDK / SPDK ❏Mellanox libvma ❏Standard libc (POSIX)
- 8. Common Pitfalls ❏Grand Unified System ❏Massively Scalable ❏Performance focussed ❏Complex Algorithms ❏Intelligent Self-managing
- 9. Filesystem ❏NFS-Ganesha vs Custom RPC ❏kernel NFS Client, libnfs, FUSE, LD_PRELOAD NFS Passive NFS Active NFS Passive Eventually Consistent Eventually Consistent Active I/O
- 10. Block Storage ❏NBD vs tgt ❏Log-structured D D D D P P P P Intel ISA-L Erasure Code DPDK + NBD | tgt
- 11. Object Storage ❏No more POSIX. Kernel free! ❏Certificate based trust (Let’s Encrypt) ❏Free to choose any language ❏Stick to Amazon S3 APIs ❏HTTP is the new TCP/IP
- 12. Cache ❏Tachyon Project ❏Memcached ❏Varnish, Squid ❏Redis ❏Opportunity for a new project:
Editor's Notes
- #4: http://www.intel.com/content/dam/www/public/us/en/documents/solution-briefs/communications-packet-processing-brief.pdf