Local file systems update
2 likes2,628 views
This document summarizes a presentation about updates to local file systems in Linux. It provides an overview of common Linux file systems like Ext4, Xfs, and Btrfs. It discusses challenges around scalability and reliability as hardware capacities increase. It then focuses on improvements made in recent versions of Xfs, Ext4 and Btrfs to address scalability and reliability issues through features like delayed logging in Xfs, bigalloc in Ext4, and checksumming in Btrfs. The document points to additional resources for learning more about updates in these file systems.
Local file systems update
- 1. Local file systems update Red Hat Luk´ˇ Czerner as February 23, 2013
- 2. Copyright © 2013 Luk´ˇ Czerner, Red Hat. as Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the COPYING file.
- 3. Agenda 1 Linux file systems overview 2 Challenges we’re facing today 3 Xfs 4 Ext4 5 Btrfs 6 Questions ?
- 4. Part I Linux kernel file systems overview
- 5. File systems in Linux kernel Linux kernel has a number of file systems Cluster, network, local Special purpose file systems Virtual file systems Close interaction with other Linux kernel subsystems Memory Management Block layer VFS - virtual file system switch Optional stackable device drivers device mapper mdraid
- 6. The Linux I/O Stack Diagram version 0.1, 2012-03-06 outlines the Linux I/O stack as of Kernel version 3.3 Applications (Processes) anonymous pages chmod(2) (malloc) write(2) open(2) read(2) stat(2) ... VFS block based FS Network FS pseudo FS special ext2 ext3 ext4 NFS coda proc sysfs purpose FS direct I/O Page (O_DIRECT) gfs ocfs pipefs futexfs tmpfs ramfs Cache xfs btrfs ifs smbfs ... usbfs ... devtmpfs iso9660 ... network Block I/O Layer LVM optional stackable devices on top of “normal” block devices – work on bios mdraid device drbd lvm ... mapper BIOs (Block I/O) I/O Scheduler maps bios to requests cfq deadline noop hooked in Device Drivers (hook in similar like stacked devices like request-based mdraid/device mapper do) device mapper targets /dev/fio* /dev/rssd* dm-multipath SCSI upper layer iomemory-vsl mtip32xx with module option /dev/vd* /dev/fio* /dev/sda /dev/sdb ... /dev/nvme#n# nvme sysfs (transport attributes) SCSI mid layer virtio_blk iomemory-vsl Transport Classes scsi_transport_fc scsi_transport_sas SCSI low layer scsi_transport_... libata megaraid sas aacraid qla2xxx lpfc ... ahci ata_piix ... HDD SSD DVD LSI Adaptec Qlogic Emulex ... Fusion-io nvme Micron drive RAID RAID HBA HBA PCIe Card device PCIe Card Physical devices The Linux I/O Stack Diagram (version 0.1, 2012-03-06) http://www.thomas-krenn.com/en/oss/linux-io-stack-diagram.html Created by Werner Fischer and Georg Schönberger License: CC-BY-SA 3.0, see http://creativecommons.org/licenses/by-sa/3.0/
- 7. Most active local file systems File system Commits Developers Active developers Ext4 648 112 13 Ext3 105 43 2 Xfs 650 61 8 Btrfs 1302 114 21
- 8. 14 1/ /0 01 13 1/ /0 01 12 1/ Ext3 Btrfs Xfs Ext4 /0 01 11 1/ /0 01 10 1/ /0 01 09 1/ /0 01 Number of lines of code 08 1/ /0 01 07 1/ /0 01 06 1/ /0 01 05 1/ /0 01 80000 70000 60000 50000 40000 30000 20000 10000 Lines of code
- 9. Part II Challenges we’re facing today
- 10. Scalability Common hardware storage capacity increases You can buy single 4TB drive for a reasonable price Bigger file system and file size Common hardware computing power and parallelism increases More processes/threads accessing the file system Locking issues I/O stack designed for high latency low IOPS Problems solved in networking subsystem
- 11. Reliability Scalability and Reliability are closely coupled problems Being able to fix your file system In reasonable time With reasonable memory requirements Detect errors before your application does Metadata checksumming Metadata should be self describing Online file system scrub
- 12. New types of storage Non-volatile memory Wear levelling more-or-less solved in firmware Block layer has it’s IOPS limitations We can expect bigger erase blocks Thinly provisioned storage Lying to users to get more from expensive storage Filesystems can throw away most of it’s locality optimization Cut down performance Device mapper dm-thinp target Hierarchical storage Hide inexpensive slow storage behind expensive fast storage Performance depends on working set size Improve performance Device mapper dm-cache target, bcache
- 13. Maintainability issues More file systems with different use cases Multiple set of incompatible user space applications Different set of features and defaults Each file system have different management requirements Requirements from different types of storage SSD Thin provisioning Bigger sector sizes Deeper storage technology stack mdraid device mapper multipath Having a centralized management tool is incredibly useful Having a central source of information is a must System Storage Manager http://storagemanager.sf.net
- 14. Part III What’s new in xfs
- 15. Scalability improvements Delayed logging Impressive improvements in metadata modification performance Single threaded workload still slower then ext4, but not much With more threads scales much better than ext4 On-disk format change XFS scales well up to hundreds of terabytes Allocation scalability Free space indexing Locking optimization Pretty much the best choice for beefy configurations with lots of storage
- 16. Reliability improvements Metadata checksumming CRC to detect errors Metadata verification as it is written to or read from disk On-disk format change Future work Reverse mapping allocation tree Online transparent error correction Online metadata scrub
- 17. Part IV What’s new in ext4
- 18. Scalability improvements Based on very old architecture Free space tracked in bitmaps on disk Static metadata positions Limited size of allocation groups Limited file size limit (16TB) Advantages are resilient on-disk format and backwards and forward compatibility Some improvements with bigalloc feature Group number of blocs into clusters Cluster is now the smallest allocation unit Trade-off between performance and space utilization efficiency Extent status tree for tracking delayed extents No longer need to scan page cache to find delalloc blocks Scalability is very much limited by design, on-disk format and backwards compatibility
- 19. Reliability improvements Better memory utilization of user space tools No longer stores whole bitmaps - converted to extents Biggest advantage for e2fsck Faster file system creation Inode table initialization postponed to kernel Huge time saver when creating bigger file systems Metadata checksumming CRC to detect errors Not enabled by default
- 20. Part V What’s new in btrfs
- 21. Getting stabilized Performance improvements is not where the focus is right now Design specific performance problems Optimization needed in future Still under heavy development Not all features are yet ready or even implemented File system stabilization takes a long time
- 22. Reliability in btrfs Userspace tools not in very good shape Fsck utility still not fully finished Neither kernel nor userspace handles errors gracefully Very good design to build on Metadata and data checksumming Back reference Online filesystem scrub
- 23. Resources Linux Weekly News http://lwn.net Kernel mailing lists http://vger.kernel.org linux-fsdevel linux-ext4 linux-btrfs linux-xfs Linux Kernel code http://kernel.org Linux IO stack diagram http://www.thomas-krenn.com/en/oss/linuxiostack- diagram.html
- 24. The end. Thanks for listening.