Linuxconeurope2011.ext4btrfs.talk

Quo vadis Linux File Systems: An
operations point of view on
EXT4 and BTRFS
Udo Seidel

Agenda
● Introduction/motivation
● EXT4 – the new member of the extfs family
● Facts, specs
● Operational considerations
● BTRFS – the newbie .. the hope
● Facts, specs
● Operational considerations
● Summary

LinuxCon Europe 26-28/10 2011 2

Me ;-)
● Teacher of mathematics and physics
● PhD in experimental physics
● Started with Linux in 1996
● Linux/UNIX trainer
● Solution engineer in HPC and CAx environment
● Joined Amadeus in 2006
● Head of a international team with people in
Erding, Miami and Sydney

File systems challenges
● Speed
● Size/growth
● Flexibility
● Sustainability


Linux file systems
● More than 50 file systems shipped with Linux
kernel
● Local
● Remote
● Cluster
● ...
● A few as standard for root directory
● EXT2, EXT3
● XFS

Operational challenges with Linux
file systems
● ReiserFS
● sun-setted
● Big installation base
● EXT3
● Stable, robust
● Slow
● XFS
● Additional costs?
● Changes in recent Enterprise distributions

Chances, possibilities, choices for
the next Linux file systems
● New version of the extfs family -> EXT4
● Marked as stable
● Shipped with Enterprise distributions
● New approach with BTRFS
● Still experimental in vanilla kernel
● Shipped as production ready by some Enterprise
distributions


th
4 extended file system
● Shipped since 2.6.19
● Stable since 2.6.28
● To overcome limits of EXT3
● Size
● Performance


EXT4 - facts
● Max volume size: 1 EB = 1024 PB (EXT3: 16
TB)
● Max file size: 16 TB (EXT3: 2 TB)
● Max length of file name: 256 Bytes
● Support of extended attributes
● No encryption
● Not really compression
● Partially 64bit

EXT4 – operational first glance
● Looks promising
● OPROC/procedure changes?
● Behaviour changes?
● Migration


EXT4 – OPROCs/procedures
● Known tools
● mkfs
● fsck

● tune2fs
● e2label

=> Easy to re-use or adapt of existing :-)


EXT4 – Behaviour changes ...
● Extents
● Improved block allocation
● Journaling
● File system check


EXT4 – from blocks to extents
● Common addressing for modern file systems
● Overdue
● Contiguous area of blocks
– Less management information needed
– Less meta data operations
– Less “fragmentation”
● Requires change of on-disk format :-(
● Migration paths?


EXT4 – delayed allocation
● Use cache information for placement
● Less fragmentation
● Risk of data loss in early versions => improved
since 2.6.30
● Easy switch on/off
● No change of on-disk format! :-)


EXT4 – “clever” allocation
● Support of system call fallocate()
● Application reserves blocks ahead
● File system ensures disk space availability
● Allocation information in extent structure
● Only usable with extents!


EXT4 – consistency via JBD2
● Transactions have checksums
● 64 bit ready
● Deactivation possible
● Keep other EXT4 features
● Comparison: EXT3 ... -> EXT2
● Reduction of file systems in use :-)


EXT4 – repair
● Improved fsck()
● No check of unused blocks
– information stored in block group header
– Information secured via checksums
– (de)activation possible at any time
● First run possibly as slow like in EXT3


EXT4 – other news
● Nano second precision time stamps
● Unix millennium bug shifted to 2514
● More subdirectories
● Up to 65000
● More than 65000 ... with limitation


EXT4 – background for migration
● 2 kind of changes compared to EXT3
● change of on-disk format:
– Extents
– Only enabled for new files via tune2fs
– Additional tasks needed
● On-disk format not relevant:
– block allocation
– Immediately enabled via tune2fs


EXT4 – general migration paths
● mkfs() and backup/restore
● Clean new file system structure
● Only way for file systems other than EXT2/3
● Extended outage
● Conversion via tune2fs
● Partial only
● Possible for EXT2/3 family only
● Faster/easier


EXT4 – migration strategy
● It depends
● For us: both options
● Significant installation base of non-EXT2/3
● Big landscape (2500+ Linux servers)


EXT4 – migration via mkfs()
● For all new deployments
● for existing file systems
● On demand
● By chance


EXT4 – migration via tune2fs
● Results in mix of EXT2/3 and EXT4 structure
● Access via EXT2/3 driver impossible
● fsck() needed
parameter description
extent Extent based block allocation
flex_bg Flexible placement of meta data
uninit_bg Flag uninitialized blocks for faster fsck
dir_nlink Infinite number of sub directories
extra_isize Timestamps with nano seconds


EXT4 – migration hints
● fsck() recommended
● /boot – booting from EXT4 possible?
● Rescue media enabled for EXT4?
● Backup/restore of EXT4?


EXT4 – summary
● Good successor of EXT3
● Manages higher amount of data
● Faster
● Performance
● recovery
● Safer
● Sufficient migration options from/to EXT2/3
=> GO!

Better/b-tree file system
● Shipped since 2.6.29
● Still experimental
● In vanilla kernel
● But .... production ready in some Enterprise
distributions
● Supposed to replace EXT3/4
● New storage management approach


BTRFS - facts
● Max volume/file size: 16 EB
● Max length of file name: 256 Bytes
● Support of
● Extended attributes
● No encryption yet
● Snapshot
● Compression
● Copy-on-Write


BTRFS – operational first glance
● OPROC/procedure changes?
● Behaviour changes?
● Migration


BTRFS – OPROCs/procedures
● Known tools
● mkfs ... with limitation
● fsck ... with limitation

● tune2fs -> nope
● XXlabel -> nope

=> No easy to re-use or adapt of existing :-(


BTRFS – behaviour changes ...
● Device management
● Snapshots
● Compression


BTRFS – device management
● Included volume manager
● RAID-0, RAID-1, ...
● Add/remove devices
● ...
● New tool
● change of OPROC's/procedures :-(
● Change to CLI/GUI or new CLI/GUI tools :-(


BTRFS – snapshots
● Not really new ... but not used so far
● Integration into package management
● automatic - no additional tasks
● Faster roll-back
● Post change tracking
● New tool
● New OPROC's anyway :-|
● Minimal human intervention :-)


BTRFS – compression
● Transparent
● Flexible:
● different algorithms possible
● Easy to switch on/off
● Operations:
● Minimal changes on admin side
● No changes on other levels :-)


BTRFS – file system check
● Yes ... there is something ....
● Reliable?
● Use of alternatives?!
● Backup/restore
– Possible longer outage :-(
– covered in OPROC's :-)
● Snapshots -> new at all :-(


BTRFS – what else
● Support of POSIX ACL's
● Online grow/shrink
● Online add/removal of disks
● SSD-aware
● Management tool evolution (btrfsctl -> btrfs)
● du/df not fully BTRFS-aware


BTRFS – migration paths
● mkfs() and backup/restore
OR
● In place from EXT3/4 via tool btrfs-convert
● Via libe2fs
● BTRFS meta data location flexible
● Old EXT3/4 organized in snapshot
● Roll-back possible to date/time of conversion


BTRFS – migration strategy
● In place via btrfs-convert
● Fast
● EXT3/4 a given in data centre by then
● Change management friendly


BTRFS – migration hints
● New file system – check everything!
● General knowledge/awareness
● /boot – booting from BTRFS possible?
● Rescue media enabled for BTRFS?
● Backup/restore of BTRFS?


BTRFS summary
● Still experimental
● Meets standard file systems requirements
● Bridges existing gaps
● Easy migration from EXT3/4 possible
● New approach to storage management
● Prospects
● SSD
● Compression

Summary
● Improvement moving to EXT4
● Safe switching to EXT4
● In place migration from EXT3 possible

● Future is BTRFS
● In place migration from EXT3/4 to BTRFS
possible


References
● http://ext4.wiki.kernel.org
● http://btrfs.wiki.kernel.org


Thank you!


Linuxconeurope2011.ext4btrfs.talk

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Linuxconeurope2011.ext4btrfs.talk (20)

More from Udo Seidel (11)

Recently uploaded (20)

Linuxconeurope2011.ext4btrfs.talk