Wafl overview
- 2. WAFL: Write Anywhere File Layout Filesystem for Improved Productivity Berkeley Fast File System/Veritas File System/NTFS/etc. – Writes to pre-allocated locations (data vs. metadata) ... WAFL – No pre-allocated locations (data and metadata blocks are treated equally). Writes go to nearest available free block. 1-2 MB Cylinders ... Writing to nearest available free block reduces disk seeking (the #1 performance challenge when using disks). 2 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 3. WAFL uses integrated RAID4 RAID4 is similar to better known RAID5: – RAID5: parity is distributed across all disks in the RAID group – RAID4: parity is contained in a single disk in the RAID group Tradeoffs with the single parity disk RAID model: – PRO: The RAID group can be instantly expanded by adding (pre-formatted) data disks. – CON: The parity disk is perceived to be the ‘hot spot’ in the RAID group, due to intensive XOR parity calculations on it. 3 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 4. WAFL eliminates the parity hot spot WAFL overcomes the ‘classic’ parity-disk hotspot issue, by the use of flexible write allocation policies: – Writes any filesystem block to any disk location (data and meta data)* – New data does not overwrite old data – Allocates disk space for many client-write operations at once in a single new RAID-stripe write (no parity re-calculations) – Writes to stripes that are near each other – Writes blocks to disk in any order * except root inode 4 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 5. Result: Minimal seeks and no hotspot Typical File System WAFL Long file1 head Short seeks head especially file1 file2 seeks on file3 across file3 parity all disk disks file2 1 file at a time Multiple files at once 5 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 6. WAFL Combined with NVRAM WAFL uses NVRAM “consistency points” (NetApp’s flavor of journalling), thus assuring filesystem integrity and fast reboots. CP flush to disk occurs once every 10 seconds or when NVRAM reaches half full. NVRAM placement is at the file system operation level, not at the (more typical) block level. This assures self- consistent CP flushes to disk. No fsck! 6 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 7. NVRAM placement is key! General-purpose NetApp NV-RAM NV-RAM TC P / o r TC P / o r File System U D P /I P U D P /I P NFS or NFS or C IF S C IF S S e ma nti File S e ma nti N VR A M System c c W r it e W r it e D Asl lk c i o N VR A M D Asl lk c i o D r iv e r D r iv e r NVRAM safe-stores NVRAM safe-stores the disk blocks the FS operation 7 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 8. Seek Example in a SAN environment Assume 4K disk blocks and 5 msec for one seek+rotate 100MB/sec FC bandwidth x .005sec = .5MB worth of data blocks not sent on the FC channel during that seek .5MB x 1 block/4KB = 128 blocks not sent Therefore a 5ms seek for just 1 block equates to a 128 block penalty Conclusion: one seek every 128 blocks or less ( ~1%) wastes at least half of your FC bandwidth! (seek 1 block) 128 blocks (seek 1 block) 128 blocks 8 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 9. The protocol overhead issue Isn’t NAS slower than local disk? • Yes, we have TCP/IP overhead. • Yes, we have double-buffering overhead. • Yes, we might well have <obscure performance gotcha>. • Despite all that, we're able to improve performance, even with databases (now over 40% of NetApp customer base). • Clearly, we're doing *something* sufficiently right to make up for the overhead. 9 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 10. The protocol overhead issue Keep the timing in perspective with today’s CPU speeds! • TCP/IP might seem to be a massive overhead, but passing packets up and down the stack turn out only to consume microseconds per request. (For example: 1Ghz CPU speed == 1 nanosecond clock cycle. So 1000 extra ____second clock cycle. CPU cycles for TCP stack = 1000x1ns = 1 microsecond) • Eliminating head seeks, which WAFL does better than any other file system thanks to its full integration with RAID, TCP over saves whole milliseconds, eg, a 1000x savings. head (5ms seek) 128 blocks (5ms seek) 128 blocks TCP overhead is small by comparison 10 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 11. NetApp Filers High speed, low latency SFS97_R1 Performance - NFS v3 TCP http://www.spec.org/cgi-bin/osgresults?conf=sfs97r1 • RAID protected • Single file system F825 F880 FAS940 FAS960 FAS960c 11 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
- 12. Summary • WAFL extracts more ops/sec from a single drive due to minimum seeks. • More ops/sec equates to faster overall performance • WAFL’s “anywhere” property makes NetApp’s RAID-4 the performance and scalability winner. • Fastest File System in the world with RAID enabled 12 July 05 © Network Appliance 2005 - Redistribution outside of an authorized NetApp distributor or reseller to third parties prohibited withou
Editor's Notes
- #3: March 19, 2012 Earlier we talked about disk capacity increasing but disk access times not. NetApp’s patented file system is called WAFL which stands for Write Anywhere Layout. WAFL always writes to the nearest available free block as opposed to a preallocated location on disk…If you look at conventional file systems, such as the Veritas Fast File System, NTFS, or the Berkeley Fast File System which is used in Solaris and HP/UX, they all write to pre-allocated locations on disk meaning that lot’s of disk seeking must occur. With WAFL we write out a stripe and hopefully not even move the heads to write the next stripe (if free space in the same cylinder). If not, it’s hopefully just one head click away. Quite simply, we minimize head seeking as much as possible.
- #4: RAID-3 typically uses a very small stripe width, sometimes as small as one byte per disk. The result: RAID-3 accesses all the disk in the group at one time, and can only execute one I/O request at a time. With RAID4, access to each disk becomes independent. The stripe size is sufficiently large that the majority of I/Os to the group will only affect a single disk. This allows the RAID-4 group to execute multiple I/O requests simultaneously (assuming they map to different member disks).
- #5: RAID-3 typically uses a very small stripe width, sometimes as small as one byte per disk. The result: RAID-3 accesses all the disk in the group at one time, and can only execute one I/O request at a time. With RAID4, access to each disk becomes independent. The stripe size is sufficiently large that the majority of I/Os to the group will only affect a single disk. This allows the RAID-4 group to execute multiple I/O requests simultaneously (assuming they map to different member disks).
- #6: March 19, 2012 15 92 48 Berkeley Fast File System (FFS) Assigns blocks to fixed disk locations, as physically close together as possible on a single disk, optimized for single-file-at-time access Apply it to NFS and the disk heads fly about madly Write Anywhere File Layout (WAFL) Writes blocks anywhere it finds convenient, close to the disk heads’ current positions The previous version of a changed block is not over-written (it’s either retained or marked free) WAFL then logically threads a single file’s current blocks by updating the block pointers – it’s easy to adjust the pointers in the “inode” The result: reduced disk seek/latency time* Figure 1 from http://www.netapp.com/tech_library/3002.html A tree of blocks
- #8: March 19, 2012
- #10: March 19, 2012 9 86 42 Unlike general purpose file systems, WAFL has an intimate understanding of its underlying physical disk configuration. WAFL caches write operations that come in from the network, and then optimizes by performing multiple write operations all together within the same RAID array stripe. The stripe is chosen based on its physical proximity to the location of the disk heads at the time of the operation. This behavior ensures that the single parity disk does not become a bottleneck within the system as it would typically do with a general purpose file system. It also allows WAFL to achieve excellent write performance, since the disk heads never have to seek very far to write client data. Fragmentation is also not a significant issue with WAFL, as data belonging to the same file is always written to adjacent locations within the stripe.
- #11: March 19, 2012 9 86 42 Unlike general purpose file systems, WAFL has an intimate understanding of its underlying physical disk configuration. WAFL caches write operations that come in from the network, and then optimizes by performing multiple write operations all together within the same RAID array stripe. The stripe is chosen based on its physical proximity to the location of the disk heads at the time of the operation. This behavior ensures that the single parity disk does not become a bottleneck within the system as it would typically do with a general purpose file system. It also allows WAFL to achieve excellent write performance, since the disk heads never have to seek very far to write client data. Fragmentation is also not a significant issue with WAFL, as data belonging to the same file is always written to adjacent locations within the stripe.