Pnfs
Download as PPT, PDF2 likes837 views
NFS v4.0 introduced mandatory strong security, easier firewall deployment, finer access control, and more robust byte-range locking. pNFS extends NFS v4 to allow clients direct access to storage devices for parallel access to files striped across storage. With pNFS, clients obtain layouts from the NFS server mapping files to storage addresses, then perform I/O directly. Layouts can be recalled, data committed, and layouts returned over the control path.
Pnfs
- 1. Introduction to NFS v4 and pNFS PRESENTATION TITLE GOES HERE David Black, SNIA Technical Council, EMC slides by Alan Yoder, NetApp with thanks to Michael Eisler and Brent Welch
- 2. NFS v4.0 Under development from 1998-2005 primarily driven by Sun, Netapp, Hummingbird some University involvement (CITI UMich, CMU) systems beginning to ship available in Linux © 2008 Storage Networking Industry Association. All Rights Reserved.
- 3. NFS v4.0 Mandates strong security be available Every NFSv4 implementation has Kerberos V5 You can use weak authentication if you want Easier to deploy across firewalls (only one port is used) Finer grained access control Goes beyond UNIX owner, group, mode Uses a Windows-like ACL Read-only, read-mostly, or single writer workloads can benefit from formal caching extensions (delegations) Multi-protocol (NFS, CIFS) access experience is cleaner Byte range locking protocol is much more robust Recovery algorithms are simpler, hence more reliable Not a separate protocol as in V3 © 2008 Storage Networking Industry Association. All Rights Reserved.
- 4. NFS v3 and v4 compared NFSv3 NFSv4 A collection of protocols (file access, One protocol to a single port (2049) mount, lock, status) Lease-based state Stateless Supports UNIX and Windows file UNIX-centric, but seen in Windows semantics too Mandates strong authentication Deployed with weak authentication String-based identities 32 bit numeric uids/gids Real caching handshake Ad-hoc caching Windows-like access UNIX permissions Bans UDP Works over UDP, TCP Uses a universal character set for file Needs a-priori agreement on names character sets © 2008 Storage Networking Industry Association. All Rights Reserved.
- 5. pNFS history Idea to use SAN FS architecture for NFS originally from Gary Grider (LANL) and Lee Ward (Sandia) Development driven by Panasas, Netapp, Sun, EMC, IBM, UMich/CITI Folded into NFSv4 minor version NFSv4.1 in 2006 © 2008 Storage Networking Industry Association. All Rights Reserved.
- 6. pNFS Essentially makes clients aware of how a clustered filesystem stripes files Files accessible via pNFS can be accessed via non-parallel NFS (and in the case of filers, CIFS, and other file access protocols) Benefits workloads with many small files very large files Three supported methods of access to data: Blocks (FC, iSCSI) Objects (OSD) Files (NFSv4.1) © 2008 Storage Networking Industry Association. All Rights Reserved.
- 7. pNFS architecture data met ada l ta o pNFS c ontr Block (FC) / Clients Object (OSD) / File (NFS) NFSv4.1 Server Storage © 2008 Storage Networking Industry Association. All Rights Reserved.
- 8. pNFS architecture data met ada l pNFS ta tro Block (FC) / con Clients Object (OSD) / File (NFS) NFSv4.1 Server Storage Only this is covered by the pNFS protocol Client-to-storage data path and server-to-storage control path are specified elsewhere, e.g. SCSI Block Commands (SBC) over Fibre Channel (FC) SCSI Object-based Storage Device (OSD) over iSCSI Network File System (NFS) © 2008 Storage Networking Industry Association. All Rights Reserved.
- 9. pNFS basic operation Client gets a layout from the NFS Server The layout maps the file onto storage devices and addresses The client uses the layout to perform direct I/O to storage At any time the server can recall the layout Client commits changes and returns the layout when it’s done pNFS is optional, the client can always use regular NFSv4 I/O layout Storage Clients NFSv4.1 Server © 2008 Storage Networking Industry Association. All Rights Reserved.
- 10. pNFS protocol operations LAYOUTGET (filehandle, type, byte range) -> type-specific layout LAYOUTRETURN (filehandle, range) -> server can release state about the client LAYOUTCOMMIT (filehandle, byte range, updated attributes, layout-specific info) -> server ensures that data is visible to other clients Timestamps and end-of-file attributes are updated GETDEVICEINFO, GETDEVICELIST Map deviceID in layout to type-specific addressing information © 2008 Storage Networking Industry Association. All Rights Reserved.
- 11. pNFS protocol callbacks CB_LAYOUTRECALL Server tells the client to stop using a layout CB_RECALLABLE_OBJ_AVAIL Delegation available for a file that was not previously available © 2008 Storage Networking Industry Association. All Rights Reserved.
- 12. pNFS read Client: LOOKUP+OPEN NFS Server: returns file handle and state ids Client: LAYOUTGET control path NFS Server: returns layout Client: many parallel READs to storage devices data path Storage devices: return data Client: LAYOUTRETURN NFS server: ack control path Layouts are cacheable for multiple LOOKUP+OPEN instances Server uses CB_LAYOUTRECALL when the layout is no longer valid © 2008 Storage Networking Industry Association. All Rights Reserved.
- 13. pNFS write Client: LOOKUP+OPEN NFS Server: returns file handle and state ids Client: LAYOUTGET NFS Server: returns layout control path Client: many parallel WRITEs to storage devices Storage devices: ack data path Client: LAYOUTCOMMIT NFS server: “publishes” write control path Client: LAYOUTRETURN NFS server: ack Server may restrict byte range of write layout to reduce allocation overheads, avoid quota limits, etc. © 2008 Storage Networking Industry Association. All Rights Reserved.
- 14. What pNFS doesn’t give you Improved cache consistency NFS has open-to-close consistency Perfect POSIX semantics in a distributed file system Clustered metadata Though a mechanism for this is not precluded © 2008 Storage Networking Industry Association. All Rights Reserved.