FILES IN TODAY’S WORLD - #MFSummit2017
1 like340 views
The document outlines the features and improvements of Open Enterprise Server 2018, including enhanced storage capabilities, support for cloud integration, and protocol upgrades. Key advancements include increased throughput for read/write operations, better support for large file systems, and a unified update channel for SLES and OES. Additionally, it highlights future developments in data tiering services with integration to various cloud storage solutions.
![Enhanced MAC user experience using
alternate data streams
Large directory: improved listing performance
SMB2 and above.
Optionally enabled/disabled
SMBv1 can be made disabled for Mac/Windows clients for data format compatibility
Improves application interoperability on Mac (refer bug 950189)
Server-side migration utility
Color code information migration for files created using AFP
AFP Resource fork, FinderInfo -- CIFS Alternate Data Streams (stored as
extended attribute in NSS)
usage : migafp2cifs <volume>:<file-path> [-r | --recursive] [-v | --verbose] [-d | --
deleteAFP] [-i | --iconRename] [-f fileNamePrefix | --fileName=fileNamePrefix]](https://image.slidesharecdn.com/4d52daed2e7ec4f8206e8df496f67cca5f1abf77-170406130650/85/FILES-IN-TODAY-S-WORLD-MFSummit2017-13-320.jpg)
FILES IN TODAY’S WORLD - #MFSummit2017
- 1. David Shepherd Senior Solutions Consultant 9 March 2017 Open Enterprise Server 2018 #MFSummit2017
- 2. The problem…
- 3. • OES 2018 (VEGA) • Release outline • Storage improvements • Protocol improvements • Cloud integration Agenda
- 5. Features Platform and ecosystem currency • Newer hardware platform and technology support – SLES 12 • Update OES to include the updated version of eDirectory – eDirectory 9 Enhancements to SMB stack – relevant capabilities of SMB 2.1 and 3.0 • Improved performance; end-to-end encryption support OES 2018
- 6. Features • Lower TCO • Unified OES update channel – SLES and OES updates from a single repository • NSS improvements – better storage allocation, support for online defragmentation*, exponentially increase the limit on number of files • Adding support for secondary cloud tier for storage of ‘cold’ / back up data* • Additional enhancement from OES community submitted via IDEAS Portal OES 2018
- 8. NSS Delayed Block Allocation Performance improvements • Read • Write • Backup/Restore Sample numbers from test data set • Write throughput improved by 169% • Read throughput improved by 403% • Backup throughput improved by 477%
- 9. NSS support for 64bit zids • zids in the volume to grow more than 32 bits. • No need for Pool Rebuild (Rezid) • NCP server and CIFS server makes use of the new 64bit zid (inode numbers) • Requires Pool Media Upgrade. • 8 Trillion files/folders in a volume
- 11. CIFS / SMB Enhancements • Authentication expiry and dynamic re-authentication • Large MTU support • Parallel reads/writes • Enhanced Mac user experience • Alternate data streams – configuration options • SMB 3.0 – security enhancements • Purge salvage over CIFS – Mac • Password change notification – Windows, Mac
- 12. Large MTU support • Feature in SMB 2.1 – enables better SMB performance for 10-gigabit ethernet networks. • This has been achieved with a feature called ‘Large MTU’, or ‘multi-credit’ operations. • In SMB 2.1 this maximum data unit was increased from 64KB to 1 MB. • Tasks such as copying large files are improved with this capability.
- 13. Enhanced MAC user experience using alternate data streams Large directory: improved listing performance SMB2 and above. Optionally enabled/disabled SMBv1 can be made disabled for Mac/Windows clients for data format compatibility Improves application interoperability on Mac (refer bug 950189) Server-side migration utility Color code information migration for files created using AFP AFP Resource fork, FinderInfo -- CIFS Alternate Data Streams (stored as extended attribute in NSS) usage : migafp2cifs <volume>:<file-path> [-r | --recursive] [-v | --verbose] [-d | -- deleteAFP] [-i | --iconRename] [-f fileNamePrefix | --fileName=fileNamePrefix]
- 14. Server side copy Sequence 1. Client issues read requests to server 2. Server reads file data off disk 3. Server sends data to client 4. Client sends duplicate file data to server 5. Server writes duplicate file data to disk Result 1. File data traverses network and disk 2. Duplicate file data stored on disk Traditional file copy Server Side Copy Sequence 1. Client issues FSCTL_SRV_COPYCHUNK requests to server 2. Server reads file data off disk 3. Server writes duplicate file data to disk Result 1. File data traverses disk only 2. Network round trip avoided 3. Duplicate file data stored on disk
- 15. Data tiering service with Cloud storage support Disclaimer: This section provides an overview of future releases; due to market changes, customer enhancement requests, and other factors, product themes and timings are subject to change.
- 16. High Level Deployment View OES Cloud Tiering Service Amazon S3 Cloud Database Server OES 2018 Server OES 2015 Server Futures
- 17. Key Goals Policy Driven Data Migration •Create/Manage Cloud Accounts, Policies, Pairs, Policy Execution •Data Migration; Isolated Stub Namespace •Multiple Jobs; Multiple Sources (OES) Support For Multiple Object Stores •AWS S3 •Ceph •Minio •S3 Scality End-User Experience •Low latency Merged View •File Re-call Other •CI/CD
- 18. The key pieces of the puzzle DTS backend infra Database Integration Chunking, Revamped REST APIs Control Data Plane Split OES side REST interfaces for CTS communication Tier Pair management, downstream communication OES interfaces for NSS/HSM file recall Upcalls; re-call flow Merged View Policy engine REST APIs Create Pairs List Jobs Policy Driven Execution of jobs Manage Webstack; Workflow View Cloud accounts View Pairs View Jobs Other CI/CD Automation TC Design Docker Encryption
- 19. Summary ‘A network service that manages my data intelligently, reduces TCO and improves flexibility’
- 20. Data tiering service with Cloud storage support = immense potential •Per-object key managementOn-premises encryption • Compression • Global de-duplication Data reduction •Post-processing also WHAT???? In-line analytics, data-aware system •Local index?Search •Data + Meta-dataLocal caches •No on-prem data transferCloud-to-Cloud data movement •S3, Glacier, Nearline, … Moving across different classes of storage within Cloud