The Enterprise File Fabric and IBM COS: S3 Drive and S3 Explorer | Solution Brief
0 likes57 views
This document provides a solution guide for integrating Storage Made Easy's S3 Drive and S3 Explorer applications with IBM Cloud Object Storage, detailing their architecture, configuration, and performance testing results. Both applications enable users to interact with cloud storage as local disks, supporting various file operations, while the certification process validates functionality and performance. Performance metrics indicate differences in upload and download speeds between on-premise and public cloud instances.
The Enterprise File Fabric and IBM COS: S3 Drive and S3 Explorer | Solution Brief
- 1. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 1 SOLUTION GUIDE IBM COS - STORAGE MADE EASY: S3 DRIVE & S3 EXPLORER INTRODUCTION This document outlines the solution architecture, configuration, functionality and performance characteristics of Storage Made Easy™ (SME) direct apps with IBM Cloud Object Storage (COS). The direct or “standalone” desktop applications, S3 Drive and S3 Explorer, work as independent desktop clients directly connecting IBM Cloud Object Storage. Configuration information was verified and performance information collected during a certification process at the IBM Cloud Object Storage Alliance Integration Team certification lab. A separate guide provides information on the full stack Storage Made Easy Enterprise File Fabric™ product. PRODUCTS TESTED Storage Made Easy software: • Storage Made Easy S3 Drive 3.5.0 • Storage Made Easy S3 Explorer 3.5.0 IBM software: • IBM Cloud Object Storage – On Premise (version 3.10.2.72) • IBM Cloud Object Storage – US South (version 3.11.0.89) • IBM Cloud Object Storage – US Region (version 3.11.0.89) S3 DRIVES The S3 Drive enables users to mount and interact with IBM Cloud Object Storage as a local disk. Users can browse and open remote files with desktop applications including the Microsoft Office Suite. The Windows File Explorer can be used to copy, move and rename objects as if they were files and folders. The S3 Drive can also be accessed through Windows batch scripts as if it was a regular local drive. S3 EXPLORERS The S3 Explorer desktop application allows users to interact with IBM Cloud Object Storage through a familiar file explorer like interface. From the menu, or using drag and drop, users can easily upload, download and delete files and folders. Copying, moving and renaming objects and folders is also supported. Objects can be quickly located through search, and sensitive data automatically encrypted on upload.
- 2. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 2 CERTIFICATION The certification exercise included installing products and executing functional and performance tests with IBM Cloud Object Storage - on premise and public cloud. Functional tests centered around uploading, downloading, copying and deleting files and folders. App specific features such as rename and searching were also tested. Performance testing focused on throughput supported by a single app simulating heavy usage by a single end user. Results were compared between on premise and cloud instances of IBM Cloud Object Storage. SOLUTION ARCHITECTURE The S3 Drive and S3 Explorer were deployed to a Windows 10 virtual machine. Other than endpoints, configuration for the different IBM COS instances was the same. IBM COS Public Cloud For testing with IBM COS in the public cloud over the Internet the apps were configured for an S3 compatible endpoint over HTTPS. IBM COS On-Premises An on premise deployment of IBM COS offers multiple endpoints for high availability. To allow the apps to use both endpoints a load balancer was used. An Nginx load balancer was used during certification but any open source or commercial load balancer (supporting HTTPS) would be appropriate.
- 3. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 3 DEPLOYMENT Client The S3 Drive application was installed on two client machines: • O/S: Microsoft Windows 10 Pro Standard Install • RAM: 16GB • CPU: 8 • HDD: 500GB (SSD) Other software installed: • Microsoft Office 2016 - used for functional testing • Fiddler 4 - used for identifying HTTP calls to IBM COS and for timing tests • Python - used for scripting operating system calls • Boto 3 - used for scripting S3 calls in Python Load Balancer An NGINX load balancer was used for IM COS On-Premise. STORAGE MADE EASY S3 DRIVE The S3 Drive enables users to mount and interact with IBM Cloud Object Storage as a local disk. For configuration information see Appendix A. FUNCTIONAL TESTING The objective of S3 Drive functional testing was to: • Verify compatibility with IBM Cloud Object Storage for both on premise and public cloud storage. • Map S3 Drive user interactions to one or more IBM COS HTTP operations for capacity planning and budgeting. • Validate and document any recommendations for optimum performance. The S3 Drive tests were performed manually through the Windows 10 File Explorer and with Microsoft Office applications. The S3 Drive Cache was cleared before each test (Settings > Browse > Delete all). S3 Drive Operations Create Root Folder – creates bucket Browse Folder Delete Root Folder – removes bucket Upload Folder Browse root-level folders – list all buckets Download Folder Create Subfolder Copy folder Remove Subfolder - hierarchy (with files) Rename File Upload File Rename Folder Download File Refresh (access from tray icon) Word - Save new document Excel - Save new document Word - Edit document Excel - Edit document
- 4. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 4 PERFORMANCE TESTING The objective of S3 Drive performance testing was to: • Calculate throughput to assist with recommendations of appropriate use cases • Validate and document any recommendations for optimum performance. A single workload was creating consisting of 161 files in 17 folders, ranging in size from 1MiB to 1GiB. Total workload size was 10.4 GiB (11,161,589,120 bytes). The S3 Drive Cache was cleared before each test (Settings > Browse > Delete all). Transfer and *Effective Transfer rates for S3 Drive Rename and Delete operations were executed without data transfer between the S3 Explorer and IBM COS. Observations: • Uploads are multi-threaded. Other operations are single-threaded. S3 Drive Mixed Workload MiB/sec Test On Prem US South US Region Upload 110 48 27 Download 20 15 13 Rename* 104 45 34 Delete* 2,129 560 138 * Effective speed (no data transferred) STORAGE MADE EASY S3 DRIVE
- 5. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 5 STORAGE MADE EASY S3 EXPLORER The S3 Explorer desktop application allows users to interact with IBM Cloud Object Storage through a familiar file explorer like interface. For configuration information see Appendix B. FUNCTIONAL TESTING The objective of S3 Explorer functional testing was to: • Verify compatibility with IBM Cloud Object Storage for both on premise and public cloud storage. • Map S3 Explorer user interactions to one or more IBM COS HTTP operations for capacity planning and budgeting. • Validate and document any recommendations for optimum performance. Testing was successful against IBM COS - both on premise and public cloud storage. Features Create Root Folder - creates bucket Copy file Delete Root Folder – removes bucket Copy folder View Root items (buckets) Move file Create Subfolder Move folder Remove Subfolder - file hierarchy Rename File Upload File Rename Folder Download File Bucket Search Browse Folder Sort files (name, modified, size) Upload Folder Properties (root folder or file only) Download Folder - file hierarchy
- 6. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 6 PERFORMANCE TESTING 161 files, 17 folders, ranging in size from 1MB to 1GB. Total 10.4 GiB (11,161,589,120 bytes). Transfer and *Effective Transfer rates for S3 Explorer Copy, Rename and Delete operations were executed without data transfer between the S3 Explorer and IBM COS. S3 Explorer Mixed Workload MiB/sec MiB/sec On Prem US South US Region Upload 46 23 12 Download 59 22 21 Copy* 110 50 37 Rename* 108 45 34 Delete* 2,661 484 201 * Effective speed (no data transferred) STORAGE MADE EASY S3 EXPLORER
- 7. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 7 APPENDIX A - S3 DRIVE CONFIGURATION To connect to cloud storage the S3 Drive requires: • Endpoint - HTTPS was used for testing. HTTP is also supported. • Access Key • Secret Key Default installation settings were used:
- 8. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 8 APPENDIX B - S3 EXPLORER CONFIGURATION To connect to cloud storage the S3 Drive requires: • Endpoint - HTTPS was used for testing. HTTP also supported. • Access Key • Secret Key All other settings were defaults after installation.
- 9. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 9 APPENDIX C – MAPPING OF S3 DRIVE FUNCTIONS TO IBM COS OPERATIONS S3 Drive Operation IBM COS Operations Create Root Folder – creates bucket PUT Delete Root Folder – removes bucket GET (bucket) GET (contents of bucket to possibly delete) DELETE (bucket) Browse root-level folders – list all buckets GET Create Subfolder PUT Remove Subfolder - hierarchy (with files) GET For each subfolder: DELETE HEAD /<subfolder>/ HEAD /<subfolder> For each file: DELETE GET /<bucket> Upload File PUT Download File For each file: GET HEAD Browse Folder HEAD Upload Folder PUT Download Folder For each file: GET HEAD Copy folder PUT folder GET folder For each file: GET PUT (x-amz-copy-source) Rename File GET PUT DELETE Rename Folder For each file: GET PUT (x-amz-copy-source) DELETE Refresh (access from tray icon) GET
- 10. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 10 APPENDIX C – MAPPING OF S3 DRIVE FUNCTIONS TO IBM COS OPERATIONS Feature Notes Word - Save new document HEAD PUT Word - Edit document HEAD - if in cache, is it out of date? GET - if not in cache, or cache out of date PUT HEAD Excel - Save new document HEAD HEAD PUT HEAD Excel - Edit document HEAD - if in cache, is it out of date? GET - if not in cache, or cache out of date PUT HEAD
- 11. SOLUTION GUIDE | © Copyright Storage Made Easy® 2019 11 APPENDIX D–MAPPING OF S3 EXPLORER FUNCTIONS TO IBM COS OPERATIONS Feature Notes Create Root Folder - creates bucket PUT Delete Root Folder – removes bucket GET (bucket) GET (contents of bucket to possibly delete) DELETE (bucket) GET / (refresh explorer listing) View Root items (buckets) GET Create Subfolder PUT Remove Subfolder - file hierarchy GET /<bucket> Upload File PUT GET (refresh parent folder) Download File GET Browse Folder GET (contents cached for period of time) Upload Folder GET (refresh parent) Download Folder - file hierarchy GET Copy file PUT (x-amz-copy-source) Copy folder PUT (x-amz-copy-source) Move file GET folder For each file: PUT (x-amz-copy-source) DELETE Move folder GET folder For each file: PUT (x-amz-copy-source) DELETE DELETE folder Rename File PUT (x-amz-copy-source) DELETE Rename Folder GET folder For each file: PUT (x-amz-copy-source) DELETE Bucket Search GET (for each 1,000 objects) Sort files (name, modified, size) None (client-side only) Properties (root folder or file only) HEAD
- 12. ABOUT STORAGE MADE EASY® The Storage Made Easy Enterprise File Fabric™ enables IT to regain control of "cloud data sprawl" to unify on-premises and on-cloud files into a single easily managed converged infrastructure with policy-based controls for governance, audit and security. The File Fabric solution provides a “blanket” enterprises wrap around all their data: on premises, within public and private clouds, as well as on third-party software vendor clouds (SharePoint Online and Salesforce for example). Customers use the SME File Fabric product to unify corporate data to enable workgroup collaboration with universal controls for security, encryption, audit, and data access. Companies will have access to a greater understanding of where and how sensitive data is stored, transferred, and accessible with particular focus on how companies are able to police and audit data to ensure enforcement of the General Data Protection Regulations (GDPR) in addition to other regional data legislation. USA +1 415 477 1053 UK +44 20 8212 7316 EU +41 435 080 078 sales@storagemadeeasy.com Free hosted and enterprise trials available from www.storagemadeeasy.com @SMEStorage /StorageMadeEasy /company/storage-made-easy