IBM Spectrum Scale Secure- Secure Data in Motion and Rest
Download as PPTX, PDF2 likes2,518 views
The document discusses the IBM Spectrum Scale's capabilities for securing data both at rest and in motion. It details features such as file encryption policies, secure deletion, and the use of remote key management, as well as data encryption during transfer across nodes in a cluster using TLS and Kerberos authentication. Additionally, it provides configuration guidelines and FAQs addressing common security concerns related to data management and encryption protocols.
IBM Spectrum Scale Secure- Secure Data in Motion and Rest
- 1. HPCXXL 2018 IBM Spectrum Scale Securing Data at Rest and Data in Motion Acknowledgement: Felipe Knop , Truong Vu, Christof Schmitt, Yong ZY Zheng, Christopher Maestas - Sandeep Patil
- 2. Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
- 3. Spectrum Scale: Breath of new features Catering to newer workloads Block iSCSI Client workstations Users and applications Compute farm Traditional applications Global Namespace Analytics Transparent HDFS Spark OpenStack Cinder Glance Manilla Object Swift S3 Transparent Cloud Powered by IBM Spectrum Scale Automated data placement and data migration Disk Tape Shared Nothing ClusterFlash New Gen applications Transparent Cloud Tier Worldwide Data Distribution(AFM) Site B Site A Site C SMBNFS POSIX File Encryption DR Site AFM-DR JBOD/JBOF Spectrum Scale RAID Compression Consolidate all your unstructured data storage on Spectrum Scale with unlimited and painless scaling of capacity and performance – ENSURING DATA SECURITY !
- 4. Protocol Nodes Protocol Nodes NSD Server NSD Server ObjectSMB NFS ObjectSMB NFS Keystone Postgres Keystone Active Directory Clients ClientsClients Secure Data at Rest Secure Erase External Key Mgmt. sudo based admin access Object ExpirationFile/Object ACL Log admin commands Immutability support Network Secure Data in Transit Kerberos SSL/TLS Firewall IPS Support sudo based admin access Secure inter-cluster communication GPFS Node(s) Hadoop Connector Hadoop client nodes Kerberos Spectrum Scale High Level Security Outlook Rich policy support for data placement as well as data isolation Secure Data in Transit SSL/TLS LDAP Directory Services GUI SSL/TLS RBAC Transparent Cloud Tier Secure Data in Transit
- 5. What we are going to cover today ? Secure Data at Rest Secure Data in Motion
- 6. Spectrum Scale Secure Data at Rest : At-A-Glance Spectrum Scale Filesystem Fileset fs 2Fileset fs1 Encryption policy rules: - which files are to be encrypted, - with which algorithm, - using which MEKs. Example encryption policy rules RULE 'myEncRule1' ENCRYPTION 'E1' IS ALGO 'DEFAULTNISTSP800131A' KEYS('1:RKM_1', '2:RKM_2’) RULE 'Encrypt files with extension doc with rule E1' SET ENCRYPTION 'E1' FOR FILESET('fs1') WHERE NAME LIKE '%.doc' External Key Manager Server (IBM SKLM or Vormetric DSM Key Server) Block iSCSI Analytics Transparent HDFS Spark OpenStack Cinder Glance Manilla Object Swift S3 Transparent Cloud SMBNFS POSIX File *.doc *.doc *.doc Encryption of Data at Rest • Files are encrypted before they are stored on disk • Master Keys are never written to disk • No data leakage in case disks are stolen or improperly decommissioned Secure Deletion • Ability to destroy files with no data remanence • No “digital shredding” secure delete is cryptographic operator *.txt *.txt *.txt Encrypted *.doc Not Encrypted *.txt NIST & FIPS • The encryption algorithms that are used for file encryption are all compliant with NIST Special Publication 800-131A. • Allows cluster to be configured in FIPS mode
- 7. Spectrum Scale Secure Data At Rest- Key-based encryption Master Encryption Key (MEK) • Used to encrypt file encryption keys • Stored in Remote Key Management (RKM) Servers • MEK’s have a unique key name that combines the name of the key and the RKM server where it resides File Encryption Key (FEK) • Used to encrypt sectors of an individual file • Unique key randomly generated • Encrypted (or “wrapped”) with one or more MEK’s and stored in the extended attribute on filesystem. • FEK must have access to MEK to be decoded • FEK can be re-wrapped to new MEK(s) in the case of a compromised key 7 FEK MEK wrap Encryptxattr Stored External Key Manager Server (IBM SKLM or Vormetric DSM Key Server) Stored
- 8. Spectrum Scale Secure Data At Rest: Native Filesystem encryption • Files encrypted before I/O submission • Encryption takes place on the node(s) from which the user drives the I/O • File content travels encrypted to the NSD server • MEKs can be accessed by nodes that have appropriate RKM credentials • Nodes that cannot access keys cannot access files, irrespective of file permissions • Granularity is per file or per fileset, as determined by encryption policies 8 Application Spectrum Scale Node(s) Spectrum Scale NSD Server / Block Device Disk Drive Data flows Encrypted
- 9. How to Enable File System Encryption ? • Encryption Policies • Manage how files are encrypted and includes the following: • Which files are to be encrypted • Which algorithm is to be used for encryption • Which MEK (or MEK’s) are to be used to wrap the FEK of a file • The mmchpolicy command is used to configure encryption and is applied at file creation time. • When a file is created, encryption rules are executed. If the file matches at least one SET ENCRYPTION rule, an FEK is generated and used to encrypt the contents of the file. • The FEK is then wrapped using the stated encryption rule using the MEK and stored in the file’s extended attribute. 9
- 10. Example Spectrum Scale Filesystem Fileset fs 2Fileset fs1 Example encryption policy rules RULE 'myEncRule1' ENCRYPTION 'E1' IS ALGO 'DEFAULTNISTSP800131A' KEYS('1:RKM_1', '2:RKM_2’) RULE 'Encrypt files with extension doc with rule E1' SET ENCRYPTION 'E1' FOR FILESET('fs1') WHERE NAME LIKE '%.doc' External Key Manager Server (IBM SKLM or Vormetric DSM Key Server) Block iSCSI Analytics Transparent HDFS Spark OpenStack Cinder Glance Manilla Object Swift S3 Transparent Cloud SMBNFS POSIX File *.doc *.doc *.doc *.txt *.txt *.txt Encrypted *.doc Not Encrypted *.txt ‘DEFAULTNISTSP800131A’ is a shortcut for “256-bit FEK, AES in XTS mode, and preprocessing the FEK by using HMAC with SHA-512.” AND “FEK is wrapped with an AES key wrap, with keys 1:RKM_1 and 2:RKM_2 combined during one round of XOR followed by one round of HMAC with SHA-512”
- 11. FAQ – Spectrum Scale Filesystem Encryption 1. Should all nodes have access to the Remote Key Manager? Yes , all nodes that may access the FS, including nodes with management roles like NSD servers, POSIX clients FS managers, etc 2. Does data travel in encrypted form from NSD servers to NSD clients (intra-cluster) when Filesystem is configured for encryption ? Yes, but only the ‘data’ and not the intra-cluster RPC. 3. Can I have selective encryption for a given set of files ? Yes, as far as you can write policies to determine those files. 4. Can I encrypt existing set of files which were present before the encryption was enabled ? No. You need to copy the files into newly-created encrypted counterparts, possibly using a migration policy. 5. Where are the Encrypted FEK stored ? Encrypted FEK are stored in the ‘gpfs.Encryption’ extended attribute of the file. 6. How do I find out if a given file is Encrypted ? Run ‘mmlsattr -n gpfs.Encryption /gpfs1/file-name’ 11
- 12. Spectrum Scale nodes Spectrum Scale : Secure Data at Motion • Data in transit, also referred as Data in Motion or Data in Flight, is data that is being accessed over a network (internal or external) and can therefore be intercepted by malicious users on the network. • Based on your business needs or on the sensitivity of your data that is being accessed over the network, one needs to protect it by encryption over the wire. Spectrum Scale nodes EMPTY (no-auth, no-sign, no-privacy) AUTHONLY* (auth) CIPHER (auth+sign+privacy)) Spectrum Scale Secure Cluster Communication AUTHONLY* perform TLS authentication, but then all RPC traffic goes in the clear.
- 13. 13 Creating the Cluster (mmcrcluster) Public & Private Keys for the entire cluster are created and distributed to all nodes. For secure data in motion for cluster: Enable Spectrum Scale RPC encryption cluster wide mmchconfig cipherList=<Supported Cipher > AUTHONLY is the default cipherList. All RPC across the cluster are now authenticated using TLS ALL GPFS RPC within the cluster are now encrypted using the selected Cipher using TLS Notes: • Use mmauth command to generate new set of Keys for the cluster. • The keys are located in /var/mmfs/ssl/ • The authentication , integrity check and encryption is based on TLS mechanism. • Expiration of the keys can be monitored by the admins using openssl commands. • Supported Ciphers • AES128-SHA • AES128-SHA256 • AES256-SHA • AES256-SHA256 Configuration Flow for Secure Data in Motion Between Nodes Within a Cluster
- 14. Configuration Flow for Secure Data in Motion Between Cluster 14 Generate Keys : mmauth genkey new Enforce Encryption: mmauth update . –l AES128-SHA256 Add the Remote Cluster that will access the filesystem and its Public Key mmauth add cluster2 -k cluster2_id_rsa.pub Grant the remote cluster secure access to a filesystem mmauth grant cluster2 -f gpfs0 CLUSTER 1 CLUSTER 2 (Remote) Public key of cluster 2 given to admin of cluster 1 /var/mmfs/ssl/id_rsa.pub Public key of cluster 1 given to admin of cluster 2 /var/mmfs/ssl/id_rsa.pub Add the cluster to be accessed along with its key and nodes used to access. mmremotecluster add cluster1 -n node1,node2,node3 -k cluster1_id_rsa.pub Add the remote filesystem that will be accessed by the cluster mmremotefs add mygpfs -f gpfs0 -C cluster1 -T /mygpfs Followed by mmmount mygpfs Generate Keys : mmauth genkey new Enforce Encryption: mmauth update . –l AES128-SHA256
- 15. FAQ - Secure Data in Motion For Cluster Communication 1. What are the supported Ciphers Type by Spectrum Scale for cluster communication? ‘mmauth show ciphers’ list all the support Ciphers. 2. How do I determine which encryption Algorithm is being used for secure cluster communication ? For intra-cluster communication: It is the value set via ‘mmchconfig CipherList ‘and can be viewed via ‘mmlsconfig cipherList’ or even via ‘mmauth show all’ (look for "Cipher list" the local cluster entry) For inter-cluster communication: ‘mmauth show all’ shows the encryption algorithm being used with remote cluster 3. How to Change the Encryption Algo being used for cluster communication ? For intra-cluster communication: ‘mmchconfig cipherlist=security_mode’ For inter-cluster communication: Update the supported Cipher type that you want to mandate the the remote cluster to use for communication . Setting takes effect for newer cpnnections. Example: Mandate remote cluster (cluster 2) with AES128-SHA for communication run ‘mmauth update cluster2 -l AES128-SHA’ on local cluster. 4. How do I change the cluster certificates of my cluster ? Use ‘mmauth’ command . Check Knowledge Center for details. 15
- 16. 5. How do I check if my certificates are going to expire ? Use openSSL commands to check your cluster certificates located in /var/mmfs/ssl ‘openssl x509 -in id_rsa_committed.cert -noout –dates’ 6. Are default certificate for a cluster created when cluster is created ? Yes, from IBM Spectrum Scale V4.2 or later mmcrcluster creates the certificates for the cluster and are propagated to all the nodes. The nodes make use of these certificate for secure communication which is based on TLS with AUTHONLY as the default security mode. 7. For inter cluster communication, can I mandate specific Ciphers to communicate with other clusters ? Yes. You can choose what kind of secure communication you want to have with a given cluster. Need to use mmauth command. Check Knowledge Center for examples. 8. If I want to use NIST complaint encryption algorithm , which Algorithms should I set to the CipherList ? ‘mmauth show ciphers’ , should show you the list which includes NIST complaint encryption algorithms. 9. What is the significance of mmchconfig nistCompliance=SP800-131A ? Setting nistCompliance governs what values to ‘CipherList’ can be set/reset using mmchconfig command. When set to SP800-131A only NIST complaint algorithms are allowed to be set. 16 FAQ - Secure Data in Motion For Cluster Communication
- 17. Spectrum Scale : Secure Data at Motion for Protocols Secure access of Spectrum Scale File Interfaces SMB SMB Client Automatic encryption Mandatory encryption Disabled encryption Secured dialect negotiation Improved signing Secured transmission NFS NFS Client Kerberos V5 (auth) Kerberos V5 (auth+integrity) Kerberos V5 (auth+integrity+privacy) Enabling squashing Using Kerberos Enabling port security Object ClientObject (S3/SWIFT) SSL/TLS HAProxy
- 18. Secure Data at Flight for NFS, SMB based on Kerberos 18 SOURCE: https://www.ibm.com/developerworks/ibmi/library/i-sso/index.html • NFS , SMB (Samba) is just another service to Kerberos. • NFS makes use of Kerberos via GSS-API to provide authentication, integrity check and per-message encryption. • Enabling per message encryption for NFS / SMB shares ensures secure data in flight for NFS and SMB access. Source: http://csis.pace.edu/~marchese/CS865/Lectures/Chap11/Chapter11.htm
- 19. Configuration– Kerberized NFS 19 NFS Configuration Steps • Ensure Prerequisites for configuring Kerberos based NFS access are in place. • NFS server configuration file changes to reflect Kerberos realm names • NFS client configuration changes to reflect Kerberos realm name and start of NFS secure Linux services • Keytab file creation in case of LDAP+MIT KDC and transfer to protocol nodes • Configuring appropriate Kerberos auth using mmuserauth based on backend directory server. • Detail Prerequsite mention in Knowledge Center - https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/b1ladm_prereqforkerbnfsaccess.htm • ‘krb5p’ SECTYPE Clientoptions need to be used with mmnfs export add | change for NFS data encryption for that NFS export mmnfs export add /ibm/gpfs0/fset1 –client “*(Access_Type=RW,Squash=no_root_squash,SecType=krb5p)” On RHEL Linux Client: mount -t nfs –o vers=4,sec=krb5p ckcluster.ad.com:/ibm/gpfs0/fset1 /mnt/4k5p -vvvv
- 20. Secure Data in Flight for SMB 20 • Spectrum Scale supports encryption for the SMB3 protocol • Only requirement is to use a client version that supports SMB3 (e.g. Windows 8, Windows Server 2012 or newer) • After successful authentication, encryption secrets are derived from the exchange information and can be used to encrypt protocol traffic. • The encryption algorithm also guarantees integrity, so it equivalent to encryption and signing. • The complete traffic for a SMB connection can be encrypted or only the traffic to certain SMB shares. • Encryption leverages CPU crypto hardware acceleration
- 21. Configuration– Secure Data in Flight for SMB 21 SMB Configuration Steps • ‘smb encrypt’ option need to be used with mmsmb export add | change for SMB data encryption (can be specified per SMB share or globally) • Values: auto - SMB encryption is offered, but not enforced. mandatory -SMB encryption is required disabled - SMB encryption can not be negotiated. Example: • mmsmb export add myExport "/ibm/gpfs0/myFolder” –option “smb encrypt = mandatory” • Note: SMB share must be mounted using Netbiois name).
- 22. Network Trace : Show Secure Data in Flight (SMB) The best illustration for secure data in rest is a network trace: • In this example we have the complete SMB connection encrypted (smb encrypt = mandatory in the global config). You see here that the connection is established, features are negotiated and authentication is done through the SESSION_SETUP exchanges. Everything following that is encrypted: 22
- 23. Spectrum Scale : Secure Data at Motion Secure access of Spectrum Scale Hadoop Connector Spectrum Scale (Hadoop Transparency Connector = Name Node+ Data Node) HDFS Client (Kerberos V5 auth only) Kerberos V5 (auth+integrity) Kerberos V5 (auth+integrity+privacy) Kerberos HTTP SPNEGO webhdfs dfs.encrypt.data.transfer = true for data encryption of Block data transfer hadoop.rpc.protection=authentication|integrity|pri vacy for secure data transfer between hadoop services & clients Between Hadoop Client and Services (NameNode) • Uses Java SASL (Simple Authentication & Security Layer) API • On both set of nodes Set hadoop.rpc.protection in core-site.xml to: authentication (default) integrity (includes authentication) privacy (includes integrity + authentication) Between HDFS Client and DataNode • On both set of nodes set following in hdfs-site.xml dfs.encrypt.data.transfer = true dfs.encrypt.data.transfer.algorithm = 3des (default) OR rc4 To use AES: dfs.encrypt.data.transfer.cipher.suites = AES/CTR/NoPadding dfs.encrypt.data.transfer.cipher.key.bitlength = 128 (default) OR 192 OR 256
- 24. Spectrum Scale : Secure Management Applications Spectrum Scale REST API (management) HTTPS Admin BrowserSpectrum Scale (Management GUI) HTTPS Secure access of Spectrum Scale Management Spectrum Scale CLI (management) Spectrum Scale Nodes ssh Administration of Spectrum Scale requires Remote Shell and Remote Copy (SSH and scp are default and recommended)
- 25. • Redpaper (refresh) – Released Sept 2018 http://www.redbooks.ibm.com/abstracts/redp5426.html?Open • Security Blogs by Developers: https://developer.ibm.com/storage • Enhanced Knowledge Center with all details. • Spectrum Scale Immutability whitepaper: http://www- 03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102620 • For Kerberized NFS with LDAP-Kerberized authentication scheme: https://www- 03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD106395 • For Kerberized NFS with AD-Kerberized authentication scheme: https://crk0acrk0a.wordpress.com/2016/07/19/ibm-spectrum-scale- kerberized-nfs-with-ad-rfc2307/ 25 IBM Spectrum Scale : Security Redpaper & Blogs
- 27. Ciphers Supported by Spectrum Scale tscomm layer c10c1apv7:~ # mmauth show ciphers Supported ciphers for nistCompliance=SP800-131A: AES128-GCM-SHA256 AES128-SHA AES128-SHA256 AES256-GCM-SHA384 AES256-SHA AES256-SHA256 DES-CBC3-SHA Supported ciphers for nistCompliance=off: AES128-SHA AES256-SHA DES-CBC-SHA DES-CBC3-SHA EXP-RC2-CBC-MD5 EXP-RC4-MD5 RC4-MD5 RC4-SHA Supported ciphers for both environments: AES128-SHA AES256-SHA 27 For NIST Compliance mode use the following: mmchconfig nistCompliance=SP800-131A • The nistCompliance variable applies to security transport (tscomm security, key retrieval) only, not to encryption, which always uses NIST-compliant mechanisms. • SP800-131A Specifies that security transport is to follow the NIST SP800-131A recommendations. For clusters at the GPFS 4.1 level or higher, this is the default.
- 28. Secure erase : Overview • Cannot be achieved with standard methods: unlink() leaves data on disk, overwriting is cumbersome and may not work (e.g. SSD) • Secure Cryptographic Erase When MEK is deleted, encrypted FEK is no longer retrievable Hence, file cannot be decrypted Regardless of cached copies, snapshots, backups, ... • Two-step operation • Files are deleted with standard file system operations (e.g. rm, unlink...) • Secure deletion committed with key management operation • Registration of new MEK • Re-encryption of FEKs that “need to stay” • Deletion of old MEK 28
- 29. Secure erase : High Level Process • At a high level, the steps for secure deletion are (assuming the files are already encrypted): • Remove the file using commands like rm / unlink • Create a new (master) key • Change the existing policy to "point" to the new key • Run a rewrap policy (using mmapplypolicy) to rewrap all files which are to be kept, making them use the new key --> that’s may be an expensive operation depending on the number of files that need to be traversed. https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.2/com.ibm.spectrum.scale.v5r02.doc/bl1adv_encryptionpolicyrule s.htm • Remove the old key from the server • After the last step, the file is considered secure-deleted. It is the last step that ensures that nobody will be able to decrypt the file, even if someone could get access to the raw content of the disk. 29
- 30. How to Change the Encryption Algorithm that you want to use for Spectrum Scale RPC Communication • For secure data in motion within the Cluster • mmchconfig cipherList=<cipher> works to set the cipher for communications within the cluster. • For remote cluster communication • Using multiple security levels for remote access and changing the Cipher types • In this example, cluster2 is accessing file systems that are owned by cluster1 by using a cipherList of AUTHONLY, but the administrator of cluster1 decides to require a more secure cipherList. The administrator of cluster1 issues this command: mmauth update cluster2 -l AES128-SHA Existing connections is upgraded from AUTHONLY to AES128-SHA. 30
- 31. Network Trace : Show Secure Data in Flight (SMB) at Share Level The best illustration for secure data in rest is a network trace: • Here is on example where traffic to one share is encrypted (smb encrypt = mandatory on one SMB share). The requests on top are visible in the trace. Only everything that relates to the smbencrypt share is encrypted. For encrypted traffic, there is a TRANSFORM header that only specifies that the following data is AES-128-CCM encrypted: 31
- 32. Key Security requirement Spectrum Scale Capability Secure Data at Rest Secure Data in Transit Authentication Authorization Secure Administration Immutability Firewall Hadoop Security Cloud Tiering Security Audit Logging Anti Virus Basic Covered Security Requirement Vs Spectrum Scale Security Capabilities
- 33. High Level Flow for Kerberos 33 SOURCE:http://user.it.uu.se/~hsander/Courses/DistributedSystems/Reports/Kerberos.pdf
- 34. Spectrum Scale : Cloud Tiering Security • Data is encrypted (AES 256) before it is pushed to Cloud Object Storage (on-premises or off- premises) • Supports two types of Encryption Key Management Providers to store the encryption key • IBM Security Key Lifecycle Manager and Java Key Store • TLS protocol is used when communicating with the cloud.