BigDataRevealed SecureSequesterEncrypt - iot easy as 1-2-3 - catalog-metadata discovery
0 likes48 views
The document outlines challenges faced by Hadoop users in creating secure data lakes, including issues with metadata loss, inadequate security measures, and difficulties in identifying sensitive data. It introduces 'BigDataRevealed' as a solution to these challenges, offering tools for metadata reconstruction, identification, and encryption of personally identifiable information (PII). The solution streamlines data integration from various sources while ensuring compliance with data protection regulations like GDPR.
BigDataRevealed SecureSequesterEncrypt - iot easy as 1-2-3 - catalog-metadata discovery
- 1. Hadoop users understand the following barriers must be overcome to have a secure functioning Data Lake. 1. Hadoop strips Cataloguing/Metadata from files as the file enters the ecosystem. - Until Cataloguing/Metadata information is rebuilt the Data Lake is of little value. 2. Big Data software products lack the sophisticated security mechanisms available with legacy databases. - As a result, Hadoop Data Lakes are soft targets for intruders to penetrate. 3. Locating Personally Identifiable Information and other Sensitive data is difficult and may require many people hours from Data Scientists. - Therefore, many Data Lakes contain undiscovered Personally Identifiable Information and Sensitive data fields that are vulnerable to attackers. Hadoop databases are designed to capture and organize incredible volumes of raw data reaching into the Peta Bytes. A properly built Data Lake can provide a company with a 360% view of it’s activities, customers and machinery, but can also supply hackers with the same bounty of information if not properly managed. BigDataRevealed was designed to address each of the above major weakness in Hadoop with limited effort from Data Scientists and Data Management folks. This is why you want BigDataRevealed on your side to help create a Useful and Protected Data Lake. - As Data streams or imports into your Data Lake BigDataRevealed’s Intelligent Catalogue will re-create catalogue data and metadata that was stripped away as well as determine the business classification form more precise columnar naming. - Again, as data streams into your Data Lake BDR’s Intelligent Catalogue will identify PII and other Sensitive data and Sequester/Encrypt the fields before writing them to HDFS or Hbase. The decryption key is safely stored outside of Hadoop. PII and Sensitive Data are never exposed. - The same processes can be run against ‘data at rest’ as well as streaming data with little effort. - BDR provides a graphical interface to connect to IoT, and Social Media data feeds directly to your Data Lake, Eliminating the need for Data Scientist to build unique connectors for every data feed you wish to process. Saving many hours of coding and testing while automating the SecureSequester/Encrypt of Personally Identifiable Information. BigDataRevealed Fills the Weaknesses Inherent in Hadoop And makes IoT as easy as 1-2-3 And all with SecureSequester/Encrypt
- 2. BigDataRevealed employs 3 components • The producer is the tool that tells BigDataRevealed about the stream of data that you wish to include in the data lake • The Intelligent Catalogue (SecureSequester Facility) defines what patterns are to be detected as potential Personal Identifiable Information (PII) and which ones have been deemed false positives • The Intelligent Catalogue (SecureSequester Facility) then takes the configuration information and applies it to streams of information intended for your data lake BigDataRevealed employs three components to protect streams of data made available by IoT and other devices that write in your data lake
- 3. I-o-T as easy as 1-2-3 Protection for EU GDPR – US and worldwide Data Protection, Sequester / Encryption 1. Producer is used to register potential streams of information to BigDataRevealed. Connections are automatically generated. 2. The SecureSequester Administrative workbench is used to define PII patterns and known false positives. Set Duration and parameters for the Stream Job. 3. SecureSequester will interrogate streams as they are introduced to the data lake, encrypt potential PII, dispatch alerts for review and sequester encrypted source data. BigDataRevealed employs three components to protect streams of data made available by IOT and other devices that write in your Data Lake
- 4. Cataloguing / Metadata / Columnar Naming 1. Executive Summary of Discovery Process and Patterns Detected. 2. BigDataRevealed creating Business Column Classification. 3. Creating the File Headers, Catalog Info and Collaborative usage.
- 5. Pattern Discovery of Private Data then Sequester / Encrypt Protection for EU GDPR 1. View File/Column where Personal Data or Discovered Data was found and select Sequester 2. Select individual columns by data type to Encrypt, or select entire file to Encrypt. Then Run Process. 3. View the results of what Columns or if the entire file is encrypted by seeing the actual data.
- 6. Pattern Discovery of Private Data then Consolidate into one Folder for further Analysis and Remediation 1. View File/Column where Personal Data or Discovered Sensitive Data was found and select Sequester / Encrypt the sensitive data. 2. Select files to be copied into a new folder containing like data for further analytics, and remediation. 3. View the results of files that where written into the New Folder as per number 2. info@bigdatarevealed.com 847-791-7838