Fluency® Security's Next Generation Database - Aladdin
0 likes91 views
Fluency's Aladdin database, showcased at RSA Conference 2018, enhances the company's AWS-based database with smart storage management and on-demand search capabilities, outperforming traditional solutions like Elastic in both speed and cost. Key benefits include a 90% reduction in total cost ownership, 50x faster indexing, and efficient cold storage access through a unique streaming design. The advancements have earned Fluency several awards for its innovative approach to log management and data retention.
Fluency® Security's Next Generation Database - Aladdin
- 1. Fluency Security Proprietary and Confidential Copyright 2018 The Magic of Fluency’s ® Aladdin = Innovation & Speed Fluency’s Next Generation Database At the RSA Conference 2018, Fluency demonstrated its latest database version, Aladdin. Aladdin builds on Fluency’s native Amazon Web Services (AWS) database by adding smart storage management and on‐demand search processing. Even from a warm start, Fluency outperforms Elastic. And for compliant long‐term retention, Fluency also provides loading from a cold start. Once searching is fully loaded, massive searches are performed in a few seconds. While other log management solutions rely on older database technologies and manual cold loads, Fluency continues to outperform and out deliver. A few of the advantages of Fluency are: • 90% Reduction on Total Cost Ownership • Cost Effective use of storage and processors • 50x Faster Indexing over Elastic • Multi‐Year Integrated Storage • Multi‐Month detailed searching in seconds • Searching is isolated from Indexing • Full PCI data retention compliance Fluency was recognized for its database innovations by being an RSA Innovation Sandbox Finalist in 2015, when we were known as SecurityDo. Cyber Defense Magazine recognized Fluency’s leap in Technology at RSA Conference 2018 by presenting its Cool Company Award, presented to groundbreaking innovative companies. Also, in 2018, Info Security PG’s Global Excellence Awards, Fluency was named the winner for offering the fastest big data solution for Security Automation & Orchestration. Fluency Out performs: Cold, Warm or Hot Cost and Speed are impacted by the type of storage used. Most solutions have an always‐on hot‐ storage to compensate for slow database technology. The customer pays twice for such a solution. First, the search window is significantly lowered to reduce costs and increase performance. Second, the customer is paying for higher‐end disk storage and search processing power, even when there is no user performing searches. Fluency is the only log management system that reduces cost and increases speed effectiveness by moving data to the appropriate storage and assigning processing when needed. In Fluency, warm data is kept on S3 storage and moved into faster EBS storage once a user begins to search. When a user requests a search
- 2. Fluency Security Proprietary and Confidential Copyright 2018 outside of the 90‐day window, the system reaches back into AWS Glacier storage and moves cold data into EBS storage. When data is placed in EBS storage, Fluency assigns a higher amount of processing power. The result of EBS searching with Fluency’s LavaDB is so fast that searching a four‐week block of warm data (2 minutes) is faster than a fifteen (15) node Elastic cluster searching the same period with always‐on hot storage (about 5 minutes). Once loaded, following Fluency searches will take 2 to 11 seconds, while the Elastic cluster takes minutes. Cold Warm Hot Data is stored in AWS Glacier, needs a few hours to transfer Very slow, but extremely cost effective storage. Cannot search here. Data is stored in S3, needs to download to EBS first Slow, but cost effective storage. Could search here, but faster if we load in to EBS. Data is stored in AWS EBS, which like local storage. The fastest of the storage types. Excellent disk IO for search but too costly to store when not in use. FIGURE ONE: TYPES OF AWS STORAGE Cold Searching While storing data in cold storage is an optimal means of keeping log retention costs down, access to cold data when needed can become an issue. Due to how traditional relational and big data databases format and index data, it’s not as simple as moving data from one storage device to the other. Often data is kept raw and then reloaded, which can take hours or days. FIGURE 2: COLD STORAGE SEARCH This is further complicated as data needs to be reindexed and often cannot be properly correlated.
- 3. Fluency Security Proprietary and Confidential Copyright 2018 Fluency addresses cold storage by its streaming design. Data remains in database format and does a simple move and then decompresses. The four‐phase process of instantiating a process (launching), moving the data (downloading), decompressing and searching can be seen in the timeline tracker of the Fluency interface (Figure 2: Cold Storage Search). At the first use of this data, the database is moved into EBS storage making future searches on this dataset extremely fast. Warm Searching Warm searching moves data from S3 data storage to EBS. To a user using Fluency, this will normally be the first search using the interface, after which the data will be in EBS, and the searches will be hot. FIGURE 3: WARM SEARCHING To the user, the warm storage search takes 118 seconds, while it covered 4 weeks and 12 Billion events (Figure 3: Warm Searching). As stated, these speeds are faster than Elastic’s hot searching. Warm searching allows that system to be efficient, keeping a 90‐day window of data for analysis, while avoiding using more costly storage. Hot Searching The user will spend most of their time performing hot searching. When the data is hot the user is interacting with a read‐only copy of the data on the fastest data storage. As seen in a process timeline of a hot search, a series of processes operate in parallel searching the dataset. Compare that to Figure 2 and see why Fluency’s hot searching is so fast. Not only is it using a fast disk, but also doing so in parallel.