Apache Ignite
0 likes301 views
Apache Ignite is an open-source, in-memory distributed database designed for scalability and high performance, supporting various data models like key-value and SQL. It allows for both in-memory and persistent storage, providing features such as ACID transactions, integration with RDBMS and NoSQL, and various deployment options. Additionally, Ignite supports machine learning capabilities and can be deployed across multiple environments like cloud services and container orchestration platforms.
Apache Ignite
- 1. What Is Apache Ignite ? ● An in memory distributed database ● A cluster based processing platform ● Open source / Apache 2.0 license ● A key / value store ● Written in Java, C#, C++, SQL ● Multiple API's available ● Multiple deployment options ● Developed by GridGain Systems
- 2. What Is Apache Ignite ? ● Persistence is turned off – A distributed in-memory database – In memory data grid – Use SQL or key-value APIs ● Persistence is turned on – A distributed, horizontally scalable database – Guarantees full data consistency – Is resilient to full cluster failures
- 3. How Does Ignite Work ? ● Persistence can be toggled ● Small data sets can be stored in memory ● Large datasets can use disk ● Memory can be used as a caching mechanism ● RDBMS or NoSQL integration ● Partitioning or replication of data across cluster ● Horizontally scaleable
- 4. How Does Ignite Work ? ● Use Ignite as a distributed cache ● Implements JCache specification (JSR 107) ● Supports ACID transactions ● Supports SQL ( with limitations ) ● Supports Java, C++, and .NET ● Has a feature rich key-value API ● Offers two user interface ( UI ) options ● Supports collocated processing
- 5. How Does Ignite Work ? ● Ignite has client and server nodes in it's cluster ● Server nodes are for storage and computation ● Client nodes are connection points to the database ● They may be embedded in client applications ● Application code written in Java, C# or C++ ● Ignite supports ODBC, JDBC and REST
- 7. Ignite DataGrid ● An in-memory distributed key-value store ● Horizontal scaleable ● Add nodes on demand ● Scale to hundreds of nodes ● View as a distributed partitioned hash map ● Every cluster node owns a portion of the data ● As cluster (server) nodes are added more data is cached
- 11. Ignite Machine Learning ● Multiple development API's ● A set of simple / efficient ML functions ● Massively scaleable ● Co-location of data and processing ● Minimised the need for costly data transfers ● Minimised ETL as ML functions act on distributed data ● Algorithms support – Classification, Regression, Clustering – Recommendation, Preprocessing
- 12. Ignite Deployment Options ● Multiple deployment options – Docker – Amazon AWS – Google Cloud – Mesos – YARN – VMWare – Microsoft Azure – Kubernetes
- 13. Available Books ● See “Big Data Made Easy” – Apress Jan 2015 ● See “Mastering Apache Spark” – Packt Oct 2015 ● See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” ● Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ ● Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020
- 14. Connect ● Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at – open-source-systems.blogspot.com/ ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration