SnapLogic Adds Support for Kafka and HDInsight to Elastic Integration Platform
2 likes860 views
SnapLogic's spring 2016 release of its elastic integration platform introduces support for Apache Kafka, enhancing capabilities for streaming data and integration with Microsoft's Azure services and Google Cloud Storage. This update aims to simplify the creation of low-latency data pipelines and includes features such as metadata snaps and parallel task execution to improve performance. Despite competition from larger firms, SnapLogic has seen growth in customer adoption, now boasting 423 customers, reflecting its increasing relevance in the iPaaS and big data integration market.
SnapLogic Adds Support for Kafka and HDInsight to Elastic Integration Platform
- 1. 451 RESEARCH REPRINT REPORT REPRINT SnapLogic adds support for Kafka and HDInsight to Elastic Integration Platform MATT ASLETT 02 JUN 2016 In the spring 2016 release of its Elastic Integration Platform, SnapLogic has added support for the Apache Kafka messaging system for streaming data, as well as furthered its integration with Microsoft Azure. ©2016 451 Research, LLC | W W W. 4 5 1 R E S E A R C H . C O M THIS REPORT, LICENSED EXCLUSIVELY TO SNAPLOGIC, DEVELOPED AND AS PROVIDED BY 451 RESEARCH, LLC, SHALL BE OWNED IN ITS ENTIRETY BY 451 RESEARCH, LLC. THIS REPORT IS SOLELY INTENDED FOR USE BY THE RECIPIENT AND MAY NOT BE REPRODUCED OR REPOSTED, IN WHOLE OR IN PART, BY THE RECIPIENT, WITHOUT EXPRESS PERMISSION FROM 451 RESEARCH.
- 2. Integration-platform-as-a-service (iPaaS) provider SnapLogic has updated its Elastic Integration Platform with the spring 2016 release, adding support for the Apache Kafka publish and subscribe messaging system for streaming data, as well as Microsoft’s HDInsight Hadoop service and Google Cloud Storage. THE 451 TAKE SnapLogic’s move to add support for Apache Kafka should provide value to its customers given the increasing demand for streaming data. The ability to create low-latency pipelines without coding, com- bined with pre-built ‘Snaps’ for common transformations, should lower the barriers to ingesting stream- ing data from Kafka. It is interesting to see the company cozy up to Microsoft’s Azure platform, while the addition of support for Google Cloud Storage shows that it is responding to changing customer demands. CO NTEXT The last time we caught up with SnapLogic, in late 2015, it had just announced a $37.5m funding round, led in part by Microsoft. No surprise then that with the latest spring 2016 release of its Elastic Integration Platform service it has continued to strengthen its integration with Microsoft’s Azure cloud services. The iPaaS provider already offered integration with Microsoft’s Cortana Analytics Suite, including connectors (or Snaps) for Microsoft Azure SQL Data Warehouse, Microsoft Azure SQL Database and Microsoft Azure Blob Storage. The spring 2016 release adds support for Microsoft’s Azure HDInsight Hadoop and Spark service (using both the Hadoop Distributed File System and Azure Blob Storage), as well as Azure SQL Bulk Load, while support for Azure Data Lake is also in the works. While support for HDInsight is a notable addition to the spring 2016 release, along with support for Google Cloud Storage to complement existing integration with Google BigQuery, perhaps the most significant addition to the new release is integration with the Apache Kafka publish and subscribe messaging system. Kafka is fast becoming the default choice for ingesting high-throughput data streams, so the addition of SnapLogic Snaps (or connectors) was somewhat inevitable as the company looks to enable users to build data integration pipelines to ingest data via Kafka, automate transformations, and persist data into databases or Hadoop for storage and historical analysis. SnapLogic is offering Snaps that enable users to both read data from and write data to Kafka, as well as the ability to use the Elastic Integration Platform Designer drag-and-drop development environment to create integration pipelines without coding that ingest, transform and feed high-throughput data; for example, clickstream data, e- commerce data and data from IoT devices.The Kafka Snaps are also designed to be used with SnapLogic’s Ultra Pipe- lines, which were launched in early 2015 to provide always-on data processing for real-time application integration. Also new in the spring 2016 release is the Metadata Snap Pack, which provides the ability to read and write metadata from existing integration pipelines, enabling the creation of new pipelines that can be exposed as APIs for auto- mated provisioning using DevOps tools such as Chef and Puppet. The Metadata Snap functionality is also likely to be of interest to SnapLogic’s hundreds of OEM customers because it will enable them to call on SnapLogic pipelines without exposing the underlying integration platform. Elastic Integration Platform spring 2016 also adds a Pipeline Execution Snap, which is designed to spawn child pipelines that execute repetitive tasks in parallel to improve pipe- line execution performance. SnapLogic now claims 423 customers in total for Elastic Integration Platform, including 169 direct and 254 indirect customers via OEMs, compared with 100 direct customers and 400 in total in December 2015. SnapLogic has also increased its headcount since December from about 150 to 200 or so. 451 RESEARCH REPRINT
- 3. 451 RESEARCH REPRINT CO MPETITION While SnapLogic is best known for its cloud-based service, it also offers a Groundplex version for on-premises in- tegration execution, as well as Hadooplex and Sparkplex for execution using Hadoop and Spark.Thus, it continues to compete with more traditional on-premises data specialists, including the likes of Informatica, TIBCO, Talend, IBM, SAS Institute and Hitachi’s Pentaho. Other competitors include smaller data integration specialists such as Syncsort and Diyotta, as well as fellow cloud-based iPaaS providers, such as Dell Boomi and MuleSoft. Support for Kafka and growing interest in high-throughput streaming data could also take SnapLogic into competition with the likes of StreamSets, Striim and Confluent. SWOT ANALYSIS STRENGTHS The company continues to make good prog- ress since it repositioned for the cloud, and is enjoying growing adoption of its iPaaS and big-data integration offerings thanks to its Snap connectors and hybrid approach to de- sign and execution. WEAKNESSES SnapLogic does not have the highest profile compared with other on-premises data inte- gration vendors, and its terminology (Sna- plex, Cloudplex, Groundplex, Hadooplex) can sometimes over-complicate what is a simple hybrid execution approach. OPPORTUNITIES Adding support for Apache Kafka should pro- vide value to its customers given the increas- ing demand for streaming data. THREATS The company is competing on multiple fronts – often with companies much larger than it is, and with more substantial installed bases.