Daniel Myers (Snowflake) - Developer Journey_ The Evolution of Data Applications
0 likes30 views
This document discusses different types of data applications and considerations for developers. It begins by stating that all applications involve ingesting, transforming, and displaying data. It then outlines three types of data applications: managed apps where the provider stores customer data, connected apps where data resides in the customer's account, and native apps that run directly in the customer's Snowflake account. The document provides examples and reference architectures for each type and compares them based on factors like data location, access, management, and costs. It aims to help developers understand options for building applications that leverage Snowflake's data platform.
Daniel Myers (Snowflake) - Developer Journey_ The Evolution of Data Applications
- 1. The Evolution of Data Applications Daniel Myers Lead, Developer Relations Snowflake
- 2. Previously, Daniel held leadership roles at several companies, including Google and Cisco. In addition, he led and founded multiple startups. Daniel has a bachelor’s degree in computer engineering and an MBA, and he is an inventor of three patents. Twitter: @jdanielmyers Daniel Myers Lead, Developer Relations, Snowflake
- 3. © 2021 Snowflake Inc. All Rights Reserved 3 DATA APPLICATIONS ALL APPS ARE DATA APPS ● Data is required to solve the problem ● Data intensive apps have different considerations ● All apps ingest, transform, and display data - just in different ways
- 4. © 2021 Snowflake Inc. All Rights Reserved DATA APP USE CASES Customer 360 Internet of Things Machine Learning & Data Science App Health & Security Analytics Embedded Analytics
- 5. © 2022 Snowflake Inc. All Rights Reserved Borderless Data with the Data Cloud File-based + Table-based Data Lake / Lakehouse Table-based Data Warehouse Evolution of Data Management
- 6. © 2022 Snowflake Inc. All Rights Reserved JSON, Parquet, Avro, ORC, XML Raw arbitrary binary data Continuous or batch loading Transactional data DEVELOPER CONSIDERATIONS All data, any speed Ingest and immediately query JSON, Parquet, Avro, ORC, XML, and even raw arbitrary binary data. Code the same way, execute faster Use familiar programming and tooling to work with build your application and streamline your architecture. Create dev environments with real data Easily create sandboxes, dev, and QA environments with real, up-to-date data. Access and share live, ready-to-query data Instantly access and share live data, cross-cloud and region, no ETL or API needed.
- 7. © 2022 Snowflake Inc. All Rights Reserved DEVELOPER CONSIDERATIONS All data, any speed Ingest and immediately query JSON, Parquet, Avro, ORC, XML, and even raw arbitrary binary data. Code the same way, execute faster Use familiar programming and tooling to work with build your application and streamline your architecture. Create dev environments with real data Easily create sandboxes, dev, and QA environments with real, up-to-date data. Access and share live, ready-to-query data Instantly access and share live data, cross-cloud and region, no ETL or API needed. * Snowpark for Python in Private Preview * Snowpark Java API in Public Preview
- 8. © 2022 Snowflake Inc. All Rights Reserved PROD Data Science DEV Zero Copy Cloning All data, any speed Ingest and immediately query JSON, Parquet, Avro, ORC, XML, and even raw arbitrary binary data. Code the same way, execute faster Use familiar programming and tooling to work with build your application and streamline your architecture. Create dev environments with real data Easily create sandboxes, dev, and QA environments with real, up-to-date data. Access and share live, ready-to-query data Instantly access and share live data, cross-cloud and region, no ETL or API needed. DEVELOPER CONSIDERATIONS
- 9. © 2022 Snowflake Inc. All Rights Reserved Secure Data Sharing No ETL or API required Automatically up-to-date Share UDFs/external functions Consumer managed compute Cross-cloud and cross-region Governed, revocable access All data, any speed Ingest and immediately query JSON, Parquet, Avro, ORC, XML, and even raw arbitrary binary data. Code the same way, execute faster Use familiar programming and tooling to work with build your application and streamline your architecture. Create dev environments with real data Easily create sandboxes, dev, and QA environments with real, up-to-date data. Access and share live, ready-to-query data Instantly access and share live data, cross-cloud and region, no ETL or API needed. DEVELOPER CONSIDERATIONS
- 10. © 2021 Snowflake Inc. All Rights Reserved TYPES OF DATA APPS Managed Apps • Builder of Application is the controller and hoster of data • Builder manages and operates everything (SaaS), B2C Applications • Data can be shared to customer from Builder’s Account via data sharing Connected Apps • Customer of Application is the controller and hoster of data • Builder provides application software for the customer to use • Builder and Customer coordinate to deploy the Application, B2B Applications • Application is provided credentials to operate with/on data in the Customer’s Account Native Apps • Customer of Application is the controller and hoster of data • Customer installs application via Snowflake Application Marketplace • Operates entirely within the Customer’s Snowflake Account, B2B Applications
- 11. © 2022 Snowflake Inc. All Rights Reserved MANAGED APPLICATION Differentiate with data sharing Unlock revenue or differentiate your product by delivering live, ready-to-query data to your consumers. How it works App provider stores and processes customer data in it’s own data platform (eg. traditional SaaS model). Best fit for ● Reaching a large number of customers ● Offering a fully managed service ● B2B or B2C SaaS Provider Code Data Sharing Customer 1 Customer 2 Customer 2
- 12. © 2021 Snowflake Inc. All Rights Reserved MANAGED APPS REFERENCE ARCHITECTURE GOOGLE CLOUD | AWS | AZURE OLTP NoSQL Operational Data Apps & Services Streaming Services Business Intelligence Machine Learning Sharing Workload Isolation Load data easily with Snowpipe Query data lake without loading data using External Tables Database Batched Data Analytical Queries Event Data 1 3 2 ETL 4 Cloud Object Storage
- 13. © 2022 Snowflake Inc. All Rights Reserved SaaS Provider Code Data Consumer 1 Data Consumer 2 CONNECTED APPLICATION Provide consumers with control Compute and storage occur in the consumer’s Snowflake account. Give them complete control of their data. How it works App provider stores and processes consumer data in customers’ data platform. Best fit for ● Deploying to 10-100s of customers ● Large/Regulated customers ● B2B
- 14. © 2021 Snowflake Inc. All Rights Reserved CONNECTED APPS REFERENCE ARCHITECTURE Connected App Streaming Services Customer Workloads Sharing Customer’s Snowflake Account Load data easily with Snowpipe Batched Data Analytical Queries Event Data 2 Builder’s Snowflake Account 3 1 Internal Stage Copy Into Customer or Builder Hosted GOOGLE CLOUD | AWS | AZURE
- 15. © 2022 Snowflake Inc. All Rights Reserved NATIVE APPLICATION Bring your app to your customer’s data Leverage Snowflake infrastructure to build, distribute, and deploy your app directly in your customer’s Snowflake account. How it works Runs directly in consumer’s account. Data never leaves consumer-controlled environment. Best fit for ● 100k+ customers ● Targeting Snowflake customer base ● Large/Regulated customers Consumer 1 Native App Code Data
- 16. © 2021 Snowflake Inc. All Rights Reserved NATIVE APPS REFERENCE ARCHITECTURE Native App Code Customer Workloads Customer’s Snowflake Account 2 Snowflake Marketplace GOOGLE CLOUD | AWS | AZURE Marketplace Native Application Native App Data Shares 1
- 17. © 2021 Snowflake Inc. All Rights Reserved APP COMPARISON Managed Apps Connected Apps Native Apps Tenancy MTT, OPT, or APT APT APT Data Location Builder Customer Customer Builder has Data Access ✅ ✅ 🚫 Customer has Data Access Data Sharing (optional) Direct Direct DDL Management Builder Builder & Customer Feature of Native App Customers Anyone Anyone w/connector support Snowflake Customers Sales/GTM Sold directly from Builder Alignment with Snowflake Sales Alignment with Snowflake Sales Credential Security Builder Builder & Customer Customer Onboarding No Dependencies/Immediate Builder & Customer Coordination No Dependencies/Immediate Operational Costs Builder pays Customer pays License and for variable Operational Costs Customer pays License and for variable Operational Costs
- 18. © 2021 Snowflake Inc. All Rights Reserved THANK YOU Daniel Myers Lead, Developer Relations Snowflake