Leapfrog into Serverless - a Deloitte-Amtrak Case Study | Serverless Conference 2017
1 like758 views
The document details Deloitte's collaboration with Amtrak to implement serverless computing solutions, illustrating a shift from traditional server-based architectures to serverless models. It discusses the benefits, challenges, and technological framework of serverless architecture, including improved processing speed and reduced costs. Key recommendations emphasize starting small, integrating existing tools, and understanding AWS limitations.
Leapfrog into Serverless - a Deloitte-Amtrak Case Study | Serverless Conference 2017
- 1. LeapfrogintoServerless A Deloitte-AmtrakCaseStudy S E R V E R L E S S C O N F 2 0 1 7 @GaryArora Cloud Solutions Architect Deloitte
- 2. 1. Share our serverless journey with Amtrak 2. How can traditional enterprises leapfrog into serverless? 3. Collect some geek swag! Objectives
- 4. 4 | Copyright © 2017 Deloitte Development LLC. All rights reserved. Road to serverless solutions Paradigm shift in computing evolution Focus on business logic Stackabstraction Serverless Unit of scale: Functions • Deploy in milliseconds • Live for seconds Containerization Unit of scale: Application • Deploy in seconds • Live for minutes/hours Virtual Machines Unit of scale: Machine • Deploy in minutes • Live for week Physical Machines Unit of scale: Physical servers • Deploy in months • Live for years Server based
- 6. Carries more riders than all airlines combined! DC Boston 85,700 Passengers/Day 30 million annual riders Serves 500 destinations over 21K miles of routes Top Speed: 150 mph $3.2 Billion in Revenue 71% Online
- 7. Challenges • Unpredictable and slow response time: Ranging from few seconds – 1 hour • Analytics & BI dashboard refreshed data every 24 hours – causing reports to be stale by 1 day • Existing systems are expensive to support and extend with new features • No single source of truth • High level of data redundancy • Lack of integration & consistency • Non standardized data storage and data access protocols
- 8. Solution Approach Serverless Integrated within AWS ecosystem Widest prod implementation & support Native mobile support & caching 1 2 Relational DB: High familiarity = Fastest GTM NoSQL: Flexible schema = future proof Select DB Design Select DB Datastore centered
- 9. There’s a lot more than Lambda in AWS Serverless stack API Gateway: Builds, deploys, and manages micro services. Lambda Runs code in response to events Integratedwithother AWScomponents Dynamo DB Managed NoSQL Database Redshift Managed, petabyte- scale data warehouse SNS Managed push notification for multiple platforms Kinesis Real-time data streaming and analytics S3 Durable, highly- scalable cloud storage
- 10. Solution Architecture Redshift ServiceTickets Ticket Process QUERY Target Systems Automated Notifications Operations Trade Partners AWS Cloud Native ServicesSource Systems Tickets Booking Tickets Migration Glacier: Archive Ref Data JBoss Booking Tickets S3: Overflow >400KB Booking Process Data Pipeline: Archive Segments 1 HOURETL ETL Booking Data Pipeline: Reporting S3: Flatfile S3: Consumer Archive DynamoDB S3: Reporting Ticket Service Process 20K/min 7 DAYS XML Data Integration Enterprise Services Lambda Kinesis Lambda Lambda Kinesis Lambda Lambda Kinesis Lambda CloudWatch Monitoring, Auditing, and Logging Audit Log Lambda: Notification SNSAppDynamics S3: History Archive 1.2K/min 20K/min API Enterprise API API Enterprise API Reports & Dashboards
- 11. TL;DR?
- 12. Simplified View Event driven and Serverless Architecture Lambda Kinesis LambdaNoSQL DB Data Transformation Lambda Kinesis Lambda Data Transformation Downstream Consumers Reporting & Analytics Data Warehouse NoSQL DBMainframe Archive RESTful JSON Microservices
- 13. Near-real-time reporting and advanced analytics
- 14. Value Delivered • Processing 1 million transactions/day with a peak load of 2K transactions/minute • near real-time reports and dashboards • Laid out the groundwork for decommissioning legacy systems • Low cost maintenance & operations: No servers to maintain, load-balance, or scale • Single source of truth & entry via JSON restful services • NoSQL schema: Future proof • Improved data accuracy by supporting edge cases that were previously missed Developed and released in six months!
- 16. Serverless meets it’s big claims! Lower infrastructure cost Lower maintenance cost Pay Per Use Faster deployment Faster performance Continuous Scaling Highly available Increased speed Reduced costs
- 17. Infrastructure Model Prod 24x7 Test 24x7 Dev 24x7 Prod 18 hours/day Test 2 hours/day Dev 8 hours/day On-Prem Stack Serverless Stack 8 Load Balanced Instances 2 Load Balanced Instances 1 Instance Production Capacity Production Capacity Production Capacity
- 18. BUT Serverless is NOT one-size-fits-all
- 19. Serverless Use cases • Long-running or complex computational tasks • Data migration from relational to NoSQL • Applications requiring significant disc space or RAM • Applications requiring SSH server access • Web applications • Mobile backend • IoT backend • Real-time analytics and data processing • Etc. Most Suited Least Suited J
- 20. Key Takeaways • Serverless = Technology + Mindset shift • Prototypes Prototypes Prototypes! • Start small • Reuse & integrate with existing tools as much as possible • Don’t forget DevOps! • Understand AWS limits • DynamoDB: 400 KB per item • 5 GSI and 5 LSI. Batch processing. No nested indexes. • Lambda: 128 KB input per event. Cold starts. • Kinesis Shards & DynamoDB partition • Exponential back off algorithm • Make CloudWatch and X-Ray your best friend • Understand how pricing works • A larger RAM lambda can be cheaper than lower if it finishes executing much quicker • IAM roles & policy • Data migration is a beast! Approach Technology
- 21. So, is Serverless for me? Should I buy a house? Should I rent a Room?My Core Objectives Packaged or à la carte services Find individual suppliers or DIY OOB FeaturesOOB Features
- 22. @GaryArora