Microservices - Is it time to breakup?
Download as PPTX, PDF0 likes319 views
Microservices: Is it time to break up? discusses when and how to transition from a monolithic application architecture to a microservices architecture. While microservices allow for improved scalability, performance, and agility, they also introduce more complexity. The document recommends using Redis data services to address scalability needs within a monolithic application initially. As needs grow, the application can then be decomposed into microservices while still leveraging Redis to manage shared data. Redis Enterprise provides high availability, security, and performance advantages for microservices architectures in the cloud.
![Simplicity: Redis Data Structures – ’Lego’ Building Blocks
Lists
[ A → B → C → D → E ]
Hashes
{ A: “foo”, B: “bar”, C: “baz” }
Bitmaps
0011010101100111001010
Strings
"I'm a Plain Text String!”
Bit field
{23334}{112345569}{766538}
Key
8
2
”Retrieve the e-mail address of the user with the highest
bid in an auction that started on July 24th at 11:00pm PST” ZREVRANGE 07242015_2300 0 0=
Streams
{id1=time1.seq1(A:“xyz”, B:“cdf”),
d2=time2.seq2(D:“abc”, )}
Hyperloglog
00110101 11001110
Sorted Sets
{ A: 0.1, B: 0.3, C: 100 }
Sets
{ A , B , C , D , E }
Geospatial Indexes
{ A: (51.5, 0.12), B: (32.1, 34.7) }](https://image.slidesharecdn.com/microservices-isittimetobreakupv0-181106225803/85/Microservices-Is-it-time-to-breakup-8-320.jpg)
Microservices - Is it time to breakup?
- 1. Microservices: Is it time to break up? DAVE NIELSEN | @DAVENIELSEN | DAVE@REDISLABS.COM HEAD OF ECOSYSTEM PROGRAMS REDIS LABS
- 4. Redis Top Differentiators Simplicity ExtensibilityPerformance NoSQL Benchmark 1 Redis Data Structures 2 3 Redis Modules 4 Lists Hashes Bitmaps Strings Bit field Streams Hyperloglog Sorted Sets Sets Geospatial Indexes
- 5. Performance: The Most Powerful Database Highest Throughput at Lowest Latency in High Volume of Writes Scenario Least Servers Needed to Deliver 1 Million Writes/Sec Benchmarks performed by Avalon Consulting Group Benchmarks published in the Google blog 5 1 Serversusedtoachieve1Mwrites/sec
- 6. Redis – First Cloud Native Database 1. Client writes data to Master in-memory 2. Master writes data to Replica in-memory for High Availability • Then Master responds to the Client • If Master fails, then Replica takes over 3. Replica saves data to disk for Disaster Recovery • (attached storage or object store) • If disaster occurs, Master & Replica recover from disk 6 2. Replica1. Master Client 3. Changes
- 7. “REDIS IS FULL OF DATA STRUCTURES!” 2 Simplicity: Data Structures - Redis’ Building Blocks
- 8. Simplicity: Redis Data Structures – ’Lego’ Building Blocks Lists [ A → B → C → D → E ] Hashes { A: “foo”, B: “bar”, C: “baz” } Bitmaps 0011010101100111001010 Strings "I'm a Plain Text String!” Bit field {23334}{112345569}{766538} Key 8 2 ”Retrieve the e-mail address of the user with the highest bid in an auction that started on July 24th at 11:00pm PST” ZREVRANGE 07242015_2300 0 0= Streams {id1=time1.seq1(A:“xyz”, B:“cdf”), d2=time2.seq2(D:“abc”, )} Hyperloglog 00110101 11001110 Sorted Sets { A: 0.1, B: 0.3, C: 100 } Sets { A , B , C , D , E } Geospatial Indexes { A: (51.5, 0.12), B: (32.1, 34.7) }
- 9. 9 University Style • 4-6 week course duration • Tutorials, Online Labs, Homework • 3 hours / week commitment (20 hours per course) • Certificate of Completion university.redislabs.com Free, online education for the Redis Comminity Next Courses start Nov-27 • RU101 Data Structures • RU201 RediSearch In Development • RU102J Redis for Java Devs • RU202 Redis Streams
- 11. Benefits of Microservices • Make it perform faster or scale better • Extend an application’s capabilities more easily • Add new features more quickly and easily • Improve maintainability • Reduce vulnerabilities 11
- 12. However, Microservices are Complicated 12 • More than meets the eye
- 13. Be Prepared for Success Now, and Later • What to do when app begins to hockey stick • Duck tape the parts when they break? • Do you rewrite your app with scalability in mind? 13
- 14. Do Both with Data Services + Microservices • Data Services - Redis solves web scale data problems 14 • Microservices - Kubernetes solves the hockey stick problem
- 15. Scale with Redis and Microservices • In many cases, Monolith plus Data Services is best • Smaller apps and small teams don’t need overhead and unnecessary complexity of Microservices Architecture • Then, when its time to scale, use Microservices with Redis 15
- 16. Scale with Redis Data Services while Re- architecting for Microservices Architecture
- 17. Scale with Redis Data Services … while Re-architecting for Microservices Architecture 17 Top Redis Data Services • Cache Long Running Queries • User Session Management • Queues/Data Ingest • Streaming • Custom Analytics / Leaderboard • Pub-Sub
- 18. User Session Store • The Problem • Maintain session state across multiple servers • Multiple session variables • High speed/low latency required Why Redis Rocks • Hashes are perfect for this! • HSET lets you save session variables as key/value pairs • HGET to retrieve values • HINCRBY to increment any field within the hash structure
- 19. Redis Hashes Example userid 8754 name dave ip 10:20:104:31 hits 1 lastpage home hash key: usersession:1 HMSET usersession:1 userid 8754 name dave ip 10:20:104:31 hits 1 HMGET usersession:1 userid name ip hits HINCRBY usersession:1 hits 1 HSET usersession:1 lastpage “home” HGET usersession:1 lastpage HDEL usersession:1 lastpage Hashes store a mapping of keys to values – like a dictionary or associative array – but faster DEL usersession:1
- 20. Managing Queues of Work • The Problem • Tasks need to be worked on asynch to reduce block/wait times • Lots of items to be worked on • Assign items to worker process and remove from queue at the same time • Similar to buffering high speed data- ingestion Why Redis Rocks • Lists are perfect for this! • LPUSH, RPUSH add values at beginning or end of queue • RPOPLPUSH – pops an item from one queue and pushes it to another queue
- 21. Redis Lists Example LPUSH queue1 orange LPUSH queue1 green LPUSH queue 1 blue RPUSH queue1 red LPUSH adds values to head of list RPUSH adds value to tail of list blue green orange .. .. red
- 22. Redis Lists Example blue green orange .. .. RPOPLPUSH queue1 queue2 red LPUSH queue1 orange LPUSH queue1 green LPUSH queue1 blue RPUSH queue1 red RPOPLPUSH pops a value from one list and pushes it to another list
- 23. Leaderboard with Sorted Sets Example • The Problem • MANY users playing a game or collecting points • Display real-time leaderboard. • Who is your nearest competition • Disk-based DB is too slow Why Redis Rocks • Sorted Sets are perfect! • Automatically keeps list of users sorted by score • ZADD to add/update • ZRANGE, ZREVRANGE to get user • ZRANK will get any users rank instantaneously
- 24. Redis Sorted Sets ZADD game:1 10000 id:1 ZADD game:1 21000 id:2 ZADD game:1 34000 id:3 ZADD game:1 35000 id:4 ZADD game:1 44000 id:3 or ZINCRBY game:1 10000 id:3 34000 id:3 35000 id:4 21000 id:2 10000 id:1 ZREVRANGE game:1 0 0 ZREVRANGE game:1 0 1 WITHSCORES 44000 id:3 + 10000id:3
- 25. Examples of Data Services • Traditional Cache • Smartcache or write through cache • User Sessions Management • Leaderboard (Netflix API use case) • Who’s Online (Pinterest use case) • Tag Server (Stack Overflow use case) 25
- 26. Scale with Microservices Architecture
- 28. Break App Up into a Microservice • Large Companies: Teams gets too big. Map team to org structure. • Conways Law: “Organizations which design systems… are constrained to produce designs which are copies of those organizations” • Reverse Conways Law: “Create your org structure to map the Microservices in your application” • Startups: Break apart for technical reasons • Parts of your app needs to scale at different rates • You want to use different programming languages 28
- 29. Examples of Microservices with Redis: By Department • Product Catalog • Product Recommendations • User Checkout and Order Processing • Order Fulfillment • Customer Loyalty 29
- 30. Examples of Microservices with Redis: By Technology • Email Campaigns - Sending 25k Emails (ex: SendGrid) • Other User Communications: (ex: email, text, web sockets, etc.) • URL Shortener (ex: Bit.ly) • Webcache: Scrape and Cache (ex: Twitter) • Machine Learning/Deep Learning (Scikit-Learn/Tensorflow) • “Autocomplete” or Search-as-you-Type (RediSearch) 30
- 31. • https://github.com/dnielsen/uploadvalidate/blob/master/website-pod.yaml • spec: –containers: • env: –name: API_IP –value: api # Service Discovery using pod label Service Discovery 31 The simple Kubernetes way
- 32. Redis can do Many Things 32
- 33. Real Time Analytics User Session Store Real Time Data Ingest High Speed Transactions Job & Queue Management Time Series Data Complex Statistical Analysis Notifications Distributed Lock Content Caching Geospatial Data Streaming Data Machine Learning Redis can do many things 33 Very Large Data Sets Search
- 34. High Availability • Cover your business for peak hour traffic • Prevent cache stampede • Deliver consistent high performance • Zero downtime scaling Security • Meet your organization’s security and compliance requirements Performance • Maximize resource utilization: Run on all CPU cores Advantages of Redis Enterprise 34 Active-Active • Maintain consistent session state across all data centers • CRDT based active-active delivers strong eventual consistency with local latency Redis on Flash • No need to evict or delete session data; allow the data to overflow to Flash memory
- 35. For any enterprise datacenter or cloud environment Fully managed, server- lessly scaling service in the VPCs of public cloud platforms Fully managed, server- less Redis Enterprise service on hosted resources in the public cloud platforms How to Get Started with Redis Enterprise? 35 or or Cloud VPC Software
Editor's Notes
- #10: Knowledge Center: boost your learning and development through quality content and best practices Relationship Forging: interact with like-minded people to exchange ideas and receive a direct line of communication to the Redis Labs team Rewards & recognition: complete challenges to earn badges and become the featured Advocate of the Month.
- #12: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/cloud/VMW_17Q3_SO_Build-Cloud-Native-Apps-with-Containers_FINAL_081617.pdf
- #13: There is so much emphasis on Microservices these days, it hardly seems like anyone should build an application another way. But starting Microservices too early can exhaust your resources, taking your focus away from the important goal of building a useful application. Many developers get caught-up on developing scalable architecture before they actually solve user problems. In the next 30 mins, I’m going to show you how Redis Enterprise + PKS can teach your whale to scale one microservice at a time. I will demonstrate how service discovery in PKS made it easy to create microservices and how Redis Enterprise made it easy to scale.” https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/cloud/VMW_17Q3_SO_Build-Cloud-Native-Apps-with-Containers_FINAL_081617.pdf
- #14: In the next 30 mins, I’m going to show you how Redis Enterprise + PKS can teach your whale to scale one microservice at a time. I will demonstrate how service discovery in PKS made it easy to create microservices and how Redis Enterprise made it easy to scale." https://www.linkedin.com/pulse/5-business-benefits-twelve-factor-app-edward-viaene/
- #15: So, I’m going to show you one approach to get from start to finish, while highlighting how Redis Labs and Pivotal Container Services help you scale your application.
- #29: Conways Law: Organizations which design systems… are constrained to produce designs which are copies of those organizations ‘reverse Conway’s Law – Org structure around project teams - requires standardization – Uber https://articles.microservices.com/microservices-refactoring-your-teams-5a949d64db2 Netflix https://www.youtube.com/watch?v=QcNqfvMeWow