"Evolution of microservice architecture on Conversations API", Alexander Malinovsky

© 2020 TWILIO INC. ALL RIGHTS RESERVED.
Microservices evolution on Conversations API
Oleksandr Malinovskyi 
Technical Lead at Conversations API

What Twilio is?

Twilio products

Conversations API
A single API for scalable, multiparty 
conversations across channels
https://www.twilio.com/conversations-api

Existing Conversations API situation
Load
Team size: 3 teams, 4-5 devs each
AVG: ~~ 4700 RPS

Organization scaling
More teams to fasten the development process
○ Ability to assign ownership of the feature to particular
team, spreads cross team communication
○ Decreases load on single code base, less chances to step
on each others toes

Simpler and faster on-call onboarding process

Team is better specialized on some domain which improves specialization and quality
○ Fewer things to own by single team

Reliability improvements
Get rid of single point of failure
○ Abusive traffic for one part of functionality doesn’t mean
degradation of the whole app for customer
○ The same is applicable to potential incidents on the other
Introduce better fallback mechanisms and retries
○ Smart retries for messaging functionality
Have opportunity to replay events after the incident

Scaling and rate limiting
● Scale up only parts which are overloaded
○ Increased load of push notifications orchestration might require scale up of the whole service
which might be inefficient and affect other parts of functionality
● Rate limiting should be smart
○ Fairness queues for async functionality
○ RPS limits on various operations

Existing architecture

Prerequisites for microservice adoption
1. Split storages
2. Common procedures for new service creation(Paved path)
3. Cloud adoption(AWS Cloud)
https://fwdays.com/event/architecture-fwdays-2021/review/migration-from-cassandra-to-dynamodb

Communication mechanisms
● Synchronous
○ Simple to implement
○ No additional infra costs
○ Lack of retries if service is down
○ Tight coupling between services
● Event-driven
○ Additional service to maintain(message broker)
○ Loose coupling
○ Opportunity to replay events / restore the state

Types of event driven architecture
https://martinfowler.com/articles/201701-event-driven.html
Point to take into account: max event size. For kafka by default it’s 1 MB!
● Event notification
● Event carried state transfer
● Event sourcing
● CQRS

Domain entity based split

Main issues with domain based split
1. Cross service requests
2. Potential inconsistencies due to lack of transactions
3. Potential issues with data replication
4. Difficult to set boundaries between the teams

Functionality based split
1. Core data access
2. Push notifications
3. Webhooks
4. Billing
5. Outbound channel fanout
6. SDK protocols

Services decommissioning
VS
1. Shared data storage
2. Shared code base which requires updates in two places
3. Complexities in testing Data cleanup service in isolation

Result of the Phase 1

Drawbacks
Increased latency + potential decrease in reliability due to additional service call

Conclusions
PROS
1. Introduced 1 more completely independent team + shared some features with 2 other teams
2. Better reliability for some of the features + additional rate limiting protection due to even
driven nature
3. Due to independence of domains each service can scale independently and based it’s own
needs
CONS
1. Time for feature development not decreased(yet?)
2. Latencies increased in some cases

QA

"Evolution of microservice architecture on Conversations API", Alexander Malinovsky

More Related Content

Similar to "Evolution of microservice architecture on Conversations API", Alexander Malinovsky (20)

More from Fwdays (20)

Recently uploaded (20)

"Evolution of microservice architecture on Conversations API", Alexander Malinovsky