Scalable Google Cloud Payroll Project- Presentation
- 1. A RESTful, Auto-Scaling Payroll API Mike, Joe, Edson
- 2. Introduction RESTful Auto-Scaling Payroll API Several Custom Components (Application, Translator, Accruals, Delivery) Google Components (Load Balancer, Auto Scaling, Pub/Sub, Cloud Spanner) Users send/receive data as JSON objects through HTTP Requests
- 4. Load Balancer Distributes our users to various applications that are in service Distributes to a Managed Instance Group Managed Instance Groups are just groups of identical instances built from an Instance Template Instance Template is an image of our Application If changes are made to the Instance Template, these changes are rolled out to all instances in the Managed Instance Group
- 5. Auto Scaling Monitors the CPU level of our Application Instances If it gets too high, new Application instances are created to help reduce the load When the traffic dies down, these instances are shut off and deleted automatically
- 6. Pub/Sub Google’s Messaging Service that allows you send and receive messages between independent applications. Allowed for easy communication between all of our components Topics: ● insert-new-company ● insert-new-employee ● insert-timesheets ● calculate-accruals ● paystub-delivery
- 7. Application Flask Server Routes Publishes To Returns createCompany insert-new-company Response addEmployee insert-new-employee Response submit insert-timesheets Response calculateAccruals calculate-accruals Response deliveryRequest delivery-request Response paystub - Your paystub
- 8. Translator This component subscribes to 3 topics: ● insert-new-company ● insert-new-employee ● insert-timesheets Parses the JSON message, generates a database insert statement, then inserts the data
- 9. Accruals Subscribes to calculate-accruals Calculates your pay stub Inserts your pay stub into the database It only calculates your pay when you request it
- 10. Delivery This instance has one job, to deliver pay stubs to the front-end application Subscribes to delivery-requests Queries the database for an employee’s pay stub Publishes your pay stub to paystub-delivery
- 11. Cloud Spanner Database Relational Strong Consistency High Availability Automatic Replication Horizontally Scalable SQL-like queries
- 12. Component Model
- 13. Design Challenges ● Designing our components. What should they do? How do they interact with the rest of our components? ● Constructing the JSON objects our users would be required to send ● Designing the database to work with the JSON ● We tried Google SQL first, but it was not horizontally scalable ● How to scale the Application server and add a Load Balancer? ● Issues with various Kafka libraries led to Google’s Pub/Sub
- 14. Conclusion Things that would be nice: ● Auto calculate accruals (chron job, every 2 weeks or so) ● Full horizontal scalability (Translator, Delivery, Accruals) ● Find alternative for database cluster component instead of Cloud Spanner ● Costs $9000 per node/year ($0.90/hour for just 1 node) ● Can’t be shut down
- 15. Demo!