Designing and coding for cloud-native applications using Python, Harjinder Mistry
0 likes668 views
The document discusses cloud native applications and their advantages. It describes how Mark and Grace build an online store as a cloud native application using microservices, containers, and horizontal scalability. This allows their application to be easily deployed, scaled, and updated. The document outlines layers of cloud native applications like functionality, data access, and deployment. It provides an example of a machine learning recommendation service and concludes that cloud native applications allow businesses to experiment quickly and react to needs.
![Images Courtesy
[1] Grace and Mark
https://image.freepik.com/free-vector/happy-university-students_23-2147531065.jpg
[2] Professor
https://image.freepik.com/free-vector/science-teacher_23-2147509483.jpg
[3] Rocket
https://image.freepik.com/free-vector/launching-business-concept_23-2147504957.jpg](https://image.slidesharecdn.com/4-170615114925/85/Designing-and-coding-for-cloud-native-applications-using-Python-Harjinder-Mistry-33-320.jpg)
Designing and coding for cloud-native applications using Python, Harjinder Mistry
- 1. Coding For Cloud Harjindersingh Mistry Bargava Subramanian Red Hat
- 2. Grace ● Smart individual ● Just finished MBA Mark ● A geek ● Just completed engineering
- 3. Grace & Mark Online Store
- 4. Hey Mark, We don’t have enough money to buy our own servers ! So, can we rent some ? That sounds like a good idea, Grace ! That’s called Cloud Computing. I remember what Prof told about it ...
- 5. Cloud Computing ● Computers on rent ● Some guy has lot of computer infrastructure ○ He wants to give it on rent to make money ● Very similar to car-rental service like Europcar ● Cloud-native Application is a program that is designed and developed specifically for cloud computing architecture
- 6. Alright, I have finished the implementation ! Cloud, Take it and go !
- 7. Hey dude, customers are reporting that our service is very slow :( Please check what is wrong there. Oh! is it ? But I had used best algo! Ahha, there seems to be a scalability issue. Let me call my Prof and get his suggestions ...
- 8. Horizontal Scalability ● Think of data in terms of standard ‘collections’ ● Write functional as much as possible ● Let the data flow through functions: ○ If data is static then we can do Parallel Processing ○ If data size is not fully known then we can be Reactive ○ There is also a possibility of mixing them together by using Stream Processing
- 9. Alright, I have done cool clustering and my architecture is reactive now! Cloud, …. take it and go !
- 10. Wow Mark, the customers seem to be happy with performance now :) But one customer has reported a bug ! Can you please fix it ? Oh! is it ? Let me check that. It might be something related to “data-access” module.
- 11. Ok ! So, I have fixed the “data-access” module. Cloud, …. take it and go one more time !
- 12. Hey dude, what are your programming ?!? There is another bug reported :( Oh! is it ? Let me check that.
- 13. Oh man, this debugging is taking forever :( I need to configure the whole system to repro the scenario ! ... Finally, I got it !!! Bug is inside “business-logic” module. I fixed it. Let me test the whole app. Come on Mark ! How much more time do you need ?
- 14. Ok ! So, the tests passed. Cloud, …. take it and go one more time ! Wait, now I remember what Prof said about Micro Services ...
- 15. Micro Services ● Divide the whole application into smaller services, called Micro Services ● It gives flexibility in testing and deploying in small independent chunks ● This improves maintainability a lot
- 16. Ok ! So, I have done Micro Services :) Cloud, …. take it and go !
- 17. Awesome Mark ! We are doing good ! How about adding one new feature into our store ! Oh yes ! That will be fun :) It has been while since I coded something meaningful !
- 18. Alright ! The new feature is ready ! But I somehow don’t like all micro services sharing same computing resources :( Let me talk to Prof ...
- 19. Containers ● Deploy services by using Containers ● Containers increase speed of deployment ● Developer can simulate production environment locally ● Moreover, each service runs in its own environment ○ So if one service goes down, others are still safe
- 20. So, now I have each micro service containerized separately ! That looks good. Cloud, take it and go !
- 21. Crux of the Story ● Current business model demands speed ○ Need to experiment: It’s ok to fail, but fail fast ○ Need the ability to react: Everything cannot be planned ● But, the software has to be good in terms of: ○ Correctness ○ Performance ○ Maintainability ○ Operability
- 22. Cloud Native Application ● It helps us experiment with an idea ● We can achieve those attributes of good software as follows: ○ Correctness: Test cases. This cannot change :) ○ Performance: Horizontal Scalability ○ Maintainability: Micro Services ○ Operability: Containers
- 23. Cloud Native Application: Example ● Movie Recommendation Service ● Cloud Native Machine Learning Application in Python ● There are two parts: ○ Offline Training Job: ○ Online Scoring Service: Movie Data Training Job ML Model Cloud Storage User Input + Model Scoring Service Recommendations
- 24. Layers of Cloud Native Application Deployment Output Channel Functionality Data Access S3 Local Files ...
- 25. Functionality Layer ● Contains core business logic ● As much ‘functional’ as possible i.e. immutability and statelessness ● Data flows through pipeline of functions: ● For example: Input Data Function-1 ... Function-N Output Result Input Data Read/Split Transform Train ML Model
- 26. Data Access Layer ● Provides abstraction on various data sources ● Helps in testing the core logic locally Abstract Class / Interface S3 Local Files ...
- 27. Output Channel Layer ● Different channels for different consumers ● For micro service, it will have REST component Functionality CLI REST ...
- 28. Deployment Layer ● Contains the scripts for deployments ○ Docker files and/or YAML files for containers ○ Python scripts with calls to vendor-specific APIs e.g. AWS API run_job_flow()
- 29. Demo ● Functionality Layer: Training ● Functionality Layer: Scoring ● Deployment Layer: OpenShift Input Data Read/Split Transform Train ML Model User Input + Model Scoring Service Recommendations
- 30. Not Covered ● Continuous Integration / Continuous Delivery ● Security ● Monitoring ● Vendor APIs for deployment
- 31. Code, Slides and Contact ● Code: ○ https://github.com/harjinder-hari/cloud_native_analytics ● Slides: ○ https://speakerdeck.com/harjinderhari/coding-for-cloud ● Contact: ○ Harjinder email: harjinder.v2@gmail.com ○ Bargava twitter: @bargava
- 32. Conclusion ● Current business model demands speed ○ Need to experiment: It’s ok to fail, but fail fast ○ Need the ability to react: Everything cannot be planned ● Cloud Native Applications is a way to achieve it ○ We saw ■ Layers involved in a cloud native application ■ Techniques to achieve the required speed ■ Everything is written in Python ! :)
- 33. Images Courtesy [1] Grace and Mark https://image.freepik.com/free-vector/happy-university-students_23-2147531065.jpg [2] Professor https://image.freepik.com/free-vector/science-teacher_23-2147509483.jpg [3] Rocket https://image.freepik.com/free-vector/launching-business-concept_23-2147504957.jpg