A Taxonomy of Clustering, or, No Container is an Island
2 likes725 views
The document outlines a taxonomy of clustering related to application deployment and management, covering various aspects such as applications' architectures, scaling, resource sharing, and the role of different tools. It discusses deployment methodologies including the use of Docker, container management, and orchestration concepts like Kubernetes, as well as considerations for reliability and resilience. Ultimately, it presents a comprehensive overview of best practices and challenges in managing clustered environments for software applications.
A Taxonomy of Clustering, or, No Container is an Island
- 1. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Ted M.Young @jitterted Sr. Software Engineer tedyoung@gmail.com Developer Productivity Group Guidewire Software, Inc.
- 2. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Ted M.Young @jitterted Sr. Software Engineer tedyoung@gmail.com Developer Productivity Group Guidewire Software, Inc.
- 3. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Disclaimer
- 4. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Disclaimer #2
- 5. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 6. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster 1. Applications with few pieces (includes monoliths) 2. Service-oriented and applications with more pieces – complex deployments 3. Need for Scaling and Resiliency 4. Sharing resources with other services/applications (usually analytics) 5. How different tools and systems help with each of these issues
- 7. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster SimpleApps
- 8. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ApplicationWeb Server Database How to Deploy Into Production?
- 9. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Single Machine (maybe aVM) Application Database Web Server
- 10. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster postgres-machine my-app-machine Application Database Web Server
- 11. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster my-app-machine postgres-machine nginx-machine Application Database Web Server
- 12. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Logical Structure my-app-machine postgres-machine nginx-machine ApplicationWeb Server Database
- 13. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Physical Structure:VMs Infrastructure Host Operating System Hypervisor/Virtualization bin/libs ruby bin/libs nginx app postgres
- 14. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster VMs and Containers Container Host OS Infrastructure Docker Engine (daemon) bin/libs ruby bin/libs nginx app postgres Infrastructure Host Operating System Hypervisor/Virtualization bin/libs ruby bin/libs nginx app postgres
- 15. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Physical Structure: Containers Container Host OS Infrastructure Docker Engine (daemon) bin/libs ruby bin/libs nginx app postgres
- 16. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Withmesofar?
- 17. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Deployment
- 18. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster application postgres nginx Application Database Web Server $ docker run -p 80:80 -p 443:443 -d nginx cf9fb97bb4fadf673a58580d $ docker run -d application 1238f2558ae2704d129cf $ docker run -d postgres 4557c7c9d991238f2558ae27
- 19. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Static Deployment application: image: application:latest links: - postgres postgres: image: postgres:9.4 (docker-compose.yml) nginx: image: nginx:latest links: apiservice ports: - "80:80" - "443:443" $ docker-compose up -d
- 20. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster application postgres nginx Application Database Web Server
- 21. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster search cache loggingapplication postgres nginx Scaling: Add Caching
- 22. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Multiple Containers application: image: application:latest links: - logstash - redis - postgres - elastic nginx: image: nginx:latest links: apiservice ports: - "80:80" - "443:443" (docker-compose.yml) logstash: image: logstash:latest links: - apiservice redis: image: redis:latest elastic: image: elasticsearch:latest postgres: image: postgres:9.4 $ docker-compose up -d
- 23. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scaling: MoreThan One?
- 24. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster load balancer Scaling: MoreThan One
- 25. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Where to PutThem?
- 26. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Availability: Dying Server
- 27. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Cluster Management
- 28. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Hi
- 29. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Dynamic Deployment
- 30. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scheduling Work
- 31. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Managing Resources
- 32. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Nouns
- 33. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Nodes
- 34. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Resources
- 35. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Workloads: Get Work Done
- 36. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster LabelingThings node = FAST!
- 37. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Reservations
- 38. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Execution
- 39. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Executor
- 40. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Placement
- 41. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scaling
- 42. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Reliability/Resilience
- 43. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GettingConcrete
- 44. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GoogleKubernetes (Greek for "helmsman")
- 45. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Lessonsfrom BorgandOmega
- 46. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster API service log forwarder service monitor attached storage Pod: Unit of Placement
- 47. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller
- 48. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller
- 49. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller
- 50. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller
- 51. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ApacheMesos
- 52. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster MesosDifferences
- 53. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Utilization
- 54. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Siloed Clusters
- 55. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Frameworks
- 56. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 57. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 58. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scalability
- 59. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Resource Allocation
- 60. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 61. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GreatTogether
- 62. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 63. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Explore, Play
- 64. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Final Questions?
- 65. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
- 66. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ThankYou!