![HASHICORP
example.nomad
# Define our simple redis job
job "redis" {
# Run only in us-east-1
datacenters = ["us-east-1"]
# Define the single redis task using Docker
task "redis" {
driver = "docker"
config {
image = "redis:latest"
}
resources {
cpu = 500 # Mhz
memory = 256 # MB
network {
mbits = 10
dynamic_ports = ["redis"]
}
}
}
}](https://image.slidesharecdn.com/sfgooglegithubhug-consulandnomad-160427094145/85/London-HUG-14-4-Deploying-and-Discovering-at-Scale-with-Consul-and-Nomad-12-320.jpg)
London HUG 14/4 - Deploying and Discovering at Scale with Consul and Nomad
- 1. HASHICORP Deploying and Discovering at Scale with Consul & Nomad
- 3. HASHICORP Agenda • Nomad Overview • Nomad Architecture
- 4. HASHICORP Agenda • Consul Overview • Consul Architecture • Prepared Queries
- 6. HASHICORP Advantages of a Scheduler Higher Resource Utilization Decouple Work from Resources Better Quality of Service
- 7. HASHICORP Advantages of a Scheduler Bin Packing Over-Subscription Job Queueing Higher Resource Utilization Decouple Work from Resources Better Quality of Service
- 8. HASHICORP Advantages of a Scheduler Abstraction API Contracts Standardization Higher Resource Utilization Decouple Work from Resources Better Quality of Service
- 9. HASHICORP Advantages of a Scheduler Priorities Resource Isolation Pre-emption Higher Resource Utilization Decouple Work from Resources Better Quality of Service
- 10. HASHICORP
- 11. Nomad HASHICORP Cluster Scheduler Easily Deploy Applications Job Specification
- 12. HASHICORP example.nomad # Define our simple redis job job "redis" { # Run only in us-east-1 datacenters = ["us-east-1"] # Define the single redis task using Docker task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 dynamic_ports = ["redis"] } } } }
- 13. HASHICORP Job Specification Declares what to run
- 14. HASHICORP Job Specification Nomad determines where and manages how to run
- 15. HASHICORP Job Specification Abstract work from resources
- 16. Nomad HASHICORP Multi-Datacenter Multi-Region Flexible Workloads Job Priorities Bin Packing Large Scale Operationally Simple
- 17. HASHICORP Thousands of regions Tens of thousands of clients per region Thousands of jobs per region
- 18. HASHICORP Built on Experience gossip consensus
- 19. HASHICORP • Cluster Management • Gossip Based (P2P) • Membership • Failure Detection • Event System
- 20. HASHICORP • Gossip Protocol • Large Scale • Production Hardened • Operationally Simple
- 21. HASHICORP • Service configuration and discovery • Monitoring at scale • High-availability
- 22. HASHICORP • Service configuration and discovery • Monitoring at scale • High-availability
- 23. HASHICORP • Service configuration and discovery • Monitoring at scale • High-availability
- 24. HASHICORP • Service configuration and discovery • Monitoring at scale • High-availability
- 25. HASHICORP
- 26. HASHICORP • Multi-Datacenter • Raft Consensus • Large Scale • Production Hardened • Coordination (Locking) • Central Servers + Distributed Clients • Network Tomography • Prepared Queries
- 27. HASHICORP Prepared Queries • Multiple instances of a given service exist in multiple datacenters • Clients can talk to any of them, and always prefer the instances with lowest latency • Policies can change, desire to not have the clients know the details of how to locate a healthy service
- 28. HASHICORP Prepared Queries • New query namespace, similar to services • Register queries to answer for parts of this namespace • Clients use APIs, or “.query.consul” DNS lookups to run queries
- 29. HASHICORP gossip consensus Mature Libraries Design Patterns No Scheduling Logic
- 30. HASHICORP Built on Research gossip consensus
- 31. Nomad HASHICORP Inspired by Google Omega Optimistic Concurrency Internal State and Coordination Service and Batch workloads Pluggable Architecture
- 32. HASHICORP Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENTDC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
- 33. HASHICORP Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
- 34. Nomad HASHICORP Region is Isolation Domain 1-N Datacenters Per Region Flexibility to do 1:1 (Consul) Scheduling Boundary
- 36. HASHICORP Evaluations ~= State Change Event
- 37. HASHICORP Create / Update / Delete Job Node Up / Node Down Allocation Failed
- 38. HASHICORP External Event Evalua?on Crea?on Evalua?on Queuing Evalua?on Processing Op?mis?c Coordina?on State Updates
- 39. HASHICORP Server Architecture Omega Class Scheduler Pluggable Logic Internal Coordination and State Multi-Region / Multi-Datacenter
- 40. HASHICORP Client Architecture Broad OS Support Host Fingerprinting Pluggable Drivers
- 41. HASHICORP Fingerprinting Operating System Hardware Applications Environment Type Examples Kernel, OS, Versions CPU, Memory, Disk Java, Docker, Consul AWS, GCE
- 42. HASHICORP Fingerprinting Constrain Placement and Bin Pack
- 43. HASHICORP Fingerprinting “Task Requires Linux, Docker, and PCI-Compliant Hardware” expressed as Constraints
- 44. HASHICORP Fingerprinting “Task needs 512MB RAM and 1 Core” expressed as Resource Ask
- 45. HASHICORP Drivers Execute Tasks Provide Resource Isolation
- 46. HASHICORP Containerized Virtualized Standalone Docker Qemu / KVM Java Jar Static Binaries Rocket
- 47. HASHICORP Containerized Virtualized Standalone Docker Rocket Windows Server Containers Qemu / KVM Hyper-V Xen Java Jar Static Binaries C#
- 49. Nomad HASHICORP Operational Simplicity: Single Binary No Dependencies Highly Available
- 50. Nomad HASHICORP Cluster Scheduler Easily Deploy Applications Job Specification
- 51. Nomad HASHICORP Higher Resource Utilization Decouple Work from Resources Better Quality of Service
- 52. Nomad HASHICORP • Million Container Challenge • hashicorp.com/c1m.html • github.com/hashicorp/c1m
- 53. HASHICORP Thanks! We’ll do a quick demo then answer questions… Jon Benson @jm_benson