Soa with consul
2 likes423 views
The document discusses service-oriented architecture (SOA) and the use of Consul for service discovery and configuration management in microservices environments. It outlines configuration examples for server and client nodes, service registration, and health checks via HTTP and DNS interfaces. Additionally, it covers topics like distributed locking, leader election, and accessing key-value data in Consul.
![Server nodes
con g.json
{
"server": true,
"data_dir": "/var/consul",
"encrypt": "2346rxJm2aFuMNxWCKWnvPBA==",
"log_level": "INFO",
"enable_syslog": true,
"bind_addr": "172.31.0.195",
"start_join": ["172.31.12.223", "172.31.28.140"],
"ui": true
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-8-320.jpg)
![services.json
{
"services": [
{
"id": "application@ip-172-31-21-186",
"name": "application",
"address": "172.31.21.186",
"port": 8080,
"checks": [
{
"name": "Application/Apache on port 8080",
"tcp": "localhost:8080",
"interval": "10s",
"timeout": "1s"
}
]
}
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-11-320.jpg)
![/v1/catalog/nodes
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "baz",
"Address": "10.1.10.11",
"TaggedAddresses": {
"lan": "10.1.10.11",
"wan": "10.1.10.11"
},
"Meta": {
"instance_type": "t2.medium"
}
} ...
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-14-320.jpg)
![/v1/catalog/services
{
"consul": [],
"redis": [],
"postgresql": [
"primary",
"secondary"
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-15-320.jpg)
![/v1/catalog/service/<service>
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "192.168.10.10",
"Meta": {
"instance_type": "t2.medium"
},
"CreateIndex": 51,
"ModifyIndex": 51,
"ServiceAddress": "172.17.0.3",
"ServiceEnableTagOverride": false,
"ServiceID": "32a2a47f7992:nodea:5000",
"ServiceName": "foobar",
"ServicePort": 5000,
"ServiceTags": [
"tacos"
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-16-320.jpg)
![/v1/catalog/node/<node>
{
"Node": {
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "10.1.10.12"
},
"Services": {
"consul": {
"ID": "consul",
"Service": "consul",
"Port": 8300
},
"redis": {
"ID": "redis",
"Service": "redis",
"Tags": [
"v1"
],
"Port": 6379](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-17-320.jpg)
![/v1/health/checks/<service>
[
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",
"Status": "passing",
"Notes": "",
"Output": "",
"ServiceID": "redis",
"ServiceName": "redis"
}
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-20-320.jpg)
![GET /v1/kv/<key>
[
{
"CreateIndex": 100,
"ModifyIndex": 200,
"LockIndex": 200,
"Key": "zip",
"Flags": 0,
"Value": "dGVzdA==",
"Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e"
}
]
PUT /v1/kv/<key>
DELETE /v1/kv/<key>](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-24-320.jpg)
![Node Lookups
<node>.node[.datacenter].<domain>
$ dig @127.0.0.1 -p 8600 foo.node.consul ANY
; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY
; (1 server found)
;; truncating comment sections for brevity
;; QUESTION SECTION:
;foo.node.consul. IN ANY
;; ANSWER SECTION:
foo.node.consul. 0 IN A 10.1.10.12
;; AUTHORITY SECTION:
consul. 0 IN SOA ns.consul. postmaster.consul. 139283](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-26-320.jpg)
![Service Lookups
[tag.]<service>.service[.datacenter].<domain>
$ dig master.redis.service.consul
; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul
;; truncating comment sections for brevity
;; QUESTION SECTION:
;master.redis.service.consul. IN A
;; ANSWER SECTION:
master.redis.service.consul. 0 IN A 172.31.28.140
;; Query time: 2 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Feb 01 10:36:28 UTC 2017
;; MSG SIZE rcvd: 88](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-27-320.jpg)
![Other nodes can discover the leader node using
the session info API
$ curl http://localhost:8500/v1/kv/<key>
[
{
"Session": "4ca8e74b-6350-7587-addf-a18084928f3c",
"Value": "Ym9keQ==",
"Flags": 0,
"Key": "<key>",
"LockIndex": 1,
"ModifyIndex": 29,
"CreateIndex": 29
}
]
If the key has no associated Session, then there is
no leader](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-31-320.jpg)
![The nodes would be watching the key to check if it
was released and try to become the leader
$ curl
http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a1
[
{
"LockDelay": 1.5e+10,
"Checks": [
"serfHealth"
],
"Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y"
"Name": "dbservice",
"ID": "4ca8e74b-6350-7587-addf-a18084928f3c",
"CreateIndex": 28
}
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-32-320.jpg)
Soa with consul
- 1. Service Oriented Architecture with Consul Rajesh Sharma @broncha VantageBit
- 3. Problems when you scale a SOA/microservices Host resolution (service discovery) Con guration updates
- 5. Consul It is a tool for discovering and con guring services in your infrastructure, with features like Service Discovery Health Checking Key/Value Store
- 6. Installation Just download a binary le and you are good to go.
- 7. Con guration
- 8. Server nodes con g.json { "server": true, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.0.195", "start_join": ["172.31.12.223", "172.31.28.140"], "ui": true }
- 9. Client nodes
- 10. con g.json { "server": false, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.21.186", "start_join": ["172.31.28.140","172.31.12.223","172.31.0.195 }
- 11. services.json { "services": [ { "id": "application@ip-172-31-21-186", "name": "application", "address": "172.31.21.186", "port": 8080, "checks": [ { "name": "Application/Apache on port 8080", "tcp": "localhost:8080", "interval": "10s", "timeout": "1s" } ] } ] }
- 13. Catalog API
- 14. /v1/catalog/nodes [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "baz", "Address": "10.1.10.11", "TaggedAddresses": { "lan": "10.1.10.11", "wan": "10.1.10.11" }, "Meta": { "instance_type": "t2.medium" } } ... ]
- 15. /v1/catalog/services { "consul": [], "redis": [], "postgresql": [ "primary", "secondary" ] }
- 16. /v1/catalog/service/<service> [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "192.168.10.10", "Meta": { "instance_type": "t2.medium" }, "CreateIndex": 51, "ModifyIndex": 51, "ServiceAddress": "172.17.0.3", "ServiceEnableTagOverride": false, "ServiceID": "32a2a47f7992:nodea:5000", "ServiceName": "foobar", "ServicePort": 5000, "ServiceTags": [ "tacos" ] }
- 17. /v1/catalog/node/<node> { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Services": { "consul": { "ID": "consul", "Service": "consul", "Port": 8300 }, "redis": { "ID": "redis", "Service": "redis", "Tags": [ "v1" ], "Port": 6379
- 18. Health Check API
- 19. /v1/health/node/<node> [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "serfHealth", "Name": "Serf Health Status", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "", "ServiceName": "" }, { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "",
- 20. /v1/health/checks/<service> [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "redis", "ServiceName": "redis" } ]
- 21. /v1/health/service/<service> [ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",
- 22. /v1/health/service/<service>?passing [ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",
- 23. The Key-Value Store API
- 24. GET /v1/kv/<key> [ { "CreateIndex": 100, "ModifyIndex": 200, "LockIndex": 200, "Key": "zip", "Flags": 0, "Value": "dGVzdA==", "Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e" } ] PUT /v1/kv/<key> DELETE /v1/kv/<key>
- 25. The DNS Interface Instead of making HTTP API requests to Consul, a host can use the DNS server directly via name lookups redis.service.us-east-1.consul master.rabbitmq.service.us-east-1.consul slave.redis.service.consul Consul will listen on 127.0.0.1:8600 for DNS queries in the consul. domain We can use dnsmasq to route DNS queries for consul. domains to 127.0.0.1:8600 Unhealthy nodes are automatically ltered
- 26. Node Lookups <node>.node[.datacenter].<domain> $ dig @127.0.0.1 -p 8600 foo.node.consul ANY ; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY ; (1 server found) ;; truncating comment sections for brevity ;; QUESTION SECTION: ;foo.node.consul. IN ANY ;; ANSWER SECTION: foo.node.consul. 0 IN A 10.1.10.12 ;; AUTHORITY SECTION: consul. 0 IN SOA ns.consul. postmaster.consul. 139283
- 27. Service Lookups [tag.]<service>.service[.datacenter].<domain> $ dig master.redis.service.consul ; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul ;; truncating comment sections for brevity ;; QUESTION SECTION: ;master.redis.service.consul. IN A ;; ANSWER SECTION: master.redis.service.consul. 0 IN A 172.31.28.140 ;; Query time: 2 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Feb 01 10:36:28 UTC 2017 ;; MSG SIZE rcvd: 88
- 28. Distributed lock and Leader election
- 29. Pick a key service/<servicename>/leader` Create a session $ curl -X PUT -d '{"Name": "rabbitmq-service"}' http://localhost:8500/v1/session/create # response { "ID": "4ca8e74b-6350-7587-addf-a18084928f3c" }
- 30. Acquire a session for given key $ curl -X PUT -d <body> http://localhost:8500/v1/kv/<key>?acquire=<session> If return value is true key was acquired and this node is the leader Else some other node already acquired that key and is the leader
- 31. Other nodes can discover the leader node using the session info API $ curl http://localhost:8500/v1/kv/<key> [ { "Session": "4ca8e74b-6350-7587-addf-a18084928f3c", "Value": "Ym9keQ==", "Flags": 0, "Key": "<key>", "LockIndex": 1, "ModifyIndex": 29, "CreateIndex": 29 } ] If the key has no associated Session, then there is no leader
- 32. The nodes would be watching the key to check if it was released and try to become the leader $ curl http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a1 [ { "LockDelay": 1.5e+10, "Checks": [ "serfHealth" ], "Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y" "Name": "dbservice", "ID": "4ca8e74b-6350-7587-addf-a18084928f3c", "CreateIndex": 28 } ]
- 33. Questions?