visualization of Docker metrics
- 1. VISUALIZATION OF DOCKER METRICS 2015/12/09 by janus_wel
- 4. TO CHECK LIFE OR DEATH ? should be covered by orchestration ex) consul
- 5. TO USE RESOURCES EFFICIENTLY !! figure out bottlenecks decision a number of containers on a machine
- 6. TO MAKE DOCUMENTS for customers for executives
- 7. TO FOLLOW THE PDCA CYCLE figure out influences of changes
- 8. COLLECTING METRICS USUALLY is important to trace back
- 11. REST LIKE API The API tends to be REST, but for some complex commands, like attach or pull, the HTTP connection is hijacked to transport STDOUT, STDIN, and STDERR.
- 12. AVAILABILITY maybe, all operations in commands are available. https://docs.docker.com/engine/reference/api/docker_remote_
- 13. FOR EXAMPLE List containers Create a container Start a container Stop a container Remove a container List Images Build image from a Dockerfile Create an image Display system-wide information Show the docker version information ...
- 14. REQUIREMENTS needs some settings for access control $DOCKER_OPTS in /etc/init/docker.conf on Ubuntu $OPTIONS in /etc/sysconfig/docker on other Linux if 192.0.2.1 is a client, DOCKER_OPTS="-H 192.0.2.1:2376 -H unix:///var/run/docker.sock" OPTIONS="-H 192.0.2.1:2376 -H unix:///var/run/docker.sock" https://docs.docker.com/engine/userguide/basics/#bind- docker-to-another-host-port-or-a-unix-socket
- 15. NOTE make sure to hear from unix domain socket. OPTIONS="-H 192.0.2.1:2376 -H unix:///var/run/docker.sock" # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ or, docker commands are not available.
- 16. FOR DOCKER MACHINE machines created by docker-machine needs https by default, in your ~/.docker/machine/machines/(machine name)/ cert.pem key.pem this path is set by "$(docker-machine env (machine name))" as $DOCKER_CERT_PATH
- 17. INFO wget http://(machine ip):2376/info -O - -q | jq . for docker-machine wget --no-check-certificate --certificate=${DOCKER_CERT_PATH}/cert.pem --private-key=${DOCKER_CERT_PATH}/key.pem https://$(docker-machine ip (machine name)):2376/info -O - -q | jq . "curl" can't handle non-checking local cert, so use "wget"
- 19. STATS API to get container's metrics, use stats API GET /containers/(id)/stats https://docs.docker.com/engine/reference/api/docker_remote_ container-stats-based-on-resource-usage
- 23. POINTS schemaless SQL like query REST like interface
- 24. SCHEMALESS there is no need to define somethings when writing data, InfluxDB needs only TABLE NAME and VALUE these are optional timestamp tags https://influxdb.com/docs/v0.9/concepts/schema_and_data_la
- 25. SQL LIKE QUERY to read SELECT * FROM cpu_load_short WHERE host=server01 AND region=us-west -- ^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- table name tags https://influxdb.com/docs/v0.9/query_language/data_explor
- 26. REST LIKE INTERFACE # CREATE DATABASE curl -G http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb" # INSERT curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_ # SELECT curl -G 'http://localhost:8086/query?db=mydb' --data-urlencode "q=SELECT valu https://influxdb.com/docs/v0.9/guides/writing_data.html https://influxdb.com/docs/v0.9/guides/querying_data.html
- 27. SYSTEM pull archtecture collect metrics minutely run with various OSs Linux Windows others can run Docker for now, test only in Linux...
- 29. METRICS COLLECTOR get metrics from Docker remote API and insert them InfluxDB in Ruby requires gem "clockwork" runs a collection minutely
- 30. VISUALIZE in HTML and JavaScript use chart.js use some EcmaScript 2015 features "let" statement Promise Object.assign http://www.chartjs.org/ now available with only Google Chrome
- 32. METRICS legend description type rx_bytes received bytes counter tx_bytes transferred bytes counter total_cpu_usage cpu usage time % counter
- 34. PROBLEMS load stats API is heavy pull architecture calling API is proportional to number of containers handling mass containers id is not human-friendly
- 35. PUSH ARCHITECTURE AND AGENT to collect with low load, we need inspect /sys cat /sys/fs/cgroup/memory/docker/<container-id>/memory.stat cat /sys/fs/cgroup/cpuacct/docker/<container-id>/cpuacct.stat pull architecture is unfit to cat local files agent is needed https://docs.docker.com/engine/articles/runmetrics/
- 36. AFTER increase collectable metrices I don't know Docker metrics means what is... agent-ize collector