Best Practices for Managing Docker Versions as presented at JavaOne 2017
Download as PPTX, PDF0 likes300 views
The document discusses best practices for Docker versioning and tagging, emphasizing the importance of maintaining both static and dynamic tags for effective image promotion and traceability. It highlights the issues surrounding mutable tags and the need for proper management of image layers and tags to prevent storage problems. The presentation concludes with practical cleanup strategies for Docker registries and the benefits of using JFrog Artifactory for image management.
Best Practices for Managing Docker Versions as presented at JavaOne 2017
- 1. @jbaruch #oraclecode Docker Version – Best Practices
- 2. @jbaruch #oraclecode About me »@jbaruch »Developer Advocate @JFrog »We might still have some t-shirts left!
- 4. @jbaruch #oraclecode Poll Time! »Heard about Docker »Can do the tutorial »PoCing, playing etc. »Production, baby!
- 8. @jbaruch #oraclecode JFrog Artifactory + Docker
- 9. @jbaruch #oraclecode »There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.
- 10. @jbaruch #oraclecode Naming Things with Docker
- 12. @jbaruch #oraclecode Why Do We Care about versions? »Pipeline Automation/Orchestration »Traceability/Communication »Maintainability/Storage Management
- 13. @jbaruch #oraclecode Those are old news, we have been using versions for years »Docker is different !! »Versions are Tags
- 14. @jbaruch #oraclecode Docker Manifest and Tags ac-image:1.0 ac-image:1.0-beta ac-image:1.0-qa Manifest sha256:252564.. Manifest sha256:462564.. ac-image:1.1 OS: SHA2 Framework: SHA2 Application: SHA2: 3 Layers Application: SHA2: 4 Tags Manife st
- 15. @jbaruch #oraclecode So let’s talk about tags in detail »Mutable and Dynamic in nature »Example: my-image:5.0 »And Latest is not latest »Confusion !!
- 16. @jbaruch #oraclecode my-image:5.0 OS layer 1.0 Framework layer 2.0 Application layer 2.0 OS layer 1.1 Framework layer 2.1 Application layer 2.1 Yesterday Today
- 17. @jbaruch #oraclecode The case for `latest` And other mutable tags
- 19. @jbaruch #oraclecode Automation: Docker Image Promotion Process across silos
- 20. @jbaruch #oraclecode Promotion ac-image:1.0- dev ac-image:1.0-qa ac-image:1.0- release Manifest sha256:252564.. OS: SHA2 Framework: SHA2 Application: SHA2: 3 L a y e r s T a g s
- 21. @jbaruch #oraclecode Pull, retag, push for every stage? It’s nuts!!!
- 22. @jbaruch #oraclecode We got you covered
- 23. @jbaruch #oraclecode The case for immutable tags
- 24. @jbaruch #oraclecode Traceability! »We always know what this image is »Trace it back to CI »Trace it back to source
- 25. @jbaruch #oraclecode Traceability: Classic Approaches »Version Names based On Git Hashes »Version Names based on Jenkins Builds »Version Names based on the packaged software version number »Version names with a unique date (the build timestamp)
- 26. @jbaruch #oraclecode So which one is better?
- 27. @jbaruch #oraclecode Static vs. Dynamic Tags »Static Tags: ⋄Assigned at image CREATION ⋄Reflect metadata that traces the tag to its build and/or contents »Dynamic Tags: ⋄Reflect an image you should use, the current known-good version ⋄Classic example is ubuntu:trusty ⋄Remember, ‘latest’ isn’t actually latest, just one tagged as such!
- 28. @jbaruch #oraclecode Love and Hate the dynamic versions Pros Cons Versions express promotion state Consistency No version – trash! Traceability
- 29. @jbaruch #oraclecode Best Practices – Best of two worlds!
- 30. @jbaruch #oraclecode Double Tag »Push every image with a static tag »Create a second, dynamic tag for the same image as appropriate ⋄Just a like a sym-link ⋄Tag name is not traceable by itself, because the “dynamic” tag may have moved since client pulled. ⋄Docker Inspect, Search, Find image for traceability »Cleanup static tags based on some policy
- 31. @jbaruch #oraclecode Static and Dynamic Tags ac-image:1.0- 2017-04-01- 111 ac-image:latest1- beta ac-image:latest1 Manifest sha256:252564.. Manifest sha256:462564.. ac-image:1.0- 2017-04-03- 222 OS: SHA2 Framework: SHA2 Application: SHA2: 3 L a y e r s Application: SHA2: 4 T a g s ac- image:latest1
- 33. @jbaruch #oraclecode Another problem! »So, if we change tags during promotion, what happens to not promoted images? »They loose their version!!! (WTF, what does that mean?)
- 34. @jbaruch #oraclecode The unlucky ones »In Java we call it SNAPSHOT-s. »In Docker – images without versions. »They are huge!
- 36. @jbaruch #oraclecode You don’t really have unlimited space
- 37. @jbaruch #oraclecode How Docker Registries Work
- 38. @jbaruch #oraclecode Docker Distribution and DTR »Checksum based storage ⋄Multiple tags with the same manifest does not use up additional disk space »Deleting a tag does NOT delete the layers from storage ⋄Actual Delete based on SHA2 reference ONLY ⋄Tag is effectively deleted if you re-push with a same tag ⋄Old Manifest remains, and still referenceable by SHA2
- 39. @jbaruch #oraclecode Deleting a tag does NOT delete the layers from storage ac-image:1.0 ac-image:1.0-beta ac-image:1.0-qa Manifest sha256:252564.. Manifest sha256:462564.. ac-image:1.1 OS: SHA2 Framework: SHA2 Application: SHA2: 3 L a y e r s Application: SHA2: 4 T a g s
- 40. @jbaruch #oraclecode Actual DELETE is based on SHA2 reference ONLY ac-image:1.0 ac-image:1.0-qa Manifest sha256:252564.. Manifest sha256:462564.. ac-image:1.1 OS: SHA2 Framework: SHA2 Application: SHA2: 3 L a y e r s Application: SHA2: 4 T a g s Delete via SHA2
- 41. @jbaruch #oraclecode Docker GC »Required to delete layers with no manifests »Required to clear up disk space »Stop the world
- 42. @jbaruch #oraclecode Docker Distribution, Best Practice »Delete layers without tags up front. ⋄HEAD Tag get SHA2 of manifest ⋄Delete SHA2 ⋄After deleting reference, run GC to clean up
- 43. @jbaruch #oraclecode Docker Distribution, Best Practice »(Not OOB) To clean up a registry with manifests that have no tag ⋄Command used to find these layers: comm -23 <(find . -type f -name "link" | grep "_manifests/revisions/sha256" | grep -v "/signatures/sha256/" | awk -F/ '{print $(NF-1)}' | sort) <(for f in $(find . -type f -name "link" | grep "_manifests/tags/.*/current/link"); do cat ${f} | sed 's/^sha256://g'; echo; done | sort) | wc –l
- 44. @jbaruch #oraclecode JFrog Artifactory »We delete any layer that is not referenced by a tag immediately »We delete manifests that is not referenced by a tag »We have an API to copy/move a docker image or change its tag without using the docker client to pull/push a second time »Configure the max number of tags per image
- 45. @jbaruch #oraclecode Built-in cleanup ac-image:1.0 ac-image:1.0-qa Manifest sha256:252564.. Manifest sha256:462564.. ac-image:1.1 OS: SHA2 Framework: SHA2 Application: SHA2: 3 L a y e r s Application: SHA2: 4 T a g s Delete via SHA2 Manifest sha256:462564.. ac-image:1.2 Application: SHA2: 5 Delete Tag
- 46. @jbaruch #oraclecode Recap »Dynamic versions are good for promotion »Static versions are good for traceability »Retagging should be done in the registry »All unreferenced objects should be collected
- 47. @jbaruch #oraclecode Q&A and Links »@jbaruch »jfrog.com/shownotes »We’re hiring!
Editor's Notes
- #12: BOM (Bills of Material) – Software name. 5 items