Don’t talk to strangers: Test isolation with containers

Editor's Notes

• #3: In order to build quality software we need: Tests so that we know it works as expected Automation that allows us to develop and experiment in a cheap and quick manner -> improve Tools can help alot Discipline is required to keep all that is good and improve further
• #4: In order to build quality software we need: Tests so that we know it works as expected Automation that allows us to develop and experiment in a cheap and quick manner -> improve Tools can help alot Discipline is required to keep all that is good and improve further
• #5: GEORGI All photos except our own images are Creative Commons license
• #6: GEORGI In order to build quality software we need: Tests so that we know it works as expected Automation that allows us to develop and experiment in a cheap and quick manner -> improve Tools can help a lot Discipline is required to keep all that is good and improve further
• #7: GEORGI Everyone want to build quality systems, but it Turns out difficult Following are examples of common pitfalls during testing We will introduce tools that mitigate the presented problems
• #8: GEORGI The first problem we want to present is the test pollution In general pollution is caused by resources left behind (as in real life). These can be env. Variables database/storage/persistence data causing an app/program to appear in certain state files (another form of state) dependencies already present on the machine
• #9: GEORGI Consuming too much cpu/memory causes other leads to starvation for other processes Running tests on the same machine may lead to delays and failures (hard to debug) Not talking for performance but for functional - unit/integration tests
• #10: GEORGI Different Kind of devices (emulators and physical hardware) landscapes (cloud) Backends Libraries / set of dependencies and programs Let’s see how we can solve this …
• #11: GEORGI We’ve seen these 3 problems. Let’s now see how all these problems can be addressed …
• #12: HRISTO Dedicated hardware for Android OS env: AWS (mostly linux), Azure (mostly Windows) Recently a few tools and technologies became popular. Let’s see how they can help solve our issues…
• #13: HRISTO In the past goods were delivered ‘as they are’ – multiple different packages Hard to load/unload on different infrastructures Solution: shipping container Engineers borrowed the same idea Isolation, Standartization, Automation
• #14: HRISTO
• #15: HRISTO
• #16: HRISTO
• #17: HRISTO
• #21: HRISTO
• #22: GEORGI Virtualization emulates all devices (memory, disk, video, ….). We can run different OS-es Containerization allows us to run only a single OS, but is much cheaper and lightweight.
• #23: GEORGI
• #24: GEORGI
• #25: GEORGI
• #26: GEORGI
• #27: GEORGI
• #28: GEORGI
• #29: GEORGI
• #30: GEORGI
• #31: GEORGI Docker is an implementation of the container idea Provides definition of container in “Dockerfile” The container is stored as image The image is stored on a site/repo - Hub
• #32: GEORGI Docker is an implementation of the container idea Provides definition of container in “Dockerfile” The container is stored as image The image is stored on a site/repo - Hub
• #38: HRISTO Run 2 (or more) mvn clean install in parallel to demonstrate they fail Now start a container with Ubuntu to show that containers actually work and show uname Whoami restart does not work –this is not a VM Run 2 containers to show that mvn clean install now works
• #39: HRISTO Now it’s a time for demo. You can find the source, container definition (dockerfile) and the presentation itself in github You can take a picture so you don’t have to remember or write down the address.
• #40: HRISTO Linux only. Windows requires different VM Discipline – cleanup still needed. CPU and memory contention is problem in containers too. This has to be addressed in the same way as with VM/standalone env.
• #41: GEORGI Software is complex- it consists of many moving parts, developed by many teams using multiple technologies Like a car in an assembly line our software needs to pass through multiple stages of build and validation until we have an end product We wan to have this assembly line fully automated -> release often and receive feedback. When producing software we have to: Deal with high complexity Have an ‘assembly line’ like process for testing Have this automated We can do everyting in a container, however we are missing one important concept …
• #42: GEORGI Pipelines can help us reach from build through testing to the final packaging and release of our product
• #43: GEORGI We can use popular solutions with containers Or use homegrown tools However … they all miss the concept of a pipeline and don’t treat it as a first class citizen That’s why ...
• #44: GEORGI CI that understands pipelines Concourse was build by the CF community out of necessity – after experimenting with Jenkins and GoCD
• #45: HRISTO
• #46: HRISTO
• #47: HRISTO Conveyor that accepts resources on the input and after processing/manifacturing outputs releases on the output. Resources: Worker Controller Integration suite Release Final-release Jobs: Worker build & unit test Controller build & mysql test Controller build & postgres test Integration test Deploy
• #49: GEORGI Show our local concourse pipeline and explain what it does Change the source of the example and demonstrate: Concourse picks the change Runs tests in parallel (in containers) Packages and releases the code
• #50: GEORGI Now it’s a time for demo. You can find the source, container definition (dockerfile) and the presentation itself in github You can take a picture so you don’t have to remember or write down the address.
• #51: GEORGI Pros Reproducible environment – configuration in pipeline.yml Native pipeline support – pipeline = resources + tasks & jobs Release process modeling – easy to model the whole (end to end) release process Cons Maturity – not matury. A lot of changes Pipeline management – build & support of pipelines is hard Scripting – a lot of scripting involved in tasks/jobs