Docker and Django Meet For A Tango - London Meetup

Copyright 2019 © systemati.co
Docker and Django
Meet For A Tango
Hendrik Frentrup

hendrik@systemati.co

15 July 2019

London, UK

Ways to get in touch
hendrik@systemati.co

@HendrikFrentrup

linkedin.com/in/frentrup/

github.com/hendrikfrentrup

medium.com/@hendrik.frentrup

Links
•GitHub repo for the code

•github.com/hendrikfrentrup/docker-django

•Walkthrough:

•medium.com/devopslinks/tech-edition-how-to-dockerize-a-django-web-
app-elegantly-924c0b83575d

•https://medium.com/@hendrik.frentrup/tech-edition-django-dockerization-
with-bells-and-whistles-and-a-tad-bit-of-cleverness-2b5d1b57e289

•Further reading:

•medium.com/panaseer-labs-engineering-data-science/how-devops-
helped-us-solve-our-big-data-development-problem-263474dfeedb

•Contact details:

•Email: hendrik@systemati.co

•Twitter: @hendrikfrentrup
•LinkedIn: linkedin.com/in/frentrup/

Code!
• This will be a pretty detailed walk-through

• There will be a lot of code in detail

• Sometimes I’ll ﬂick through things pretty quickly

• I’ll make the slides available

The Problem
• Moving code from development environments to staging or
production can break things

• The state of environments drifts and is hard to manage

• For lack of access, developers have no insight into the speciﬁcs of
production environments

Why?
• Ease the pain of moving from Dev to Prod

• Step towards being Cloud Native

• Twelve Factor App (https://12factor.net/)

• Infrastructure-as-Code

• Productivity

From here
Application
Database
Local machine

To here
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver
*.conf
+ static/
pg-data/

• Move a locally running app into a container

• Setup database, admin and nginx services

• Run migrations

• Logging
• Implement Infrastructure-as-code

• Bring development and production
environments close together

• Increase productivity & working collaboratively
Outline

Simple app running locally
python manage.py runserver
db.sqlite3
Local machine

10 commands to set up
> git init
> git checkout -b develop
> virtualenv env
> source env/bin/activate
> pip install Django>=2.1
> echo 'Django>=2.1' >> requirements.txt
> django-admin startproject my-site
> git add * && git commit -m "..."
> python mysite/manage.py runserver

App running in container
Docker
web
db.sqlite3
Local machine

App inside Docker
> touch Dockerfile
> docker build -t django-docker:0.0.1 .
> docker run -p 8001:8001 docker-django:0.0.1
______Dockerﬁle____________________________________________________________
FROM python:3.6.7-alpine
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD ./ /code/
RUN pip install -r requirements.txt
CMD ["python", "mysite/manage.py", "runserver", "0.0.0.0:8001"]

Success looks like …

Local storage
Docker
web
db.sqlite3db.sqlite3
Local machine

Moving to docker-compose
> touch docker-compose.yml
> docker-compose up
______docker-compose.yml__________________________________________________
version: "3"
services:
web:
build: .
command: python mysite/manage.py runserver 8001
ports:
- “8001:8001”
volumes:
- ./mysite/:/code
maps ports and container code/db to local ﬁlesystem

Database service
Docker
web
python
manage.py
runserver
db
postgres
Local machine

Adding a database service
version: "3"
services:
web:
build: .
command: python mysite/manage.py runserver 8001
ports:
- “8001:8001”
volumes:
- ./mysite/:/code
db:
image: postgres
networks:
- backend
networks:
backend:
driver: bridge

Handling env. variables
______settings.py____________________________________________________________
POSTGRES_PASSWORD = os.environ.get('POSTGRES_PASSWORD')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': POSTGRES_PASSWORD,
'HOST': ‘db',
'PORT': '5432',
},
______.env__________________________________________________________________
POSTGRES_PASSWORD=secret
POSTGRES_USER=postgres
DJANGO_SECRET_KEY=secret
> touch .env
> echo 'psycopg2-binary' >> requirements.txt

Credentials in envvars
version: "3"
services:
web:
[...]
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
db:
[...]
environment:

Rebuilding the web image
______Dockerﬁle____________________________________________________________
FROM python:3.6.7-alpine
ENV PYTHONUNBUFFERED 1
RUN apk update &&
apk add --virtual build-deps gcc python-dev musl-dev &&
apk add postgresql-dev
RUN mkdir /code
WORKDIR /code
ADD ./ /code/
RUN pip install -r requirements.txt

A bit of admin
> docker-compose exec web python mysite/manage.py migrate
> docker-compose exec web python mysite/manage.py createsuperuser
Run that every time we tear down our containers?

Note down the credentials we’ve used?

Data migrations
______0001_initial.py___________________________________________________________
import os
from django.db import migrations
class Migration(migrations.Migration):
dependencies = []
def generate_superuser(apps, schema_editor):
from django.contrib.auth.models import User
DJANGO_SU_NAME = os.environ.get('DJANGO_SU_NAME')
DJANGO_SU_EMAIL = os.environ.get('DJANGO_SU_EMAIL')
DJANGO_SU_PASSWORD = os.environ.get('DJANGO_SU_PASSWORD')
superuser = User.objects.create_superuser(
username=DJANGO_SU_NAME, email=DJANGO_SU_EMAIL,
password=DJANGO_SU_PASSWORD)
superuser.save()
operations = [
migrations.RunPython(generate_superuser),
]

More credentials
version: "3"
services:
web:
[...]
environment:
DJANGO_DB_NAME: ${DJANGO_DB_NAME}
DJANGO_SU_NAME: ${DJANGO_SU_NAME}
DJANGO_SU_EMAIL: ${DJANGO_SU_EMAIL}
DJANGO_SU_PASSWORD: ${DJANGO_SU_PASSWORD}

web, db and admin services
Docker
web
python
manage.py
runserver
db
postgres
Local machine
db-admin
pgadmin server

Adding a DB admin service
[...]
db-admin:
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL}
PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD}
depends_on:
- db
ports:
- "8080:80"
networks:
- backend
[...]

pgadmin in action

Local app for debugging
Docker
web
python
manage.py
runserver
db
postgres
Local machine
db-admin
pgadmin
server
python
manage.py
runserver

Run a local app alongside
______settings.py____________________________________________________________
from decouple import config
SECRET_KEY = config(‘DJANGO_SECRET_KEY')
DEBUG = config('DJANGO_DEBUG', default=False, cast=bool)
WEB_HOST = config('DJANGO_WEB_HOST', default='localhost')
DB_HOST = config('DJANGO_DB_HOST', default='localhost')
DB_NAME = config(‘DJANGO_DB_NAME’)
[...]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': POSTGRES_USER,
'PASSWORD': POSTGRES_PASSWORD,
'HOST': DB_HOST,
'PORT': '5432',
},

Run a local app alongside
______docker-compose.yml_____________________________________
[...]
environment:
DJANGO_DEBUG: "True"
DJANGO_SECRET_KEY: ${DJANGO_SECRET_KEY}
DJANGO_DB_HOST: “db"
[...]

With nginx
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver

Add nginx server
______default.conf_______________________________
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://web:8001;
}
}
______settings.py________________________________
[...]
ALLOWED_HOSTS = [WEB_HOST, ‘localhost']
[...]
> echo “gunicorn” >> requirements.txt

Add nginx server
[...]
web:
command: gunicorn mysite.wsgi --bind 0.0.0.0:8001
[...]
nginx:
image: nginx:latest
ports:
- 8088:80
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
networks:
- backend
depends_on:
- web
[...]

Static ﬁles & persistent data
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver
*.conf
+ static/
pg-data/

Serving static assets
______settings.py_______________________________________
[...]
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
> python mysite/manage.py collectstatic
______default.conf_____________________
server {
[...]
location /static {
alias /code/static;
}
}

Serving static assets
[...]
nginx:
[...]
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
- ./mysite/static:/code/static
[...]
db:
volumes:
- ./postgres-data:/var/lib/postgresql/data
[...]

Mapped ports
Docker
web
8001
db
5432
Local machine
db-admin
8080
nginx
8088
8000

Splitting YAML ﬁles
______docker-compose.override.yml_____________
[...]
volumes:
- ./mysite:/code
environment:
DJANGO_DEBUG: “True"
[...]
______docker-compose.yml_______________
[...]
web:
build: .
command: gunicorn mysite.wsgi
--bind 0.0.0.0:8001
depends_on:
- db
networks:
- backend
[...]

Logs

Logging locally
• To STDOUT!

Logs in shared envs
• Complex topic and no one size ﬁts all

• Log aggregator (SumoLogic, Splunk, ELK etc.)

• A complete ELK stack in one container

• Various logging driver options

• Fluentd

• Gelf
Log aggregatorFluentd
LogStash
ElasticSearch
Kibana

Logging
Local machine
Staging environment
Build
server
Log aggregator
Production
Code
repo
Local machine
push
trigger
tail
deploy
stream
deploy
stream

That’s all!
Thanks

Docker and Django Meet For A Tango - London Meetup

More Related Content

What's hot (19)

Similar to Docker and Django Meet For A Tango - London Meetup (20)

Recently uploaded (20)

Docker and Django Meet For A Tango - London Meetup