![# When deployed to the server, read the config from the encrypted files on S3
if [[ "$APP_ENV" == "development" || "$APP_ENV" == "staging" || "$APP_ENV" == "production" ]]; then
export AWS_DEFAULT_REGION=$AWS_REGION
S3_CREDENTIALS_BUCKET=$(cat $CONFIGPATH/package.json | jq "(.kelsus.s3_credentials_bucket)" —raw-
output)
# Retrieve encrypted credentials file from S3
aws s3 cp $S3_CREDENTIALS_BUCKET$APP_ENV/$SERVICE_NAME.config $SERVICE_NAME.config.enc
# Decrypt
SERVER_CONFIG_JSON=`aws kms decrypt --ciphertext-blob fileb://$SERVICE_NAME.config.enc --query
Plaintext --output text | base64 --decode`
# Remove tmp file
rm $SERVICE_NAME.config.enc
if [ "$CONFIG_INTO_ENV_VARIABLES" == "true" ]; then
# Extract all entries in the json and put them on environment variables
echo $SERVER_CONFIG_JSON | jq -r '.config | to_entries[] | [.key, .value] | @tsv' > values.txt
while IFS=$'t' read -r key value; do export "$key=$value"; done < values.txt
rm values.txt
else
# Write the server config file on the config/environment folder
echo $SERVER_CONFIG_JSON | jq -r '.config' > ./dist/config/environment/server.json
fi
fi
Grabbing secrets from S3](https://image.slidesharecdn.com/gluecontalk-180517205800/85/A-battle-tested-CI-CD-Pipeline-15-320.jpg)
A battle tested CI/CD Pipeline
- 1. A battle tested CI/CD Pipeline Featuring AWS and Docker with Jon Christensen of Kelsus Gluecon 2018
- 2. We were using Heroku or Elastic Beanstalk Users were the first to notice issues We had minimal test coverage We had disorganized logs We were hackers This wasn’t going to cut it at scale
- 3. • There are too many choices. • AWS alone adds 5 new features a day. • So the best way to start CI/CD is gradually. • Just like your software itself, CI/CD is always a work in progress.
- 4. I truly believe that one of the best benefits of Docker is that it forces less experienced developers on teams to confront head-on some of the things they don’t know about how computers work and learn them and incorporate them into their body of knowledge
- 5. "True" continuous deployment requires no manual intervention or human involvement. Requires two pillars: 1. Comprehensive, automated integration tests with broad coverage 2. The ability to rollback quickly (ideally automated) in case a bad build gets deployed (because your verification wasn't complete enough)
- 6. Dev Test Deploy Docker Docker Docker Gitflow Gitflow Gitflow CircleCI Volume Mounts S3 - KMS S3 - KMS AWS ECS Code Reviews AWS ECRUnit Tests Current CI/CD Tools and Processes
- 7. Volume Mounts for Hot Reloads version: "2" networks: docker-dev: external: name: dev services: api: build: api/ env_file: .env networks: default: {} docker-dev: aliases: - my-restful-service volumes: - ./shared:/var/www/my-restful-service/shared ports: - "8080:80" links: - db db: build: db/ env_file: .env ports: - "32099:5432" volumes_from: - data data: image: postgres restart: "no" command: "true" volumes: - /var/lib/postgresql test: build: api/ environment: APP_ENV: test command: "./run_tests_docker.sh" volumes: - ./shared:/var/www/my-restful-service/shared ports: - "8080:80" links: - test-db test-db: build: db/ ports: - 5432 ###################### # The below container is provided # as a developer convenience. # It mounts the host volume at the # project root (so that both container and # host are reading same files). This # allows for hot reloading of code changes. ###################### api-dev: build: api/ env_file: .env volumes: - .:/var/www/my-restful-service ports: - "8080:80" links: - db
- 8. Testing has to be a moral mandate In order to succeed at always writing tests, the organization as a whole must treat *not* testing as unthinkable
- 10. # Circle CI 2.0 Specification version: 2 jobs: build: # Docker image selected. docker: - image: 'kelsus/circleci-node-6.10:1.0' A walkthrough a CircleCI file The core thing we do is a build
- 11. environment: # The Service Name must be the same you have # in your package.json, Dockerfile & # docker-compose.yml. SERVICE_NAME: customer-dashboard-server # Git User Email & Name GIT_USER_EMAIL: ci@kelsus.com GIT_USER_NAME: CircleCI A walkthrough a CircleCI file Then we set some environment variables like a GitHub account
- 12. steps: # Checking out the project from the # repository. - checkout - run: name: Run tests with docker compose. command: sh ./tests-up.sh A walkthrough a CircleCI file Then we checkout code and run tests!
- 13. - run: name: Creating a coverage folder. command: mkdir -p ./coverage - run: name: Copy artifacts from container. command: docker cp project_test_run_1:/var/www/ $SERVICE_NAME/shared/artifacts/coverage ./coverage # Prepare Coverage Report on Circle CI - store_artifacts: path: ./coverage destination: coverage A walkthrough a CircleCI file Get coverage information and store it
- 14. - deploy: name: Deploy if branch is dev or staging command: ./deploy/deploy.sh A walkthrough a CircleCI file Deploy!
- 16. Why ECS? excerpt from hangops slack
- 17. CI/CD is always a work in progress You’re always about half done We still need to work on • Improving database migrations • Getting off of bash scripts • Updating how we store secrets • Look at AWS Code Pipeline • Better tests — always better tests
- 18. Thank you! — Jon Christensen Kelsus Inc www.kelsus.com www.prodockertraining.com @jonxtensen