Nuvola: a tale of migration to AWS
2 likes1,089 views
This document describes Matteo Moretti's company Madisoft's migration of their Nuvola application from on-premise servers to AWS. It involved moving over 60 servers, 1000 databases totaling 4.5TB of data, and 25M media files to AWS using Ansible automation. The migration provided benefits like increased flexibility, horizontal scaling, infrastructure as code, and cost optimization. It took place over multiple environments using tools like Ansible, AWS EC2, RDS, S3, Route53, and was completed in around 50 minutes with no issues for users.
![VPC
infrastructure_nuvola_vpc.yml
- name: INFRASTRUCTURE NUVOLA VPC | vpc peering route
ec2_vpc_route_table:
vpc_id: "{{ vpc['vpc']['id'] }}"
tags:
Name: "nuvola_{{ nuvola_env }}_to_nuvola_default"
subnets:
- "10.0.{{ ec2_vpc_subnet }}.0/24"
routes:
- dest: 0.0.0.0/0
gateway_id: "{{ vpc.igw_id }}"
http://docs.ansible.com/ansible/ec2_vpc_route_table_module.html](https://image.slidesharecdn.com/nuvola-a-tale-of-migration-to-aws2-161018065941/85/Nuvola-a-tale-of-migration-to-AWS-24-320.jpg)
![EC2 backend
infrastructure_nuvola_ec2.yml
count_tag:
nuvola_type: "{{ nuvola_env }}_backend"
exact_count: '{{ nuvola_backend_ec2_instances }}'
vpc_subnet_id: "{{ vpc['subnets'][0]['id'] }}"
assign_public_ip: yes
termination_protection: "{{ delete_lock }}"
volumes:
- device_name: /dev/sda1
volume_type: gp2
volume_size: "{{ ec2_volume_size_backend }}"
delete_on_termination: true
instance_profile_name: "{{ ec2_instance_role }}"
register: ec2_backend
http://docs.ansible.com/ansible/ec2_module.html](https://image.slidesharecdn.com/nuvola-a-tale-of-migration-to-aws2-161018065941/85/Nuvola-a-tale-of-migration-to-AWS-27-320.jpg)
![Provisioning
./provision_nuvola_backend.sh --limit "tag_nuvola_type_${ENV}_backend"
./provision_nuvola_dbserver.sh --limit "tag_nuvola_type_${ENV}_database"
./provision_nuvola_routine.sh --limit "tag_nuvola_type_${ENV}_routine"
if [ "$ENV" != "prod" ]; then
./nuvola-init-not-prod-env.sh --env ${ENV}
./deploy_nuvola.sh --limit "tag_nuvola_type_${ENV}_backend" --env $
{ENV}
fi](https://image.slidesharecdn.com/nuvola-a-tale-of-migration-to-aws2-161018065941/85/Nuvola-a-tale-of-migration-to-AWS-28-320.jpg)
Nuvola: a tale of migration to AWS
- 1. Nuvola: a tale of migration to AWS Ansible + AWS: victory is mine!
- 2. Who am I? Matteo Moretti
- 3. Who I am? CTO @ website: madisoft.it tech blog: labs.madisoft.it
- 4. It’s a story ●It’s our story ●It’s about a migration ●We did it! ●We’ve learnt a lot ●We want to share it with you
- 5. Nuvola ●~ 2M users ●~ 1000 databases ●~ 350GB of mysql data ●~ 25M of media files ●~ 4.50TB of media files ●~ 60 servers
- 6. Why a migration? No automation, flexibility and autoscaling
- 7. Need of change ●Flexibility ●Horizontal scaling ●Infrastructure as code ●Multiple environments ●All services by one provider ●Cost optimization
- 8. pixabay.com
- 9. Change is coming isn’t it?
- 10. pixabay.com
- 11. Obstacles ●It’s a distributed app ●Learn an entire new ecosystem ●Causing no troubles for users ●Few weeks of time
- 13. AWS ●100% automation ●Tons of services ●Very well integrated with Ansible ●IaaS services ●Autoscaling / Reserved instances / Spot instances
- 14. AWS Autoscaling + reserverd instances + spot instances Optimizing services while reducing costs
- 15. AWS
- 16. Ansible ●IT automation tool ●Easy lo learn ●No coding skills. It uses YAML ●No agents on target machines ●Ready-made AWS modules ●Can be easily idempotent
- 17. Ansible & AWS AWS Azure Cloudstack Digital Ocean Google 86 18 31 5 10 Openstack Ovh Rackspace Softlayer Vmware 52 1 26 1 26 http://docs.ansible.com/ansible/list_of_cloud_modules.html
- 18. Ansible & AWS ec2 - Create, terminate, start or stop an instance in ec2 http://docs.ansible.com/ansible/ec2_module.html ec2_asg - Create or delete AWS Autoscaling Groups http://docs.ansible.com/ansible/ec2_asg_module.html ec2_elb_lb - Creates or destroys Amazon ELB. http://docs.ansible.com/ansible/ec2_elb_lb_module.html ec2_snapshot - creates a snapshot from an existing volume http://docs.ansible.com/ansible/ec2_snapshot_module.html ec2_tag - create and remove tag(s) to ec2 resources. http://docs.ansible.com/ansible/ec2_tag_module.html s3 - manage objects in S3. http://docs.ansible.com/ansible/s3_module.html route53 - add or delete entries in Amazons Route53 DNS service http://docs.ansible.com/ansible/route53_module.html
- 19. Nuvola
- 20. Multiple environments ● ./infrastructure_nuvola_env_aws.sh --env prod ● ./infrastructure_nuvola_env_aws.sh --env dev ● ./infrastructure_nuvola_env_aws.sh --env lavorazione
- 21. Infrastructure ./infrastructure_nuvola_env_aws.sh --env prod #!/bin/bash . libs/limit_option_parser.sh …. ansible-playbook --vault-password-file secrets/infrastructure_nuvola_env.secret ansible/infrastucture_nuvola_env.yml -e"$EXTRA_OPTIONS" ….
- 22. Infrastructure playbook infrastructure_nuvola_env.yml tasks: - include: .../infrastructure_nuvola_vpc.yml tags: vpc - include: .../infrastructure_nuvola_ec2.yml tags: ec2 - include: .../infrastructure_nuvola_elb.yml tags: elb - include: .../infrastructure_nuvola_destroy.yml when: destroy == "true" and nuvola_env != "prod"
- 23. VPC tasks/infrastructure_nuvola_vpc.yml - name: INFRASTRUCTURE NUVOLA VPC | setting up vpc ec2_vpc: state: present cidr_block: 10.0.0.0/16 resource_tags: { Name: "nuvola_{{ nuvola_env }}_vpc", nuvola_env: '{{ nuvola_env }}', nuvola_role: "vpc", billing: "{{ billing_tag_value }}" } az: eu-west-1a internet_gateway: True register: vpc
- 24. VPC infrastructure_nuvola_vpc.yml - name: INFRASTRUCTURE NUVOLA VPC | vpc peering route ec2_vpc_route_table: vpc_id: "{{ vpc['vpc']['id'] }}" tags: Name: "nuvola_{{ nuvola_env }}_to_nuvola_default" subnets: - "10.0.{{ ec2_vpc_subnet }}.0/24" routes: - dest: 0.0.0.0/0 gateway_id: "{{ vpc.igw_id }}" http://docs.ansible.com/ansible/ec2_vpc_route_table_module.html
- 25. ELB infrastructure_nuvola_elb.yml - name: INFRASTRUCTURE NUVOLA ELB | Setup ELB ec2_elb_lb: state: present name: 'nuvola-{{ nuvola_env }}-elb' listeners: - protocol: http load_balancer_port: 80 instance_port: 80 - protocol: https load_balancer_port: 443 instance_protocol: http instance_port: 80 ssl_certificate_id: '{{ output.stdout }}' http://docs.ansible.com/ansible/ec2_elb_lb_module.html
- 26. EC2 backend infrastructure_nuvola_ec2.yml - name: INFRASTRUCTURE NUVOLA EC2 | Init backend instances ec2: key_name: '{{ ec2_key_name }}' instance_type: '{{ backend_instance_type }}' instance_tags: nuvola_type: "{{ nuvola_env }}_backend" nuvola_env: '{{ nuvola_env }}' nuvola_role: "backend" billing: "{{ billing_tag_value }}" image: "{{ ec2_ami_id }}" zone: "{{ ec2_zone }}" wait: yes wait_timeout: 600 group: "nuvola_{{ nuvola_env }}_backend_sg" http://docs.ansible.com/ansible/ec2_module.html
- 27. EC2 backend infrastructure_nuvola_ec2.yml count_tag: nuvola_type: "{{ nuvola_env }}_backend" exact_count: '{{ nuvola_backend_ec2_instances }}' vpc_subnet_id: "{{ vpc['subnets'][0]['id'] }}" assign_public_ip: yes termination_protection: "{{ delete_lock }}" volumes: - device_name: /dev/sda1 volume_type: gp2 volume_size: "{{ ec2_volume_size_backend }}" delete_on_termination: true instance_profile_name: "{{ ec2_instance_role }}" register: ec2_backend http://docs.ansible.com/ansible/ec2_module.html
- 28. Provisioning ./provision_nuvola_backend.sh --limit "tag_nuvola_type_${ENV}_backend" ./provision_nuvola_dbserver.sh --limit "tag_nuvola_type_${ENV}_database" ./provision_nuvola_routine.sh --limit "tag_nuvola_type_${ENV}_routine" if [ "$ENV" != "prod" ]; then ./nuvola-init-not-prod-env.sh --env ${ENV} ./deploy_nuvola.sh --limit "tag_nuvola_type_${ENV}_backend" --env $ {ENV} fi
- 29. Provisioning PHP7 roles/php7/tasks/php7_prod.yml - name: PHP7 ALL | install php packages apt: pkg={{ item }} state=latest update_cache=yes with_items: '{{ php7_packages }}' - name: PHP7 ALL | Set php.ini CLI template: src=roles/php7/templates/nuvola/php.ini.cli.j2 dest=/etc/php/7.0/cli/php.ini - name: PHP7 ALL | Set php.ini php-fpm template: src=roles/php7/templates/nuvola/php.ini.web.j2 dest=/etc/php/7.0/fpm/php.ini
- 30. Multiple env: how do I find it? Route 53 ● Public DNS ○ nuvola-prod-backend-3.ops.madisoft.it ○ nuvola-prod-database-24.ops.madisoft.it ○ nuvola-dev-database-34.ops.madisoft.it ○ nuvola-issue8978-database-34.ops.madisoft.it ● Private DNS ○ local-prod-backend-0.ops.madisoft.it ○ local-prod-cache-sessioni-0.ops.madisoft.it ○ local-dev-database-14.ops.madisoft.it ○ local-issue8978-backend-0.ops.madisoft.it pixabay.com
- 31. DNS infrastructure_nuvola_ec2.yml - name: INFRASTRUCTURE NUVOLA EC2 | Assign backend dns route53: command: create zone: "{{ domain_tld }}" record: "nuvola-{{ nuvola_env }}-backend-{{ item.0 }}. {{ domain_tld }}" type: A value: '{{ item.1.public_ip }}' overwrite: yes ttl: "{{ ttl_expire }}" with_indexed_items: '{{ ec2_backend.instances }}' nuvola-prod-backend-3.ops.madisoft.it http://docs.ansible.com/ansible/route53_module.html
- 32. DNS: local infrastructure_nuvola_ec2.yml - name: INFRASTRUCTURE NUVOLA EC2 | Assign database local dns route53: command: create zone: "{{ domain_tld }}" record: "local-{{ nuvola_env }}-database-{{ item.0 }}. {{ domain_tld }}" type: A value: '{{ item.1.private_ip }}' overwrite: yes with_indexed_items: '{{ ec2_database.instances }}' local-dev-database-14.ops.madisoft.it http://docs.ansible.com/ansible/route53_module.html
- 33. DNS: local infrastructure_nuvola_ec2.yml - name: INFRASTRUCTURE NUVOLA EC2 | Assign database local dns route53: command: create zone: "{{ domain_tld }}" record: "local-{{ nuvola_env }}-database-{{ item.0 }}. {{ domain_tld }}" type: A value: '{{ item.1.private_ip }}' overwrite: yes with_indexed_items: '{{ ec2_database.instances }}' local-dev-database-14.ops.madisoft.it http://docs.ansible.com/ansible/route53_module.html
- 34. Ready to move?
- 35. Warm up Moving: - static files from a shared NAS to S3 - external standalone services to ec2 - Jenkins CI to AWS - ELK stack to AWS - (and testing) Nuvola stage environment
- 36. Switch of - Stop current app - Create prod env infrastructure - App deployment - Copy db data
- 37. Infrastructure ./infrastructure_nuvola_env_aws.sh --env prod #!/bin/bash . libs/limit_option_parser.sh …. ansible-playbook --vault-password-file secrets/infrastructure_nuvola_env.secret ansible/infrastucture_nuvola_env.yml -e"$EXTRA_OPTIONS" ….
- 39. Db data migration Many small databases on diferent machines Use of parallelization Mysql_migrate_dbserver.sh …. ansible-playbook -l $LIMIT ansible/mysql_migrate_dbserver.yml -e "nuvola_env=$ENV" --vault-password-file ./secrets/provision_nuvola_dbserver.secret ….
- 41. Switch of Total time: ~ 50m
- 43. WE ARE HIRING!(wanna join? ask us at the end of the talk or visit our website)