Deployment and Management on AWS:  A Deep Dive on Options and Tools

DEPLOYMENT AND
MANAGEMENT ON AWS: 
A DEEP DIVE ON OPTIONS
AND TOOLS
Danilo Poccia – Solutions Architect

AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

AWS
ELASTIC BEANSTALK
QUICKLY DEPLOY
AND MANAGE
APPLICATIONS

Application
Application
Version Environment
!
!Environment
Conﬁguration
Conﬁguration
Template

NEW FEATURES
SINGLE INSTANCE
WORKER ROLE

.ebextensions/01install.config
packages:
yum:
libmemcached: []
ruby-devel: []
gcc: []
rubygems:
chef: '0.10.2'
apt:
mysql-client: []
packages:
msi:
mysql: http://dev.mysql.com/.../mysql-connector-net-6.6.5.msi/.../

.ebextensions/02copy.config
files:
"/home/ec2-user/myfile" :
mode: "000777"
owner: ec2-user
group: ec2-user
source: http://foo.bar/myfile
"/home/ec2-user/myfile2" :
mode: "000777"
owner: ec2-user
group: ec2-user
content: |
# this is my file
# with content

.ebextensions/03run.config
commands:
test:
command: myscript.py
cwd: /home/ec2-user
env:
myvarname: myvarvalue

.ebextensions/04run_after.config
container_commands:
collectstatic:
command: "django-admin.py collectstatic --noinput"
01syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
02migrate:
command: "django-admin.py migrate"
leader_only: true
99customize:
command: "scripts/customize.sh"

AWS Elastic Beanstalk
Resource Types Reference
AWS Resource Resource Type Identifier
Amazon CloudWatch AWS::CloudWatch::Alarm
DynamoDB Table AWS::DynamoDB::Table
Amazon ElastiCache Cache Cluster AWS::ElastiCache::CacheCluster
Amazon ElastiCache Security Group AWS::ElastiCache::SecurityGroup
Amazon ElastiCache Security Group Ingress AWS::ElastiCache::SecurityGroupIngress
Amazon SNS Subscription AWS::SNS::Subscription
Amazon SNS Topic AWS::SNS::Topic
Amazon SQS Queue AWS::SQS::Queue

AWS Elastic Beanstalk
Built-in Resources
Resource Name Description
AWSEBAutoScalingGroup
The name of the Auto Scaling group that Elastic Beanstalk
uses when it launches Amazon EC2 instances.
AWSEBAutoScalingLaunchConfiguration
The name for the launch configuration settings that Elastic
Beanstalk uses when it launches EC2 instances.
AWSEBEnvironmentName The name of the Elastic Beanstalk environment.
AWSEBLoadBalancer
The name of the elastic load balancer used in the Elastic
Beanstalk environment.
AWSEBRDSDatabase The name of the Amazon RDS database.
AWSEBSecurityGroup
The name for the EC2 security group that Elastic Beanstalk
uses when it launches EC2 instances.

Production
Environment
Test
Environment
master
branch
test
branch
Git
Repository
git commit
git aws.push
git aws.push
Swap
URLs

AWS OPSWORKS
MODEL AND MANAGE
THE ENTIRE APPLICATION

LIFE CYCLE EVENTS
setup configure deploy undeploy shutdown

CONTINUOUS CONFIGURATIONnew/stopped
online
requested
pending
booting
setup
configure
terminating
shutting
down
deploy
configure

GATHER AND SHIP LOGS TO
AMAZON S3 AS SOON AS
THE CPU LOAD
IS TOO HIGH

AWS OPSWORKS STORES
1-MINUTE METRICS
IN CLOUDWATCH

EVERY INSTANCE
CREATES AN ALARM
FOR HIGH CPU LOAD

CLOUDWATCH
ALARM ACTION: 
WRITE TO
SNS TOPIC

WATCHER
INSTANCE
POLLS 
SQS FOR
NOTIFICATIONS

ON ALARM
NOTIFICATION, 
CALL OPSWORKS
API TO …

EXECUTE A
SCRIPT ON THE 
AFFECTED
SERVER

LOGS ARE
GATHERED AND 
WRITTEN TO
AMAZON S3

AWS
CLOUDFORMATION
INFRASTRUCTURE IS CODE

Stack Template
Conditions
Resources
References
Mappings
Outputs
Parameters

Conditions
Resources
References
Mappings
Outputs
Parameters
"Parameters" : {!
"InstanceType" : {!
"Type" : "String",!
"Default" : "t1.micro",!
"AllowedValues" : ["t1.micro", "m1.small", "m1.large"],!
"Description" : "Enter t1.micro, m1.small, or m1.large. Default is t1.micro."!
}!
}

Conditions
Resources
References
Mappings
Outputs
Parameters
"Mappings" : {!
"RegionMap" : {!
"us-east-1" : {!
"AMI" : "ami-76f0061f"!
},!
"us-west-1" : {!
"AMI" : "ami-655a0a20"!
},!
"eu-west-1" : {!
"AMI" : "ami-7fd4e10b"!
},!
"ap-southeast-1" : {!
"AMI" : "ami-72621c20"!
}!
}!
}

Conditions
Resources
References
Mappings
Outputs
Parameters "Parameters" : {!
"EnvType" : {!
"Description" : "Environment type.",!
"Default" : "test",!
"Type" : "String",!
"AllowedValues" : ["prod", "test"]!
}!
},!
!
"Conditions" : {!
"CreateProdInstance" : {"Fn::Equals" : [{"Ref" : "EnvType"}, "prod"]}!
}!
!
"ProductionInstance" : {!
"Type" : "AWS::EC2::Instance",!
"Condition" : "CreateProdInstance",!
"Properties" : {!
"InstanceType" : "c1.xlarge",!
"SecurityGroups" : [ { "Ref" : "ProdSecurityGroup" } ],!
"KeyName" : { "Ref" : "ProdKeyName" },!
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}!
}!
}

Conditions
Resources
References
Outputs
Parameters
"Resources" :{!
"MySimpleImage" : {!
"Type" : "AWS::EC2::Image",!
"Properties" : {!
"ImageId" : "myLinuxBundle-2011-12-30",!
}!
}!
}
Mappings

Conditions
References
Outputs
Parameters
"Parameters" : {!
"MyURL" : {!
"Type" : "String",!
"Default" : "http://aws.amazon.com"!
},!
!
...!
!
"Outputs" : {!
"URL" : {!
"Value" : { "Ref" : "MyURL" }!
}!
}
Mappings
Resources

Conditions
Outputs
Parameters
"Outputs" : {!
"URL" : {!
"Value" : "http://aws.amazon.com/cloudformation"!
}!
}
Mappings
Resources
References

Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
Intrinsic
Functions

Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
Resources : {!
"MyVolume" : {!
"Type" : "AWS::EC2::Volume",!
"Properties" : {!
"Size" : "4",!
"SnapshotId" : "snap234",!
"AvailabilityZone" : "us-east-1a"!
}!
}!
}
Intrinsic
Functions

Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
"Outputs" {!
"MyStacksRegion" : { "Value" : { "Ref" : "AWS::Region" } }!
}
Intrinsic
Functions

Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources "Outputs" : {!
"URL" : {!
"Value" : { "Fn::GetAtt" : [ "MyLoadBalancer", "DNSName" ] }!
}!
}
Intrinsic
Functions
Name
Fn::Base64
Fn::FindInMap
Fn::GetAtt
Fn::GetAZs
Fn::Join
Ref

Stack Template
Conditions
Resources
References
Mappings
Outputs
Parameters
Pseudo
Parameters
Resource
Properties
Intrinsic
Functions

{

"Description"
:
"Create
RDS
with
username
and
password",

"Resources"
:
{
!

"MyDB"
:
{

"Type"
:
"AWS::RDS::DBInstance",

"Properties"
:
{

"AllocatedStorage"
:
"500",

"DBInstanceClass"
:
"db.m1.small",

"Engine"
:
"MySQL",

"EngineVersion"
:
"5.6",

"MasterUsername"
:
"MyName",

"MasterUserPassword"
:
"MyPassword"

}

}

}
}

"AWS::CloudFormation::Init"
:
{

"config"
:
{
!

"packages"
:
{

"yum"
:
{

"mysql"

:
[],

"mysql-‐server"
:
[],

"httpd"

:
[],

"php"

:
[],

"php-‐mysql"

:
[]

}

},

"sources"
:
{

"/var/www/html"
: 

"https://s3.amazonaws.com/my-‐builds/build-‐v4.zip"

}

}

{

"Parameters"
:
{

"KeyName"
:
{

"Description"
:
"Name
of
an
existing
EC2 

KeyPair
to
enable
SSH 

access
to
the
instance",

"Type"
:
"String"

}

},
}

aws ec2 describe-instances!
--filter Name=instance-state-name,Values=running,!
Name=tag:aws:autoscaling:groupName,Values=XXX!
--query Reservations[].Instances[].PublicDnsName!
--output text

for ip in $(!
!
--output text!
!
); do ssh -t ec2-user@$ip "sudo yum update"; done!
!

for ip in $(!
!
--output text!
!
); do ssh -t ec2-user@$ip "cd /app ; git pull"; done!
!

for ip in $(!
!
--output text!
!
); do ssh -t ec2-user@$ip "/app/update.sh"; done!
!

AWS CLI
http://aws.amazon.com/cli/!
https://github.com/aws/aws-cli!
+!
./jq!
https://github.com/stedolan/jq

aws sns list-subscriptions |!
jq -c '.Subscriptions[] |!
select(.Protocol == "sqs")' |!
while read s!
do!
if [ "$(aws sqs get-queue-url —queue-name!
$(echo $s | jq -r .Endpoint |!
sed ’s/^arn:aws:sqs:[^:]*:[^:]*://')!
2>&1 >/dev/null ;!
echo $?)" != "0" ]!
then!
echo "Unsubscribing $s ..."!
aws sns unsubscribe —subscription-arn!
$(echo $s | jq -r .SubscriptionArn)!
echo "Done."!
fi;!
done

Deployment and Management on AWS:  A Deep Dive on Options and Tools

More Related Content

Viewers also liked (15)

Similar to Deployment and Management on AWS:  A Deep Dive on Options and Tools (10)

More from Danilo Poccia (20)