![Bucket Policy
Allow Jason or Mark to do anything in my S3 bucket
when coming from the Office internal network
{"Statement":[{
"Effect":"Allow",
"Principal":{"AWS":["4649-6425", "5243-0045"]},
"Action":"*",
"Resource":"/mybucket/*",
"Condition":{
"IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}
}}]}
©2011 Amazon Web](https://image.slidesharecdn.com/awsintroductiontocloudstorage-111104161108-phpapp02/85/Aws-introduction-tocloudstorage-14-320.jpg)
Aws introduction tocloudstorage
- 1. Amazon Web Services Introduction to Cloud Storage Kevin Wittkopf Enterprise Solutions Architect…
- 2. Cloud Storage AWS offers multiple cloud-based storage options. • Amazon S3 (Simple Storage Service) • Amazon EBS (Elastic Block Storage) • Amazon EC2 Local Instance Store (Ephemeral volumes) • Amazon Cloudfront Each has a unique combination of performance, durability, cost, and interface Often used to form a Storage Hierarchy
- 3. Storage by the Numbers Cloudfront S3 EBS Local Disk Scale Unlimited Unlimited Storage 1TB per volume 147 – 1690GB Temporary 5TB objects ~44TB per 1-5 volumes Storage 100 name spaces server (buckets) Durability N/A 99.999999999% .1-.5% AFR Ephemeral (standard) design requires 99.99% application-level (RRS) redundancy Latency Low High Low Very Low Management Zero Zero Maintenance Snapshots, Manual Maintenance RAID, Redundancy Consistency Locality Global Region-Centric AZ-Specific In-machine Interface REST or SOAP REST or SOAP Block Device Block Device Web Services Web Services
- 4. S3 Amazon Simple Storage Service ©2011 Amazon Web Services May not be reused or redistributed without permission
- 5. Amazon EC2 Amazon Amazon Amazon Amazon EBS S3 RDS SimpleDB ©2011 Amazon Web Services May not be reused or redistributed without permission
- 6. Amazon Simple Storage Service Amazon S3: Simple Storage Service Highly scalable data storage in-the-cloud Programmatic access via web services API Is a Web Store, not a file system Optimized for WORM Eventually consistent Fast, highly available and durable Economical ©2011 Amazon Web
- 7. Is Your Data Storage This Resilient? Note: Conceptual drawing only. Actual number of nodes & datacenters may vary Region Datacenter Datacenter Datacenter Node Node 1 n ... ©2011 Amazon Web
- 8. Amazon S3 Adoption Rate: Billions of Objects Stored Peak Requests: 290,000+ 600 per second 6 months! 500 490 400 300 262 200 102 100 40 14 2.9 0 Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q2 2011 ©2011 Amazon Web Services May not be
- 9. Amazon S3 Namespace Amazon S3 bucket bucket object object object object bucket object object ©2011 Amazon Web Services May not be reused
- 10. Amazon S3 Namespace Amazon S3 mculver-images media.mydomain.com Beach.jpg 2009/maui/sun.jpg img1.jpg img2.jpg public.blueorigin.com Installer.msi img/pic1.jpg ©2011 Amazon Web Services May not be
- 11. Reduced Redundancy Storage Saves $$ Designed to provide 99.99% durability and 99.99% availability of objects over a given year This durability level corresponds to an average annual expected loss of 0.01% of objects Designed to sustain the loss of data in a single facility ©2011 Amazon Web
- 12. S3 Pricing has Three Dimensions Storage Data Transfer Requests Tier Norma RRS Tier Price Type Price l Frist 1 $0.14/G $0.093/GB All transfer in Free PUT, $0.01 TB/mo B COPY, per POST, or 1,000 LIST Next 49 TB $0.125/G $0.083/GB First 1 GB out Free GET and $0.01 B all other per requests 10,000 Next 450 $0.11/GB $0.073/GB <= 10 TB out $0.12/G TB B Next 500 $0.095/G $0.063/GB Next 40 TB $0.09/G TB B out B Next 4000 $0.08/G $0.053/GB Next 100 TB $0.07/G TB B out ©2011 Amazon Web B
- 13. Access Control Lists • Control access to bucket or to object • No inheritance from bucket to object • Set at create, write, or any time • List of up to 100 grants by: – Owner – Email – Any AWS account – Anyone • Permissions: – READ – Object or bucket – WRITE – Create, overwrite, delete objects in bucket – READ_ACP – Read ACL for bucket or object – WRITE_ACP – Overwrite ACL for bucket or object – FULL_CONTROL – All of the above ©2011 Amazon Web Services May not be reused or redistributed without permission
- 14. Bucket Policy Allow Jason or Mark to do anything in my S3 bucket when coming from the Office internal network {"Statement":[{ "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"} }}]} ©2011 Amazon Web
- 15. S3 Multipart Uploads Improved throughput—upload parts in parallel Quick recovery from any network issues—Smaller part size minimizes the impact of restarting a failed upload due to a network error. Pause and resume object uploads— upload object parts over time Once you initiate a multipart upload there is no expiry must explicitly complete or abort the multipart upload Begin an upload before you know the final object size— can upload as you are create. ©2011 Amazon Web
- 16. Even More Amazon S3 (not so simple, is it?) Website support Versioning Browser Upload to S3 Logging Requestor Pays Signed URLs Bittorrent support IAM Meta-data ©2011 Amazon Web
- 17. AWS Import/Export Get your data into AWS faster - load it onto a portable storage device and ship it to an Amazon data center Faster than Internet transfer and more cost effective than upgrading your connectivity Use cases: data migration, offsite backup, direct data interchange, disaster recovery ©2011 Amazon Web
- 18. AWS Import/Export vs. S3 upload costs $1,200 $1,000 $800 $600 Import/Export S3 upload $400 $200 $0 100Gb 500Gb 1 Tb 2 Tb 3 Tb 5 Tb * 10 Tb **
- 19. AWS Import/Export vs. S3 upload time savings 60 50 40 Days 30 Import/Export 20 S3 upload 10 0 100 Gb 500 Gb 1 Tb 2 Tb 3 Tb 5 Tb
- 20. Amazon EC2 Amazon Amazon Amazon Amazon EBS S3 RDS SimpleDB ©2011 Amazon Web Services May not be reused or redistributed without permission
- 21. EBS Features Persistent storage Volume lifetime is independent of any particular EC2 instance. General purpose Raw, unformatted, block device. Use from Linux, Solaris or Windows. Performance Equal to or better than local EC2 drive. High reliability Built-in redundancy within availability zone. AFR (Annual Failure Rate) between 0.1% and 1%. Scalable Volume sizes ranging from 1 GB to 1 TB. Easy Easy to create, attach, back up, restore, and delete volumes. ©2011 Amazon Web Services May not be reused or redistributed without permission
- 22. Amazon Web Services: Durable & Available Note: Conceptual drawing only. The number of Availability Zones may vary Japan US East Region EU West Region Availabilit Availabilit Availabilit Availabilit y Zone A y Zone B y Zone A y Zone B Availabilit Availabilit y Zone A y Zone B Availabilit US West Region Availabilit Singapore y Zone C y Zone C Availabilit Availabilit Availabilit Availabilit y Zone A y Zone B y Zone A y Zone B ©2011 Amazon Web Services May not be reused or redistributed without written permission
- 23. EBS Volumes are in a Single Availability Zone EBS Volume 1 EBS Volume 2 Availability Zone A Availability Zone B Note: Conceptual diagram only ©2011 Amazon Web Services May not be reused or redistributed without permission
- 24. How EBS Interacts With EC2 and S3 AWS Cloud EC2 Availability Zone S3 Storage Create Snapshot EBS Snapshot EBS EBS EBS EBS EBS EBS EBS Snapshot Clone From Snapshot EBS Snapshot EC2 EC2 EC2 EBS Snapshot EBS Snapshot Internet ©2011 Amazon Web Services May not be reused or redistributed without permission
- 25. What’s the Difference Between Amazon S3 and Amazon EBS? Amazon S3 EBS Paradigm Object store File system Performance Very fast Very very fast Redundancy Across data centers Within data center Security Public Key / Private Visible only to your Key EC2 Pricing $0.14/GB/Mo. stored $0.10/GB/Mo. allocated Access from the Net? Yes(1) No(2) Typical use case Write once, read many It’s a disk drive (1) Only with proper credentials, unless ACLs are world readable (2) Accessible from Net if mounted to server and set up as FTP, etc. ©2011 Amazon Web Services May not be reused or redistributed without permission
- 26. EBS Pricing EBS Volumes $0.10 per GB-month of provisioned storage $0.10 per 1 million I/O requests No charge for mounting/unmounting volume EBS Snapshots to Amazon S3 $0.14 per GB-month of data stored $0.01 per 1,000 PUT requests (when saving a snapshot) $0.01 per 10,000 GET requests (when loading a snapshot) ©2011 Amazon Web Services May not be reused or redistributed without permission
- 27. ©2011 Amazon Web Services May not be reused or redistributed without permission EBS Volume Lifecycle Call CreateVolume 1 GB to 1 TB Create Call AttachVolume to affiliate with one EC2 instance Vast amounts of unused space Attach • Format from EC2 instance Attached OS & • Mount formatted drive Snapshot to Amazon S3 In Use Deleted CreateSnapshot Call DeleteVolume Detach Call DetachVolume
- 28. How Do Snapshots Work? Time Snapshot 2 Snapshot 3 Snapshot 1 Chunk 1 Block 1 S3 Chunk 2 Block 2 EBS Volume Chunk 3 Block 3 Chunk 4 Block 4 ©2011 Amazon Web Services May not be reused or redistributed without permission
- 29. Things You Should Know… (Part I) EBS is a network resource Networks have variable performance Striping will improve performance, although it’s usually fine to start with 4-8 drives per logical volume is observed sweet spot Great approach for high-performance database workloads EBS is already redundant behind the scenes. It makes no sense to implement RAID on top of EBS ©2011 Amazon Web Services May not be reused or redistributed without permission
- 30. Things You Should Know… (Part II) Lazy loads from snapshots When you create a volume, space is allocated immediately Data from snapshot written to disk on 1st request for maximum performance For maximum IO performance, run dd to touch all blocks Like all of AWS EBS is constantly innovating. These tips may not be appropriate in a few months ©2011 Amazon Web Services May not be reused or redistributed without permission
- 31. AWS Import/Export Get your data into AWS faster - load it onto a portable storage device and ship it to an Amazon data center Faster than Internet transfer and more cost effective than upgrading your connectivity Use cases: data migration, offsite backup, direct data interchange, disaster recovery
- 32. Amazon CloudFront Easy-to-use content delivery network Same pay-as-you-go pricing as all of AWS – No up-front contracts – No long-term commitments – Self-service sign-up Uses Amazon S3 as the origin store Worldwide network of edge locations – Seattle, Palo Alto, Los Angeles, St. Louis, Dallas, Ashburn, Newark, Miami, Amsterdam, Dub lin, Frankfurt, London, Hong Kong, Tokyo