Basic understanding of aws
Download as DOCX, PDF4 likes1,570 views
Amazon Web Services (AWS) provides on-demand access to computing resources and services through a global network of data centers. AWS allows organizations of all sizes to use distributed IT infrastructure to deliver a variety of use cases. Customers pay only for the resources they consume on a pay-as-you-go basis. AWS offers advantages like flexibility, cost effectiveness, and scalability. It provides a variety of computing, storage, database, analytics, and other services. AWS ensures security of customer data and applications using features like virtual private clouds, identity and access management, encryption, and monitoring tools.
Basic understanding of aws
- 1. Amazon Web Services (AWS) AmazonusesdecentralizedordistributedITinfrastructure tomake several ITresourcesavailable on demand. Use Cases:- 1. Small Organizationcanleave the ITmanagementtoAWS. 2. Large Organizationcanuse to delivertrainingtodistribute workforce. 3. Architecture consultingcompany- Togethighcompute renderingof constructionprototypes. 4. Mediacompany- Can provide differenttypesof contenttoworldwide customers. Pay-as-you-Go:- Paymentwouldbe made onthe basisof usagesof resources. 1. Computing. 2. ProgrammingModules. 3. DB storage. 4. Networking. 5. Developmentplatforms. Advantages of AWS:- 1. Flexibility:- a> More time forcore businessthroughthe instantavailabilityof new featuresandservices. b> Effortlesshostingof legacyapplications. c> Partlydeploymentof ITinfrastructure. 2. Cost Effectiveness:- a> No upfrontinvestment. b> Long termcommitment. c> Minimumexpenses. 3. Scalability/Elasticity:- Ability forasystemto expandaccordingtoworkloaddemands. AutoScaling ReducedCost | | Users | Leadsto | | | ElasticLoad Balancing IncreasedSatisfaction Namesof only fewamazonapplications:- 1. AmazonS3 (itis already designedas highlyavailable andfaulttolerant). 2. Elasticcompute cloud(EC2). 3. AmazonMachine Image (AMI). 4. Relational Database Service (RDS). There are waysto scale applicationslike EC2,AMI,RDS etc. as below:- Proactive CycleScaling:- Scale outbasedon “known” peakperiods. Proactive Event-BasedScaling:- Scale outin anticipationof increase demand.
- 2. Auto ScalingBasedon Demand: - Scale out basedonmetricssuchas CPU utilization,Network utilizationetc. Itallowourapplicationtoadaptanincrease indemand.However,itdoestakes several minutesforan auto scalingapplicationtorecognize the increase inourmetrics. Itdoes takescoupleof minutes foran EC2 instance toregisterandbecome part of your application. 4. FaultTolerant:- Abilityfora systemtooperate withoutinterruptioninthe eventof service failures. a. AutoScaling. b. AvailabilityZones. c. Multiple Regions. Oneof the reputed customer(NETFLIX) of AWSservice use a toolcalled ChaosMonkeyto check thefaulttolerancebehaviorof services by randomly disabling EC2production instances. 5. Security:- AWS provide end-to-endsecurityandprivacytoitscustomers. Itsvirtual infrastructure offersoptimumavailabilitywhile managingfull privacyforcustomers&isolation of theiroperations. AWScanensure Confidentiality,Integrity,andAvailability. AWS providessecuritytotheirglobal infrastructurealongwithavarietyof featuresforsecuringcritical data incloud. Controls AWS: - Supervisors Physical Accessof Datacenters& Networks. Audits Customercan performbelowactivities:- 1. Manage credentials. 2. Accesscontrol List. 3. Configure aVirtual Private Cloud(VPC). 4. Configure &control the OS inAWS virtual servers. 5. Configure asecuritygroupas a virtual firewallforincoming andoutgoingtraffic. 6. State a keypairwhile introducingvirtual serverinAWSwhichisbeingusedtoencryptthe login information. AWSIT infrastructure has beendesigned& managedaccording to the bestsecuritypractices certifications& IT securitystandards- a. Service OrganizationControl(SOC). b. Federal InformationSecurityManagementAct(FISMA). c. Departmentof Defense (DOD) cloudcomputingsecurityrequirementsguide. d. Criminal Justice informationservices(CJIS). e. National Institute of Standards&Technology(NIST). f. PaymentCardIndustry(PCI). The Industryspecificstandardsforcustomerstodeploytheirsolutionsinclude:- 1. HealthInsurance Portability&Accountability(HIPPA).
- 3. 2. CloudSecurityAlliance (CSA). 3. MotionPicture associationof America(MPAA). Ways-to-accessthe AWS:- 1. AWS managementconsole(GUI). 2. AWS CommandLine Interface (CLI). 3. Commandline tools- Operatesthroughcommandstomanage individual products. 4. AWS SDKs->Specifictothe usedprogramminglanguage. 5. QueryAPIs->AccessiblethroughHTTPrequest. Featuresof AWSFree usage Tier:- Amazonoffersfree usage of AWScloudservicesfor12 months. 1. AmazonS3 data storage infrastructure withstandardstorage of 5 GB facilitating20000 get requests &2000 put requests. 2. AmazonEC2 forresizingcomputingcapacityinthe cloudwith750 hourspermonth eachof Linux,RHEL etc. 3. AmazonDynamoDB with25 unitseach of Readand Write capacity and 25GB storage.It does not expire atthe endof 12 months. 4. AWS IoT,device tocloudconnectorthat can publishand/ordeliver250000 messagesevery month. 5. AmazonEC2 containerregistrythatfacilitatesstorage &retrieval of Dockerimageswitha storage capacityof 500MB per month. AWSPricing:- 1. Monthlycalculatorto estimate the cost. 2. Differentregionshave differentprices. Amazon Data Centers:-Amazon havingitsdatacentersin Asia, Europe,Australia, NorthAmericaand SouthAmerica.Each data centersite termedas Regionand eachregionconsistsof several distinctsites termedas AvailabilityZonesor AZ. Everyzone isdesignedtoremainisolatedfromfailuresdetectedin otherzonesdespite havinganeconomicallow latencyconnectionswithotherzonesinthe region.Soby placingresourcesatdifferent AZ,customercanshieldtheirdata, site orapplicationfromthe failure at one location. The AWS cloudoperatesin 32AZ and within 12 geographical locationsacrossthe globe.The AWShas plannedtoexpandtheirreal estate to 11 more AZs and5 more Regions inthe comingyear (Indiawill be one of those 5 regions). Characteristics ofRegion &AvailabilityZones:- Region:- 1. It isan isolatedphysical locationora geographical areainthe world. 2. It isusedto:- a. Run applicationsandworkloads. b. Minimize the gapbetweenrequest& response timeorlatencyforend-users.
- 4. c. Manage longtermcommitments. d. Tackle challengestoscale &manage a global infrastructure. 3. It consistsof minimum2availabilityzones connectedthroughlow latencylinks. AvailabilityZones:- 1. It isan isolated locationwithsingle ormultiple advanceddatacenters. 2. The presence of multipleAZsenable the customerstodistribute theircomputingresources amongseveral tier1internetservice provider. Security MeasuresProvidedBy AWS:- 1. CloudSecurity:- AWS providesdatasecuritybyemployingstate of the art datacentersand networkarchitecture thathelpyoumeetsecurityrelatedobjects suchas:- a> Visibility. b> Auditability. c> Manageability. d> Alertness. These featuresenablesyoutoobtainthe securitywithoutpaying the additional operationalcostof an on premise environment. 2. InformationSecurity:- AWSdeliversthe informationrelatedtothe implementedsecurityusing differentmediumsuchas:- a> Reports. b> Papers. c> Certifications. d> ThirdParty Attestations. Shared ResponsibilityModel forSecurity:- 1. AWSundertakesthe global infrastructureforservicesthatruninthe cloudwhichreferredtoas Security “Of” the cloud. 2. Customermanagestheirdata and applicationsusingthe AWSservices whichreferredtoas Security “in” the cloud. The Sharedresponsibilitymodel reducesthe customeroperationalburdenasAWSoperates,manages, and controls componentsof the hostoperatingsystemandvirtualizationlabtothe physical securityof facilitiesinwhich the servicesoperate. Physical Security of Data Centers: - Followingare the listof measurestakencare byAWS team – 1. DeployingtrainedsecurityGuards. 2. Two factorauthentication. 3. Allowingonlythe individualswithapprovedandauthorizedaccessandprovidingthe non-stop monitoring,loggingandauditingof physical accesscontrols. AWSMonitoring Tools:- AWS service providessecurityforall software andhardware productsusing differentmonitoringtoolsandtheykeepaneye on
- 5. 1. Usage of networkandserver,portscanning, unauthorizedintrusionattemptsetc. 2. Denial of Service (DOS) Attack. 3. Flooding. 4. Software orLogic attacks. MeasuresimplementedbyAWSMonitoringtools:- 1. Use SSL or secure APIendpointsforencryptedtransmissionsoverHTTPS. 2. Allowonlyusersandsoftware withcryptographickeysandcertificatestoaccessan AWS API. 3. Control external accesstoEC2 instancesusingbuilt-infirewalls,calledsecuritygroups. 4. Create individual useraccountinthe IdentityandAccessManagement(IAM) tool. 5. Enable multi-factorauthenticationof MFA withthe helpof hardware tokenora software app. 6. Offerdataencryptionof filesandobjectsstoredusingAWSservicessuchasAmazonS3, Amazon Glacier,AmazonRedshift,Oracle RDSandothers. Amazon Virtual Private Cloud (VPC):- Itoffersthe facilitytologicallyisolate aportionof AWScloudand thenintroduce the AWSresourcesinthe definedvirtual network. Thisfacilitygivesthe completecontrol on virtual networksettingssuchasIP selectingrange,creatingsubnetsandconfiguringroutingtables and networkgateways. AmazonVPCaddsa networksecuritylayertooverandabove data instances.Itcreates IPsecVPN tunnel betweenthe customernetworkandVPC. Identityand Access Management:- It providesthe below setof facilities- 1. Control the user’sservice level access. 2. SetsecurityforusersaccessingAWSservicesandresources. 3. Work withAWSusersand groups. 4. SetpermissionsforusersaccessingAWSresources. 5. Create usersandgroups. 6. Define roles. 7. SetIAMaccess control policies. 8. Define groupfunctionsrelatedtousermanagement. North Virginiaisthedefaultregion forall the new accountof AWS,which is also known as US-East-1or US Standardregion. Edge Locations:- An Edge location isan AWS datacenterwhichdoesnotcontainAWSservice.Itisused to delivercontenttopartsof the world.For example-Asof now there isno AWSregionexistinIndia.If any enduserrequestfora content,insteadof connectingtothe anyotherregionwhichisfar awayfrom Indiaitwill connecttothe closest Edge locationandreceive the cachedcontentfromthatdatacenter. Thus reducesthe amountof latencyrequiredforarequestfromotherpart of the world. AWSservicesare grouped togetherin the followingcategories:- 1. Compute andNetworking. 2. Storage and contentDelivery. 3. Database Services. 4. Analytics.
- 6. 5. APPServices. 6. DeploymentServices. 7. ManagementServices. 1.Compute andNetworking:- AWSprovidesarobustofferingof compute andnetworkingservices. Namesof fewsuchservicesasbelow:- a. ElasticCompute Cloud(EC2). i> AutoScaling. ii>Elasticload balancer. iii>Route 53. b. Virtual Private Cloud(VPC). c. AmazonRoute 53. Elastic Compute Cloud(EC2):- It providesscalablevirtual serversinthe cloud.Virtualserverscanrun differentOSbutmost commonlyruna flavorof Linux or Windows. PricingModels:There are differentkindof modelslike Reservedinstances,On-DemandInstances,Spot Instances. Auto Scaling:- AutoScalingisa service andmethodprovidedbyAWSinorderto increase the numberof instanceson-demandbasedoncertainmetrics. Thisisknownas“Elasticity”inthe AWSenvironment. Elastic Load Balancer:-Itis a service byAWSEC2 that allowstoadd instancesanddistribute traffic amongthose instances. Route53:- It isa domainmanagementservice whichwill hostinternal andexternal DNSfor your applications. Amazon Machine Image (AMI):- It isa template thatcontainsa pre-builtsoftware configuration.AMI are usedwithautoscalinganddisasterrecovery. There are differenttypesof AMIstorage types,like- Instance Store-backedInstances(Ephemeral Storage):- Blocklevel temporarystorage overthe lifeof aninstance. Livesforas longas your instance isNOTturnedoff/shutdown(youcanreboottoobutnot turn off). EBS Backed Instance(ElasticBlock Store):- Networkattachedblockstorage. Easy to backupwithsnapshotsstoredonAmazonS3 and alsoallow forpointintime snapshots. Can be attachedto one instance at a time but notin differentavailabilityzone.
- 7. Pic: -Architecture for Compute and Networking 2.Storage and ContentDelivery: - AmazonS3 (Simple Storage Service) isanobjectstorage service.Itnot onlyserve objectsthroughaCDN to CloudFront,manage accesstospecificobjects,enableversioning but itcan alsoserve HTML fileswithRoute 53. It isa simple key-valuestorage designedforunlimited objectstorage. Designedfor“11 nines” (99.999999999%) durabilityand99.99% “availability”. Apart fromthe AmazonS3 storage,there isanothertype of storage called ReducedRedundancy Storage (RRS) whichiscost effectiveandonlyfor“easilyreproducibledata”butwithlessdurability i.e99.99% comparedto “11 nines”. AmazonGlacier:- a. Archival storage type. b. Usedfor data not frequentlyaccessed. c. IntegrateswithAmazonS3lifecycle policiesforeasyarchiving. d. 0.01/gig per month. AmazonStorage Gateway: - It connectslocal datacentersoftware appliancestocloudbasedstorage such as AmazonS3. There are two types- Gateway-CachedVolumes:- Create storage volumesandmountthemasiSCSIdevicesonthe onpremise servers. The gatewaywill store the datawrittentothis volume inAmazonS3and will cache frequently access data onpremise inthe storage device.
- 8. Gateway-StoredVolumes:- Store all the data locallyinstorage volumes. Gatewaywill periodicallytake snapshotsof the dataas incremental backupsandstoresthemon AmazonS3. AmazonImport/Export:- AmazonImport/Exportgivesthe abilitytotake onpremise dataand physically mail itto AWS. AWSwill importthe datato eitherS3,EBS (Elastic Block Store) or Glacierwithin one BusinessDayof the physical device arrivingatAWS. Advantages of Import/Export:- Off-site backuppolicy. Quickmigrate LARGE amountsof data to the cloud. Disasterrecovery(AWSwill eventake S3data and shipitback to customer). 3.Database Services:- AmazonRDS (RelationalDatabase Service) isafullymanageddatabase service for relational databases.Thismeansthataccessto the underlyingOSisnotallowedandsoftware patches and managementare handledbyAWS. Database supportedbyRDS:- MySQL. Oracle. PostgreSQL. MS SQL. Aurora (A home grownrelational database forked,fullycompatiblewithMySQL). AmazonElastiCache:- Itis a fullymanaged,in-memorycache engine. Availableenginesthatpower ElastiCache are MemCachedandRedis.It is usedtoimprove performance bycachingresultsof queries, managingwebsessionsandcachingdynamicallygenerateddata. AmazonDynamoDB: - A fullymanagedNoSQLdatabase serviceprovidedbyAWS.Itissimilarto MongoDB but a home grownsolution.EasilyintegrateswithotherservicessuchasElasticMapReduce. AmazonRedshift:-A petabyte-scale datawarehousingservice. 4.AnalyticsServices: - AmazonElasticMapReduce (EMR) isa HadoopClusteringtool thatmakesiteasy to manage and integrate withHadoopClusters.HadoopisusedforbigdataanalyticsthroughElastic MapReduce.Itcan integrate easilywithotherservicessuchasRedShiftandDynamoDBfordata analytics. 5. APP Services:- AmazonSimpleWork FlowService (SWF):- Itis a longtermprocessingworkflow solution.E.g.A job whichneedstoputtogethermultipleimagesandwe needthatjobtoexecute overaspecifictime frame.It couldgo fromstartingthe job of puttingthose imagestogethertoevenahuman component that isrequire toapprove that piece of images.All of these needstocreate awork flow.The 1st stepor task of that workflowwouldbe toput those imagestogetherandthe 2nd stepcouldbe uploadit to some specificserver.The 3rd stepcouldbe to notifyan employee thattheyneedtoapprove the process.
- 9. That Work flowsservice allowsustotake up a task and stepthroughit.We can do that taskfor evenup to a year. Characteristicsof SWF are as below:- AWS control panel abilitytomonitortaskworkflow. Consistentexecution. Scalable Parallel EC2processing. Guaranteesexecutionof workflow. AmazonSimpleQueue Service (SQS):- A service whichdecoupledthe infrastructure system.E.g.Foran image processingjobwhenthe useruploadthe image SQSservice addamessage (message couldbe anythinglike asimple textof say256 kb or smallersize) inthe queuewhichwill be pickedupbyworker instance i.e. EC2 instance.If the workerinstance failsoroverloadedalsousercanstill uploadimage and lateron those messageswillbe pickedbyworkerinstance.Thisiscalledapplicationorinfrastructure decoupling. AmazonSimpleNotificationService(SNS):- A service whichcanbe usedto coordinate andmanage of deliveryorsendingof messagestospecificendpoints.Itcanalsobe usedforpublishingof IOS/Android app notifications. Namesof few endpointsare SQS,Email, SMS,and HTTPS etc. 6. DeploymentServices:- AmazonElasticBeanstalk (EB): - It helpstodeploycomplete environments i.e.dev/test/qa/prod automaticallywiththe helpof EBcommandline tool whichintegrateswithgitrepositories. AmazonCouldFormation: - A tool that allowsto code for an infrastructure anddeployresourcesbased on a pre-buildtemplate.Thisgivesthe advantage of easybackupanddisasterrecoveryandevenversion controllingthe AWSinfrastructure. 7. ManagementServices:- Identity Access Management:- A webservice thatallowsmanagingpermissionstoAWSresources. CloudTrail:- An APIloggingservice thatlogsall APIcallsmade toAWS.It doesnot matterif the API calls fromthe commandline,SDKorconsole. CloudWatch:- ItisusedtomonitorAWS servicessuchasEC2 and helpstocentralize the performance metricsintoinstancessuchas CPU usage,Networkusage andmore. Autoscalingisheavilyusedwith CloudWatch.E.g.To auto scale more workerEC2 instancesif aqueue size becomestoolarge. Details about RDS (Relational Database Service):-A fullymanagedRelational Database service in the cloud. It doesnotallowaccessto the underlyingOS. Abilitytoprovision/resize hardware ondemandforscaling. Multi AZ deploymentsforhighlevel of faulttolerance. Read replicasi.e. creatingareplicaof a database onlyforreadingpurpose,write againstthat replica database isnot allowed.
- 10. Characteristicsof RDS:- It has owninstance. Diskspace minimum5GB and Maximum3TB. Benefitsof RDSinstance:- I>Automaticminorupdates. Ii>Automaticbackups. Iii>Multi-AZwithasingle click. IV>Automaticrecoveryineventof a failover. The tradeoff of usingRDS overEC2 is,as we don’thave the access of underlyingOSsowe can’t do any customize configurationas perour requirement.Forexample,settingupMySQL cluster,settingthe swapspace forbinlogetc. AutomaticAZfailover,Multi-AZsynchronousreplicatesdatatothe backupinstance locatedinthe AvailabilityZone. I> Automaticfailovertostandby instance incase of primaryfailure. Ii> Allowsmanual failover. Backupsare takenagainstthe standbyinstance toreduce I/Ofreezesandslow downif Multi-AZis enabled. Automatedpointintime backupisallowedagainstthe RDSinstance. Backup ondatabase enginesonlyworkcorrectlywhenthe database engine is“transactional”. ReadReplicas:- Can be replicatedfromanotherreadreplica. Multiple readreplicascanhave the same source. ReadreplicasallowforelasticityinRDS. MonitorreplicationlogusingCloudWatch. CurrentlysupportMySQL,PostgreSQL,andAurora. AllowedonlyinMySQL:- Replicate anonpremise database toRDSand vice-versa. Can replicate acrossregion. Whento Use ReadReplicas:- Highnon-cacheddatabase readtraffic(elasticity). Runningbusinessfunctionsuchasdata warehousing. Rebuildingindexesinareadreplicaand promote itto a primaryinstance. RDS CloudWatch/Notifications:- Subscribe tobe notifiedwhenspecificeventstakes place. I>Snapshots. Ii>Parametergroupchanges. Iii>Optionchanges. IV>Securitygroupchanges. IntegrateswithCloudWatch I>CPU Utilization,Free ableMemory, SwapUsage.
- 11. Ii> Database connectionsandbinarylogdiskusage. Iii>Read/Write IOPS. IV>ReadReplicate latencylog. v> Read/Write throughput. Detailsabout Virtual Private Cloud(VPC):- “AmazonVPCenablesyoutolaunchAWSresourcesintoa virtual networkthatyouhave defined.This virtual networkcloselyresemblesatraditional networkthatyouwouldoperate inyourowndatacenter, withthe benefitsof usingthe scalableinfrastructure of AWS”. A VPCresemblesOn-premise:- Private datacenters. Private corporate network. Pic:- blocking traffic (subnet or Security group) Benefitsof Virtual Private Cloud:- Abilitytolaunchinstancesintoasubnet. AbilitytodefinecustomCIDR(IPaddressrange) inside eachbucket.
- 12. Abilitytoconfigure route tablesbetweensubnets. Abilitytoconfigure internetgatewaysandattachthemto subnets. Ability tocreate a layerednetworkof resources. More securitysettingstoprotectcloudassets. Extendyournetworkintothe cloudwithVPN/VPGandan IPSecVPN tunnel. Layeredsecurity I>Instance securitygroup. Ii>SubnetnetworkACLs(Essentiallyafirewall forincomingpacketsonthe subnetlevel). Understandingof defaultVPCfromAWS:- It comeswitheveryAWSaccount,whose setupisdifferentthananon-defaultVPCs. It ismeantto allowthe usereasyaccess to a VPCwithouthavingtoconfigure itfromscratch. It has an internetgatewayattached. Each instance has a defaultprivate andpublicIPaddress(definedonthe subnetsettings), rememberpublicIPaddressesare attached/routedtoanENI(ElasticNetworkInterface) thathasa Private IPaddressattachedto an instance(NAT). VPCPeering:- VPCPeeringenablethe abilitytocreate a directnetworkroute betweenone VPCandanother.This allowsthe sharingof resourcesbetweentwosubnetsasif itwason the same network.Basically,at a highlevel itcreatesa linkbetweenthe two. VPCpeeringcanoccur betweenotherAWSaccountand otherVPCswithinthe same region. VPCpeeringconnectionscannotoccurbetweentworegions. Scenarios:- i>Peering2 VPCs– Companyrunsmultiple AWSaccountsandyouneedto linkall the resourcesasif theywere all underone private network(assumingresourcesinthe same region). ii>Peeringtoa VPC – Multiple VPCscanconnecttoa central VPCbutcannot communicate with each other,onlycommunicationcanoccurbetweenthe peeredVPCandthe primary.Thisuse case couldbe if thirdpartywas sharinga resource that the customersneededtoconnectto(file sharing, customeraccess,active directory). Limitsof VPC:- 5 VPCsperregion(more availableuponrequest). 5 internetgateways(equal toVPClimitbecause youonlyhave one internetgatewayperVPC). 50 customergatewaysperregion. 50 VPN connectionsperregion. 200 route tablesperregion/50 entriesperroute table. 5 elasticIPaddresses. 100 securitygroups. 50 rulespersecuritygroup. Securitygroups pernetworkinterface (remembersecuritygroupsare onthe VPClevel).