How to scale up, out or down in Windows Azure - Webinar

Editor's Notes

• #3: Hello good mornign afternoon of evening depending on the time zone you may be, as nancy just said, we I will be talking about Windows Azure, and things to consider when you want to Scale up, Out Or Down.
• #4: This are the items we will be talking about todayScalabilityWe will talk about linear scaling and how to achieve itDifferences between Scaling up And Scaling Out in Azure.Different characteristics and Sizes of VM’sAlso we will Talk about Caching and how it helps to increase performanceDifferent approach that can be used And finally we will talk about Azure elasticity, how to take advantage of it, and what to consider to manage load.
• #5: What scalability mean?Basically, Scalable applications are those that you can add more computer resources, and you get more computer power.Ideally that happens in a linear fashion, which is key to maintaining very scalable applications.Specially if you have thousands of nodes running your platform.Do not confuse scalability with performance.
• #6: There are Two general ways to approach scaling. Basically Two types of scaling.We talk generally about computational units. (banks of machines) in case of azure they are virtual machines.Scale up vertically entail : bigger boxes. More resources to same computational unitScale out horizontally: more machines. More computational units. Have them work together.Most of the Big scale web sites , Bing, Facebook, Twitter, follow the Scale out Architecture.
• #7: For small scenarios scaling up is pretty easy to do, you see this all the time in the enterprice.Generally if you can get a faster machine , place more memory and more disk, your application start performing better.But it has the limitations that you can only scale up so far, and they do not necessary scale in a linear fashion.Also it is more expensive to have 1 X 64 Way Server Than 64 X 1 way servers, Economies of scaling come to play.So for larger scenarios, Scaling out is how the big applications are build, Bing , Hotmail, etc, If cheaper, and well architected it can scale infinitelly.And this is one of the features that Azure bring to the table to do very easily.
• #8: Touch point Here is where you want to get linear scale, for every computational unit I am equally increasing the capacity the same amount of performance.You want to increase the capacity and also increase the performance you have to Scale out.Load testing.
• #9: Right away you are going to realize that Scalability is not the same as performance.Some times you need to Sacrifice raw performance, for scalability Asp.Net session State is an example of that, in order to scale we move our session state into other machines, I cases of big scale out applications the session state is partitioned across many machines.So you can think about grouping a set of machines to manage the state on a certain partition, this can take to your process some extra milliseconds to manage, but you gain the scalability to be able to do it, in many nodes to manage big loads.
• #10: How do we archive linear scale.For instance if every machine in your farm have to go and query state on SQL database you start having resources issues if you do not partition you data when you get to thousand of nodes, you have to start reduction or elimination of hot share resources. Partition of your data is key.Minimize the use of transactional behaviour, in this case, windows Azure leverage queues and you can used them to help scaling you application, you must be careful with this be cause it requires that you maintain this principle of idempoten, Idempoten is a big word, but it just means that if you have a function, you should be able to call that function 1,2,3 times over and over without incurring in concurrently issues of what that function is actually doing. The end state of that function stores will stay the same.By default you get round robing with windows azure, so the idea is all you nodes need to be Homogeneous and stateless, to be able to continue with the process even its change to a different node.By having multiple nodes you are more fault tolerant in case a node failure and by doing this you also have less reliance on expensive hardware.
• #11: Slide1You can create a Create a bunch of roles to do the work you need. And you can create as many instances of those roles as you need.Most of the solutions we see come out with multiple differrent roles, and is ok, you can scale this way and you can see we have some roles a clean up role, a web roles, worker role, some of this have a couple of instances.Two instances of a web site role and two of the worker role, what endsup happening if I lost and instanceSlide 2Is I lost one instance then I lost 50 % of my processing.Slide 3Instead of that you can consolidate, some of the roles and in this case if one of my instances goes down I just lost 25% of capacity.The key here is how to decide what roles can be consolidated in an azure VM and how many instances of those services do I need?And this is something that you have to balance based on you specific aplication and you can decide if you need to scale out to multiple instance or if you need to scale up to a bigger VM.
• #12: And in azure we have Multiple VM sizes, there is a Extra Small In beta. But basically you have.Small. To Extra large, 1 to * cpus and of course Different cost, A single Azure unit has a top 8 cpu core. There are some thing to consider when selecting a VM Size.So with windows azure eventually you will optimize your architecture based on the cost.The network is Share but not necesarally burstable, in cases of the Extra Large you can burst thru the 8 core, making the most of the machine. Determine how much bandwith you needWe guarantee 1gb, you 100mbs guarenteed, you share bandwith between swichesFor video streaming go for a bigger machine.
• #13: Some thing important to remember, If it does runfaster on multi… in the cloudParallel task library or any or the other optimization.
• #14: Don’t just throw big VMs at every problemScale out architectures have natural parallelismTest various configurations under loadSome scenarios will benefit from more coresWhere moving data >$ parallel overheadE.g. Video processingStateful servicesDatabase server requiring full network bandwidth
• #15: Lets talk about caching
• #16: The Key in Cashing is that ir is the easiest way to improve performanceCaching is like indexing in SQL Server, it can improve performance greatly.It is important to think about moving the data closer to consumer, you can move the data out to the client, or out to memory.And it gives you the advantage of reducing the load on the data tier, having huges ramifications in the back end, allowing for more scale.Is it valuable to cache data for just a second. Sub seconds calls make a difference.
• #17: Caching helps a lot when you have Largely static data, images , etc, you can also cash entire pages.Here you can use azure blob storage to serve those image and you save in processing and bandwith. Lowering cost and having a better performance.The idea here is to serve content once, avoid round trips to the database, lower load on web roles.There are a couple of ways you can do this you will that here soon.
• #18: The other scenario is with more dynamic content like RSS feeds.Most of people think I can not use caching for dymanic data, of course you can, the idea is to stic to the goals, Minimise traffic on the wire, fewer transactions and less hits to the database.In windows azure you pay for transactions to the database so if I have a an use that really like my RSS feeds, and is downloading it constanty, this can affect my cost, so you need to architect you solution to control this situations.So you can definitylly use caching to your advantage and control this cases.And we will see some strategies on hoe to control that.
• #19: The two strategies that you can use areStatic Content generation or Client Side CachingClient Side Caching, has to do with using the different headers available on windows Azure, to encouraging the client to cache the content localy, to minimize the amount of round trips that ocurr back into your server.There are also unique approaches to minimize visit to you sever to update content by increasing the amount of caching tags. Place there for clients to consume content inteligently.On the Static Content generation you can generate content and storage it as a cache resource and directly access it from storage.
• #20: Lets ilustrate this for client CashingWe have to scenarios here one is to officialy cache data, out of the web role and the other is cache data out of the blob storage.So a client could restfulltechniqques to go directly to the Blog and table storage and for many scenarios this is a way to minimize your cost. Also actually if I am making use of the CDN, because actually CDN Technology pushes the content to the CDN Network and increases the performance.
• #21: Generally Client Caching involve the use of etags.You can think of etags as a way to version content. It makes content version aware. They are just like a resource version number for that particular content.It is a header that is added to the Http Response, and a client will make a conditional code based on the maching of the etag, if that etag does not match the updated content is delivered.You can implent that as a conditional Put Or get that is done in a restful way behind the scene to update the content when update are present. By using blogs you can store those tags, and have the content served automatically,the windows azure blobs and tables actually use those tags under the hud, so it is a great way to implement client side caching natively.The benefit is that etags can definitelly reduce unneceserally downloads.
• #22: If I have the most current content because the etag is telling me that, I don’t need to go and download the whole contentIt will only download the content when it changes.You just need to generate the etag, just when content change on the database.
• #23: We also have a Cache Control header in the client side, it works with the CDN, its basically tells the client the how much to wait before requesting this content again.I is great for static content or content that you know it will not change in X amount of seconds to reduce the amount of unnecesary request to the server and is easy to set on Blobs just by using the XML at the bottom on the slide.
• #24: If files change how do I refresh the content, How do I expire content. On cdnYou can target clients to different containers by the data on the cache control headers.Xdrive. Your content in one page.Do I need to provision your , cloud source studio.The cloud control headers.
• #25: Static Content generationsIs about reducing the times you process content.And increase the performance.So the idea is to cache the content and storage it and send it to the client from there. This to reduce the trips to the database. The you can use all the caching headers techniques we just talk about to deliver de content efficiently.
• #26: So the idea is to cache the content and storage it and send it to the client from there. This to reduce the trips to the database. The you can use all the caching headers techniques we just talk about to deliver de content efficiently.You can spin up worker roles to update the content.You can queue messages during the day to process those updates and take advantage of lower load periods and make the most of the spare CPU cycles of VM.Content could be many things, Full pages, images, PDF files, or just portion of your content data.Blog storage can behave like a web server and deliver de storage as dynamic content.Also by using the storage you are significantly reducing the cost y you compare that with the cost of processing and accessing the data on the Database.
• #27: Manage your stale data.If they are not pulling data because of the etag, there are less transactions less cost.
• #28: Update the blog just if the etag change
• #29: Order of magnitud cheaper.
• #31: Describe the different patterns.On off: most of the scientific calculations/ simulations, that run for a period of time ,and then the data is analized for weeks, and no more process is required for while.Groing Fast, new web sites of sites that have a history of increase demandUnpredictable bursting: sites affected by sudden news or unpredictable behaviourAnd the Predictable bursting of: the pizza store every Saturday night.So how do you deal with variable load.
• #32: There ate two forms to deal with variable loadYou have to maintain excess capacity what we call Headroom, you are going to pay a little bit for excess capacity, to trade that off wit faster availability when you need it, you can maintain some asyncronous work to provide buffer but in most cases a good head room will be about 10 to 15%.The other form is to add or remove capacity when needed, this usually takes time, this is why is important the head room, you can use your current analitics to determine when to add or remove capacity. Takes two form, maintaing excess capacity, keep head room. 15% margingTo give enough time to add.Adding removal capacity.
• #33: For the head room, approach, For Web roles: you need to add additional instances to allow you to get that additional processing when is needed for when picks happend, also you need to monitor your load to be prepare to handle additional load before performance degrades.On worker roles: Try to buffer into queues when is possible to manage picks, star additional roles if levels of latency are increasing or if the queues are not clearing, and the same if the workers are idle and the queues are empty, decrease the amount of roles.How to go about this?
• #37: You need to use the Service management and diagnostics, API available in Azure.Based on the metric you obtain and we will see how to get them just in a moment, you create to me time based rules to turn on or off the amount of instances required to manage load.In case of unpredictable demand, you have to monitor some metrics and react to their behaviour.
• #38: What are the metrics we need to consider.We need to monitor de actual work done, Request per second, messages processed in the queue how often?Another level is to monitor CPU, Amount of messages in the queue, response time,And also , Rate or change in queue length, monitor how much latency in the queue is created based on the load during the day.Now how you can get this information.
• #39: You can capture information via de management API, this will give you information about Infraestructure logs, events and IIS logs and performance Counters.In some case you may need to average some measures. Also depending on how your out come is, you want to consider evaluating some business rules.
• #40: What to do when?Request are taking too longDo I have may jobs in the queueAlso ask the question, Am I soending to much money?How to improve the processeShould I automated, can I use a rules engine?This answers will allow you to decide.
• #41: When to take Action.And add or remove instances, you can use the service management API or develop you automated process.Change roles sizes, Remember to send notifications on every case, use email, instant messaging, but keep people in the loop of what is happening.And mostly manage the momentum, do not overshoot because at the end it cost you money.
• #42: Well, we see that designing for multiple instances allow you have a better elastic solution that can scale acondingly to the demand.We talk about how caching can improve performance and some techniques on how to implement it in windows Azure.And also we have describe various options to manage the different loads your environment may have.With this a finish the webinar.
• #44: If you have any questions please feel free to post them on our blog we will be more that happy to answer them, Thanks.Nancy.