Running in the Cloud - First Belgian Azure project
1 like1,725 views
The document discusses how ChronoRace, a company that provides timing services for sports events, migrated their infrastructure to Windows Azure to handle unpredictable traffic bursts during large events. Key aspects covered include identifying current infrastructure limitations, migrating the VS2003 website and SQL database to Azure, implementing auto-scaling functionality, and addressing issues with video streaming and PDF generation. The migration allowed ChronoRace to scale their infrastructure as needed for events while reducing monthly costs compared to their previous setup.
Running in the Cloud - First Belgian Azure project
- 2. Running in the CloudChronoRaceMaarten Balliauw – RealDolmenE-mail: maarten.balliauw@realdolmen.comBlog: http://blog.maartenballiauw.beTwitter: @maartenballiauw
- 3. Who am I?Maarten BalliauwAntwerp, Belgiumwww.realdolmen.comFocus on webASP.NET, ASP.NET MVC, PHP, Azure, VSTS, …MVP ASP.NEThttp://blog.maartenballiauw.behttp://twitter.com/maartenballiauw
- 5. AgendaCloud computing?Windows Azure?The ChronoRace caseChronoRace?InfrastructureThe projectResourcesQ&A
- 6. What’s in the cloud?Cloud Computing
- 7. Cloud computingOn-demand self serviceStandardized accessLocation independant resource poolingRapid elasticityPay per use
- 8. IT as a ServicePrivate(On-Premise)Infrastructure(as a Service)Platform(as a Service)You manageApplicationsApplicationsApplicationsYou manageRuntimesRuntimesRuntimesSecurity & IntegrationSecurity & IntegrationSecurity & IntegrationManaged by vendorYou manageDatabasesDatabasesDatabasesServersServersServersManaged by vendorVirtualizationVirtualizationVirtualizationServer HWServer HWServer HWStorageStorageStorageNetworkingNetworkingNetworking
- 9. What’s in the cloud?Windows Azure
- 10. The Windows Azure PlatformWindows AzureSQL AzureWindows Azure platform AppFabricMicrosoft Codename “Dallas”Microsoft Codename “Sydney”
- 11. Windows AzureFlexible application hostingLights-out service managementProvide code & service model, hit ENTERStorage at massive scaleBlobs, tables, queuesComputeStorageManagement
- 12. Windows Azure
- 13. Storage optionsBlobs, tables, queuesWindows Azure Drive (a.k.a. XDrive)Virtual NTFS volume that can be mounted.vhd formatUse existing NTFS API’sEasier migrationStored on blob storage provides quick mount/unmount in other VM
- 14. SQL AzureRelational database as a serviceHighly available, automatically maintainedExtension of the SQL Server Data PlatformReportingData SyncBusiness AnalyticsDatabase
- 15. Windows Azure AppFabricSecure connections between servicesAcross organizational boundariesClaims-based access controlServiceBusAccessControl
- 17. Microsoft Codename “Dallas”Content brokerage and discovery platform Available as a CTPMicrosoft Codename “Dallas”Information Services
- 18. Microsoft Codename “Sydney” - ConnectivityWINDOWS AZURE PLATFORMENTERPRISEData ServiceService BusAccess Control ServiceCode Name “Project Sydney”
- 19. North America Region Asia Pacific RegionEurope RegionN. Europe Sub-region N. Central – US Sub-region E. AsiaSub-region W. Europe Sub-region S. Central - US Sub-regionS.E. AsiaSub-region
- 24. What’s in the cloud?Running in the CloudThe ChronoRace case
- 25. ChronoRace
- 26. ChronoRaceDelivering infrastructure for timing sports eventsRunning, biking, triathlon, mountainbike, …Founded 10 years ago, Malmédy, 4 personsTiming of every participantVideo footage of every participantPDF certificates for every participantAround 340 events per year!Belgium, Luxemburg, France, Germany, Netherlands, …Small eventsLarge eventsDwars door Brugge 8.000 participantsAntwerp 10 Miles 17.000 participants20km of Brussels 30.000 participants
- 27. What’s in the cloud?InfrastructureThe ChronoRace case
- 29. Google Analytics from Antwerp 10 Miles 2010
- 30. Big event…20km of Brussels upcoming! (30/05/2010)Expected 30.000 participantsDownloading their certificate 30.000 hits processing a PDFWatching results online 30.000 hits*Having a video online 30.000 x +/- 10MB (= 292 GB)Watching their video 30.000 video viewsAnd showing off to family and friends… 30.000 x 10MB x 10 views (avg.) = 2,92 TB bandwidth in 1 dayAbout 35 MbpsOver a 10 Mbps line…But there’s an option to a 100 Mbps line!
- 32. How to overcome these predictable peaks?Make videos only available after a day or 2Many hits to the results page can be handledVideos online during week: traffic spreadInvest in streaming?Costly, big capacity only needed for +/- 7 events per yearThe videos are not the only problem
- 33. How to overcome these predictable peaks?A better WAN connection?CostlyCurrently the server can handle all traffic because of the 10 Mbps capBuy additional hardware?CostlyEstimated use of additional hardware is +/- 7 events per year14 days of required uptime342 days of money burning
- 34. “Growing Fast“ “On and Off “ InactivityPeriod Compute Compute Average UsageUsageAverageTime Time On & off workloads (e.g. batch job)
- 35. Over provisioned capacity is wasted
- 36. Time to market can be cumbersome
- 37. Successful services needs to grow/scale
- 38. Keeping up w/ growth is big IT challenge
- 39. Complex lead time for deployment“Unpredictable Bursting“ “Predictable Bursting“ Compute Compute Average Usage Average Usage Time Time Unexpected/unplanned peak in demand
- 40. Sudden spike impacts performance
- 41. Can’t over provision for extreme cases
- 42. Services with micro seasonality trends
- 43. Peaks due to periodic increased demand
- 44. IT complexity and wasted capacity Workloadpatternsoptimalforcloud
- 45. What’s in the cloud?The ProjectThe ChronoRace case
- 46. Where did it start…Microsoft salesguy is a runnerAnnoyed by the slow / failing websiteDecided to give a callChronoRace interested in the cloud storyRealDolmen chosen as partnerAlready investing heavily in Windows Azure for 1,5 yearMostly building PoC’s to see what fits / what notExperience
- 47. Step 1: Identify the possible use case for cloudObviously: peaks, mostly predicted
- 48. Step 2: Identify possible pitfallsCurrent websiteVS2003 Web SiteDatabase1 GB or 10 GB?PDF generation32 bit component (Azure is 64 bit)License key requires registry accessStreamingPossible from Azure?Directly from storage?ScalingNot automatic on AzureDeploymentSite is +/- 300 MB, deploy this every time?PricingIs this all worth the migration?
- 49. Current web siteVS2003 Web SiteShould be easy!File -> New -> CloudAdd a web applicationCopy the VS2003 web site inRight click -> Convert to web applicationDone!
- 51. Current web siteVS2003 Web Site in a VS2010 Web ApplicationWould not compile (no namespaces)Migration took some timeAdd namespaceRestructure classesThrow out stuff that was no longer usedActually migration allowed us to get a cleaner application
- 52. DatabaseCurrent database size: around 2,5 GBSqueeze stuff into 1 GB?Data retentionRestructure database?
- 53. DatabaseDecided to go for 10 GB versionMigrationUsing open-source: SQL Azure Migration Wizardhttp://sqlazuremw.codeplex.comWorked surprisingly good!2 stored procedures could not be migratedTest migration took 7 hours initially!Don’t migrate on your home connection
- 54. DatabaseCurrent website at www.chronorace.be runs on SQL AzureAs easy as changing the connection string!After installing SQL Server 2008 R2 clientAfter installing SQL Server 2008 R2 management objects
- 55. DatabaseBackups?Do you NEED backups on Azure anyway???Not supported in current version, backups manually via SQL Azure Migration Wizard to local SQL ServerOne of the upcoming SQL Azure versions will support database backups
- 56. PDF generationUsing “WebSuperGoo ABCpdf”Requires registry access for the license keyNot allowed on AzureWe built a small PoC in Development Fabric which worked!Deployment to production learned that this was false optimism Current version: 32 bitsAzure: 64 bitsWe tried to run the non-licensed version on AzureDid not work: author has disabled 64 bits support
- 57. PDF generation64 bits version…Decided to keep this on the current web server for nowMigrate to an open-source PDF component in the future
- 58. StreamingCurrently a Flash playerDownloads chunks of +/- 1 MBShould work on blob storage!Cheap storageCheap trafficCDN supportAPI supports downloading chunks
- 59. StreamingGetting a chunck took a few seconds for each video…What if a participant watches the video 10 times?Decided to cache the video to computeDuring the advert that is shownNext view of the video is instant!Demo: http://bit.ly/d8koz6
- 60. ScalingDone by doing this:Would be awesome to have this:Not possible!“When” should it scale?“How” should it scale?“Who” / “What” is responsible for scaling?<Instancescount="2" /><InstancesminInstances="3" maxInstances="10" />
- 61. Auto-ScalingAuto scaling would be a big benefitDuring events all 4 people are busyIt is the cloud after all? The scale-up/down thing?We decided to build something for thatApplication should be able to monitor itselfOnly one instance should be responsible for that
- 62. Auto-Scaling – Scaling logicScaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)To implement per applicationJust a suggestion!Scaling logic provider uses sensor data to suggest an action (up/fast-up/down/stable)To implement per applicationJust a suggestion!
- 63. Auto-ScalingUses all the standard API’s of Windows AzureMaster election Role EnvironmentPerformance counters Diagnostics APIQueue length Storage APIScaling (changing # instances in config) Management API
- 64. Auto-Scaling in ChronoRaceSensor based on concurrent video streamsScales between 2 and 20 instances automaticallyHard to verify! The Windows Azure portal is not willing to disclose the number of instances during scale.MMC Snap-In does give useful info!http://code.msdn.microsoft.com/windowsazuremmc
- 65. DeploymentSite is +/- 300 MB, deploy this every time?Deployment through portal seems to take foreverUploading 300 MB for a very small change is crazyWe decided to move images and static content to blob storage and serve it from thereCode and dynamic content are on Windows Azure. Making it a 3,5 MB application.
- 66. DeploymentDeployment can be done in 2 ways“Upgrade”“VIP swap”Website will mostly run on 1 or 2 instancesCase 1 upgrade will bring the site down!Process: use VIP swap
- 67. Pricing – Current pricingServer lease for 2 serversIncluding 10 Mbps bandwidth+/- 450 EUR / month
- 68. Pricing – Windows AzureStorage30 GB required x $ 0,15 = $ 4,50 / monthSQL Azure10 GB database x $ 99,99 = $ 99,99 / month
- 69. Pricing – Windows AzureComputeOn average 1 instance = $ 43,20 / monthWorst case 15 days 3 instances = $ 129,60 / monthTraffic3 TB x $ 0,15 / GB = $ 450,00 / month**will probably be less
- 70. Pricing – Windows AzureEstimated worst-case scenario would cost:Current setup: 450 EUR / monthWindows Azure worst-case: 590 EUR / month
- 71. What’s in the cloud?SummaryThe ChronoRace case
- 72. ChronoRace summaryPitfalls were identified and mostly tackledExcept for the PDF generationMost hours spent with VS2003 web site to VS2010 web appDatabase migrationOne test run of 7 hours (unattended)Final migration took 1 hourDevelopment using Windows Azure Storage APIMax. 1 day spentWebsite on Azure will be put to test next weekend:20 km of Brussels is D-DayConfident that it will not explode this time
- 73. ResourcesWhere do I find more info?
- 74. ResourcesWindows Azurehttp://www.azure.comSteve Marxhttp://blog.smarx.com/PDChttp://microsoftpdc.comMy bloghttp://blog.maartenballiauw.be/category/Azure.aspxRealDolmen blogs is running on Azure!http://www.realdolmenblogs.comRealDolmen will be happy to work with you in identifying if cloud computing is beneficial to your applications.Stop by our booth for a chat!
- 75. AZUG.BE – Azure User Group BelgiumFocus on the Azure Services PlatformCommunity-drivenFocus on architectureAnd developmentAnd the business value of Azurewww.azug.be
- 76. QuestionsPossibly AnswersQ&A Session also later today,check your session scheduleor stop by our booth.
Editor's Notes
- #20: Datacenter pairing!