Meetup 2020 - Back to the Basics part 101 : IaC
- 2. 2
- 3. Innovative technology consulting for business. 3 Back to the Basics part 101 Infrastructure as Code Meetup Cloud & DevOps @Devoteam 3rd of March 2020 Damien Carpy & David Frappart
- 4. 4 About the speakers ● David Frappart ○ Career = “Cloud Black mage +7000 XP” ○ MainCharacteristic = “Talk too much, waaaaayyyy tooo fast!” ○ Description = $Career ○ AdditionalInfoTag = concat(“Azure MVP because”, $MainChararacteristic) ● Damien Carpy ○ Career = “15y XP in network / 7y in private Cloud / 1y in public Cloud” ○ AdditionalInfoTag = “IaC / Terraform / AWS enthusiast” ○ Favourite_quote = “Groumph, so old world”
- 5. agenda. ● Why did people need IaC? ● IaC keypoints ● Focus on Terraform ● Live demo ● Final thoughts ● Q & A 5
- 6. Why did people need IaC? 6
- 7. 7 ...OK, actually less than ten years ago (at best)... ...administrating infrastructure was hairy. ● There was almost no automation : admins had to repeat boring tasks over and over (like Sisyphus). ● Engineers may have written some obscure custom automation scripts… ○ ...But generally wouldn’t share them, or would finally leave the company with them, or sometimes just simply lose them. ● Given which silo engineers worked in, devices (appliances and/or servers) would rarely expose API to drive them, usually interactive interfaces, hence the difficulty of writing custom scripts. nce upon a time…
- 8. 8 Mission: impossible ● Staging new devices and reprogramming them was sometimes tricky due to huge scale deployments and/or lack of skills and/or less than short timetable… sometimes it just seemed overwhelming... ● Due to this heavy vertical siloing, engineers would eventually have to wait for others to end up their task before beginning theirs. ● Ultimately, those granted with datacenter access would proceed directly in racks… with more or less success (and cleanliness). ● Evolution & refresh was often a synonym of clean slate, “do it all over again”.
- 9. 9 Major failure detected ● Managing high availability was such a huge challenge no one would really test full-scale applicative automatic failover, because there was so much at stake, they would only schedule manual failovers. ● And when huge incidents would eventually happen (remember Murphy’s law), infrastructure would not behave as expected and fail, mostly because engineers from each silo wouldn’t talk to each other, and also because in time, engineers would configure devices differently, without any consistency..
- 10. 10 And then Cloud happened. ● With cloud offerings - private as well as public - native API availability exploded for all infrastructure layers. ● RESTful APIs are now a de facto standard, all sorts of functions can now be called from a variety of speaking-API softwares, from a simple web browser to specialized tools. ● Specifically JSON language is now used for most APIs (over XML).
- 11. 11 And then network got automated too. ● Even old gen appliances also began proposing an overlay API (with more or less success). ● Even lowest, stable layers like network and security began publishing APIs. ● They were the last layers to open themselves to automation, specifically with Software-Defined Network and network virtualization turning points.
- 12. "Infrastructure as code (IaC) = manage IT infrastructure using config files." Wikipedia free interpretation 12 Now the whole IT infrastructure stack is open to full automation, we can state some definition: ...but there’s still one challenge to beat: consistency
- 13. IaC keypoints 13
- 14. 14 Benefit #1 : speed of execution, therefore cost-effectiveness ● Through heavy automation, IaC makes it feasible to handle huge provisioning, updates, or any change involving tens or hundreds of devices in short time range, with a regular-sized team. ● Ultimately, IT infrastructure lifecycle may now be directly controlled by applications (Software- Defined DataCenter) and integrate applications’ own lifecycle by becoming a component among others within a CI/CD platform. ● Doing so, IaC enables IT infrastructure to join the DevOps movement. ● Much money is saved by reducing the duration of these lifecycles and increasing productivity. ● Engineers are freed from repetitive tasks, and can now focus on improving their code and introducing added-values features.
- 15. 15 Downside #1: Do more with less ● Costs reductions inevitably lead to teams’ size reduction. ● Not everyone embraces the turning point of code management (see Benefit #2): people who don’t adapt will simply lose their jobs : it’s evolve (quickly) or die.
- 16. 16 Benefit #2: Consistency (1/2) ● Using code implies inheriting multiple properties & possibilities: ○ Code is versionable through source control (see our next meetup “Back to the Basics: GitOps”) ○ Git repository becomes the Single Source Of Truth for everyone using versioned IaC scripts. ○ Changes should be traceable, only if authentication is correctly set up. ○ Authorize only IaC tools and code reduces errors induced by human interactions that should be prohibited. ○ Any unwanted human modification should be logically be overwritten by applying IaC script again (manually or periodically), in order to avoid any discrepancies (configuration drift) and unwanted configuration state.
- 17. 17 Benefit #2: Consistency (2/2) ○ Applying well-crafted code, e.g. using modules, variables, etc. provides deterministic results = fire & forget. ○ Tools should bring idempotence : applying the same script multiple times shouldn’t build multiple times the same object or stupidly replace it.
- 18. 18 Downside #2: Code this ! ● Even if code versioning is most highly recommended and helps rolling back in case of deployments errors, using IaC requires rigor & discipline. ● IaC scripts can wipe out an entire infrastructure at once and cause service outage and / or data loss. ● Adopting an IaC tools implies a more or less complex learning curve, given the profile of the engineer and the complexity of the tool’s language. ● Scripts can become complex to write & maintain because of the wide variety of infrastructure services to drive : skills must evolve accordingly.
- 19. 19 Methods, approaches, languages ● 2 methods for dispatching code: push (agentless) or pull (with agent). ● 2 approaches of code style: either imperative (how to compute things to achieve a goal) or declarative (what to compute, i.e. the goal to reach), sometimes both. ● Tools use different scripting languages, either general-purpose (YAML, Ruby, PowerShell, etc.) or domain-specific (HCL, etc.). ● Tools themselves are built using different programming languages such as C, Python, Go, etc.
- 20. 20 Main tools & usual area of expertise ● Main competitors are Ansible, Chef, Puppet & Terraform. ○ Some of their features overlap, but they don’t follow the same objective. ○ For now, let’s just just accept the fact Terraform has no purpose being used as a system configuration & management tool. ● All 3 major public cloud providers use their own tool, but they all suffer from being vendor-related: ○ AWS CloudFormation. JSON & YAML accepted. ○ Microsoft ARM Templates. JSON accepted. ○ Google Cloud Deployment Manager. Jinja (YAML-like) & Python accepted.
- 22. ● Created by HashiCorp in 2014, Terraform is an open source & free software written in Go. ● Hashicorp Configuration Language (HCL) used for scripting is API-agnostic and declarative. (but Hashicorp kinda broke the rules with Terraform v0.12 using loops, that are imperative-style). ● Designed for driving multiple Cloud & infrastructure providers, using “providers” plugins. ○ Everyone can write their own provider plugin using Go. ● Stores the desired state configuration AND the actual state of ressources into a simple JSON- formatted file (tfstate), A.K.A. the Precious… because it holds all your secrets. ○ There’s a lock function when working with state file on a remote location. 22 Terraform main features (1/2)
- 23. 23 Terraform main features (2/2) ● Ensures idempotence : reduced risk of overprovisioning and credit card meltdown. ● Offers changes preview (plan) before applying: control THEN power. ● Changes are applied incrementally: implies fast changes. ● Handles destroying resources: useful for ephemeral environments. ● Resources are usually grouped in modules, that can also be custom-made for later use. ● Resources dependencies are automatically built using graph theory. Users don’t have to worry about resources declaration ordering + API calls are parallelized: more faster changes.
- 24. 24 Terraform pros... ● Free unless you wish to pay :-) ● Ensures all best features required for IaC. ● Lightweight & cross-platforms. ● Official site is OK but needs improvements. ○ e.g. missing outputs on some items, out-to-date examples, etc.. ● Large supporting community ○ Mostly on Stack Overflow & such. ● Fast learning curve with HCL. ○ Investment is even more cost-efficient if you manage multiple providers.
- 25. 25 Terraform cons... ● Terraform is still young but is evolving rapidly. ○ Some minor-numbered releases are in fact major releases (hello Terraform v0.12). ○ With v0.12 huge changes happened… But Hashicorp included an upgrade tool. ○ Cases were followed very seriously on their GitHub/issues section. ● Declarative approach would mean no loops (in fact: doable using “count”) ○ Since v0.12 you can use both : it may bring confusion (what about future upgrades ?). ● Not stupid-proof: “with great power, etc.”: ○ If you don’t know: stick to guidelines & best-practices, ○ If you think you know: try somewhere not sensitive first. ○ tfstate file holds your secrets (even your login & password if you’re not careful)
- 26. Live demo 26
- 27. 27 Live demo on Microsoft Azure
- 28. 28 Live demo: what do we do? ● Let’s try the classical n-tiers architecture. ● What do we need? ○ A RG ○ A VNet ○ Some Subnet and NSG because Micro-segmentation is cool ○ A public IP ○ A Load Balancer ○ Front End VM ○ Backend VM PaaS Database system if we have time ○ And that would be nice for one night
- 30. 30 Final thoughts ● IaC is made possible only if the whole IT stack offers API driving capabilities. ● Embracing IaC implies handling everything through code: ○ Code can be versioned in your favorite Single Source of Truth repository. ○ Deployments are fast, but so are errors that can wipe everything out. ○ Engineers must learn crafting code if they want to leverage IaC. ■ Therefore they must learn GitOps to work in teams and secure operations. ● Amongst other IaC tools, Terraform is (one of) the most complete and becomes the de facto standard choice for public Cloud deployments.
- 33. Damien Carpy damien.carpy@devoteam.com contact us. 33 David Frappart david.frappart@devoteam.com