Introduction to Puppet Enterprise

Editor's Notes

• #2: Sales Presentation Deck April 2016 – v17
• #3: As we’ve mentioned a few times now, all of this doesn’t matter if it’s not delivering value. Marcus does a great job talking about how Puppet Enterprise has delivered value to phoenixNAP, detailing the savings across various projects where they used Puppet Enterprise for automation. https://puppetlabs.com/blog/we-saved-over-1-million-with-puppet
• #5: —We live in a software-driven world. —Software is everywhere. It’s on our wrists, on our walls, and in our cars. It’s changed the way we shop, the way we work, the way we heal, and the way we stay connected to those we love. —In this software-driven world, every company is a software company. —For example John Deere engineers can alter the horsepower of a standard physical engine using software alone; and, their vehicles let dealers remotely wirelessly monitor and upload fixes and upgrades. Walmart has acquired a host of software companies as they make their digital transformation. They acquired Tasty Labs for software to connect with social networks, Inkiru for data analytics, Torbit for website optimization, and Luvocracy, a social product recommendation app.
• #6: The demand is relentless and coming from all sides. —Customer expectations are higher than ever. Software that is great today is out of date tomorrow. —Technology is evolving at a compounding, astounding rate. —Today’s reality (i.e. existing applications and infrastructure) must be maintained. —Need to move faster without sacrificing reliability, scale & security. —Today’s tools and practices for managing change don’t cut it. —The result of this pressure shows up in many teams in the form of mounting backlogs, projects on hold, consultants being hired to meet deadlines, etc. It’s easy to feel like we’re at a breaking point, and the waves keep coming: the rate of change will continue, with new technology continuing to come at us faster.
• #7: In order to realize the potential of software, to gain a competitive edge, and to be great at delivering differentiated software to your users, things must change. Those who don’t change will become disrupted and irrelevant. If you don’t think so, your competitors do. Change is necessary. It’s necessary to differentiate. Necessary to succeed. Companies need to —Change practices to deliver with more agility, reliability and predictability. —Change how teams together to become more collaborative, productive and innovative. —Change tooling to support these efforts, simply, at scale and securely. We are here to help companies navigate that change.
• #8: We help great companies become great software companies. We help them use automation to —deliver fantastic experiences to their users —provide better software, faster —simply, at scale and securely
• #9: Why people choose us. —We have a track record at helping customer succeed, in production, at scale, in the enterprise —We provide the leading platform (more than 30k companies use our technology) —It’s the datacenter standard. The standard way for teams to deliver, operate and collaborate around the distributed applications and global infrastructure they manage —We are the bridge to the future, giving companies a common language for providing and managing current and future technology. There will always be a future cool thing; Puppet is the way to get to it faster, with confidence. Whether your next step is cloud adoption, DevOps, or microservices and containers, Puppet is the bridge to your future.
• #10: With Puppet you gain situational awareness and drive change with confidence. Here’s how. —Define with a common language —Gain situational awareness —Orchestrate change intelligently —Ensure security & compliance —Manage every device, across your stack Let’s dig into each of these key areas.
• #11: Whether you write your own code or choose from thousands of free modules, Puppet gives you a common language to define your infrastructure. A standard language that's easy to understand, write, and share across teams —Easy to read, understand, write and share across teams —Write once, use everywhere —Testing built in —No code clobbering —Choose from thousands of free modules that are backed by a vibrant ecosystem
• #12: Puppet Enterprise Console With Puppet you get the insights, traceability, visibility and reporting to know exactly what is going on with all your software so you can drive change with confidence. The result, they can deliver more software faster than ever, while maintaining quality, security, and compliance. —Unique interactive visualization necessary for teams to manage distributed apps and global infrastructure —Graphing to help troubleshoot, optimize code, collaborate better —Real-time change visibility —Rich reporting capabilities built in —Integrations with leading tech like Splunk (see hidden slides for examples showing Grafana and Splunk)
• #13: Grafana With Puppet you get the insights, traceability, visibility and reporting to know exactly what is going on with all your software so you can drive change with confidence. The result, they can deliver more software faster than ever, while maintaining quality, security, and compliance. —Unique interactive visualization necessary for teams to manage distributed apps and global infrastructure —Graphing to help troubleshoot, optimize code, collaborate better —Real-time change visibility —Rich reporting capabilities built in —Integrations with leading tech like Splunk (see hidden slides for examples showing Grafana and Splunk)
• #14: Splunk With Puppet you get the insights, traceability, visibility and reporting to know exactly what is going on with all your software so you can drive change with confidence. The result, they can deliver more software faster than ever, while maintaining quality, security, and compliance. —Unique interactive visualization necessary for teams to manage distributed apps and global infrastructure —Graphing to help troubleshoot, optimize code, collaborate better —Real-time change visibility —Rich reporting capabilities built in —Integrations with leading tech like Splunk (see hidden slides for examples showing Grafana and Splunk)
• #15: Puppet Whether you schedule a change or push it out directly from HipChat, Git or Jenkins, Puppet gives you control, visibility, and automated intelligence to orchestrate change across your apps and infrastructure. —Real-time and direct change orchestration —Change throttling —Control change from your favorite tools, in your existing CI/CD workflows —Intelligent, ordered deployments of distributed apps and global infrastructure —Ongoing enforcement and automatic drift remediation (see hidden slides for examples showing git, jenkins and hipchat)
• #16: Git Whether you schedule a change or push it out directly from HipChat, Git or Jenkins, Puppet gives you control, visibility, and automated intelligence to orchestrate change across your apps and infrastructure. —Real-time and direct change orchestration —Change throttling —Control change from your favorite tools, in your existing CI/CD workflows —Intelligent, ordered deployments of distributed apps and global infrastructure —Ongoing enforcement and automatic drift remediation (see hidden slides for examples showing git, jenkins and hipchat)
• #17: Jenkins Whether you schedule a change or push it out directly from HipChat, Git or Jenkins, Puppet gives you control, visibility, and automated intelligence to orchestrate change across your apps and infrastructure. —Real-time and direct change orchestration —Change throttling —Control change from your favorite tools, in your existing CI/CD workflows —Intelligent, ordered deployments of distributed apps and global infrastructure —Ongoing enforcement and automatic drift remediation (see hidden slides for examples showing git, jenkins and hipchat)
• #18: HipChat Whether you schedule a change or push it out directly from HipChat, Git or Jenkins, Puppet gives you control, visibility, and automated intelligence to orchestrate change across your apps and infrastructure. —Real-time and direct change orchestration —Change throttling —Control change from your favorite tools, in your existing CI/CD workflows —Intelligent, ordered deployments of distributed apps and global infrastructure —Ongoing enforcement and automatic drift remediation (see hidden slides for examples showing git, jenkins and hipchat)
• #19: Puppet helps make security and compliance inherent and automatic. With Puppet you get the automation needed to continually enforce policies and the traceability required to prove compliance. —Define and deploy security and compliance policies —Continually monitor and enforce, automatically remediate drift —Prove compliance
• #20: Our goal is to make it possible to manage nearly any IP-connected device (we don’t cover it all today, but that is our vision, and we have the broadest coverage). Plus, you go beyond breadth and can bring automation to your entire stack, orchestrating change for core infrastructure and applications. —We cover all your technology (RedHat, Windows, AIX, AWS, Azure, networking, storage, etc.) —Rich ecosystem & key partners with the vendors you rely on —We do this through the stack, automating the delivery and operation of distributed applications and global infrastructure
• #21: And going back to “why”? Because automating helps you deliver value to the business faster and more reliably. Here are a few examples. This stat focuses on the speed gains Sony sees with Puppet Enterprise. For more point customers at https://puppetlabs.com/presentations/keynote-decentralize-your-infrastructure-alan-green-sony-computer-entertainment And here are some internal only details you should be familiar with. About Sony Computer Entertainment Worldwide Studios Sony Computer Entertainment Worldwide Studios (SCE WWS) was formed in 2005 by Sony Computer Entertainment to develop games for the PlayStation family, including the best-selling Gran Turismo and Uncharted series. The Global Platform division, a team of 2,000 people distributed worldwide, provides global support for all WWS titles and oversees performance across its hosting regions. The systems team is tasked with consolidating and automating infrastructure across regions to save time and effort, and managing the infrastructure that delivers the games and applications to millions of customers. Challenge As long-time users of Puppet Open Source, the SCE WWS team already understood the value of infrastructure as code. As the company expanded globally, it needed the reliability and support of an enterprise-ready solution that could scale with the business. To keep up with the growing demand for new games, it also needed to deploy applications faster. In order to scale, SCE WWS needed a solution that could address the following challenges: Management overhead - Various backend tools and homegrown systems created a management nightmare and prevented the team from proactively managing its time and infrastructure. Global visibility - With 15 development studios worldwide, SCE WWS needed a common management interface and greater visibility into their infrastructure. Multi-platform support - SCE WWS needed a solution that could seamlessly manage its VMware, OpenStack and AWS infrastructure. Puppet Enterprise Delivers Agility and Visibility Among the solutions they evaluated, Puppet Enterprise emerged as the clear choice. The approachable Puppet language allowed their operations teams to define and manage infrastructure code, freeing their developers to focus on building awesome games. Migrating from Puppet Open Source to Puppet Enterprise proved a huge time savings. Rather than spending time manually packaging dependencies for Puppet Open Source, the team got more than 40 integrated and supported open source projects right out of the box, including MCollective, PuppetDB, and Hiera. Other key benefits included: Consolidation - SCE WWS eliminated several backend tools and homegrown systems after implementing Puppet Enterprise. For example, it replaced an outdated asset management tool with PuppetDB. Visibility - The global team needed to have a single, shared view of infrastructure. The Puppet Enterprise console provided visibility into inventory and certificate management. Agility - With more apps under management, SCE WWS can focus on building the next generation of games to drive revenue for the company. Productivity - SCE WWS saw immediate gains in productivity by not having to manage libraries and dependencies. Seamless Management of Physical & Cloud Infrastructure AWS For SCE WWS, one of the biggest advantages of Puppet Enterprise is the ability to seamlessly manage infrastructure across diverse operating systems, cloud environments (physical, virtual, cloud) and devices (load balancers, network, storage). SCE WWS uses OpenStack for their private cloud and AWS for bursting. The process for spinning up new environments is the same for both, which has helped them increase agility and maintain consistency across all of their environments. As a new system comes up, it talks to Puppet to gather information about the environment and configure services like the package repository, DNS and monitoring. This makes it just as easy to deploy applications on their OpenStack infrastructure as it is to spin up 200 instances in AWS when they launch a new game (as well as take them down in two weeks). Results: Using Puppet Enterprise, SCEA WWS has saved time managing its infrastructure, resulting in 150 percent more deployments, faster iterations, and higher innovation. With applications under Puppet-management, the team is able to respond faster to change requests and be more proactive about hardening their systems.
• #22: Staples: Enabling DevOps & Self-Service Cloud Provisioning with Puppet Enterprise Industry: Retail Challenge Automate private cloud used by application development teams; create self-service provisioning for developers; speed deployment cycles. Solution Puppet Enterprise for automating cloud management & creating a PaaS-like provisioning service. Automation of common IT operations tasks provides consistency & frees IT team to innovate. Results Deployment cycles have gone from weeks to hours, from days to minutes. Developers can provision their own environments as needed. About Staples Staples Inc. is the world’s largest office products company and one of the biggest internet retailers. The company opened its first store in 1986, began offering its stock publicly in 1989, began selling online in 1998, and has grown to $22.5 billion in annual revenue. Top outcomes of using Puppet Enterprise Fast provisioning of cloud resources for internal application development teams. Automated systems management makes IT ops teams faster & more efficient. Increased stability and reliability. Starting environment A private cloud service for the company’s development teams consisting of thousands of virtual machines in multiple data centers. Private cloud runs on Red Hat Enterprise Linux. Additional technology includes Red Hat Satellite, Apache, Tomcat, NodeJS, MongoDB, Oracle and Redis. Why Puppet Enterprise? Staples needed to automate its private cloud spanning multiple data centers. The company also wanted to enable self-service provisioning for application development teams to free up IT Operations from having to provision developer environments so they can focus activities that add more value to the business. Cloud Automation Managing and making optimal use of Staples’ private cloud was the primary motivation behind finding a good configuration management tool, said Tom Sabin, IT manager Cloud and Automation, and Jeff Quaintance, Senior Cloud and Automation Engineer at Staples. The private cloud was set up as a service for Staples’ internal application development teams, to give them the flexibility to quickly acquire the development and test environments they need. The private cloud runs on Red Hat Enterprise Linux, and RHEL environments are managed with Red Hat Satellite. Other technologies used within the private cloud include Apache, Tomcat, NodeJS, MongoDB, Redis and Oracle. Companies turn to cloud to speed up application delivery. The hope is that cloud will deliver VMs quickly and efficiently, so developers can focus on developing business value. Automation helps realize this value, as Staples has found. “Staples had just a limited number of people with a set of very specialized skills who could automate,” said Jeff. “We had a lot of development teams just chomping at the bit to start expanding the number of tools that they used in house. They wanted that automated.” The answer was to find a configuration management system that would be accessible for developers, platform engineers and others — not just sysadmins. “We needed to be able to say, ‘Here’s a configuration management tool; we don’t expect you be experts in it, but if you can get started in your own sandbox or development environment, and contribute back to us, that would help us build and scale out our configuration management capabilities,“ said Jeff. “That’s where Puppet came in.” Today, with configuration managed by Puppet Enterprise, Staples’ IT team has created a user interface that allows colleagues on other teams to order new VMs equipped so they can start working right away. “We’re not just handing out servers; we’re handing out servers with middleware or database on top of it, moving up the stack,” said Jeff. “Puppet has really helped us get to that point.” Choosing the Right Configuration Management Platform When Jeff and other Staples teams started analyzing several open source configuration management tools, they found an application development team at Staples that had already deployed Open Source Puppet to manage application code, and that was a strong advocate for the technology. There were several factors that made Puppet highly attractive to the Staples IT team, including: Puppet has a strong community of users producing modules, patterns and ideas that the Staples team could learn from and build on. “Other tools really didn’t have that,” Jeff said. It was easier for people to quickly get to at least an intermediate level with Puppet. “In the past, other automation tools ended up being so complex that other groups couldn’t consume them without a dedicated person, and staffing for those skill sets is a significant challenge,” Jeff said. “We needed to identify a configuration management tool that other people in the organization who aren’t necessarily advanced experts could consume, use and become part of the automation community at Staples.” The Puppet skill set was not difficult to hire for, particularly people who had experience with Open Source Puppet. The application team used Open Source Puppet on a couple of different application stacks, hosted both internally and with a public cloud service. The application engineers opened requests, and platform engineers provisioned servers with Open Source Puppet and helped the developers manage their configurations. While Open Source Puppet was working out well for this particular application team, Jeff and Tom knew they would need to expand quickly to other teams, and also from Linux-only to both Linux and AIX. That made Puppet Enterprise the best choice. Plus, getting bootstrapping help from Puppet Labs professional services engineers, and knowing there would be ongoing professional support, made the choice easy. “It was the knowledge transfer [from professional services engineers] that really set us on the right track, and got us up and going a lot faster than if we’d been doing it on our own,” Jeff said. The new private cloud service was a big priority, so getting it launched with good processes and practices mattered, and as quickly as possible. Puppet Labs professional services engineers showed the Staples engineers how to write Puppet code that was portable and reusable across applications, to make it faster and easier to scale beyond the original application development team’s specific needs. Puppet Training Puppet Labs’ public and onsite private training helped get Staples engineers quickly comfortable with Puppet Enterprise. A few Staples engineers were naturals, though, and picked up Puppet quickly. “I know the Learning VM on the Puppet website has been quite helpful for people as well,” said Tom. “Some people used that [and other Puppet Labs resources] to learn Puppet on their own, and there are quite a number who have been successful.” Jeff was one of those who dived in and learned Puppet independently, but he advocates for training. “One thing to watch out for with self-training are gaps where you skipped some core things while learning on your own,” he observed. Building Credibility for the IT Ops Team Staples’ IT team is now using Puppet Enterprise to manage the VMs in its private cloud, which runs on Linux. While system engineers are certainly saving a lot of time on routine tasks, the biggest benefit is the vastly increased speed of getting a package developed and installed. “If we have the package already developed, what took days before now takes literally minutes,” Jeff said. “For a new capability — say a new application server container we need to install — what took several weeks is now down to a week.” That huge change has made a big change in the perception of IT Operations and its role within Staples. “We can turn this stuff around really, really fast, because we’ve done a good job with building the configurations within Puppet,” Jeff said. “People are amazed at how quickly we’re turning things around. We’re doing a good job of keeping pace with our development teams as they try to roll out new functionality for our business, and that’s helped me and my team build credibility within the global technology organization here at Staples.” DevOps and Infrastructure as Code Engineers have always automated with scripts, but everyone does it differently. Where sysadmins might favor shell scripts or Perl, developers might write scripts in Python. The problems in delivering quality automation arises not just from these technical differences, but also from differences between people’s individual scripting styles, even when using the same language. Puppet has brought consistency to how Staples engineers in different departments automate. It’s made it possible for more people with different skill sets and jobs — application developers, system engineers, testing engineers — to automate work, spreading the responsibility to a much larger group. Just as important, it’s now possible to put automation code, whether for applications or the systems that support it, into a source control tool. “With a centralized center of excellence around Puppet, it allows us to put that governance in place, allows us to put a process in place, so we have the right level of review prior to anything going out to production,” Tom said. This shared responsibility for delivering code that moves the business forward is, of course, what DevOps is about. “I look at it as bringing operations and development closer together to solve business problems, regardless of whether that’s a software release or you have a priority incident to address,” Tom said. Puppet has helped Staples improve its flow of code through each stage, from development through to production. The team uses r10k to manage distribution of Puppet code across all Puppet masters in Staples’ multiple data centers. Code is stored in a repository, and then after code review and approval, it’s merged. Here’s where r10k comes into play again, pushing approved, merged code to development and QA environments. Once testing is complete, a separate process is used to push code that’s passed tests into production. “I really do see Puppet as a DevOps tool, because it helps bridge that gap between development and operations, and it really level-sets,” Tom said. “Anybody can contribute Puppet code, whereas in a legacy environment, you might actually need to have root access to be able to do certain configurations or install certain software packages. Puppet allows you to abstract that away, and provides enablement for our development teams to do some work that otherwise they’d be restricted from, from an access standpoint.” Staples development teams have become interested in Puppet. “We’ve had some adoption; recently, we’ve had some teams going out and trying to do some things in Puppet on their own, and we’re working with them,” Tom said. "What we’ve pushed on is, if you have something net new, where there was no standard beforehand, and we’re doing it for the first time in Puppet, let’s push to get as much managed by Puppet as we possibly can. Now there are a couple of examples out there, where it’s almost just click and go — they can build out an entire cluster with some piece of middleware on it, and Puppet is literally managing all aspects of that configuration. As they add new servers to the cluster, Puppet is automatically picking that up and making everyone else in the cluster aware of that.” Once people see a few of these “shining examples,” as Tom calls them, “people see that and say, ‘Okay, I get it.’” Self-Service Provisioning for Application Developers When people talk about implementing DevOps, one improvement they’re often looking for is self-service provisioning for developers. “At the end of the day, we want to get something like a PaaS [Platform as a Service] offering,” Tom said. “I don’t necessarily mean an actual PaaS offering, just complete infrastructure as code, so that when developers deploy, they don’t have to think about [infrastructure]; they don’t have to worry about what’s being configured. They can literally just deploy their code,” and the infrastructure needed to support it will be deployed correctly and automatically. Full automation of application code and infrastructure code together is not far off. “We’re working on getting to the point where the development team can literally just inject a Hiera file,” said Jeff. “It will completely configure that stack for them, so they can just deploy that configuration right along with their application artifacts on new builds.” Infrastructure Automation for Consistency, Stability & Efficiency Achieving the faster deployments enabled by DevOps is terrific. But on a more basic level, most system administration teams adopt Puppet Enterprise simply to save time on common IT operations tasks. The Staples team uses Puppet for automating configurations across Linux servers. “That’s kind of an important one, because Puppet is a great way to check the box back to management and say, ‘Yes, that configuration is up and running on all of our servers,’” Tom said. While launching new processes with Puppet Enterprise has been fairly straightforward, retrofitting Puppet Enterprise to servers with existing middleware has been a challenge, yet has yielded significant savings in time and bother. “Take the example of installing an agent,” Jeff said. “Before Puppet, you could look at 50 servers, and somehow [the sysadmins] figured out how to install the agent probably 29 different ways. We puppetized the agent install, and now for all of our cloud systems it’s very consistent — just an email, and an hour later it’s done. You just classify the server and walk away, let the Puppet agent run. This is a big improvement over the inevitable delays coordinating across multiple teams with various tickets.” Another great outcome for the team is stability. “Everyone wants to have stability while still being fast; that’s one of the main objectives we have for configuration management,” Tom said. “Puppet as an automation tool helps us to get to that point, so we can quickly build and configure systems that are standardized — and when you become standardized, you become more stable.” The Future of Puppet Enterprise at Staples Eventually, Tom would like to expand Puppet to all systems at Staples, whether they’re running Linux, AIX or Windows. “The private cloud was a greenfield project, and could be carefully planned from scratch but it's a whole different ballgame when you’re applying Puppet to existing running systems that have been built under a different set of standards,” Tom said. “We’re working on trying to retrofit and build up as much momentum behind the rest of our enterprise as we have with our private cloud.” The team is eager to roll out several time-sensitive tools across the enterprise. Puppet Enterprise is installed, and nearly ready to automate rollout of one of these tools. Others will be rolled out in the same fashion once the process has been proven on the first tool. Adopting Red Hat’s Satellite management system is another project now in the planning stage, and this too will rely on Puppet Enterprise. “The PuppetDB backend and exporter resources really enable us to have a lot more power and granularity over the configuration management,” Jeff said. Beyond these projects, Jeff would like to manage network devices with Puppet Enterprise. “Puppet creates a lot of automation opportunities that we will look to leverage in the future,” he said.
• #23: As we’ve mentioned a few times now, all of this doesn’t matter if it’s not delivering value. Marcus does a great job talking about how Puppet Enterprise has delivered value to phoenixNAP, detailing the savings across various projects where they used Puppet Enterprise for automation. https://puppetlabs.com/blog/we-saved-over-1-million-with-puppet
• #24: So where do you start? Start with something straightforward. Start automating the configurations of your core infrastructure. Think things like laying down OSs, configuring them, configuring core things like NTP, DNS and SSH. Things like firewall configurations. Configuring users and groups. After that, move to application infrastructure. Databases, web servers, app servers. Then bring automation to your provisioning practices. Whether it’s laying down OSs on bare metal or spinning up new AWS environments, automate provisioning of infrastructure. And then put all the pieces together and automate application orchestration by modeling and deploying your applications and the services they use.