Lessons Learned from Xen (Texas Linux Fest 2013)
- 1. How to (Almost) Kill a Successful Project and Bring It Back to Life Again Russell Pavlicek Xen Project Evangelist Citrix Systems Russell.Pavlicek@XenProject.org Lessons Learned from the Xen Project
- 2. A guy with a lot of experience and a really big mouth So Who is the Old, Fat Geek Up Front?
- 3. • Linux user since 1995; Linux desktop since 1997 • Linux advocate before I ever saw the software • Early advocate in DEC, Compaq • Former columnist for Infoworld, Processor • Former panelist on The Linux Show • Wrote book, Embracing Insanity: Open Source Software Development • Speaker at 40+ conferences • Currently Xen Project Evangelist employed by Citrix About the Speaker...
- 4. • We will spend a couple minutes reviewing the project • We will spend a few minutes considering its history • But we will spend the bulk of the time considering lessons to take away • We are not here for the project's history; we are here for your future About This Talk
- 5. • The premier Open Source Hypervisor • Powering some of the biggest Clouds in the industry (Amazon, Rackspace, Terremark) • Celebrating its 10th Anniversary • Now a Linux Foundation Collaborative Project – Sponsoring organizations include: Google, AMD, Intel, Samsung, Cisco, Oracle, Amazon, Verizon and more What is the Xen Project?
- 6. • The Xen Hypervisor, including ARM servers – Type 1 (Bare Metal) • Xen Cloud Platform & XAPI – Cloud readiness out of the box • Other Projects – Mirage – ARM Hypervisor for mobile devices What Does the Xen Project Produce?
- 7. • It was the first industrial-strength Open Source Hypervisor • It enjoyed a very high rate of adoption • It employed excellent technology • It had a FOSS-friendly corporation behind it • And, yet, 2 years ago, it ran the risk of being abandoned by the FOSS community before its 10th birthday The Xen Project Story (30 second version)
- 8. • The project, though viable, developed an inward focus – Reach out to the rest of the Open Source community was limited – Reach out to its users was minimal – Code development continued, but the community became insulated and stagnated – No one stepped up to contradict the rumor that Xen was dying technology, overcome by competitors How Did This Happen?
- 9. • The project forgot the importance of working with its ecosystem – Upstream projects (Linux Kernel, QEMU) were branched rather than engaged with patches – The project decided that others in the ecosystem (i.e., the distributions) would have to carry the burden of maintaining and supporting those differences – This went on too long, and the ecosystem got fed up How Did This Happen? (continued)
- 10. • The corporation backing it (XenSource) was sold to a company with a long closed source software history (Citrix) • The new corporation was interested in the technology, but had no particular interest in the project itself How Did This Happen? (continued)
- 11. • It was not about malice • It was not about fear • It was about disconnection – The project became disconnected from the FOSS community – The project became disconnected from the users – The new company became disconnected from the needs of the project, because, in part, the project never really explained what it needed from the company Why Did This Happen?
- 12. • The prognosis was not good • Xen Hypervisor had been overtaken by a commercial offering in IT mindshare • Xen Project had been overshadowed by another Open Source Hypervisor in the community • Distributions stopped facilitating Xen • The FOSS Community began to forget Xen About Two Years Ago: Battling for a Future
- 13. • About 2 years ago, a new direction was plotted • Citrix decided it wanted to understand FOSS and reinvigorate the Xen community • The company began to hire FOSS-savvy people to reconnect with the community and with users • Brought about efforts to birth Apache CloudStack, OpenDaylight, and to move the Xen Project under the Linux Foundation A Conscious Reversal in Direction
- 14. • Common themes heard at FOSS events: – What is Xen? – Xen is dead, right? – Isn't Xen closed source? – No one uses Xen anymore Reality Two Years Ago: Xen Who?
- 15. • Linux kernel 3.0 contains all that Xen needs to exist by default • Most Linux distributions are Xen-enabled – CentOS has a project to give RHEL6 users a Xen option • Xen Project now part of Linux Foundation • Launch of a new user-friendlier website (XenProject.org) Reality Today: Xen is Back!
- 16. So What Did We Learn?
- 17. • It is possible to die while you are winning – Being first is not enough – Great technology is not enough – Having a FOSS-friendly corporation behind you is not enough • A project must stay vibrant as an Open Source organism or it will perish Lesson 1
- 18. • Disconnection from users can make you a “Dead Project Walking” – You can be adding functionality, issuing new releases, and still be dying – The connection between project and users is essential – Focusing on software alone is not enough • If you are not interacting with your users, you are at serious risk Lesson 2
- 19. • Connecting with your developers != connecting with your users – You need information sources for both users and developers – If users have to dig through technical websites, wikis, etc. to answer simple questions, you are in trouble – Even Linux kernel development – arguably one of the most insular projects – cannot thrive in a vacuum Lesson 2 (continued)
- 20. • Never ignore your project's Open Source root structure – Cut flowers are beautiful – until they die – Living plants need their roots – FOSS community is the root structure, and it must spread wide – The project team cannot stand alone • Pay attention to your partners in FOSS: libraries, kernel, packaging Lesson 3
- 21. • Never ignore your support structure – Xen needed cooperation from Distributions to be properly supported – The relationship with the distributions was allowed to stagnate; it was not continuously cultivated – When one distribution invested in another Open Source virtualization solution, other distributions were swayed • Your distribution route can be critical to success Lesson 4
- 22. • Having corporate backing is not enough – The corporation has its own set of goals, and they rarely align exactly with the project's goals – When the project and the company don't mesh, friction can occur – This isn’t about good versus evil; business and projects are just two separate animals with different needs Lesson 5
- 23. • Having a FOSS company backing you is no guarantee – Even FOSS-centric companies can be sold – Sometimes they are sold to other FOSS companies (e.g., JBoss, Gluster) – Sometimes they are sold to closed source companies (e.g., MySQL, Xen, Cassatt) • If a project won’t survive without FOSS company backing, consider options (e.g., Linux Foundation) Lesson 6
- 24. • In FOSS, there is no such thing as autopilot – Intent is critical – If you are not planning to succeed, you are planning to fail – Great software is not enough; you can have the best technical solution, but if a “big dog” starts throwing its weight around, you need to be able to respond – If you're not looking at your whole ecosystem, you are inviting failure Lesson 7
- 25. • If it ain't growing, it's dying – If your project team is seeing no new blood over time, be concerned – Open Source organisms must move and grow – New folks are needed from time to time to add new ideas and keep focus on what users need Lesson 8
- 26. • Know where your project could fit in the world and make a plan to get there – Competition means you probably won't be best fit for every situation – It may not be possible to have every feature your competitors have (especially if they have much corporate backing) – Figure out who your users are, what they need, and what you need for them to use your code Lesson 9
- 27. • Competition Increases Innovation – Lack of competition can cause stagnation (consider Unix CDE) – Competing technologies keep the ball moving forward continuously – Xen's competition with KVM and VMware insured that new virtualization capabilities would keep flowing – A competing project has to stay on top of its game or it won't make it Lesson 10
- 28. • Major new features can keep your mindshare alive in the community – Large advances (e.g., ARM and Mirage) generate attention from the FOSS ecosystem and the userbase – If you aren’t making headway, your root and support structures may stop working to give you what you need – Periodic advances keeps the project growing Lesson 11
- 29. • Sometimes, perception really is reality – You can have the best code in the world, but if no one cares, it’s useless – If the rumor arises that you are dying, outmoded, or outdone by some other project, you must fight that perception – The unchallenged lie will become fact for many people Lesson 12
- 30. • In contrast, KVM managed perceptions well – It could have looked like a purely Red Hat/IBM business play when Red Hat purchased Qumranet – The relationship between Red Hat and the KVM project was well-defined & appropriate; no disconnect occurred – FOSS community embraced KVM project – Clearly, Red Hat and IBM are still focusing major business initiatives on KVM, but the community accepts that because it was done correctly Lesson 12 (continued)
- 31. • Go to local FOSS events – Submit talks • Get used to rejection and learn from it • Talk to the track chairman – “I don’t like speaking” – get over it; calculus was way harder than this – Get an ORG booth for cheap • Stock it with flyers, CDs, business cards, stickers • Shoot your mouth off – Blogs – A usable website – Podcasts (TLLTS) – Social Media • Twitter, Facebook, Google+, LinkedIn • More mouthing off – YouTube demos and tutorials – Write for Linux.com LXer, LWN.net • Get a “kick-*aas” mascot! – But our buddy the Xen panda is taken! • Shout out and live, or shut up and die! – Passion is your ally – Let it leak over everyone – Don’t imitate the suits; do what fits you Crash Course in Perception Management
- 32. • There's a new reality for FOSS projects: the corporate connection – Projects used to be primarily volunteers working nights and weekends – Today, corporations play a big role in development • You need to have a good grip on what your corporate sponsors want from you, and what you need from them; disconnection can be fatal Lesson 13
- 33. • Manage the relationship between business and project – Prevent the loss of the project’s identity – If the project appears “owned” by a business, the FOSS community might become suspicious and back away – In this case, perception is as dangerous as reality – If you forget what the project is, every else will too – Project ecosystem will wither away; only the business remains Lesson 13 (continued)
- 34. • Establish a symbiotic relationship – Business provides user feedback, resources – Project provides overall focus, goal, direction, labor – Both sides need to color in the lines – Otherwise, you get “fake Open Source:” the code is open, but there is no community, no support, no ecosystem Lesson 13 (continued)
- 35. • Make sure your project addresses its entire ecosystem: – Is the code good? – Are you reaching out to your users? – Is your development community active, engaged, and growing? – Are reaching out to the FOSS community? – Have you insured you have proper support (libraries, distros, kernels, etc.)? Final Thoughts
- 36. • If you are in a relationship with a corporation, is that relationship healthy? – Do you have freedom to do what you need to do? – Are you getting user feedback to seed new growth in the project? – Is your project's identity getting lost in the corporate identity? (if so, consider a foundation route) • Whatever else, don't give up! Final Thoughts (continued)