#GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"
1 like2,236 views
The document outlines the evolution and significance of the Apache Software Foundation (ASF) and its contributions to big data through community-driven projects. Dr. Justin R. Erenkrantz emphasizes the importance of open-source collaboration, transparency, and meritocracy in the ASF's philosophy, alongside the guidance of the Apache incubator for nurturing new projects. He encourages involvement in the Apache Geode community to maintain code quality and fosters a spirit of inclusiveness and shared governance.
#GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"
- 1. >>>>>>>>>>>>>>>>>>>>> CREATING THE FUTURE OF BIG DATA THROUGH "THE APACHE WAY” WHY THIS MATTERS TO THE COMMUNITY Dr. Justin R. Erenkrantz, Bloomberg LP justin@erenkrantz.com / @jerenkrantz
- 2. WHY SHOULD I PAY ATTENTION? » Mentor to Apache Geode and HAWQ » Commi5er to Apache HTTP Server, APR, Subversion, Serf » Former President and Director of The Apache SoBware FoundaDon » Ph.D. from University of California, Irvine » DissertaDon: "ComputaDonal REST: A New Model for Decentralized, Internet-Scale ApplicaDons” » Head of Compute Architecture at Bloomberg LP » ~50 billion Dcks DAILY flow through our systems 2
- 3. TECH @ BLOOMBERG: OPEN SOURCE 3 » The core of our Bloomberg Professional plaorm has evolved away from proprietary code » FoundaDons of our next-generaDon infrastructure - OpenStack, Ceph, Hadoop, Spark, Solr, Chromium, Chef - are all open-source » No longer can vendors tell us that they won’t fix a criDcal bug » Places a lot of pressure on our partners to collaborate openly » Giving back to the community - h"ps://github.com/bloomberg/ » Allows us to innovate at the higher levels – helping our customers make sense of the firehose of informaDon that is available to them
- 4. TECH @ BLOOMBERG: OPEN CAN BE HARDWARE TOO! 4
- 6. PHILOSOPHY OF THE APACHE SOFTWARE FOUNDATION 6 » Let the contributors do what they do best: contribute. FoundaDon exists to do the rest. » Does not pay for contribuDons » Many are sponsored by a third-party » Staff ASF has are focused on infrastructure/PR/etc » Does not pick “winners” or “losers” » “CompeDDon” between ASF projects perfectly acceptable as long as there are healthy communiDes…think Geode and Ignite (!)
- 8. ROLE OF APACHE INCUBATOR 8 » Each project (TLP) is run relaDvely autonomously » Project karma does not automaDcally carry over » If I can commit to Geode, it doesn’t mean I can commit to Ignite! (But, I could likely earn it easily!) » Incubator was formed in 2003 as we were struggling to scale the foundaDon and repeat the model. It worked. » If a podling does not have a healthy community, it’ll never graduate. That’s OK. If the podling does become a TLP, but later loses its community, it’ll end up in the Arc. That’s OK, too.
- 9. TRANSPARENCY & MERITOCRACY 9 » Roy’s Mantra: "If it's not on the list, it didn't happen.” » Apache in the age of GitHub, JIRA, ReviewBoard, etc. » Is the mailing list doomed? » Generation gap may mean email isn’t preferred » Tools are always secondary to process » Transparency is the aim: allows others to have a voice » The tools and process are never about prohibiting face-to- face contact - but, ensuring that there is equal access for participation and permitting asynchronous decision making » Making decisions in a synchronous echo chamber (Slack, IRC, etc.) is not conducive to transparency
- 10. MAKING DECISIONS 10 » Voting is the way contributors are (and feel) empowered » “Binding” votes from recognized contributors (PMC) » Vote on code, ideas, and, most importantly, releases » Minimum acceptable quorum: 3 voters » Minimum acceptable time frame: 72 hours » The power of the dreaded “-1” (veto) » Code can be vetoed, but not releases » Veto should be cast as a last resort; used to foster discussion
- 14. NORMS OF THE COMMUNITY 14 » Over the years, most disputes I have seen come down to norms that were not agreed upon or documented » Forming an explicit consensus on release versioning and compaDbility rules up-front is so incredibly helpful. » Projects always have a tension between “new features” and compaDbility. Decide where the community wants to be early on. » The Geode wiki secDon is great. Keep it up!
- 17. Join the Apache Geode Community! • Check out: http://geode.incubator.apache.org • Subscribe: user-subscribe@geode.incubator.apache.org • Download: http://geode.incubator.apache.org/releases/
- 18. 18 THANKS! Dr. Justin R. Erenkrantz, Bloomberg LP justin@erenkrantz.com / @jerenkrantz