Managing Open Source Software in the GitHub Era
3 likes989 views
This document summarizes a presentation on managing open source software in the GitHub era. It discusses how open source development and distribution has evolved from centralized models to a decentralized model exemplified by GitHub. This shift has introduced new challenges for open source compliance, such as tracking the large number of dependencies between projects and properly attributing and licensing snippets of code. The presentation provides best practices for organizations to reduce risks, such as vetting dependency sources and embedding license information.
Managing Open Source Software in the GitHub Era
- 1. December 3rd, 2014 – Silicon Valley, CA
- 2. Managing Open Source in the GitHub Era Agenda • IntroducEon • Understanding basis issues for OSS compliance • Understanding new issues for OSS compliance in the GitHub Era • Best pracEces to reduce risk • Latest trends for process and tools to manage open source compliance • QuesEons
- 3. Managing Open Source in the GitHub Era Most Common OSS License ObligaEons • Copyright and license noEces • ANribuEon obligaEons • “CopyleP” obligaEons – Licensing of derivaEve works – Change noEces – Offer to provide source code • Carve out for OSS in your license agreements
- 4. Managing Open Source in the GitHub Era Key OSS Compliance Challenges • Tracking acquisiEon and use of open source • GeWng OSS informaEon from suppliers • Delivering OSS informaEon to customers – ANribuEon noEce creaEon and delivery – CopyleP -‐ source code packaging and delivery
- 5. Managing Open Source in the GitHub Era The “GitHub Era” • Decentralized and distributed model of Git represents many of the new OSS trends • More individuals engaged directly • Smaller projects/components with many more dependencies • Forking is encouraged à exponenEal growth in number of copies of popular components • Explosion in the number of disEnct OSS components used in products and systems -‐ from dozens to hundreds to thousands or more
- 6. Managing Open Source in the GitHub Era Growth of component repositories • In January 2011 there were less than 80K components available in the main component repositories (Maven, CPAN, Pypi, RubyGems) • In December 2014 there are more than 500K components and counEng (including NPM, Bower, Godoc, Packagist, NuGet) • In 2014, new components have been added to these repositories at the rate of over 10,000 new component-‐versions per month. Source: hNp://www.modulecounts.com/
- 7. Managing Open Source in the GitHub Era GitHub – more background • Provides Git-‐based services. • Git is a version control and content management tool from Linus Torvalds (GPL v2) • GitHub key aNributes are easy code sharing and collaboraEon • JavaScript is dominant – Other languages: Ruby, Java, PHP, Python, C/C++ • Started in 2008 – Over 17 million repos and 7.8 million users claimed • Over the last 12 months, new public open source components repositories: – Over 350K created per month (excluding forks) – Over 10,000+ created daily Source: nexB research data, Github API, 2013-‐11/2014-‐11
- 8. EvoluEon of OSS Development OLD OSS • Centralized development • CVS, Subversion • Project leader is benevolent dictator • Fewer larger components • Push releases • C/C++, Java • SourceForge, Maven • L/GPL v2, BSD, MIT • Desktops and servers NEW OSS • Decentralized development • Git / GitHub • Each developer forks code at any Eme • More smaller components • Pull releases • JavaScript, Ruby, Scala, Go • RubyGems, NPM • MIT, Apache, L/GPL v3 • Mobile and Cloud
- 9. EvoluEon of OSS Compliance Challenges OLD OSS • Components without a license • OSS code downloaded to internal codebase and compiled locally (vendored) • DistribuEon means shipment or download • Snippets NEW OSS • Many more components without a license • Deep external dependencies provisioned live from the web at deployment or runEme • DistribuEon via network / Internet deployment • Many more snippets
- 10. Managing Open Source in the GitHub Era Challenges -‐ Missing licenses • No license from copyright holder means that you do not have a right to copy or re-‐use the soPware • License at project / README level helps, but… • Ambiguous without noEces in source files • License informaEon is lost when code is parEally copied • Not a new problem, but scale is increasing rapidly
- 11. Managing Open Source in the GitHub Era SoluEons – Missing licenses
- 12. Managing Open Source in the GitHub Era Challenges – Snippets • Many snippet-‐sharing / educaEonal web sites have vague or no license terms – Someone who posts a code snippet or code example does not usually think about an explicit license – Terms of service are the typical default • StackOverflow example – Major source of advice about coding including code snippets – StackOverflow license is CC-‐BY-‐SA which is effecEvely copyleP
- 13. Managing Open Source in the GitHub Era Challenges – JavaScript example • Accelerated usage on server and clients – esp. mobile • Very common to mash up snippets of JavaScript from mulEple origins and compile/minify them in a single file for execuEon efficiency – License informaEon oPen lost when extracEng snippets – Most restricEve license applies to the JavaScript file • jQuery core components are MIT-‐licensed, but components named jquery-‐xxxxx may be copyleP-‐ licensed – ExecuEng JS on client could be considered distribuEon – And could have copyleP impact on server-‐side code
- 14. Managing Open Source in the GitHub Era Healthcare.gov case study • Healthcare.gov uses JavaScript code from Datatables (jQuery plug-‐in under BSD 3-‐clause or GPL v2) • Weekly Standard accused HHS of removing copyright & license noEces from the borrowed code • Our analysis determined that the developers did not remove noEces – they created their own Datatables.js file from snippets from other Datatables project files that did not contain license noEces • HHS quickly corrected this case, but the error indicates poor guidance to developers See hNp://www.dejacode.org/healthcare_case_study.html
- 15. Managing Open Source in the GitHub Era Challenges – Managing dependencies • Java, JavaScript, Ruby, Go and many newer languages automate provisioning of required components, aka dependencies • AutomaEon is convenient for developers, but adds risk – Dependent components may not be provisioned unEl deployment or runEme – Dependencies may be deep and recursive – AutomaEcally provisioned components may contain “hidden” security, quality or licensing issues – Accurate ANribuEon for OSS components may be very complex
- 16. Managing Open Source in the GitHub Era SoluEons – Dependency Management • A basic soluEon is “vendoring” – explicitly control provisioning of third-‐party components • SoP vendoring – define exact list of third-‐party component-‐ versions from known/veNed repositories • Hard vendoring – physically copy the third-‐party component-‐ versions to a /vendor folder in your codebase • Different repositories / plarorms provide different tools • Maven and others for Java • .gitmodules file for Git • Godep for Go, NPM for Node.js, Bundler for Ruby, etc.
- 17. Managing Open Source in the GitHub Era Compliance in the GitHub Era • Open source code is evolving and expanding too quickly for tradiEonal source code scanning and matching techniques – Number of possible matches increase with each fork – Many or most of the open source components may not actually be in your codebase (dependencies) • Risk focus on components over snippets even more important • AcceleraEng proliferaEon of languages, plarorms and repositories requires acEve management and coordinaEon from business, engineering and legal teams 17
- 18. Managing Open Source in the GitHub Era Compliance in the GitHub Era • Adapt policies to specific languages and plarorms upfront: – Define acceptable licenses in context of the technology and usage • Distributed as soPware package or Cloud-‐based service? • What does copyleP mean in context? – Create Light-‐weight process for idenEfying and resolving provenance gaps / issues – Evaluate preferred sources for provisioning components – Determine best dependency management approach for each technology 18
- 19. Managing Open Source in the GitHub Era Compliance in the GitHub Era • Embed open source provenance data in your codebase – As close to the code as possible – Adapt techniques to leverage exisEng tools and data from each plarorm / repository – Use simple approach to document provenance data if missing from original project – Instrument your build processes to idenEfy components that you actually use in each deployed product • Most accurate way to track and fulfill OSS obligaEons • Fully automate aNribuEon documentaEon • RedistribuEon, if applicable, has extra steps See also hNps://github.com/dejacode/about-‐code-‐tool 19
- 20. Managing Open Source in the GitHub Era Compliance in the GitHub Era • Establish central database of open source and third-‐ party components – Collect provenance data for all products across languages and plarorms – Document all effecEve component dependencies – Harmonize open source compliance by product across languages and plarorms • Current soluEons from several vendors, but no OSS soluEon is available today See also hNps://enterprise.dejacode.com/landing/ 20
- 21. Managing Open Source in the GitHub Era QuesEons
- 22. Managing Open Source in the GitHub Era About nexB Inc. • nexB offers: – DejaCode – a central business system for managing soPware components – SoPware analysis/audit services for products and for acquisiEons • 350+ soPware audit projects completed to-‐date – Aggregated audited codebases > 3 billion lines of source code – Aggregated value of the acquisiEons transacEons > $5B • See DejaCode at www.dejacode.com
- 23. Managing Open Source in the GitHub Era Contacts • O’Melveny & Myers Heather Meeker hmeeker@omm.com +1 650 473 2635 Subscribe to news and events alert at hNp://heathermeeker.squarespace.com/ • nexB Inc. Michael Herzog mjherzog@nexB.com +1 650 380 0680
- 24. Managing Open Source in the GitHub Era Resources – OSS Licensing Trends • Neil McAllister -‐ Study: Most projects on GitHub not open source licensed hNp://www.theregister.co.uk/2013/04/18/github_licensing_study/ • MaN Asay -‐ Open Source Is Old School, Says The GitHub GeneraEon hNp://readwrite.com/2013/05/15/open-‐source-‐is-‐old-‐school-‐says-‐the-‐github-‐ generaEon • Richard Fontana -‐ Post open source soPware, licensing and GitHub hNp://opensource.com/law/13/8/github-‐poss-‐licensing • Simon Phipps -‐ GitHub finally takes open source licenses seriously hNp://www.infoworld.com/arEcle/2611422/open-‐source-‐soPware/github-‐finally-‐ takes-‐open-‐source-‐licenses-‐seriously.html • Armin Ronacher -‐ Licensing in a Post Copyright World hNp://lucumr.pocoo.org/2013/7/23/licensing/ 24
- 25. Managing Open Source in the GitHub Era Resources – OSS Language / Repo Trends • GitHub growth and language trends hNp://redmonk.com/dberkholz/2013/01/21/github-‐will-‐hit-‐5-‐million-‐users-‐within-‐a-‐ year/ hNp://redmonk.com/dberkholz/2014/05/02/github-‐language-‐trends-‐and-‐the-‐ fragmenEng-‐landscape/ hNp://beust.com/weblog/2014/05/03/language-‐popularity-‐on-‐github/ hNp://redmonk.com/dberkholz/2014/09/26/githubs-‐vanishing-‐acceleraEon/ • Repository package growth staEsEcs hNp://www.modulecounts.com/ • GitHub Users Worldwide hNp://aasen.in/github_globe/ 25