The Development Infrastructure of the TYPO3 Project
0 likes888 views
The document discusses the development infrastructure of the TYPO3 open source project. It describes how TYPO3 uses various open source tools for collaboration and communication (mailing lists, forums), source code versioning (Git, Gerrit for code reviews), quality assurance and testing (Jenkins, PHP linting tools, TravisCI), and documentation (reStructuredText, Sphinx). The talk provides examples of how these tools are implemented and integrated in the TYPO3 development process. It also addresses requirements for open source projects like collaboration, source code management, quality control, and documentation.
The Development Infrastructure of the TYPO3 Project
- 1. Brussels, 02. - 03.02.2013 1 The Development Infrastructure of the TYPO3 project Lizenz: CC BY-SA 3.0 (pictures of other authors excluded)
- 2. Brussels, 02. - 03.02.2013 What to expect from this talk? 2 ✘ Learn to set up a perfect development infrastructure ✘ See the perfect infrastructure of the TYPO3 project ✔ Learn how people in another FLOSS project work ✔ See an infrastructure built from 100% open-source ? Find inspiration for your project ? Get in touch with me/us for collaboration admin@typo3.org - we’re happy to share!
- 3. Brussels, 02. - 03.02.2013 Steffen Gebert 3 • TYPO3 CMS Core Team Member (2010-2012) • TYPO3 Server Admin Team Member (since 2011) • PhD Student in Computer Science (Communication Networks, since 2011) • E-Mail: steffen.gebert@typo3.org Twitter: @StGebert
- 4. Brussels, 02. - 03.02.2013 Questions! 4 • Happy to answer your questions! • Come down here after the talk! comments included! • Slides available Foto “Question mark”: Yello-Dog http://www.sxc.hu/photo/1279316 http://slideshare.net/stephenking/ nice that you found the slides - have fun ☺
- 5. Brussels, 02. - 03.02.2013 What’s TYPO3? 5 • TYPO3 CMS • Started: 1997, first release: 1999 • More than 5M downloads • PHP-based, GPLv2 • New products • TYPO3 Flow: PHP framework • “TYPO3 Flow and the Joy of Development” by Robert Lemke from yesterday • TYPO3 Neos • New CMS based on TYPO3 Flow - alpha state
- 6. Brussels, 02. - 03.02.2013 Who’s TYPO3? 6 • TYPO3 is community-driven open-source • no company behind, TYPO3 Association for financial support • Core teams steering development (99.x% volunteer work) • CMS with ~25ppl., Flow/Neos with ~12ppl. Photo: Tobias Liebig
- 7. Brussels, 02. - 03.02.2013 7 Requirements of an Open Source Project Photo by brokenarts: http://www.sxc.hu/photo/207291
- 8. Brussels, 02. - 03.02.2013 Requirements 8 • Collaboration & Communication • Source Code & Versioning • Quality Assurance & Testing • Documentation & Translation Photo by brokenarts: http://www.sxc.hu/photo/207291 There are couple of temas working on tasks other than development - Design, Usability, Marketing, Screencast, Event teams - Teams and committees of the TYPO3 Association (Board, Expert Advisory Board, Budget Control Committee)
- 9. Brussels, 02. - 03.02.2013 Using own infrastructure 9 • We’ve been there before Github ☺ • Sourceforge.net initially used for CVS • Got hacked, often unreliable • Owning our data (esp. issue tracker) • Own account management • Provide central place and connected services for the TYPO3 community • Nevertheless, open towards great, free hosted services for less critical things • Human resources are a problem, of course
- 10. Brussels, 02. - 03.02.2013 10 Communication and Collaboration Image by fangol: http://www.sxc.hu/photo/1269811
- 11. Brussels, 02. - 03.02.2013 Communication in the TYPO3 Project 11 • Mailing lists and news groups used since 2003 • Ten years later (last Monday): forum.typo3.org
- 12. Brussels, 02. - 03.02.2013 Forum and Official Communication 12 • Forum • Based on FUDforum • Bi-directional sync with news groups • Easier access to official channels (we hope..) • Decisions take place in lists • Transparency towards more people • Avoid hidden communication • Avoid surprises (well, we still fail too often) Image by svilen001: http://www.sxc.hu/photo/1165445
- 13. Brussels, 02. - 03.02.2013 Forge Platform 13 • Based on Redmine • Project space for 2400+ projects • Core projects, Community Extensions, Other teams • Features Want to contribute? ➪ See who’s responsible • Membership listing not the • Issue tracker best one ☹ • Wiki, Repo Browser, Roadmap News, Time Tracking forge.typo3.org It’s often a bit chaotic in a big open source community. Being able to find the persons in charge is important! Forge is the place where to search. We know that we’re running a very outdated redmine version. We’re sorry and ashamed. We’re working on an upgrade.
- 14. Brussels, 02. - 03.02.2013 14 Source Code and Versioning Source: http://nvie.com/posts/a-successful-git-branching-model/
- 15. Brussels, 02. - 03.02.2013 Source Code Versioning 15 • TYPO3 is using Git since 2010/11 • .. but we use a special flavor of Git • Unfortunately still lot of extensions restricted to subversion Yes.. it’s true. The high Subversion usage is our (=admin team) fault.. we still haven’t launched official git support for community extensions. But it’s near.
- 16. Brussels, 02. - 03.02.2013 16 Quality Assurance and Testing Photo by bredmakers: http://www.sxc.hu/photo/1280927
- 17. Brussels, 02. - 03.02.2013 Motivation for Code Reviews 17 • Bugs do exist • Ensure code quality through peer-reviews (4 eyes in our case) • Discussing code improves it • Reviews before merge • Learn from each other • Make development transparent, involves community • Using Gerrit Code Review as Git server and review tool Photo by asifthebes: http://www.sxc.hu/photo/912841 every change has to be reviewed by at least two other persons, one of them being team member
- 18. Brussels, 02. - 03.02.2013 Gerrit Code Review 18 review.typo3.org All code-changes for core projects are peer-reviewed by at least two people (one of them team member) (Disclaimer: except trivial changes)
- 19. Brussels, 02. - 03.02.2013 Gerrit Code Review 19
- 20. Brussels, 02. - 03.02.2013 Gerrit Code Review 20 • Diff view review.typo3.org
- 21. Brussels, 02. - 03.02.2013 More About Gerrit 21 Interested in more Gerrit details? “Using Gerrit Code Review in an open-source project” by myself :-) Here at 15:40
- 22. Brussels, 02. - 03.02.2013 Continuous Integration and Code Quality 22 • ci.typo3.org runs our Jenkins server • PHP lint (syntax check) • PHP CodeSniffer (enforcement of Coding Guide Lines) • PHP Mess detector • Automatic down-vote in Gerrit in case of violations • metrics.typo3.org runs Sonar • Code Metrics • For Core and Extensions, 5600+ projects
- 23. Brussels, 02. - 03.02.2013 TravisCI - More Continuous Integration 23 • TravisCI is »a hosted continuous integration service for the open source community« • What it does? It executes tests! • Fetches your code from Github • Supported languages: C++, Clojure, Erlang, Go, Groovy, Haskell, Java, JavaScript (with Node.js), Perl, PHP, Python, Ruby, Scala To execute post-merge unit tests, we are replicating our Git repo to Github, from which Travis fetches it. In order to let Travis execute them for every patch set, we are currently building a tool to push and fetch Travis’ results and add them to Gerrit.
- 24. Brussels, 02. - 03.02.2013 TravisCI 24 • Spins up virtual machines (using Vagrant and Chef) • Runs your unit tests in different an environments • Example: php: -‐ 5.2 -‐ 5.3 -‐ 5.4 PHP 5.2 PHP 5.3 PHP 5.4 env: -‐ DB=mysql MySQL ✔ ✔ ✔ -‐ DB=pgsql PostgreSQL ✔ ✘ ✔
- 25. Brussels, 02. - 03.02.2013 25 Documentation and Translation Photo by topfer: Photo by ftibor: http://www.sxc.hu/photo/969875 http://www.sxc.hu/photo/533821
- 26. Brussels, 02. - 03.02.2013 Translating TYPO3 26 • Pootle-based • Core + Extensions can be translated • Language packages generated, provided for download translation.typo3.org Photo by ftibor: http://www.sxc.hu/photo/533821
- 27. Brussels, 02. - 03.02.2013 Documentation 27 • Was a raw point for long time • Used OpenOffice 1.x .sxw files since ages • Migration to DocBook • started / planned, but format turned out to be too complicated • Migration to reStructuredText (reST) • Started ~2011 • Milestone reached with TYPO3 6.0 (Oct. 2012): docs.typo3.org • TYPO3 CMS Core documentation converted to reST Photo by topfer: http://www.sxc.hu/photo/969875 - During .sxw times, we used a wiki page to write down pending documentation changes, which sucked - docbook format is based on xml, pretty heavyweight. Probably because of markdown, people feel better with plain-text?
- 28. Brussels, 02. - 03.02.2013 reStructuredText 28 created for • Plain-text format, similar to markdown docs.python.org • Parsed and rendered through sphinx • Rendering chain at TYPO3 • Managed through TYPO3 Flow app lower barrier for adoption • Can import old .sxw files • You can view sources on docs.typo3.org! • Changes reviewed through Gerrit Documentation is parsed and rendered through the tools docutils and sphinx (both created for docs.python.org). http://docutils.sourceforge.net/rst.html Intersphinx allows cross-manual referencing Being able to import old manuals hopefully lowers the barrier to migrate documentation from sxw to reST
- 29. Brussels, 02. - 03.02.2013 Solutions at a Glance 29 • Collaboration & Communication • Mailing lists / NNTP / Forum using FUDforum, Redmine • Source Code & Versioning • Git with Gerrit as server and review tool • Quality Assurance and Testing • Gerrit, Jenkins with couple PHP tools, TravisCI • Documentation & Translation • reStructuredText, sphinx, Pootle Honestly, we use couple of more tools: - mailman runs our mailing lists.. what else (ah, we have a sympa server vor TYPO3 Assoc member lists, but I wouldn’t recommend that software) - etherpad-lite for meeting notes and collaborative writing: http://notes.typo3.org - BigBlueButton web conferencing server: http://bigbluebutton.typo3.org - and we have been using Mantis as bug tracker for years. Mantis (http://bugs.typo3.org) is now archived and Redmine (http://forge.typo3.org) acts as tracker for *all* TYPO3 projects. - yes, we are still running a subversion server for some thousand projects -
- 30. Brussels, 02. - 03.02.2013 Questions! 30 • Happy to answer your questions! • @StGebert on Twitter • steffen.gebert@typo3.org comments included! • Slides available Foto “Question mark”: Yello-Dog http://www.sxc.hu/photo/1279316 http://slideshare.net/stephenking/
- 31. inspiring people to share inspiring people to share