Building XWiki
0 likes380 views
Le document présente une série de présentations sur le projet XWiki, un wiki d'entreprise open source, et ses processus de développement, d'intégration continue et de libération. Il décrit l'importance des tests automatisés, des contrôles de qualité et de la gestion des versions tout en détaillant le flux de développement et la gouvernance communautaire. Les sujets incluent également l'utilisation de Jenkins pour la CI et l'automatisation des tests pour assurer la qualité et la compatibilité des versions.
Building XWiki
- 1. 27 au 29 mars 2013Vincent Massol,April 2019 Building XWiki
- 2. Vincent Massol • CTO XWiki SAS • Open source developer • My Projects • XWiki (community-driven open source project) • Past: Maven,Apache Cargo,Apache Cactus, Pattern Testing • Other Credentials: • LesCastCodeurs podcast about Java news • Creator of OSSGTP open source group in Paris • 3 books: JUnit in Action, Maven:A Developer’s Notebook, BBWM
- 3. Agenda • Part 0: XWiki • The project • Part 1:The XWiki Build • Automated quality checks • Different types of tests • Part 2:The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 4. Agenda • Part 0: XWiki • The project • Part 1:The XWiki Build • Automated quality checks • Different types of tests • Part 2:The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 5. What is XWiki? (1/2) • A structured open source enterprise wiki
- 6. What is XWiki? (2/2) • A platform for developing content-based web applications
- 7. Project Stats Source: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth
- 8. Motivation • Change the world! • Needs impact • Needs max number of users • Open source • Needs to show progress • Release often • Needs developers / contributors • Community-driven • Requires Time-boxing • XWiki releases every month (3 weeks for RC1, 1w for final) • Requires integration between all parts • Requires CI tool • Requires quality-control • Requires automated Tests • Requires releases as automated as possible • Requires automated Build • Requires good communication
- 10. Governance • Complete separation from XWiki SAS and XWiki.org • Only individuals working on the open source project • Rules similar to the ASF • Committership, voting (0, +1, -1), lazy consensus • xwiki.org governance and company advertising: sponsoring companies Source: http://dev.xwiki.org/xwiki/bin/view/Community/Governance
- 11. Agenda • Part 0: XWiki • The project • Part 1: The XWiki Build • Automated quality checks • Different types of tests • Part 2:The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 12. Build • Maven-based with several custom plugins • Active Quality vs Passive Quality. • Strategy: if it’s not in the build it doesn’t exist! • Common to everyone • If something is important it must fail the build, otherwise it’s not important • Try to reduce CI code to the maximum for local reproducibility and to be less tied to the CI Source: http://dev.xwiki.org/xwiki/bin/view/Community/Building Build = implement quality
- 13. Automated Checks • Standard ones: compilation, tests, javadoc linter, etc • Test execution: unit, functional, configuration,WCAG, HTML validation • Backward compatibility checks • Code quality checks • Best practices checks • Test coverage checks • Test quality checks
- 14. Tests (1/3) • Unit tests with JUnit5 & Mockito • XWiki is Component-based and we have some JUnit5 Extensions to make it easy to test/mock dependent components @ComponentTest public class DefaultVelocityConfigurationTest { @InjectMockComponents private DefaultVelocityConfiguration configuration; @Test public void getToolsReturnsDefaultTools() { assertEquals(ListTool.class.getName(), this.configuration.getTools().get("listtool")); } }
- 15. Tests (2/3) • Functional tests with Selenium/WebDriver • Using a PageObjects strategy
- 16. Tests (3/3) • Configuration tests, based on Docker • Based on TestContainers • Supported Browsers, Servlet Engines & Databases • Usable on dev machine, in the IDE! • Various other more exotic configurations: LibreOffice server, Clustering, External SOLR server • Supports Docker Out Of Docker (DOOD) @UITest(database = Database.MYSQL, databaseTag = "5", servletEngine = ServletEngine.TOMCAT, servletEngineTag = “8", browser = Browser.CHROME) public class MenuIT ...
- 17. Backward Compatibility Strategy • Check in the build with Revapi • When wanted failure, add to ignore list in pom.xml • @Deprecated then move to Legacy module using AspectJ • Use @Unstable + @Since for young APIs • Custom checkstyle check in build to prevent @Unstable from staying more than 1 cycle • {{backwardCompatibility}} xwiki macro in release notes Source: http://dev.xwiki.org/xwiki/bin/view/Community/DevelopmentPractices#HBackwardCompatibility
- 18. Code Quality Checks • Checkstyle with custom rules. For example • Verify that Script Services are not located in the internal package • Verify that @since javadoc tags have the correct format • Verify that @Unstable annotation don't stay too long • Verify that components.txt contains all Components • Verify that JUnit tests don't output content to stdout or stderr. • Verify header licenses • And more…
- 19. Best Practices Checks • Spoon checks • Verify none of the listed methods are called in our Java code • File#deleteOnExit() - Causes memory leaks since not guaranteed to be called. Not good with server software. • URL#equals() -Very slow, access the host with HTTP calls • Verify that we don't use Commons Lang < 3 (i.e. that commons- lang:commons-lang artifact is forbidden) • Verify we don't use Commons Logging or Log4j (since we use SLF4J) • And a lot more…
- 20. Test Coverage Checks - Local • Using Jacoco and Clover • Strategy - “Ratchet effect”: • Each Maven module has a threshold • Jacoco Maven plugin fails if new code has less coverage than before in % • Dev is allowed to increase threshold • Global Coverage addressed in CI (see later)
- 21. Test Quality Checks • Using PIT/Descartes Maven plugin • Concepts of PIT • Modify code under test (mutants) and run tests • Good tests kill mutants • Generates a mutation score similar to the coverage % • Descartes = extreme mutations that execute fast and have high values
- 22. Mutation - Example result = (getId() == macroId.getId() || (getId() != null && getId().equals(macroId.getId()))) && (getSyntax() == macroId.getSyntax() || (getSyntax() != null && getSyntax().equals( macroId.getSyntax())));
- 23. Mutation Example @Test public void testEquality() { MacroId id1 = new MacroId("id", Syntax.XWIKI_2_0); MacroId id2 = new MacroId("id", Syntax.XWIKI_2_0); MacroId id3 = new MacroId("otherid", Syntax.XWIKI_2_0); MacroId id4 = new MacroId("id", Syntax.XHTML_1_0); MacroId id5 = new MacroId("otherid", Syntax.XHTML_1_0); MacroId id6 = new MacroId("id"); MacroId id7 = new MacroId("id"); Assert.assertEquals(id2, id1); // Equal objects must have equal hashcode Assert.assertTrue(id1.hashCode() == id2.hashCode()); Assert.assertFalse(id3 == id1); Assert.assertFalse(id4 == id1); Assert.assertFalse(id5 == id3); Assert.assertFalse(id6 == id1); Assert.assertEquals(id7, id6); // Equal objects must have equal hashcode Assert.assertTrue(id6.hashCode() == id7.hashCode()); } Not testing for inequality! Improved thanks to Descartes!
- 24. Mutation Limitations • Takes time to find interesting things to look at and decide if that’s an issue to handle or not. Need better categorisation in report (now reported by Descartes): • Strong pseudo-tested methods:The worst! No matter what the return values are the tests always fail • Pseudo-tested methods: Grey area.The tests pass with at least one modified value. • Multi module support - PITmp • But slow on large projects (e.g. 7+ hours just for xwiki-rendering)
- 25. Test Quality Checks - Strategy • Fail the build when the mutation score of a given module is below a defined threshold in the pom.xml • The idea is that new tests should, in average, be of quality equal or better General goal with coverage + mutation: maintain qualitythan past tests. • Other idea: hook on CI to run it only on modified code/tests. • Still some hiccups regarding the mutation score stability! General goal with coverage + mutation: maintain quality
- 26. Agenda • Part 0: XWiki • The project • Part 1:The XWiki Build • Automated quality checks • Different types of tests • Part 2: The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 27. XWiki’s CI - Jenkins • XWiki Jobs are Pipeline jobs • Using “GitHub Organization” type of jobs • Autodiscovering of all Jenkinsfile in a GitHub organization • Automatic handling of branches (job creation/deletion) • Shared Pipeline library for all jobs • Clover Pipeline job to compute global TPC • Docker Pipeline jobs for running Docker-based tests on all configurations • Moving to Docker agent and DOOD (Docker Out of Docker)
- 28. Shared Pipeline Library • Features • Maven build • Check for flickers both environment flickers and test flickers and don’t send false positives emails in this case. Examples of environment flickers: • JVM crash • GitHub connection issue • X Display not ready for UI tests • Uses JIRA to log test flickers • Display screenshot of failing test in job report
- 29. Standard Pipeline Jobs • 1 job = several builds (13+ in our case) • Validates different things: • “Main”: compile and test execution including functional tests • “Quality”: Revapi checks, checkstyle, Descartes, Jacoco • “TestRelease”: Simulate a release.Verifies Maven Central requirements (javadoc, etc) • “Flavor*”:Various types of functional tests • Parallel execution for some jobs
- 30. Clover Pipeline • Issue: Local coverage can increase and global decrease • Removed code with high TPC • Code tested indirectly by functional tests and code refactoring led to different paths used • New module with lower TPC than average • Strategy: Fail the CI build if Global TPC decreases for the current version
- 31. Docker Pipeline Jobs • Currently a separate pipeline from the main one • Two issues • Requires Jenkins Docker agent to be used (Docker doesn’t work inside vServer that we use). Migration in progress. • Long to execute since it tests all configurations and thus should only be executed once per day. • Finds all Docker test modules and run Maven on each of them, passing the configuration as system properties.
- 32. DOOD • Jenkins agent running as a Docker container • Pipeline (and thus Maven) executing inside Docker • Thus Functional tests based on Docker executing inside Docker • To make it work: • Mount Docker socket (only Docker client inside Docker) • Don’t mount volumes: copy data (in and out) • Requires special care when writing the JUnit5 Test Container extension
- 33. Jenkins Docker Cloud Configuration • 3 volumes: • Docker socket • Maven settings.xml since it contains passwords • SSH keys (private data)
- 35. Agenda • Part 0: XWiki • The project • Part 1:The XWiki Build • Automated quality checks • Different types of tests • Part 2:The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 36. Release Process • Ongoing Release Notes and reference documentation • Marked in JIRA with 2 custom fields • Rolling Release Managers Source: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/ • Create Release Plan for the release
- 37. Release Plans (1/2) Source: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/
- 38. Release Plans (2/2) • Release in JIRA • Check that all issues are documented • Check Release Notes • Import translations • Build the Release • Create mail announcement • Push to Maven Central • Update Docker official image • etc Source: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/ReleasePlan845
- 39. Agenda • Part 0: XWiki • The project • Part 1:The XWiki Build • Automated quality checks • Different types of tests • Part 2:The CI (Jenkins) • The various XWiki pipelines • Part 3: Release Process • Putting it all together • Future
- 40. Future - Build Level • Move Test configurations to the build • Fix UI flickerings which are a plague • Idea: Change our DSL so that all calls always wait on something
- 41. Future - CI Level • Docker pipeline integrated into main Pipeline library • To handle branches (a pain right now) • Needs Parametrized Scheduler Plugin • Move to CD, i.e. implement the release steps in the CI • Need to resolve Maven versions and put on staging & promote when we want • Auto deploy on myxwiki.org • STAMP: DSpot on diffs, Evocrash (JIRA -> CI)
- 42. Q&A Me