The OpenEuropa Initiative
0 likes104 views
The document discusses the OpenEuropa Initiative, which aims to strengthen the adoption of open source tools and practices at the European Commission. It describes moving from a monolithic approach to a component-based one using loosely coupled, reusable software components. An example given is the Europa Component Library Drupal theme, which allows sites to have a consistent look and feel. OpenEuropa allows building a streamlined software as a service platform to power the next generation of the European Union web presence using these reusable open source components.
The OpenEuropa Initiative
- 1. The OpenEuropa Initiative An open source approach to software as a service at the European Commission Antonio De Marco nuvole.org - info@nuvole.org - @nuvoleweb
- 2. About me Hello, I’m Antonio. ● Director and co-founder of Nuvole ● From 2015 consulting part-time for Nuvole at the European Commission ● From 2018 co-founder and team leader of the OpenEuropa Initiative at the Directorate-General for Informatics (DIGIT) Disclaimer: the use case we are presenting today is built and maintained by a large team of consultants, working for different companies. 2
- 3. About Nuvole ● From 2010, a proud Drupal company ● Complex Drupal projects Interfacing with other systems or featuring complex front-end requirements ● Working with NGOs and public institutions An international team rooted in NGOs and comfortable working with large NGOs and public institutions (World Food Programme, European Commission, OECD, World Bank) https://nuvole.org 3
- 4. Nuvole in the Drupal Community ● Long time Drupal contributors Our open-source Drupal modules were downloaded more than 1 million times ● Drupal core contribution Co-leading the official Drupal 8 Configuration Management Initiative ● Drupal events Invited as presenters at several annual EU/US Drupal conferences (DrupalCon) ● Drupal trainers 10+ times official DrupalCon trainers + large organizations (HP, European Commission, etc.) 4
- 5. The European Union web presence 5
- 6. The European Union web presence ● 2 main institutional sites: ○ The European Commission: https://ec.europa.eu ○ The European Union: https://europa.eu ● 40 European agencies, such as: ○ EFSA, the European Food Safety Authority ○ Frontex, the European Border and Coast Guard Agency ● 27 Commission's Representations sites, such as ○ Italy: https://italy.representation.ec.europa.eu ○ Belgium: https://belgium.representation.ec.europa.eu ● Hundreds more: the Europass site, the COVID-19 response site, etc. 6
- 7. 7
- 8. 8
- 9. 9
- 10. 10
- 11. Managing institutional content is hard ● Multilingualism, supporting the 24 official languages in the EU ● Integration with the EU internal translation services ● Advanced content moderation workflow ● Scheduling content publication for sensitive news ● Harmonization of content categorization via shared taxonomies ● Centralized authentication and authorization system ● Integration with ECAS, the European Single Sign-On Service. ● Integration with the European Union Webtools widgets ● etc... 11
- 12. Drupal at the European Commission 12
- 13. Drupal at the European Commission ● 2012: a pilot project, the Multisite Platform, based on Drupal 7: ○ A common web management infrastructure ○ Rationalise the web presence of the EC ○ A coherent corporate image ○ Cost sharing amongst the different Directorate Generals ● 2014: the Europa platform, based on Drupal 7: ○ A Drupal 7 distribution built on the Multisite Platform experience ○ Aimed at migrating away from the then outdated web publishing platform ● 2015-2017: Migration of hundreds of websites to the Europa platform ○ Including the two main institutional sites 13
- 14. The Europa platform: a monolithic approach ● A SaaS built as a Drupal 7 distribution ● Organised in Drupal features, using the Features module ● Possibility for site owners to enable and disable features from the UI ● A first, genuine, attempt to business requirement orchestrations ● Allowed for extra customizations, on the final client’s expense 14
- 15. The Europa platform: a monolithic approach Drupal 7 Core Installation Profile Feature A Feature B Feature C Feature D Site custom code 15
- 16. The reality of a monolithic approach ● Large code base, difficult to maintain ● Long tests runs, not 100% covered ● New features might break existing sites ● Hard to involve people to contribute ● High deployment time ● Used more as a framework than a service 16
- 17. Other drawbacks ● A single codebase with tens of developers contributing to it ● Clients were hiring their own teams to manage the site ● 100 sites / 100 ways of implementing the same thing ○ Clients could extend it by hiring their own development team in house ○ Custom functionalities would require significant overrides ○ Development would then happen on a separate repository ● 100 ways of implementing the same thing / 1000 developers ○ Hard to maintain technical governance ○ Hard to monitor security ○ High cost of maintainability and quality assurance processes 17
- 19. Drupal 8: an opportunity In 2018 the migration of the Europa platform to Drupal 8 presented an opportunity for a paradigm shift by leveraging: ● The maturity of a proper package management system ● The adoption of semantic versioning ● The raise of containerised applications 19
- 20. The OpenEuropa Initiative OpenEuropa is a European Commission initiative aimed at strengthening the adoption of open source tools and practices, particularly focused to: ● Build and maintain loosely-coupled, reusable software components, ● Build fully-fledged solutions for the European Institutions ● Establish reusable technical governance guidelines ● Contribute back to upstream open source projects, such as Drupal https://github.com/openeuropa/documentation 20
- 21. 21
- 22. A “component-based” approach (1/2) Each component (e.g. a Drupal module, or a PHP library): ● Has its own repository ● Enjoys an independent release cycle ● Is tested separately ● Has a dedicated continuous integration pipeline ● Is semantically versioned (x.y.z) ● Describes its own dependencies 22
- 23. A “component-based” approach (2/2) Each component has everything that a developer needs to kickstart development, bundled right in its repository. For example, in case of a Drupal module: ● Running composer install builds a fully functional site, right within the component root directory, allowing to kickstart development ● Running docker-compose up -d spins off a fully functional environment, with all needed services (web server, database, etc.) ● Behat and functional PHPUnit tests are run against the target site ● Context-switching for developers is reduced to the minimum 23
- 24. Project setup Fully functional development site Continuous Integration and deployment 24
- 25. Advantages A component-based approach: ● Leverages modern dependency package manager, like Composer ● Leverages Drupal's modularity as a point of strength ● Makes it easier to write and maintain tests ● Allows for a shorter “time-to-market” per component ● Facilitates intra-team contributions ● Facilitates external contributions ● Facilitates the adoption of open source tools and practices 25
- 26. Breaking up the monolith Core Profile A B Site specific Site A Core Profile C Site specific Site B A 1.1.0 A 1.1.1 A 2.0.0 C 0.0.1 C 0.0.2 C 0.1.0 B 0.1.0 B 0.2.0 B 0.2.1 B 1.0.0 B 1.0.1 C 1.0.0 Time 26
- 27. 27
- 28. 28
- 29. 29
- 30. 30
- 31. 31
- 32. Building and maintaining OpenEuropa 32
- 33. An example: the EU’s visual identity The Europa Component Library (ECL) is a comprehensive style guide containing the design elements and visual standards to harmonize the European Union web presence. https://ec.europa.eu/component-library 33
- 34. EC EU 34
- 35. Aim: provide an ECL Drupal theme The OpenEuropa Theme component must: ● Allow Drupal sites to feature an ECL look and feel ● Allow a site owner to choose between the EC and the EU branding ● Reuse the components in their native Twig implementation 35
- 36. Theme architecture Three main components working together: ● ECL Twig Loader: loads ECL components using a custom Twig namespace ● OpenEuropa Theme: themes Drupal frontend using ECL components and exposes them as UI Patterns ● Composer Artifacts: allows projects to use a fully built theme artifact which contains ECL templates and compiled frontend assets Other than “the usuals”: the Task Runner and the Code Review component 36
- 37. Europa Component Library ECL Twig Loader Provides reusable components for an harmonized look and feel Imports ECL templates in any PHP application using Twig {% include '@ecl/...' with { 'variant' : 'default' , 'title': title, 'detail' : detail, } %} {% include '@ecl/...' with { 'Primary_image' : image, 'title': title, 'detail' : detail, } %} {% include '@ecl/...' with { 'secondary_image' : image, 'title': title, 'detail' : detail, } %} OpenEuropa Theme Drupal theme providing ECL components integration UI Patterns Field formatters View displays Entity view modes 37
- 38. Using the theme’s artifact ECL ECL Loader Theme Site builders Site developers Core components 38
- 39. Building a SaaS on top of OpenEuropa 39
- 40. A new SaaS for the EU web presence OpenEuropa allows for a streamlined way of building and maintaining SaaS platforms. Since 2018 we have been building a SaaS that is powering the next generation of the European Union web presence. The SaaS platform is: ● A private Drupal distribution ● Built using OpenEuropa components ● Tweaked by custom modules and themes, shipped within the distribution 40
- 41. Stitching it all together Public components Private distribution Theme Corporate Content Multilingual support Editorial Workflow OpenEuropa Modules Translation client Composer artifacts OpenEuropa Libraries Search API Paragraphs Contributed modules Drupal profile Custom module 1 Custom module 2 Custom theme 41
- 42. An economy of scale A few non-functional requirements were clear since the beginning: ● The new SaaS platform is built to serve hundreds of sites ● Updates need to be rolled out all at the same time ● Each site needs to be built and managed by the site owners, without employing any dedicated developer ● Eventually, a site owner can decide to “break free” from the flock, and start its own development, while still keeping the possibility of upgrading to newest SaaS versions 42
- 43. How we did it ● We have one repository called the master site: ○ A Composer-based Drupal site repository ○ It requires the SaaS Drupal profile ● Each other site instance gets its very own repository ● Each site instance repository is a git-fork of the master site ● A new release of the master site triggers a waterfall release pipeline, which releases all the other site instances 43
- 44. Release one, deploy hundreds 44 Master site Release manager tags a release Site A Site B Site C Site D Site A Site B Site C PROD PROD Deployment pipeline Test pipeline
- 45. Where we stand now 45
- 46. OpenEuropa in numbers ● More than 70 components, including Drupal modules, PHP libraries, Docker images, development tools, templates, etc. ● More than 100 developers, divided in 20 teams, contribute daily to the initiative ● Around 100 sites on the SaaS platform, out of which 50+ are currently live, in production ● Tens of custom (non-SaaS) sites have been built using the OpenEuropa components 46
- 47. Thank You nuvole.org - info@nuvole.org