Software Architecture Erosion and Modernization
- 1. Stop the Software Architecture Erosion Bernhard Merkle Frederic Madiot R&D Software Products Manager SICK AG, Germany OBEO, France Contact us on linkedin.com or xing.com Page: 1
- 2. Agenda – Software Architecture – Detect Architectural Erosion – Eclipse + Open Source Projects – Model-Driven Reverse-Engineering Page: 2
- 3. Architecture, Erosion Page: 6
- 4. Software-Architecture: Definitions IEEE 1471-2000: The fundamental organization of a system, embodied in its components, their relationship to each other and the environment, and the principles governing its design and evolution. Page: 7
- 5. Architectural Erosion “Sometimes the developers manage to maintain this purity of design through the initial development and into the first release. More often something goes wrong. The software starts to rot like a piece of bad meat”. Uncle Bob: “Agile Software Development” Page: 9
- 6. Architectural Erosion Why should we care ? – In (lots of) Projects, Architecture decay happens – We are not alone, as we‘ve some good representatives… ;-) There is/was a plan Page: 10
- 7. X-raying Software… Page: 11
- 8. Findbugs: the first years 0.7.2 0.8.6 (03/2004) (10/2004) Page: 12
- 9. Findbugs: first erosion 0.8.7 (05/2005) Page: 13
- 10. Tangle increase… 0.8.8 1.0.0 (05/2005) (06/2006) Page: 14
- 11. Tangle increase… 1.3.0 Page: 15 (07/2007)
- 12. ONE BIG Tangle… 1.3.8 (03/2009) Page: 16
- 13. Page: 17
- 14. Modeling Subsystems: Page: 20
- 15. Fixing Architectural Violations Page: 21
- 16. Fixing Architectural Violations Page: 22
- 17. Fixing Architectural Violations Page: 23
- 18. Fixing Architectural Violations Page: 24
- 19. Fixing Architectural Violations Page: 25
- 20. Fixing Architectural Violations Page: 26
- 21. Fixing Architectural Violations Page: 27
- 22. Tools for Architecture-Analysis – SotoArc SonarJ – Bauhaus Lattix – Structure101 Klocwork, Coverity – http://code.google.com/p/architecturerules/ – ODASA, CodeCrawler, Codecity, SeeSoft, XRadar, XDepend, SonarSource, Kalistick, Sqoring, … – Eclipse-Based: MoDisco (+Modernization of Applications) http://se-radio.net/ episode-115-architecture-analysis Page: 28
- 23. Basic Approaches (Dependencies) – makedepend, jdepend – Eclipse (Java Build Path) – OSGI (Dependencies) Page: 29
- 24. Basic Approaches PDE Dependency Visualization Page: 30
- 25. Missing in basic approaches: Architecture Analysis (Deviation) Drill-Down + Aggregation Displaying results Monitoring changes, trends Rating of Architecture Requirements for Architectural Analysis Tools Page: 31
- 26. Architecture Analysis (deviation) Requirements Architecture- Should- Design Architecture Comparison “Diff-” Actions Architecture Extraction Is- •Violations •Conformance Architecture Existing Code Page: 32
- 27. Displaying results: Page: 33
- 28. Architecture: critical for OS-projects How to augment How to replace the development key people ? and support load ? Open-Source Project Marketplace How to maintain How to develop confidence with users? a market ? Page: 34
- 29. Openess limits Erosion Developers expose their reputation Names are associated to the architecture Community can provide feedback Warnings, Recommendations, ... Page: 35
- 30. Risks of Erosion in FOSS Contributors from several organizations Different cultures, processes, tools, … Lower pressure from management Indirect Business Hazardous funding Difficulties to calculate costs and benefits Page: 36
- 31. Eclipse: Architectural Analysis Page: 37
- 32. Eclipse: 10 Years Legacy System Eclipse Helios = 33M lines of code Page: 38
- 33. Eclipse Architecture Page: 39
- 34. Eclipse Architecture Page: 40
- 35. Eclipse Architecture Page: 41
- 36. E3.4: Plattf:Ant JDT:* Page: 42
- 37. E3.4: Plattf:Ant JDT:UI Page: 43
- 38. Plattform: CVS Workb (internal) Page: 44
- 39. Team-UI UI-workbench (internal) https://bugs.eclipse.org/bugs/show_bug.cgi?id=90582 Page: 45
- 40. Antipatterns Page: 46
- 41. Dependent BaseClass – Type: • Design Problem – Problem: • one of more Methods shall implement different behavior, depending on the type, passed in – Context: • make “extensible” systems, frameworks – Forces: • Programming languages offer, instanceof/typeid funcs. – Antipattern: • Methods of the baseclass, depend on derived classes, e.g. accessing their members, doing switch/case depending on type information Page: 47
- 42. AntiPatterns / Bad Smells: Metrics/1000 Classes Eclipse JDK Dependent Baseclass: 1 16 Multiple Interface Inher. 4 18 Abstractable Methods 80 60 Abstractable Attributes 45 170 Unused Classes 50 150 Unused Methods 950 2500 Unused Attributes 30 20 Page: 48
- 43. Code Code Clones Clones Page: 49
- 44. Code Clones – identical Files • E2.0 JDT,CDT – jdtdebuginternaluidialogfieldsListDialogField.java – cdtdebuginternaluidialogfieldsListDialogField.java • E3.4 CDT: identical packages – cdtdebuginternaluidialogfields, cdtdebugmiinternaluidialogfields – variation of former identical Files • E3.4 JDT,CDT – jdtdebuginternaluidialogfieldsListDialogField.java – cdtdebuginternaluidialogfieldsListDialogField.java Page: 50
- 45. Rating Eclipse Architecture Minor Erosions, Code Duplication – large codebase, – OSGI helps a lot – API Police, – Guidelines – upcoming PDE tools – Processes and Tools Page: 51
- 46. Architecture Council Recommend. •Architectural Recommendations •Community Practices • Be asynchronous • Engage your user community • Think API • Long-running operations should be cancelable •Software Development Practices • Separate policy and mechanism • Unit Test, Unit Test, Unit Test • Keep simple things simple • Continuous Integration • Create Unittests early • Integrate in small steps • Minimize plug-in dependencies • Be aware of the deployment context • Package coherence • Putting only related things together http://wiki.eclipse.org/Architecture_Council/Top_Ten_Project_Development_Practices http://wiki.eclipse.org/Architecture_Council/Top_Ten_Recommendations Page: 52
- 47. Yearly Simultaneous Releases Rules Projects should leverage only published APIs of dependencies Version number ends with « qualifier » Source code must use ICU4J classes The project must contain an « about.html » file Packages name should start with the plug-in ID Plug-in must not contain JARs files Plug-in should contain only one « message.properties » and « Message.java » files Etc… http://www.eclipse.org/indigo/planning/EclipseSimultaneousRelease.php Page: 53
- 48. New Eclipse projects … driving to new architectures Components that are more reusable + customizable Service-oriented programming model GUI represented by a model and configurable with CSS Enabling Javascript app to be executed by Eclipse runtime Framework to build SWT app declarativeley Ajax-enabled web applications by using a subset of RCP APIs Eclipse development model Plug-ins & Eclipse workbench extension points Widget toolkit using SWT API Browser-based tools to develop for the web, in the web Client: loosely coupled components written in JavaScript Server: services exposed via REST-oriented HTTP APIs Page: 54
- 49. Manage architectural transition? ? Architecture A Architecture B Page: 55
- 50. Evolution to e4 Backward Compatibility Layer • Eclipse 3.x plug-ins run on e4 Challenges • Not clean APIs usage? Refactoring • Using e4 development model? Migration • Single sourcing? Forward Compat. Layer (Session: Singlesourcing for Eclipse 4.x and Eclipse 3.x) Page: 56
- 51. Evolution to RAP Single sourcing with RAP is not automatic • API differences • Missing extension points • Application startup and Activator scope • API Differences • Field validation Refactoring • SWT ressources • Singletons and Scopes • Jobs and background threads • Internationalization and localization (http://eclipsesource.com/fileadmin/doc/2009_product/single_sourcing_guide.pdf) Page: 57
- 52. How to modernize existing Eclipse plug-ins? Page: 58
- 53. Architectural Modernization Process Transformation Reference Tooling Tests Modernization Transformation Strategy & Integration Non-Regression Audit Tests Legacy Modernized Software Software System System Page: 59
- 54. Anatomy of an Eclipse 3.x Plug-in folders files .project .classpath ! ity MANIFEST.MF e ne g Eclipse Plug-in build.properties te ro He plugin.xml plugin.properties Source code Page: 60
- 55. Model-Driven Reverse-Engineering of Eclipse plug-ins Page: 61
- 56. MoDisco use Models to represent and manipulate artifacts of existing systems Discover ExistingUnderstand Transform Software System Software artifacts : - source code New Models Viewpoints - configuration files Software System - tests - database -… http://www.eclipse.org/MoDisco/ Page: 62
- 57. MoDisco Architecture Supported Technologies Java JSP XML EclipsePlugin Metamodel Metamodel Metamodel Discoverer Discoverer Discoverer Metamodel Generator Generator Generator Discoverer Transfo. to KDM Discovery Model Customization OMG/ADM Manager Browser & Extensibility Standards Plug and orchestrate Navigation Definition of Pivot transformations through specific Metamodels complex models Viewpoints (SMM & KDM) Infrastructure Eclipse Modeling projects Page: 63
- 58. Using EMF to describe a Plug-in folders Project’s structure (KDMSource) files .project (XML) .project .classpath (XML) .classpath it y! manifest e MANIFEST.MF en plugin build.properties og (eclipseplugin) om (XML) Eclipse Plug-in build.properties (KDMCore) Hextensions plugin.xml plugin.properties plugin.properties (KDMCore) Java source code Source code (Java) Page: 64
- 59. Using EMF to describe a Plug-in Page: 65
- 60. What can you do with the EMF model of a plug-in ? Page: 66
- 61. Leverage Eclipse Modeling components Inspect Inspect (MoDisco Browser, EEF) (MoDisco Browser, EEF) Query Query (EMF Query) (EMF Query) Vizualize Vizualize (GMF, Graphiti) (GMF, Graphiti) EMF Model of an Eclipse plug-in Compare Compare (EMF Compare) (EMF Compare) Transform Transform (ATL, QVTo) (ATL, QVTo) Generate code Generate code (Acceleo, Xpand, Jet) (Acceleo, Xpand, Jet) Page: 67
- 62. Architectural Modernization Process Transformation Transformation Reference Reference Tooling Tooling Tests Tests Modernization Modernization Transformation Transformation Strategy Strategy & Integration & Integration Non-Regression Non-Regression Audit Tests Tests • Quality Analysis • Volumetry • Cartography Legacy Legacy Modernized Modernized Software Software Software Software System System System System Page: 68
- 63. Validation Page: 69
- 64. Architecture Visualization GMF diagram created with ObeoDesigner • purple: EMF Model • green: UI Page: 70
- 65. Architectural Modernization Process Transformation Reference Reference Tooling Tests Tests Modernization Modernization Transformation Transformation Strategy Strategy & Integration & Integration Non-Regression Non-Regression Audit Audit • Parsing Tests Tests • Transformation rules • Re-generation Legacy Legacy Modernized Modernized Software Software Software Software System System System System Page: 71
- 66. Refactoring & Migration Model Model Existing of the existing Transformed of the migrated Plug-in Plug-in Plug-in Plug-in Transformation Rules What to change + How to change • Renaming • Changing code structure (inheritance, attributes, methods, etc) • Replacing method calls • Changing instructions order • Etc Page: 72
- 67. Architectural Modernization Process Transformation Transformation Reference Tooling Tooling Tests Modernization Modernization Transformation Transformation Strategy Strategy & Integration & Integration • Tests coverage analysis Non-Regression Non-Regression Audit Audit Tests Tests Legacy Legacy Modernized Modernized Software Software Software Software System System System System Page: 73
- 68. Test coverage analysis Page: 74
- 69. Architectural Modernization Process Transformation Transformation Reference Reference Tooling Tooling Tests Tests Modernization Modernization Transformation Strategy Strategy & Integration Non-Regression Non-Regression Audit Audit Tests Tests • Automatic transformation • Manual transformation • Build Legacy Legacy Modernized Modernized Software Software Software Software System System System System Page: 75
- 70. Build IDM++ Research Project (ANR) -> Sept 2011 Plug-ins to build Model of plugins to build Team information (CVS, SVN, etc) constraints solver B3 Model (build configuration) Model of Update sites update sites content (p2) Build Strategies Page: 76
- 71. Summary Stop the Software Architecture Erosion ? Analysis is Required – Evaluate the architectural situation – Take the right decisions at the right time Modernization is Required – Correct erosion consequences – Go with architectural Evolutions => Continuous Analysis + Modernization Page: 77
- 72. Thank you ! Page: 78