Software quality-libreplan
- 1. LibrePlan Software Quality attributes Javier Morán Rúa <jmoran@igalia.com> November 2012
- 2. Software quality ● When has a software quality? Two types of quality: ● Functional quality: How well a program conforms to the requirements (functional requirements). – Measured with tests. ● Non-functional quality: Structural quality and other things not related with the program behavior. Examples: Documentation, Internationalization, Scalability, Robustness, Usability, Testability ... – Analyzed with analsys of code and also with tests. www.libreplan.com
- 3. Software tests ● Difference between failure/error/malfunction: ● Error/Defect: Wrong or missing function in the code ● Failure: is the result of executing a software when an error happens. ● Malfunction. The system does not do the expected in the specification. www.libreplan.com
- 4. Software tests ● Testing definition: Execute the software with the target of finding errors and malfunctions. ● Does a software with tests has more structural and functional quality on average? Yes ● Guarantee there are not errors/malfunctions. ● Measure and know if you meet the expectations. www.libreplan.com
- 5. Software tests ● With tests you do two processes: verification and validation. ● Verification. ● Is the product right? ● checks that the software is working correctly. Assures that implementation is right for an specification. www.libreplan.com
- 6. Software tests ● Validation. ● Are we building the right product? ● checks if user requirements and the results got match. Reviews if the system is the correct one. www.libreplan.com
- 7. Software tests ● Types of tests – Classification 1: ● Black Box: testing interaction is based on inputs and outputs. ● White Box: centered in testing the logic of the software and its structure – All sentences are executed (coverage) – All conditions are checked – All loops are checked – All paths are checked www.libreplan.com
- 8. Software tests ● Types of tests – Classification 2: ● Unit tests. Isolated modules or classes. (verification) ● Integration tests. Interaction of modules and classes (verification) ● System tests. Hardware and software (verification, validation) ● Acceptance test. User validates against user requirements (validation) www.libreplan.com
- 9. Software tests ● Types of tests – Classification 2 ● Regression test. Assurance of changes. (verification) ● Stress test. Performance. (verification, validation) ● Usability test. Human interaction. (verification, validation) ● Functional GUI testing. Tests interface and can be a type of validation tests. (validation) ● Validation tests. Against software requirements (validation). www.libreplan.com
- 10. Software tests ● Types of tests – Classification 3 ● Manual. Perform by a person. ● Automatic. Perform by another program. www.libreplan.com
- 11. Software tests ● LibrePlan has white box and black box tests. ● LibrePlan has unit tests,integration tests, regression tests, acceptance tests, functional GUI tests, stress tests, validation tests, acceptance tests ● LibrePlan has not systematized usability tests, nor systematized system tests. ● LibrePlan has manual and automatic tests. www.libreplan.com
- 12. Unit and Integration tests ● Benefits: ● Increase code coverage ● Increase team productivity ● Detect regressions and limit debugging ● Improve implementation ● Document expected behavior www.libreplan.com
- 13. Unit and Integration tests ● LibrePlan uses JUnit and SpringTestContext to do integration and unit tests. ● JUnit is the de facto standard for unit testing in Java platform. ● It was created by Kent Beck and Erich Gamma ● Kent Beck. Creator of Extreme Programming and Test Driven Methodologies. ● Erich Gamma. One of the authors of Design Patterns: Elements of Reusable Object-Oriented Software. www.libreplan.com
- 14. Unit and Integration tests ● SpringTestContext is a module of Spring. In LibrePlan we use to simplify: ● Management of Hibernate sessions. ● Automatic management of transactions. Rollback after each test. ● Integration with Spring. We have access by DI to the repositories. www.libreplan.com
- 15. Unit and Integration tests ● In LibrePlan we have a lot of unit and integration tests: ● org.libreplan.ganttzk : 306 tests ● org.libreplan.business : 1884 tests ● org.libreplan.web: 400 tests www.libreplan.com
- 16. Test coverage ● Definition: It describes the degree to which the source code of a program has been tested. ● In which does it help? ● To know the untested parts of the codebase. ● In LibrePlan we measure the code coverage of our unit and integration tests. www.libreplan.com
- 17. Test coverage ● In LibrePlan we use Cobertura for measuring the code coverage of the unit and integration tests. ● Cobertura features: ● Generates reports in HTML and XML. ● Does several code coverage measurement types: – Statement coverage. % of lines executed. – Branch coverage. % of total branches tested. – Other aggregations: % of classes, files, methods www.libreplan.com
- 18. Test coverage ● Current test coverage LibrePlan measurements: ● Whole project: – Statement coverage: 12% – Branch coverage: 15% – Files: 50% of files with some test. – Methods: 20% of methods – Classes: 29% of classes www.libreplan.com
- 19. Test coverage ● Current test coverage LibrePlan measurements: – Statement coverage: 12% – Branch coverage: 15% – Files: 50% of files with some test. – Methods: 20% of methods – Classes: 29% of classes ● These numbers are much better in the domain layer and persistence layer. www.libreplan.com
- 20. Web UI tests ● Web UI testing is a type of GUI software testing for applications with web interface. ● GUI software testing: Type of tests which use the GUI of a program to perform the sequence of steps of the use cases of the application. In this way it is tested: ● That the program fulfills the requirements ● That the GUI has the way of interaction as designed. www.libreplan.com
- 21. Web UI tests ● Web UI testing can be: ● Automatic. Advantages: – Repeatability. – Speed of execution. ● Manual ● Automatic UI testing mimics the actions of a user through the interface www.libreplan.com
- 22. Web UI tests ● Web UI testing is a black box technique with a lower coverage than integration and unit tests. ● To automate or not automate ? ● Advantages: Frequent regression testing, rapid feedback for developers, unlimited execution of tests cases, support extreme and agile technologies, discipline in the documentation of test cases www.libreplan.com
- 23. Web UI tests ● To automate or not automate ? ● Disadvantages: – Tight deadline. Short time – Interface changes a lot www.libreplan.com
- 24. Web UI tests ● LibrePlan has automatic Web UI tests. ● It has been a challenge to find a framework to make UI tests with LibrePlan. ● We wanted a tool that allowed to test LibrePlan in several browsers. ● LibrePlan is a RIA application with a lot of JS, HTML5, CSS (some of the latest standards in web) ● With ZK we have to deal with an generated id HTML attribute for each page rendering www.libreplan.com
- 25. Sahi ● In LibrePlan we use Sahi as framework to build the automatic web UI tests. ● Sahi aims to make easy test automation in web technologies addressing most of the problems and limitations in this area. ● Advantages over Selenium: recorder is browser independent, waits for AJAX and page loads, it does not use XPath intensively, works well with dynamic ids, frames, iframes, popups. www.libreplan.com
- 26. Sahi architecture ● Architecture www.libreplan.com
- 27. Sahi ● Explaining the architecture: ● Need to simulate browser events. The way of Sahi: Use JS injected to do it. Why JS? – Support to access the DOM. – Runs in the browsers. ● The injected JS is generated and inserted by the Sahi proxy. www.libreplan.com
- 28. Sahi ● Explaining the architecture: ● Role of the proxy server: – Recording sessions. – Does playback of sessions. – Create reports, save data to database,... ● It is used Sahi scripting (an extended JS to define the tests) www.libreplan.com
- 29. Sahi ● Programming the tests in Sahi script has the next advantages: ● It is a programming language. Nothing better than the power of programming to define tests. ● The best API to interact with the DOM. ● Sahi has a powerful human relative API to locate things: _in, _near, _under, _table ... www.libreplan.com
- 30. i18n ● Internationalization (i18n): It is the process of designing an application so that it can be adapted to different languages and regions without engineering changes. ● Characteristics of an internationalized software: ● You can add localized data and run the same executable. ● Textual elements are not hardcoded ● Support for cultural dependent data, such as currencies and dates. www.libreplan.com
- 31. i18n ● Localization: It is the process of adapting the software for a specific language or region (translating and adding specific UI components if needed). ● LibrePlan is an internationalized software: ● Currently present in English, Spanish, Galician, Italian, French, Dutch, Catalan 100% ● Almost fully translated: Czech, German ● On-going translations: Polish, Portuguese, Russian www.libreplan.com
- 32. Stress tests ● Stress tests in software: Tests to put the system to test under heavy load to know the behavior of the system regarding to availability, robustness and error handling. ● In LibrePlan we do stress tests sometimes when we want to: ● Know the best alternative to implement a feature. ● The response times under different load conditions. www.libreplan.com
- 33. Stress tests ● At LibrePlan we use the tool JMeter to do stress tests. ● JMeter is a free software application designed initially to do stress load tests of web applications. ● Now it can test several server types: – Web: HTTP, HTTPS – SOAP – Database via JDBC, LDAP, JMS,... www.libreplan.com
- 34. Stress tests ● Example of task where we used stress tests: To appraise if the activation of the Hibernate second- level cache was worth. – http://wiki.libreplan.org/twiki/bin/view/LibrePlan/ItEr ● Results analysis: ● A 75% of gain in the maximum response time and second-level cache was activated. www.libreplan.com
- 35. i18n ● In Java i18n is provided in the Java SE. ● Internationalization done with Java SE: ● Locale class ● ResourceBoundles. ● Property files ● In LibrePlan we do not use the standard Java i18n facilities. www.libreplan.com
- 36. i18n ● In LibrePlan we use GNU Gettext ● GNU Gettext. Toolset to provide a framework to help to internationalize other programs. The tools include: ● A set of conventions about how programs should be written to support message catalogs. ● A directory and file name organization for the message catalogs themselves. ● A runtime library to get internationalized messages. www.libreplan.com
- 37. i18n ● GNU Gettext. Toolset to provide a framework to help to internationalize other programs. The tools include: ● A few stand alone programs to help translators with the translation of messages. ● A library supporting the parsing of source code and the creation of files containing messages to translate (POT files). www.libreplan.com
- 38. i18n ● Advantages of using GNU Gettext in LibrePlan: ● Programmers have not to be worried of adding explicitly messages to property files with ids. ● The generation of the messages to translate is automatic. www.libreplan.com
- 39. i18n ● LibrePlan uses a online internationalization tool called Transifex, that has helped to increase the number of languages in which LibrePlan is available. ● Transifex is free for open source projects. www.libreplan.com
- 40. Continuous Integration ● Definition: It is the process of frequently integrating one's new or changed code with the existing repository. ● It is one of the practices of XP (Extreme Programming). ● Advantages: detect and fix integration problems continuously, immediate unit testing of all changes, promote frequent check-in pushes developers doing less complex code. www.libreplan.com
- 41. Continuous Integration ● Effective practices to achieve CI: ● Maintain a single source repository. ● Automate the build. ● Make your build self-testing. ● Everyone commits to the mainline daily. ● Every commit should build the commit in an integration machine www.libreplan.com
- 42. Continuous Integration ● Effective practices to achieve CI: ● Keep the build fast. ● Test in a clone of the production environment. ● Make it easy for everyone get the latest executable. ● Everyone can see what is happening. ● Automate deployment. www.libreplan.com
- 43. Continuous Integration ● Continuous Integration can be set up using a Continuous Integration server. ● CI server definition:Tool that lets the user plan automatic builds: ● Periodically. ● Triggered after each commit. www.libreplan.com
- 44. LibrePlan CI server ● LibrePlan developers use continuous integration and use as continous integration server Jenkins ● LibrePlan continuous integration server is publicly accessible: http://jenkins.libreplan.org ● Two automatic builds: ● LibrePlan Postgresql ● LibrePlan MySQL www.libreplan.com
- 45. LibrePlan CI server ● Jenkins configuration at LibrePlan: ● The codebase that is built is master branch (unstable branch that originates new releases). ● The software is built once per day (2:00 AM) if during the last 24 hours some new changes has been pushed into the master branch. – If the build fails an e-mail is sent to libreplan-dev mailing list. www.libreplan.com
- 46. LibrePlan CI server ● After build the unit and integration tests are passed. www.libreplan.com
- 47. LibrePlan CI server ● Test code coverage is analyzed with Cobertura. www.libreplan.com
- 48. LibrePlan CI server ● If some unit or integration tests fails, then an automated mail is sent to libreplan-dev mailing list. ● As soon as a developer sees a fail (because a software build crash or some tests not passing), this gets maximum priority and managed the fixing. ● Doing it by himself if he is able to do. ● Looking for help and entrusting the responsible to fix it. ● Many times the first thing done is to revert the conflicting push and set Jenkins back to normal. www.libreplan.com
- 49. LibrePlan CI server ● Currently in LibrePlan the Sahi automate web UI tests are not integrated with Jenkins in the build cycle. ● Web UI tests are passed manually always before each release as part of the release process. ● Other occasions too. www.libreplan.com
- 50. Sonar ● Definition: It is an open source web platform to manage code quality. ● It is helped by other code static analysis tools like CheckStyle, FindBugs, ... ● Languages covered: Java, C, C#, Natural, PHP, Cobol and Visual Basic. www.libreplan.com
- 51. Sonar ● It measures the quality in 7 axis: ● Comments ● Coding rules ● Potential bugs. ● Complexity. ● Unit tests. ● Duplications. www.libreplan.com
- 52. Sonar ● In LibrePlan we use Sonar to control and monitor the quality of the source code. ● LibrePlan Sonar installation is publicly available: http://sonar.libreplan.org www.libreplan.com
- 53. Sonar ● Size metrics: ● Lines of code: Physical lines. ● Comment lines: Number of comments. ● Density of comment lines. ● Number of packages, number of methods. ● Number of accessors (getter and setters) www.libreplan.com
- 54. Sonar ● Tests metrics: ● Number of tests. ● Number of failures. ● Test success density. ● Code coverage: – Line coverage – Branch coverage www.libreplan.com
- 55. Sonar ● Duplication metrics: ● Duplicated lines: Number of lines touched by a duplication. ● Duplicated blocks: Number of duplicated blocks of lines. ● Duplicated files: Number of files involved in a duplication. ● Duplication density: Percentage of duplicated lines. www.libreplan.com
- 56. Sonar ● Complexity metrics: ● Cyclotomatic complexity (McCabe Metric). ● Method distribution complexity. ● Average class complexity. ● Average file complexity. www.libreplan.com
- 57. Sonar ● Violations metrics: ● Total number of violations ● Weighted violations = Sum of the violations weighted by a coefficient of severity. ● Rule Compliance Index (RCI) = 100 – (weighted violations/Number of Lines of Code)*100 www.libreplan.com
- 58. Sonar ● LibrePlan uses Sonar to monitor the quality of the project. ● If there are blocker violations, these are solved. ● LibrePlan Sonar installation: http://sonar.libreplan.org www.libreplan.com
- 59. Sonar ● LibrePlan Sonar dashboard www.libreplan.com