Should Auto-Generated Code Be Included in Code Coverage Reports?
On one project, we used QueryDSL to interact with our database. QueryDSL generates Java classes that allow SQL queries to be constructed using plain Java code. However, we noticed that some of these auto-generated classes weren’t covered by our automated tests, which negatively impacted our code coverage statistics.
This raised a fundamental question: Should auto-generated code be included in code coverage reports?
The common viewpoint is: “It’s auto-generated and assumed to be correct, so there’s no need to test it. Including it just distorts the coverage numbers.”
But we saw it differently.
If this code is part of the application and is never executed during testing, what does that say about the completeness of our test suite?
Upon investigation, we discovered that primarily only the setters in the generated classes were executed—very few getters were called. This made sense: our backend system mainly aggregated data from sources like XML and CSV files and then wrote that data to the database using QueryDSL. Data retrieval, on the other hand, happened mostly in a separate application. As a result, the getters in the generated classes were rarely triggered during our automated tests. Had we excluded the generated code from our coverage reports, we would have missed this critical insight. We wouldn’t have realized that some setters were never called, making it impossible to improve our test input data—and, by extension, both our code and data coverage. To tackle this, we used DBUnit to verify the contents of the database. But we needed more than just data snapshots. We developed a custom verification layer that used reflection to call the getters in the QueryDSL-generated classes—specifically for the columns being validated by DBUnit. This helped us identify which database fields were never accessed or verified during tests. With that insight, we were able to create more comprehensive DBUnit snapshots and refine our test data, significantly increasing our data coverage.
It’s risky to ignore or exclude parts of codebase simply because they are auto-generated. If such code isn’t used, perhaps it shouldn’t be generated? But if it is used, we must ensure it’s exercised during tests! This may require unconventional methods like using reflection, but the benefit is clear: we generate more accurate and actionable coverage reports that help us identify weaknesses in our test suite and improve both code and data coverage across the system.