A Tool for Rejuvenating Feature Logging Levels via Git Histories and Degree of Interest
0 likes504 views
The document presents a method for automatically evolving feature logging levels in software development by mining git repositories to assess the 'interestingness' of logs. This approach aims to address issues of information overload due to excessive logging, aligning log levels with stakeholder interests and improving the manageability of logging systems. Preliminary evaluations demonstrate the effectiveness of the technique, achieving high success rates in updating logging levels and successfully integrating with existing software development tools.
![Introduction Motivation Approach Evaluation Conclusion Logging Issues
Logging Issues
Source: Stuart Pilbrow / CC BY-SA
(https://creativecommons.org/licenses/by-sa/2.0)
Too much logging causes
information overload [Yuan
et al., 2012].
Makes postmortem analysis
difficult.
Understanding system behavior
in production and diagnosing
problems can be challenging.
Also challenging during
development as logs pertaining
to auxiliary features are tangled
with those under current
development.
Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 4 / 10](https://image.slidesharecdn.com/presentation-220510235700-a725ddd5/85/A-Tool-for-Rejuvenating-Feature-Logging-Levels-via-Git-Histories-and-Degree-of-Interest-4-320.jpg)
![Introduction Motivation Approach Evaluation Conclusion
Feature Logging Statement Level Evolution
Logging statements are typically associated with a log level.
Dictates if the log should be emitted, if at all.
Example
logger.log(Level.FINER, "Health:" + systemHealthStatus());
Outputs system health iff the run time level of logger ≤ Level.FINER.
As software evolves, logging statements levels correlated with
surrounding feature implementations may also need to be modified.
Developers often fail to initially set log levels appropriately, and then
alter them afterward [Yuan et al., 2012].
Ideally, feature log levels would evolve with the system as it is
developed.
Higher log levels (e.g., INFO) being assigned to logs corresponding to
features with more current stakeholder interest.
Lower log levels for those with less interest (e.g., FINEST).
Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 5 / 10](https://image.slidesharecdn.com/presentation-220510235700-a725ddd5/85/A-Tool-for-Rejuvenating-Feature-Logging-Levels-via-Git-Histories-and-Degree-of-Interest-5-320.jpg)
![Introduction Motivation Approach Evaluation Conclusion Overview Implementation
Automation Approach Overview
Figure: Logging Level rejuvenation approach overview (details in paper).
Automatically evolve feature logging statement levels.
Mine Git repositories to discover the “interestingness” of code
surrounding feature logging statements.
Adapt Mylyn degree of interest (DOI) model [Kersten and Murphy,
2005].
Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 6 / 10](https://image.slidesharecdn.com/presentation-220510235700-a725ddd5/85/A-Tool-for-Rejuvenating-Feature-Logging-Levels-via-Git-Histories-and-Degree-of-Interest-6-320.jpg)
![Introduction Motivation Approach Evaluation Conclusion
Evaluation Overview
18 Java projects, ˜3 MLOC, and ˜4K logging statements.
Fully-automated analysis running-time:
10.66 secs per analyzed logging statement.
0.89 secs per KLOC changed.
Developers do not actively think about how their logging statement
levels evolve with their software.
Successfully analyzes 99.26% of candidate logging statements.
Increases log level distributions by an average of ˜20%.
Ideally transforms log levels in bug contexts ˜83% of the time.
Preliminary pull request study successfully integrated into 2 large
and popular open-source projects (comparable to related work [S. Li
et al., 2018]).
More details in the paper!
Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 9 / 10](https://image.slidesharecdn.com/presentation-220510235700-a725ddd5/85/A-Tool-for-Rejuvenating-Feature-Logging-Levels-via-Git-Histories-and-Degree-of-Interest-9-320.jpg)
![Appendix Additional Material
Related Work
Source: Jonathan Joseph Bondhus / CC BY-SA
(https://creativecommons.org/licenses/by-sa/3.0)
Existing approaches [Chen and
Jiang, 2017; Hassani et al.,
2018; He et al., 2018; Kabinna
et al., 2018; H. Li et al., 2017]
are inclined to focus on either
new logging statements or log
messages.
Logger hierarchies [Apache
Software Foundation, 2020;
Oracle, 2018] may be but still
require manual maintenance.
Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 3 / 5](https://image.slidesharecdn.com/presentation-220510235700-a725ddd5/85/A-Tool-for-Rejuvenating-Feature-Logging-Levels-via-Git-Histories-and-Degree-of-Interest-14-320.jpg)
A Tool for Rejuvenating Feature Logging Levels via Git Histories and Degree of Interest
- 1. A Tool for Rejuvenating Feature Logging Levels via Git Histories and Degree of Interest Yiming Tang1 Allan Spektor2 Raffi Khatchadourian2,3 Mehdi Bagherzadeh4 1 Concordia University, Canada 2 City University of New York (CUNY) Hunter College, USA 3 City University of New York (CUNY) Graduate Center, USA 4 Oakland University, USA International Conference on Software Engineering May 11, 2022, Pittsburgh, PA, USA
- 2. Introduction Motivation Approach Evaluation Conclusion Logging Issues Logging in Modern Software in the Big Data Era Logging is pervasive in the modern software. Big data systems deal with high-volumes of transactions. Source code is tangled with scattered logging statements capturing important event information. Essential for reporting security and privacy breaches. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 2 / 10
- 3. Introduction Motivation Approach Evaluation Conclusion Logging Issues Feature Logging Statements Modern software is feature-heavy, implementing hundreds of features. Logging statements—although more informational—also capture important aspects of feature implementations. Useful for validating feature implementations and diagnosing unintended interactions with other features. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 3 / 10
- 4. Introduction Motivation Approach Evaluation Conclusion Logging Issues Logging Issues Source: Stuart Pilbrow / CC BY-SA (https://creativecommons.org/licenses/by-sa/2.0) Too much logging causes information overload [Yuan et al., 2012]. Makes postmortem analysis difficult. Understanding system behavior in production and diagnosing problems can be challenging. Also challenging during development as logs pertaining to auxiliary features are tangled with those under current development. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 4 / 10
- 5. Introduction Motivation Approach Evaluation Conclusion Feature Logging Statement Level Evolution Logging statements are typically associated with a log level. Dictates if the log should be emitted, if at all. Example logger.log(Level.FINER, "Health:" + systemHealthStatus()); Outputs system health iff the run time level of logger ≤ Level.FINER. As software evolves, logging statements levels correlated with surrounding feature implementations may also need to be modified. Developers often fail to initially set log levels appropriately, and then alter them afterward [Yuan et al., 2012]. Ideally, feature log levels would evolve with the system as it is developed. Higher log levels (e.g., INFO) being assigned to logs corresponding to features with more current stakeholder interest. Lower log levels for those with less interest (e.g., FINEST). Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 5 / 10
- 6. Introduction Motivation Approach Evaluation Conclusion Overview Implementation Automation Approach Overview Figure: Logging Level rejuvenation approach overview (details in paper). Automatically evolve feature logging statement levels. Mine Git repositories to discover the “interestingness” of code surrounding feature logging statements. Adapt Mylyn degree of interest (DOI) model [Kersten and Murphy, 2005]. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 6 / 10
- 7. Implementation Details Implemented as an open-source plug-in to the Eclipse IDE. May also be used with popular build systems via plug-ins. Supports two popular logging frameworks, SLF4J and JUL. Integrates with JGit and Mylyn. Available at https://git.io/fjlTY.
- 9. Introduction Motivation Approach Evaluation Conclusion Evaluation Overview 18 Java projects, ˜3 MLOC, and ˜4K logging statements. Fully-automated analysis running-time: 10.66 secs per analyzed logging statement. 0.89 secs per KLOC changed. Developers do not actively think about how their logging statement levels evolve with their software. Successfully analyzes 99.26% of candidate logging statements. Increases log level distributions by an average of ˜20%. Ideally transforms log levels in bug contexts ˜83% of the time. Preliminary pull request study successfully integrated into 2 large and popular open-source projects (comparable to related work [S. Li et al., 2018]). More details in the paper! Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 9 / 10
- 10. Introduction Motivation Approach Evaluation Conclusion Conclusion Feature logging statements document important values and track progress of feature implementations. As feature interest evolves, feature logging levels may also require modification to combat information overload. Our approach discovers and rectifies mismatches between feature interest levels and logging levels. Results show that the technique is promising in alleviating the burden of manually evolving logging levels. Future Implementation Work Classify feature logging statements using more advanced AI techniques. Incorporate dataflow analyses for levels in variables. Integrate with existing Mylyn task contexts. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 10 / 10
- 11. Introduction Motivation Approach Evaluation Conclusion Conclusion Feature logging statements document important values and track progress of feature implementations. As feature interest evolves, feature logging levels may also require modification to combat information overload. Our approach discovers and rectifies mismatches between feature interest levels and logging levels. Results show that the technique is promising in alleviating the burden of manually evolving logging levels. Future Implementation Work Classify feature logging statements using more advanced AI techniques. Incorporate dataflow analyses for levels in variables. Integrate with existing Mylyn task contexts. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 10 / 10
- 12. Appendix Additional Material For Further Reading I Apache Software Foundation (2020). Log4j. Log4j 2 Architecture. url: http://logging.apache.org/log4j/2.x/manual/architecture.html#Logger_Hierarchy (visited on 06/12/2020). Chen, Boyuan and Zhen Ming (Jack) Jiang (2017). “Characterizing and Detecting Anti-Patterns in the Logging Code”. In: International Conference on Software Engineering. ICSE ’17. Buenos Aires, Argentina: IEEE Press, pp. 71–81. isbn: 9781538638682. doi: 10.1109/ICSE.2017.15. Eclipse Foundation, Inc. (2020). JGit. url: http://eclip.se/gF (visited on 03/02/2020). Hassani, Mehran et al. (Mar. 2018). “Studying and detecting log-related issues”. In: Empirical Software Engineering. issn: 1573-7616. doi: 10.1007/s10664-018-9603-z. url: https://doi.org/10.1007/s10664-018-9603-z. He, Pinjia et al. (2018). “Characterizing the Natural Language Descriptions in Software Logging Statements”. In: International Conference on Automated Software Engineering. ASE 2018. Montpellier, France: ACM, pp. 178–189. isbn: 9781450359375. doi: 10.1145/3238147.3238193. Kabinna, Suhas et al. (Feb. 2018). “Examining the Stability of Logging Statements”. In: Empirical Softw. Engg. 23.1, pp. 290–333. issn: 1382-3256. doi: 10.1007/s10664-017-9518-0. Kersten, Mik and Gail C. Murphy (2005). “Mylar: a degree-of-interest model for IDEs”. In: International Conference on Aspect-Oriented Software Development. Chicago, Illinois: ACM, pp. 159–168. isbn: 1-59593-042-6. doi: 10.1145/1052898.1052912. Li, Heng, Weiyi Shang, and Ahmed E. Hassan (Aug. 2017). “Which Log Level Should Developers Choose for a New Logging Statement?” In: Empirical Softw. Engg. 22.4, pp. 1684–1716. issn: 1382-3256. doi: 10.1007/s10664-016-9456-2. Li, Shanshan et al. (2018). “Logtracker: Learning Log Revision Behaviors Proactively from Software Evolution History”. In: International Conference on Program Comprehension. ICPC ’18. Gothenburg, Sweden: ACM, pp. 178–188. isbn: 978-1-4503-5714-2. doi: 10.1145/3196321.3196328. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 1 / 5
- 13. Appendix Additional Material For Further Reading II Oracle (2018). Logger (Java SE 10 & JDK 10). url: http://docs.oracle.com/javase/10/docs/api/java/util/logging/Logger.html (visited on 02/29/2020). Yuan, Ding, Soyeon Park, and Yuanyuan Zhou (2012). “Characterizing logging practices in open-source software”. In: International Conference on Software Engineering. ICSE ’12. IEEE. Zurich, Switzerland: IEEE Computer Society, pp. 102–112. isbn: 978-1-4673-1067-3. doi: 10.1109/ICSE.2012.6227202. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 2 / 5
- 14. Appendix Additional Material Related Work Source: Jonathan Joseph Bondhus / CC BY-SA (https://creativecommons.org/licenses/by-sa/3.0) Existing approaches [Chen and Jiang, 2017; Hassani et al., 2018; He et al., 2018; Kabinna et al., 2018; H. Li et al., 2017] are inclined to focus on either new logging statements or log messages. Logger hierarchies [Apache Software Foundation, 2020; Oracle, 2018] may be but still require manual maintenance. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 3 / 5
- 15. Appendix Additional Material Rename Refactorings & Copying Program elements (e.g., methods) changed in Git may no longer exist in current project version. Must process rename refactorings. Maintain a data structure that associates rename relationships between program elements, e.g., method signatures. Use lightweight refactoring approximations. Use copy detection features of Git at the file level. New copy “inherits” old DOI values. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 4 / 5
- 16. Appendix Additional Material Classifying Feature Logging Statements Logging levels are often used to differentiate various logging “categories” (e.g., severe errors, security breaches). Need to distinguish between these and feature logs. Derive a set of heuristics based on first-hand developer interactions. Also distinguish between less-critical debugging logs (e.g., tracing) using a keyword-based approach. Goals Focus on only manipulating logging statements tied to features to better align them with developers’ current interests. Yiming Tang, Allan Spektor, Raffi Khatchadourian, Mehdi Bagherzadeh Rejuvenating Feature Logging Levels via Git Histories and DOI 5 / 5