An Empirical Analysis of Technical Lag in npm Package Dependencies
0 likes346 views
The document analyzes technical lag in npm package dependencies, highlighting that a significant portion of packages suffer from outdated dependencies that are several months behind the latest releases. The study finds that 27% of dependencies are outdated, and technical lag is influenced by strict dependency constraints. Recommendations include waiting a few days before updating to new patch releases to avoid immediate bugs.
An Empirical Analysis of Technical Lag in npm Package Dependencies
- 1. An Empirical Analysis of Technical Lag in npm Package Dependencies Ahmed Zerouali, Eleni Constantinou, Tom Mens, Gregorio Robles and Jesus M. Gonzalez-Barahona The 17th International Conference on Software Reuse May 21-23, 2018 - Madrid
- 3. /background Packages Releases Dependencies (runtime) +700K +4.5M +20M +145K +825K +2M +130K +840K +2.3M Libraries.io by March 2018 RubyGems
- 4. /background Open PRs Active Bugs +2.3M - +2M - - +120K by January 2018 - https://octoverse.github.com/
- 5. /background Technical lag*: the increasing difference between deployed software packages and the available upstream packages Measurement: version updates, bugs, vulnerabilities, line of code, commits, etc. (*) Gonzalez-Barahona, et al. "Technical Lag in Software Compilations: Measuring How Outdated a Software Deployment Is." IFIP International Conference on Open Source Systems. Springer, Cham, 2017. Gold standard: stability, security, functionality, etc.
- 6. /background Example: different kinds of “gold standards” for Debian Gold standard Scenario Candidate Stability Isolated system, stable functionality Debian Stable Functionality Cloud application Latest upstream Security Reused containers Stable upstream
- 7. /background
- 8. /aims Decan A, Mens T, Grosjean P. An empirical comparison of dependency network evolution in seven software packaging ecosystems. EMSE2017.
- 9. /aims Not update Update “How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript” - https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
- 10. /aims Goal: Analyze technical lag in a wide ecosystem of reused of packages.
- 11. /aims / research questions RQ0: How do packages manage their dependencies? RQ1: How often do packages release new versions? RQ2: What is the technical lag induced by outdated dependencies? RQ3: How often do dependencies inducing technical lag release new versions? RQ4: What is the appropriate moment to update dependencies?
- 12. /method /dataset
- 13. /method /dataset Open Data: - Libraries.io gathers data from 36 package managers and 3 source code repositories. - They track over 2.7m unique open source packages, 33m repositories and 235m interdependencies between them.
- 14. /method /dataset - 610K packages - 4.2M releases - 44.9M dependencies by Nov 2017 from
- 15. /method /semantic versioning Examples: 0.0.1, 1.0.0, 1.2.3, 1.2.3-beta
- 16. /method /semantic versioning Other: *, ==1.2.3, >1.2.3, <1.2.3, 1.2.x, 1.x.x
- 17. /method /technical lag - Measurement = version updates, time - version lag : version updates difference - time lag: time difference - Gold standard = being up to date.
- 18. /method /technical lag 1.0.1 1.1.0 2.0.01.2.1 2.1.0 Dependency: D npm package version Technical lag
- 19. /method /technical lag - time lag = date(latest) - date(used) - version lag = (∆Major, ∆Minor, ∆Patch)
- 20. /method /technical lag 1.0.1 1.1.0 2.0.01.2.0 2.0.1 Dependency: D npm package version Technical lag - time lag= date(2.1.0) - date(1.1.0)
- 21. /method /technical lag 1.0.1 1.1.0 2.0.01.2.0 2.0.1 Dependency: D npm package version Technical lag 1 minor - time lag= date(2.1.0) - date(1.1.0)
- 22. /method /technical lag 1.0.1 1.1.0 2.0.01.2.0 2.0.1 Dependency: D npm package version Technical lag - time lag= date(2.1.0) - date(1.1.0) 1 minor 1 major
- 23. /method /technical lag 1.0.1 1.1.0 2.0.01.2.0 2.0.1 Dependency: D npm package version Technical lag 1 minor 1 major 1 patch - time lag= date(2.1.0) - date(1.1.0) - version lag= (1,1,1)
- 24. /method /technical lag 1.0.1 1.2.0 2.0.1 3.6.0 4.1.04.0.0 5.0.0 2.0.0 2.1.0 npm package: P dependency: D ^1.0.0 Technical lag * ^1.0.0 ^2.0.0 ^1.0.0 = [ 1.0.0, 2.0.0 [ allowed
- 25. /method /technical lag 1.0.1 1.2.0 2.0.1 3.6.0 4.1.04.0.0 5.0.0 2.0.0 2.1.0 npm package: P dependency: D ^1.0.0 Technical lag * ^1.0.0 ^2.0.0 allowed ^1.0.0 = [ 1.0.0, 2.0.0 [
- 26. /method /technical lag 1.0.1 1.2.0 2.0.1 3.6.0 4.1.04.0.0 5.0.0 2.0.0 2.1.0 npm package: P dependency: D ^1.0.0 Technical lag = 0 * ^1.0.0 ^2.0.0 allowed ^1.0.0 = [ 1.0.0, 2.0.0 [
- 27. /results /RQ0: How do packages manage their dependencies? 68.2% 15.7% 7.8% 4.1% - Developers are concerned with backward compatibility - There is a potential of too strict dependency constraints leading to technical lag. 4.3%
- 28. /results /RQ0: How do packages manage their dependencies? - New dependencies are mainly added in major and minor releases. - Dependencies are removed almost exclusively in major releases. - Most packages do not appear to change their dependencies over time.
- 29. /results /RQ0: How do packages manage their dependencies? Number of updated dependencies between package releases, classified by release type of the update.
- 30. /results /RQ1: How often do packages release new versions? - Patch: 80% - Minor: 16% - Major: 04% - Dependent packages in npm mainly benefit from patch releases. - Technical version lag is mainly occurring at the patch level.
- 31. /results /RQ1: How often do packages release new versions? Time needed to update a package to a patch, minor or major release - The average time to release a patch, minor and major versions corresponds to 13 days, 1 month and 2 months respectively.
- 32. /results /RQ2: What is the technical lag induced by outdated dependencies? - 27% of 44.1M dependencies are outdated. - The outdated dependencies are used by 60% of all considered packages. 57% 28% 12% 3%
- 33. /results /RQ2: What is the technical lag induced by outdated dependencies?
- 34. /results /RQ2: What is the technical lag induced by outdated dependencies? - Outdated dependencies induce a median of time lag of three months and a half, and median version lag of one minor and two patch versions.
- 35. /results /RQ3: How often do dependencies inducing technical lag release new versions? - Packages that are required as dependencies and are outdated have more frequent releases than other required packages.
- 36. /results /RQ4: What is the appropriate moment to update dependencies? - Developers should not start using newly available packages immediately because they may still contain bugs that need new patches.
- 37. /limitations - If the libraries.io dataset is incomplete, then there is a risk of underestimating technical lag. - We did not differentiate between package characteristics, such as age, size, type, etc. - The results are related to the measurement used to quantify for technical lag. - npm semver had some issues in the past.
- 38. /conclusion/ summary Analyzed technical lag induced by outdated dependencies: - A large number of packages suffer from technical lag. - Outdated dependencies are several months behind the latest release. - Technical lag caused by the specific use of dependency constraints, - Maintainers should wait a few days before updating to the new patch dependency release.
- 39. /conclusion/ future work - Consider other measurements of technical lag and other gold standards. - Validate the results with bug fixes, vulnerabilities and issues. - Consider other ecosystems. - Carry out cross-ecosystem comparisons.
- 41. Questions