PhD public defense: A Measurement Framework for Analyzing Technical Lag in Open-Source Software Ecosystems

PhD Thesis A Measurement Framework for Analyzing Technical Lag in Open-Source Software Ecosystems
1
16:00 Ahmed Zerouali
Public PhD Defense
“A Measurement Framework for Analyzing Technical Lag in Open-Source
Software Ecosystems”

A Measurement Framework for
Analyzing Technical Lag in
Open-Source Software Ecosystems
Presented by:
Ahmed Zerouali
Advisor:
Dr. Tom Mens
Public PhD Defense
Software Engineering Lab, Université de Mons - Belgium, 4 September 2019
Jury:
Dr. Olivier Delgrange
Dr. Alexandre Decan
Dr. Jesus Gonzalez-Barahona
Dr. Alexander Serebrenik

Belgian Excellence of
Science Research Project 2018- 2021
https://secoassist.github.io/
3

Background
4

Focus
IssuesUp-to-date
5

Focus
Don’t updateUpdate
How can we help software developers to
decide when and why they should update
6
?

Published papers
7

Technical Lag
8

Technical lag
> Background
Technical lag¹: the increasing diﬀerence between
deployed software packages and the ideal available
upstream packages.
¹ Gonzalez-Barahona, et al. "Technical Lag in Software Compilations: Measuring How
Outdated a Software Deployment Is." IFIP International Conference on Open Source
Systems. Springer, Cham, 2017.
➢ Ideal: stability, security, functionality, recency, etc.
➢ Diﬀerence: version updates, bugs, vulnerabilities, lines
of code, commits, etc.
9

Technical lag
> Background
Credits: https://exploring-data.com/vis/npm-packages-dependencies/
+20M
dependencies
10

Technical lag
> Background
package.json
11

Technical lag
> Background
12
Semantic Versioning
Examples: 0.0.1, 1.0.0, 1.2.3, 1.2.3-beta

Technical lag
> Background
Other: *, ==1.2.3, >1.2.3, <1.2.3, 1.2.x, 1.x.x
13

Technical lag
> Example
14
1.0.1
3.6.0
*
1.2.0
allowed
4.0.0
^1.0.0
^1.0.0 = [ 1.0.0, 2.0.0 [
dependent
package P
required
package D
up-to-date
dependency

Technical lag
> Example
15
1.0.1 1.2.0 2.0.1
3.6.0
4.0.0
2.0.0
*
^1.0.0 = [ 1.0.0, 2.0.0 [
allowed
4.1.0
missing updates
dependent
package P
required
package D
outdated
^1.0.0
dependency

Technical lag
> Example
16

Followed research approach
Qualitative analysis:
surveys and interviews
Tooling
Quantitative
analysis
Technical lag
formal framework
Mixed-methods
approach
17

Tooling
Quantitative
analysis
Mixed-methods
approach
Technical lag
formal framework
18

A Framework for Technical Lag
> Qualitative analysis
Semi-structured Interviews:
➢ 5 software practitioners
➢ Place: FOSDEM 2019
➢ Highly educated interviewees with an average of 10
years of experience
19
Technical Lag is important, especially if we mix
between the benefits of updating and the effort
needed to do that.

> Qualitative analysis
Online surveys:
➢ 17 candidates (from facebook groups)
➢ Highly educated interviewees with an average of 3
years of experience
MCQ: What would be the most appropriate (ideal) version
of a software library to use?
20
★ Most stable (14)
★ Latest available (9)
★ Most documented (7)
★ Most secure (5)

Tooling
Quantitative
analysis
Mixed-methods
approach
Technical lag
formal framework
21

Technical lag
22
∆ version
∆ time
∆ bugs
∆ vulnerabilities

● is a set of component releases
● is a set of possible lag values
● ideal : → is a function returning the “ideal” component release
● delta : x → is a function computing the difference between two
component releases
● agg : is a function aggregating the results of a set of lags
23

24
Given a technical lag framework , we define:
Aggregated Technical lag
Technical lag
Let be a set of components, then:

Tooling
Quantitative
analysis
Mixed-methods
approach
Technical lag
formal framework
25

> Framework Instantiation
26

Technical Lag in npm Packages
27

28
Time-based instantiation of :
➢ Ideal: Highest available version
➢ Delta: Time Lag = date(ideal) - date(used)
➢ Aggregation: Max

29
Version-based instantiation of :
➢ Ideal: Highest available version
➢ Delta: Version lag = (∆Major, ∆Minor, ∆Patch)
➢ Aggregation: Sum

30
> Time-based instantiation
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependent
package
Technical lag
time lag = date(2.0.1) - date(1.1.0)
Required
package

31
> Version-based instantiation
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependent
package
Technical lag
1 minor
Required
package

32
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependent
package
Technical lag
1 minor
1 major
Required
package

33
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependent
package
Technical lag
1 minor
version lag = (1 major, 1 minor ,1 patch)
1 major 1 patch
Required
package

> Example
Time-based technical lag for the package debug:
- ideal (2.6.9) = 3.1.0
- time_lag(2.6.9) = 26-09-2017 - 22-09-2017 = 4 days
- version_lag(2.6.9) = (1,1,1)
34

> Example
Time-based technical lag for the package ms:
- time_lag(2.0.0) = 198 days
35

> Example
Aggregated Time-based technical lag for the package release
youtube-player@5.5.0:
- time_lag(debug@2.6.9) = 4 days
- time_lag(ms@2.0.0) = 198 days
➔ agglag({debug@2.6.9, ms@2.0.0}) = max (4 days, 198 days) = 198 days
36

New package releases have an increased technical lag.
Technical lag is induced by version constraints
Time lag induced by direct dependencies in npm package
releases:
37

Time lag induced by transitive runtime dependencies in npm
package releases:
Technical lag is accumulated from a level to another in the dependency
tree.
38

Technical Lag in npm applications

Technical Lag in npm applications
Time lag induced by direct dependencies in GitHub applications:
Technical lag in GitHub applications is higher than in npm
package releases
40

Docker Containers
41

Technical Lag in Docker Containers
> Background
42
● Containers are isolated bundles
of software packages
● Docker is one of the main tools
for containerisation
● DockerHub is the largest repository
for container images

> Background
43
What are the biggest barriers to putting containers in a
production environment? - ClusterHQ

> Background
44

> Focus
45

> Background
1.0.1 1.1.0 2.0.01.2.1 2.1.0
Docker container
C
Technical lag
technical lag =
∆ Versions (freshness)
∆ Vulnerabilities (security)
∆ Bugs (stability)
Ideal Version
deployed
container
Included
Package
version
Available
releases of a
Debian package
46

> Software Freshness
The majority of packages in Debian containers is up-to-date...
… but most of the images contain outdated packages.
How outdated are images?
IDEAL = LATEST
47

> Software Freshness
Outdated Debian packages in Docker containers induce a
median version lag of 1 version.
What is the version lag induced by the used Debian package releases?
IDEAL = LATEST
48

> Software Security
The number of vulnerabilities depends on the Debian release, and is
moderately correlated with the number of outdated packages in a container.
49

> Software Security
Few packages (2.5%) need to be downgraded in order to have the
most secure version.
Can we reduce security lag in DockerHub container images?
IDEAL = Most Secure
50

> Survey
51

52
1 out of 3 depending on a vulnerable npm package never update
their dependency and remain vulnerable
A. Decan et al. “On the impact of security vulnerabilities in the npm package
dependency network”, MSR 2018.
"37% of websites include a JavaScript library
with a known open source vulnerability”
T. Lauinger et al. "Thou Shalt Not Depend on Me: Analysing the Use of Outdated
JavaScript Libraries on the Web", NDSS 2017.
So what about Docker containers having npm packages?
Security vulnerabilities in npm JavaScript

53
★ Old Node images might be missing many updates, including
one major update.
★ All oﬃcial Node-based images have vulnerable npm packages,
with an average of 16 security vulnerabilities per image.
★ Older images are more likely to have more vulnerabilities.

Tooling
Quantitative
analysis
Mixed-methods
approach
Technical lag
formal framework
54

ConPan: A tool to analyze packages in software containers
> Existing tools
55

(*): https://github.com/neglectos/ConPan
ConPan(*): “CONtainer Packages ANalyzer”
> Overview
56

Example: $ conpan -p debian -c google/mysql -d ~/ConPan/data/debian/
> From the CLI
57

> From the API
58

> From the API
59

> From the API
60

Summary and Outlook
61

Summary
62

Future Work
● Other instantiations of the technical lag
○ Effort needed to reduce the technical lag
● Extend and enhance ConPan
● Cross ecosystems comparison
● Promote technical lag to be used by software developers
63

Conclusion
The technical lag could help open source software developers
and deployers to keep their software in a healthy shape.
64

https://github.com/neglectos/PhD_Dissertation

66
Evolution of Dependency constraints in npm packages
- Caret (^) usage is increasing over time.
- Caret introduction coincides with Major version lag increase.

67
The usage of strict constraint is much higher in external
applications
Evolution of Dependency constraints in GitHub applications

PhD public defense: A Measurement Framework for Analyzing Technical Lag in Open-Source Software Ecosystems

More Related Content

Similar to PhD public defense: A Measurement Framework for Analyzing Technical Lag in Open-Source Software Ecosystems (19)

More from Ahmed Zerouali (11)

Recently uploaded (20)

PhD public defense: A Measurement Framework for Analyzing Technical Lag in Open-Source Software Ecosystems