Managing the Software Supply Chain: Policies that Promote Innovation While Optimizing Security and Compliance

Managing the Software Supply Chain
Policies that Promote Innovation While
Optimizing Security and Compliance
Jeff Luszcz
@jeffluszcz
jluszcz@flexera.com

What is the Software Supply Chain?
• Commercial Proprietary software companies (e.g. Microsoft, Oracle)
• Projects hosted on Open source repositories (e.g. project on Github or Sourceforge.net)
• Major Open source projects (eg. Apache, Jboss )
• Commercial Open Source companies (e.g. MySQL, Sugar CRM)
• University and personal websites
• “The Web” (Blogs, websites, StackOverflow, etc..)
• Books and Magazines
• Competitors
• “Yourself” – other business units’ products or tools

What Are Companies Concerned About?
• Vulnerabilities
• Non-compliant usage of GPL and other Copyleft Licenses (esp GPL v3.0)
• Affero GPL
• Commercial Content and Libraries
• Restrictions on commercial use or field of use (e.g. no Military use)
• Cryptography
• Code with Unknown Licenses
• What is the % of undisclosed or non-compliant content
• Who can fix the problems that are found?

GNU Bash
• Potentially affects hundreds of
millions of computers, servers and
devices
• Shellshock can be used to remotely
take control of almost any system
using Bash
• Typical age: 5 years
old (seen 13 years!)
Linux GNU C Library (glibc)
• Affects almost all major  
Linux distributions
• Millions of servers on the Internet
contain this vulnerability
• Typical age: 3 years
OpenSSL
• 17% of the Internet's secure web
servers (500M) believed to be
vulnerable to the attack
• Allowed theft of the servers'
private keys, users' session
cookies and passwords
• Typical age: 3-4+
years old
Apache Struts2
• Remote Code Execution (RCE)
vulnerability in the Jakarta Multipart
parser
• Allows attacker to execute malicious
commands on the server when
uploading files
• Exploits are publicly available, simple
to carry out, and reliable
Heartbleed
CVE-2014-0160
Shellshock
CVE-2014-6271
Ghost
CVE-2015-0235 CVE-2017-5638
Known Vulnerabilities

5
Knowledge of the Components that are in use
Most Organizations
Typically Know
LESS THAN 10%
Of What Is
Actually Used
2013
2014
2015
2016
560
454
252236
27
8
2925
Customer Estimate Flexera Audit Results
Source: Flexera Professional Services Audit Data 2011 - 2016

How has the supply chain changed over time?
0
200
400
600
800
2011 2012 2013 2014 2015 2016 2017
Found
Disclosed
Found Disclosed % Disclosed
2011 135 18 13%
2012 221 25 11%
2013 236 25 11%
2014 252 29 12%
2015 454 8 2%
2016 560 27 5%
2017 718 12 2%

D I S C L O S U R E T O
A W A R E N E S S
A W A R E N E S S T O  
R E M E D I A T I O N
Known Vulnerabilities and the Risk Window 
7
0 20 40 60 80 100 120 140 160 180 200
DISCLOSURE AWARENESS REMEDIATION
D

T I M E T O 1 s t
E X P L O I T
A W A R E N E S S T O  
R E M E D I A T I O N
The Risk Window: Patch Tuesday Example 
8
0 20 40 60 80 100 120 140 160 180 200
DAYS
156
DISCLOSURE
AWARENESS
REMEDIATION
D
• Average time to first Exploitation: 30 days1
• Risk Window: 156 days
1 – Source: “2016 Data Breach Investigation Report” Verizon http://www.verizonenterprise.com/verizon-insights-lab/dbir/2016/
2 – Source: “2016 State of Vulnerability Risk Management” NopSec http://info.nopsec.com/sov

Common areas of concern for Vulnerabilities
While it is best to have a “Full” accounting of all third party software, certain
components may have a higher priority than others.
1) Highly targeted “Famous” components (Struts 2, Openssl, etc..)
2) Cryptographic components – often highly targeted, and also have legal
tracking requirements for export anyway
3) Compression components – similar to cryptography in terms of usage and
programming techniques. Often highly targeted.
4) Multi-media components - Wildly used, often contains crypto and
compression routines themselves.
5) Applications Platforms – widely used, often contain crypto and compression,
complex
6) Databases – central to all systems, complex
7) Operating Systems / Linux Stack related components

Why do you need an Open Source license?
Copyright law (in many places) means that all source is explicitly
copyright the original author EVEN if not marked
You have no right to use someone else’s code without permission
Open Source (and commercial) licenses are the way of giving permission
to use source code
Lack of license shows lack of maturity for the OSS project, often a sign of
other problems!
It is not Open Source if you don’t have a license

Common Misunderstandings
• Just because code is available, this does not give you any permission to
use it.
• “Freely Available” != Open Source
• “Public Domain” is different from “Open Source”
• You are not “safe” if you don’t ship your product (unlike some licenses
like the GPL)
• The open source author does not owe you an answer (let alone an
answer you like) if you ask about licensing
• The author is not necessarily an expert on licensing

Company Risks for Unknown Third Party Code
• Open Source is NOT a problem! (not managing or following the obligations
is)
• Unmanaged Vulnerabilities
• OpenSSL / Heartbleed
• Struts 2
• License Infringement
• Copyleft implications for your source code (GPL)
• Envelope issues
• Forced to Stop production and replace code
• Monetary Damages – (Commercial or OSS)

Open Source – Obligations
• Open Source is commonly confused with “Free” as in no cost software
• Open source may be Free of Cost, but is not Free of Obligations
Common referred to as “Free as in Speech, not Free as in Beer”
• Open Source licenses have a list of obligations that you must follow in order to legally use the open source library
under that license
• The act of following these obligations is called OSS Compliance or License Compliance
• Your Compliance actions depends on how you are using these OSS components
• Most licenses have Multiple Obligations

Commonly Seen Obligations
Obligation Type of Obligation Definition
Share Source Copyleft aka Viral Author requires user to share source code
Give Credit Notice or Attribution Name of author must be reported in About Box, Documentation,
Website, etc…
Share Patents Patent Clause Author requires permission to use patents or license patents in this
open source project
Restrict use Restrict who can use this code Restriction on military use, restriction on nuclear facilities, geography/
countries, commercial use etc..
Vanity/One Off Licenses Give me free beer, say a prayer, Do No Evil Requests by the author to do some sort of action not typically seen in
contracts or licenses
Preserve Attribution Attribution Requires attribution/copyrights to be preserved in the source code
Disclaimer Disclaimer Explain that the open source author is not responsible for the use of
the software, even if it has defects
Supply Original License text License Text Requires text of entire license to be provided to users
Commercial Pay for use of code Classic software business model license

How can we stop this lack
of knowledge from
happening again?

Create a Process that works for your company

Security & IT Ops
Develop & Test
Proactive OSS/TPS Governance Program = Get Clean & Stay Clean
Open Source Review Board
Plan, Define & Design
Legal & IT Audit
Govern, Release & Maintain
“SET GOVERNANCE POLICY”
Develop corporate policies and standard
operating procedures (SOP) for usage of
OSS and licenses.
“GET CLEAN”
Conduct scans and review findings to
produce an accurate bill of materials of
OSS within your codebase.
“STAY CLEAN”
Proactively address new findings based
on codebase changes or planned
introduction of new OSS materials with
governed approval processes.
Codebase Audit
• Delta Scan + Results
Release Management
• License obligations compliance
• License compatibility analysis
• Third-party notices
• Source distributions
• Copyright notices
• Attribution flow-through
• Contributions back to OSS
• Patent issues & royalties
“STAY CLEAN”
Proactively address new findings based
on codebase changes or planned
introduction of new OSS materials with
governed approval processes.
Remediation
Project Management
• Configure scan settings
• Configure development tool
connectors (SCM + Build)
OSS/3rd Party Requests
• Disclosed Inventory (Proactive)
• Detected Inventory (Reactive)
Codebase Audit
Remediation
Inbound Review
“GET CLEAN”
Conduct scans and review findings to
produce an accurate bill of materials of
OSS within your codebase.
Inbound Review
• Detailed review of all inbound
third-party materials
• Third-party notices
• License obligations
• Corporate policies
“SET GOVERNANCE POLICY”
Develop corporate policies and standard
operating procedures (SOP) for usage of
OSS and licenses.
Project Management
• Start clean or copy existing
• Provision project users
• Define project workflow
• Configure development tools
connectors (ALM)

Create Clear, Dynamic
and Enforced policies

Properties of good OSS Policies
Clear:
Everyone is trained on current policies, at least once a year
Policies are easily available on the corporate intranet / wiki
Dynamic:
Policies quickly change as needs change
Enforced:
Products should not ship if OSS obligations are not met

What should we know in
order to set policies?

What should we know to set Policies (deployment style)?
What is the high level architecture of your product:
SaaS vs shipping product
Embedded Linux vs classic Application
Client / Server
Any Mobile applications?
Web front ends
External resources such as web services

Where do the OSS Packages live?

Product Stacks have many owners
The software development team is often different than the release team / IT team that puts a
product into production.
Things often fall in the gaps between these teams.
They often have different management, legal contacts, understanding of OSS licensing.
Companies many know some OSS from the release team (e.g. Linux, Apache httpd, MySQL, etc..)
or some OSS from the software team (zlib, openssl, etc..) but not always from both.
A “good” list for the IT team is often confused for a “good” list for the actual product.
Linux distributions often lead to long lists of OSS components but not always a clear
understanding of the company’s OSS choices in the product.

What should we know to set Policies?
Do you have a large patent portfolio?
Usage of certain licenses may affect your legal and patent
options due to Patent Grants and Patent Retaliation clauses
Do you understand how your product is distributed?
Do you understand how your product is linked?
Does your architecture require certain types of linking?
Do you use anyone else’s SDK?

Increase inbound
awareness of Licensing

What can we do to increase inbound awareness?
Encourage your developers to always look / ask for the license
of an OSS project
Train them on where licensing info is found
Ask explicit questions to your developers to gauge their
understanding of OSS licensing
Record all inbound usage from snippets to packages!
Log bugs against projects w/ no licenses
• 90%+ of OSS authors will pick a license when prompted
( Example https://github.com/etherkit/JTEncode/issues/1 )

Steps to set up a process
to discover and manage
your use of open source

How to Start Looking for OSS and TPS?
Get people together! The “Conference Room Effect” is very helpful to uncover
large unknown OSS and TPS components
Ask pointed questions (see list on next slide)
• What databases do we use?
• What encryption libraries?
• Etc..
Check with your commercial contract people for known Commercial
dependencies
REQUIRE: Commercial products to deliver current accurate third party
disclosure
Perform classic code reviews; keep reviewing
Perform initial OSS Code scans; keep scanning

Questions to ask your development team
❑Do we have a list of the open source and commercial libraries we are using?
❑How deep have we looked? How complete is this list?
❑What Cryptography, Compression, Multimedia and Application Server libraries are we using?
❑Does this lists include all libraries brought in though repository managers like RPM / Maven / Ruby
Gems / npm, etc…?
❑Do we have a list of all the web services we depend on? (e.g. credit card processors, stock price
lookup, time servers, etc…)
❑What Databases are we using? (including sql, nosql, embedded, hosted etc..)
❑Do we ship VMs, Containers or Applications to our customers? What OS, OSS components and
software stack are we shipping?
❑What is the “Full Stack” required to run our product – including the OS, DB, etc…?
❑Do we have a “Disclosure List” from our commercial vendors

Setting up an Open
Source Review Board

Setting up an OSRB
• An OSRB should be composed of technical, security, business and
legal representatives
• The job of the OSRB is to set and potentially enforce the Open
Source policy and act as an institutional memory
• The OSRB works best when people meet in person, but can be
virtualized as necessary
• OSRBs get better with time, don’t get discouraged
• OSRBs should be expected to change their minds based on new
information or discovered complexity

How to we get developers to buy in?
It’s tough: Developers want to ship and often route around process, esp. if it
not clear why the process exists.
“We should give credit where credit is due”
Attribution
“We are required to”
Legal Obligations
War stories (busybox, grsecurity, etc..)
Development gates
“We should help the Open Source project who help us”
Open Source Ethos
Time and Financial Support
If we don’t know we’re using how can we support them?

IP Risk Management: Levels of Analysis
Overview Audit
• High level BOM
• “Big Rocks”
• e.g. zlib and JBoss
Detailed Overview
• Third Party library BOM
• All Copyrights
• Large cut & paste
findings
Forensic
• Strong overview plus
• All cut & paste evidence
• Modification Review
• Commercial scf leakage
20 % Findings 80% Findings 100 % Findings

Software Supply Chain
While more companies are performing “pedigree scans” of individual OSS projects, this is
still a rare activity for most companies.
At first these were license oriented, but vulnerability analysis has been added
Data shows that the average OSS package has ~7 subcomponents.
Infrastructure pieces (e.g. Hadoop) will have dozens to hundreds of top level components
(not counting these envelope issues)
We expect more emphasis and contractual enforcement of supply chain disclosures to only increase.

What Is “Good Enough”?
• “Analysis Paralysis” is a real thing
• The Community is getting more varied and vocal, you should listen
• The “Community” includes commercial vendors who have different compliance views
• Customers are demanding more; at delivery and at contract signing
• More internal emphasis on tracking down source for pre-compiled binaries – compliance,
long term maintenance and disaster recovery
• Scanning is occurring at internal and external touch points
• More historical versions being reviewed at M&A time
• A supplier to my supplier is MY supplier! (Software Supply Chain concerns)

Have you disclosed or found everything? [Probably Not!]
• Java: Maven and external binary repositories are more prevalent
• C/C++/etc…: Github remote repositories
• Commercial Source compiled on laptop
• Binary analysis bar is being raised due to supply chain concerns
• Where did all these binaries come from? 1000 to 10,000+
• Web services are often overlooked
• Post acquisition discovery of missing code

IP Assets – Managing
How can you make this process easier to manage for yourself?
• Make it clear you expect Compliance
• Education
• Set a clear policy concerning third party inclusion
• Vet products before they are brought into the codebase
• Require comments to explain the origin of copied code
• Keep pristine copies of licensed code
• Audit often to catch surprises early
• Have a remediation plan in place

Managing the Software Supply Chain: Policies that Promote Innovation While Optimizing Security and Compliance

More Related Content

What's hot (17)

Similar to Managing the Software Supply Chain: Policies that Promote Innovation While Optimizing Security and Compliance (20)

More from FINOS (20)

Recently uploaded (20)

Managing the Software Supply Chain: Policies that Promote Innovation While Optimizing Security and Compliance