SlideShare a Scribd company logo

Analysis of-quality-of-pkgs-in-packagist-univ-20171024

C
Clark Everetts

The document presents an analysis of the quality of libraries available in the Packagist ecosystem, emphasizing the importance of reliable and secure packages for PHP applications. It discusses the challenges developers face in finding quality libraries and offers tools and methods for assessing library quality, including static analysis tools. The author, Clark Everetts, provides a PHP script for running several quality analysis tools on codebases to aid developers in evaluating their dependencies.

Software
Related topics:
Software Quality Insights
1 of 33
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 1 An Analysis of the Quality of Libraries in the Packagist Universe Clark Everetts Sr. Professional Services Consultant 24 October 2017
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 2 Slides, Joind.in, and Stuff ‱ Rate & comment: https://joind.in/talk/377ec ‱ Slides: https://www.slideshare.net/clarkphp ‱ Tweets: #zendcon2017 ‱ Twitter: @clarkphp
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 3 INTRODUCTION/ BACKGROUND
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 4 As you know, Composer is
 ‱ Knows what packages your application or library depends upon ‱ Obtains those packages, and all of their dependencies, and installs appropriate versions of them into your project (and local cache) ‱ When requested, checks for updates compatible with your project, and downloads them into your project (and local cache) ‱ Allows you to pin multiple applications/libraries to the same or different versions of the packages they use. Composer makes it easier to manage application dependencies. 
 a per-project PHP dependency manager
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 5 Packagist.org Repository Package Archivist Just a Composer (“type”) Repository
 ‱ 
 but it is the primary repository for open source packages ‱ Best Practice for Open Source Projects: register it at packagist.org ‱ Searchable / Browsable ‱ Less work for people to find and use your package. ‱ Many, many, many packages available. There is duplication in functionality and – I suspect - a wide range of quality.
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 6 https://packagist.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 7 Questions of interest to me ‱ Composer & Packagist are GREAT! ‱ Easy to consume 3rd party libraries ‱ Easy to publish libraries How to find libraries that are ‱ Reliable ‱ Well-built ‱ Tested ‱ Secure How many projects available via Packagist?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 8 Why so important? Your Project Project == Application == Library DEPENDENCIES, PACKAGES, LIBRARIES needs A, B, C, D A B C D needs E E F needs E You are responsible for the application, and for the choice in which dependencies you use. HG needs G, H
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 9 No, really! Consumer-only. Why important? Higher quality libraries are ‱ Easier to understand ‱ Easier to maintain ‱ Easier to test ‱ Get updated sooner ‱ Bugfixes ‱ New features ‱ Don’t break as easily or as often ‱ Greater interest/participation from dev team and users *Campbell County Kentucky Public Library
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 10 SECURITY
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 11 Security – Known Vulnerabilities
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 12 "conflict": { "adodb/adodb-php": "<5.20.6", "amphp/artax": ">=2,<2.0.6|<1.0.6", "aws/aws-sdk-php": ">=3,<3.2.1", "bugsnag/bugsnag-laravel": ">=2,<2.0.2", "cakephp/cakephp": ">=3,<3.0.15|>=2,<2.4.99|>=2.5,<2.5.99|>=2.6,<2.6.12|>=1.3,<1.3.18|>=2.7,<2.7.6|>=3.1,<3.1.4", "cart2quote/module-quotation": ">=4.1.6,<=4.4.5|>=5,<5.4.4", "cartalyst/sentry": "<2.1", "codeigniter/framework": "<=3.0.6", "composer/composer": "<=1.0.0-alpha11", "contao-components/mediaelement": ">=2.14.2,<2.21.1", "contao/core": ">=2,<3.5.28", "contao/core-bundle": ">=4,<4.4.1", "doctrine/annotations": ">=1,<1.2.7", "doctrine/cache": ">=1,<1.3.2|>=1.4,<1.4.2", "doctrine/common": ">=2,<2.4.3|>=2.5,<2.5.1", "doctrine/dbal": ">=2,<2.0.8|>=2.1,<2.1.2", "doctrine/doctrine-bundle": "<1.5.2", "doctrine/doctrine-module": "<=0.7.1", "doctrine/mongodb-odm": ">=1,<1.0.2", "doctrine/mongodb-odm-bundle": ">=2,<3.0.1", "doctrine/orm": ">=2,<2.4.8|>=2.5,<2.5.1", "dompdf/dompdf": ">=0.6,<0.6.2", "drupal/core": ">=8,<8.3.7", "drupal/drupal": ">=8,<8.3.7", etc., etc. Avoiding packages with known vulnerabilities
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 13 Security – The Unknown Vulnerabilities ‱ OWASP Discussion on Source Code Analysis Tools – https://www.owasp.org/index.php/Source_Code_Analysis_Tools ‱ Static Application Security Testing – Security flaws are currently better than the tools – IDEs, catch potential vulnerable code during development ‱ Strengths – Scalability – Reliable for buffer overflow, SQL injection ‱ Weaknesses – Configuration affects security – False positives – Many flaws not amenable to automatic discovery ‱ Authentication/Access Control ‱ Poor use of crypto
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 14 Security – Some Tools for PHP ‱ RIPS – http://sourceforge.net/projects/rips-scanner/ ‱ phpcs-security-audit – PHP_CodeSniffer sniffs for core PHP and Drupal 7 – https://github.com/FloeDesignTechnologies/phpcs-security-audit ‱ VisualCodeGrepper (VCG) – C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments indicative of insecure code – http://sourceforge.net/projects/visualcodegrepp/ ‱ (Plug for KlocWork from Rogue Wave, for C, C++, Java, C#) – https://www.roguewave.com/capabilities/static-code-analysis
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 15 OTHER TOOLS
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 16 Some static analysis tools for PHP ‱ PHPLOC – size metrics, complexity, dependencies, structure – https://github.com/sebastianbergmann/phploc ‱ Copy/Paste Detector – finds duplicated code (exact dups) – https://github.com/sebastianbergmann/phpcpd ‱ PHP Depend - – https://pdepend.org/ ‱ PHP Mess Detector – https://phpmd.org/ ‱ Phan – https://github.com/phan/phan ‱ Exakat – https://www.exakat.io/ ‱ PHPMetrics – http://www.phpmetrics.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 17 WHAT MEAN QUALI-TAY?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 18 What do we mean by “quality”? Concepts/words ‱ Reliable, well-built, tested, secure What “observables” indicate higher quality? Quantifiable vs. Non- Quantifiable Download totals indicate popularity, but download trend would be more indicative. Download totals also show developers trying out a package, not necessarily continued usage. Stars or Likes or ratings
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 19 Identifying a quality package ‱ Does it do what we think it does/expect it do to, and at what level of confidence? ‱ How many defects are known to exist, or have existed, in it? – Relative to size of the package codebase (defect density)? ‱ How many defects are known to have been corrected, and at what level of confidence? ‱ What is the rate of defect discovery? Of defect removal? ‱ What are the densities of various severity levels of defects?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 20 Context! Critical Programming errors can lead to catastrophic outages or performance degradations that make a system unusable. Such programming errors at the system level are 90% of production issues. Such errors at the unit level, though far more numerous, are less than 10% of production issues.
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 21 A SIMPLE ONE-STOP ANALYSIS SCRIPT
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 22 A PHP script to run some QA tools $ cat analyse-codebase.php <?php // analyse-codebase.php // Runs several PHP static analysis tools; assumes *.phar names // This could be done with a shell/batch script, instead of php. // If running under Windows, use paths like 'C:desiredpathtofile' // If running under *nix, use paths like '/desired/path/to/file' // If PHP binary is not in your command path, you need to specify path to it. // const PHP_EXE = '/the/path/to/your/PHP/executable'; const PHP_EXE = 'php'; // the location where the QA tools reside (anywhere you like) // const QATOOLS = '/a/folder/containing/this/analyse-codebase.php/script'; const QATOOLS = 'C:Usersclark.ePHP-QA-Tools’;
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 23 PHP QA tools script: setup continued // the directory (folder) ENCLOSING the code you wish to analyze // const CODEBASE_ROOT = '/path/to/parent/of/your/source/code/appname'; const CODEBASE_ROOT = 'C:Usersclark.escratchpadqasamples'; // where to write the analyses OUTPUT files const OUTPUT_DIR = 'C:Usersclark.escratchpadqa'; // used to give a name to the output files $appName = 'samples'; // this is a folder containing the code to analyze. // It is APPENDED to CODEBASE_ROOT, allowing for analysis of // a particular folder within CODEBASE_ROOT), if needed. $codebaseLocation = '';
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 24 PHP QA tools script: setup continued // list of subfolders to exclude from counting (can be empty, but you often should exclude some folders) // PLEASE check this list and edit accordingly, BEFORE running the script. // Note, for PHPLOC, any directory in the tree with a name in this array is ignored, not just the top-level dirs. $excludeDirs = [ 'cache', 'config', 'documents', 'logs', 'public', 'sql_source', 'third_party', 'test', 'vendor', 'views', ];
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 25 PHP QA tools script: calling the tools // becomes part of the output file names $timestamp = date('YmdHis'); $targetPhpVersion = '7.1'; print phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; //print phpCompatibility($appName, $timestamp, $excludeDirs, $codebaseLocation, $targetPhpVersion) . PHP_EOL;
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 26 PHP QA tools script: Lines of Code function phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phploc'; $options = '--log-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --log-csv=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.csv" . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 27 PHP QA tools script: Copy/Paste Detector function copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpcpd'; $options = '--log-pmd=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --no-ansi' . ' --no-interaction' . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 28 PHP QA tools script: PHP Depend function pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'pdepend'; $options = '--dependency-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- dependencies.xml" . ' --jdepend-chart=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.svg" . ' --jdepend-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --overview-pyramid=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- overview-pyramid.svg" . ' --summary-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- summary.xml" . ' --coderank-mode=inheritance' // how is this used? . ' --coverage-report=' . OUTPUT_DIR . "$appName- $tool-$timestamp-coverage.xml" . ' --ignore=' . implode(',', $excludeDirs);
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 29 PHP QA tools script: PHP Depend $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 30 PHP QA tools script: PHP Mess Detector function messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpmd'; $ruleSets = ['cleancode', 'codesize', 'controversial', 'design', 'naming', 'unusedcode']; foreach ($ruleSets as $ruleSet) { $options = "text $ruleSet --reportfile " . OUTPUT_DIR . "/$appName-$tool-$timestamp-$ruleSet.out" . ' --exclude ' . implode(',', $excludeDirs); // --strict: also report those nodes with a @SuppressWarnings annotation shell_exec('php ' . QATOOLS . "/$tool.phar " . CODEBASE_ROOT . "/$codebaseLocation $options"); print $ruleSet . PHP_EOL; } }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 31 Other Resources ‱ PHP The Right Way - http://www.phptherightway.com/ ‱ OWASP Tools - https://www.owasp.org/index.php/Source_Code_Analysis_Tools ‱ Survive the Deep End: PHP Security (2013, but relevant) - http://phpsecurity.readthedocs.io/en/latest/ ‱ SonarQube – “Water Leak” approach to code quality – https://www.sonarqube.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 32 So, who is this guy? Clark Everetts, ZCE ‱ Rogue Wave Software (acquired Zend October 2015) ‱ PHP since 2005 ‱ Professional Services Consultant – Architecture and Performance Audits – PHP, Zend Framework Training – Application Development, Best Practices, etc. – IBM i ‱ clark.everetts@roguewave.com @clarkphp +ClarkEveretts
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 33 THANK-YOU clark.everetts@roguewave.com @clarkphp +ClarkEveretts Tweet: #zendcon2017 Rate, comment, get slides https://joind.in/talk/377ec Your feedback is invaluable!

More Related Content

PDF
Php Dependency Management with Composer ZendCon 2017
Clark Everetts
 
PPTX
InSpec Workshop DevSecCon 2017
Mandi Walls
 
ODP
2017 DevSecCon ZAP Scripting Workshop
Simon Bennetts
 
PPTX
Banfootguns devseccon 2019
Morgan Roman
 
PDF
DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...
DevSecCon
 
PDF
Configuration as Code in Jenkins. What's new? Nov 2016
Oleg Nenashev
 
PPTX
DevOOPS: Attacks and Defenses for DevOps Toolchains
Chris Gates
 
PPTX
Using Chef InSpec for Infrastructure Security
Mandi Walls
 
Php Dependency Management with Composer ZendCon 2017
Clark Everetts
 
InSpec Workshop DevSecCon 2017
Mandi Walls
 
2017 DevSecCon ZAP Scripting Workshop
Simon Bennetts
 
Banfootguns devseccon 2019
Morgan Roman
 
DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...
DevSecCon
 
Configuration as Code in Jenkins. What's new? Nov 2016
Oleg Nenashev
 
DevOOPS: Attacks and Defenses for DevOps Toolchains
Chris Gates
 
Using Chef InSpec for Infrastructure Security
Mandi Walls
 

What's hot (20)

PDF
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
 
PDF
The Seven Habits of Highly Effective Puppet Users - PuppetConf 2014
Puppet
 
PDF
Getting out of the Job Jungle with Jenkins
Sonatype
 
PPTX
Prescriptive Security with InSpec - All Things Open 2019
Mandi Walls
 
KEY
Continuous Integration & Drupal
LimoenGroen
 
PPT
Getting Started With Jenkins And Drupal
Philip Norton
 
PDF
LasCon 2014 DevOoops
Chris Gates
 
PPTX
Adding Security to Your Workflow With InSpec - SCaLE17x
Mandi Walls
 
PDF
Composer - The missing package manager for PHP
Tareq Hasan
 
PDF
JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...
CloudBees
 
PDF
Drupal Continuous Integration (European Drupal Days 2015)
Eugenio Minardi
 
PPTX
Adding Security and Compliance to Your Workflow with InSpec
Mandi Walls
 
PDF
Choosing the Right Framework for Running Docker Containers in Prod
Josh Padnick
 
PDF
Create Disposable Test Environments with Vagrant and Puppet
Gene Gotimer
 
PPTX
Drupal Continuous Integration with Jenkins - The Basics
John Smith
 
PPTX
Drupal Continuous Integration with Jenkins - Deploy
John Smith
 
PPTX
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
Chris Gates
 
PDF
Open Canary - novahackers
Chris Gates
 
PPTX
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
ZIP
Drupal Deployment
Jeff Eaton
 
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
 
The Seven Habits of Highly Effective Puppet Users - PuppetConf 2014
Puppet
 
Getting out of the Job Jungle with Jenkins
Sonatype
 
Prescriptive Security with InSpec - All Things Open 2019
Mandi Walls
 
Continuous Integration & Drupal
LimoenGroen
 
Getting Started With Jenkins And Drupal
Philip Norton
 
LasCon 2014 DevOoops
Chris Gates
 
Adding Security to Your Workflow With InSpec - SCaLE17x
Mandi Walls
 
Composer - The missing package manager for PHP
Tareq Hasan
 
JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...
CloudBees
 
Drupal Continuous Integration (European Drupal Days 2015)
Eugenio Minardi
 
Adding Security and Compliance to Your Workflow with InSpec
Mandi Walls
 
Choosing the Right Framework for Running Docker Containers in Prod
Josh Padnick
 
Create Disposable Test Environments with Vagrant and Puppet
Gene Gotimer
 
Drupal Continuous Integration with Jenkins - The Basics
John Smith
 
Drupal Continuous Integration with Jenkins - Deploy
John Smith
 
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
Chris Gates
 
Open Canary - novahackers
Chris Gates
 
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
Drupal Deployment
Jeff Eaton
 
Ad

Similar to Analysis of-quality-of-pkgs-in-packagist-univ-20171024 (20)

PDF
Php Dependency Management with Composer ZendCon 2016
Clark Everetts
 
PDF
Open source software: The infrastructure impact
Rogue Wave Software
 
PPTX
How to migrate SourcePro apps from Solaris to Linux
Rogue Wave Software
 
PPT
Case study
karan saini
 
PPTX
Continuous security: Bringing agility to the secure development lifecycle
Rogue Wave Software
 
PPTX
Programming languages and techniques for today’s embedded andIoT world
Rogue Wave Software
 
PPTX
Securing Container Deployments from Build to Ship to Run - August 2017 - Ranc...
Shannon Williams
 
PDF
stackconf 2024 | How to hack and defend (your) open source by Roman Zhukov.pdf
NETWAYS
 
ODP
DevOps, CLI, APIs, Oh My! Security Gone Agile
Matt Tesauro
 
PPTX
Xebia labsperforce final
Perforce
 
PDF
Monitoring Attack Surface to Secure DevOps Pipelines
Denim Group
 
ODP
Effective DevSecOps
Pawel Krawczyk
 
PDF
Good Practices for Developing Scientific Software Frameworks: The WRENCH fram...
Rafael Ferreira da Silva
 
PDF
What is the Secure Supply Chain and the Current State of the PHP Ecosystem
sparkfabrik
 
PDF
Collaborative security : Securing open source software
Priyanka Aash
 
PDF
Shift Left Security
gjdevos
 
PPTX
AppSec DC 2019 ASVS 4.0 Final.pptx
TuynNguyn819213
 
PPTX
AppSec DC 2019 ASVS 4.0 Final.pptx
Josh Grossman
 
PDF
Open source software governance with DejaCode
nexB Inc.
 
PPTX
[Wroclaw #5] OWASP Projects: beyond Top 10
OWASP
 
Php Dependency Management with Composer ZendCon 2016
Clark Everetts
 
Open source software: The infrastructure impact
Rogue Wave Software
 
How to migrate SourcePro apps from Solaris to Linux
Rogue Wave Software
 
Case study
karan saini
 
Continuous security: Bringing agility to the secure development lifecycle
Rogue Wave Software
 
Programming languages and techniques for today’s embedded andIoT world
Rogue Wave Software
 
Securing Container Deployments from Build to Ship to Run - August 2017 - Ranc...
Shannon Williams
 
stackconf 2024 | How to hack and defend (your) open source by Roman Zhukov.pdf
NETWAYS
 
DevOps, CLI, APIs, Oh My! Security Gone Agile
Matt Tesauro
 
Xebia labsperforce final
Perforce
 
Monitoring Attack Surface to Secure DevOps Pipelines
Denim Group
 
Effective DevSecOps
Pawel Krawczyk
 
Good Practices for Developing Scientific Software Frameworks: The WRENCH fram...
Rafael Ferreira da Silva
 
What is the Secure Supply Chain and the Current State of the PHP Ecosystem
sparkfabrik
 
Collaborative security : Securing open source software
Priyanka Aash
 
Shift Left Security
gjdevos
 
AppSec DC 2019 ASVS 4.0 Final.pptx
TuynNguyn819213
 
AppSec DC 2019 ASVS 4.0 Final.pptx
Josh Grossman
 
Open source software governance with DejaCode
nexB Inc.
 
[Wroclaw #5] OWASP Projects: beyond Top 10
OWASP
 
Ad

Recently uploaded (20)

PPTX
history of c programming in notes for students .pptx
Vijayakumari829030
 
PDF
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
PPTX
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
PDF
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
PDF
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
PDF
System and Network Administraation Chapter 3
jaramharo
 
PDF
AI in Product Development-omnex systems
omnex systems
 
PDF
System and Network Administration Chapter 2
jaramharo
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PPTX
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
PDF
medical staffing services at VALiNTRY
Tiyan Grayson
 
PDF
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
history of c programming in notes for students .pptx
Vijayakumari829030
 
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
Which alternative to Crystal Reports is best for small or large businesses.pdf
Varsha Nayak
 
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
System and Network Administraation Chapter 3
jaramharo
 
AI in Product Development-omnex systems
omnex systems
 
System and Network Administration Chapter 2
jaramharo
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
medical staffing services at VALiNTRY
Tiyan Grayson
 
PTS Company Brochure 2025 (1).pdf.......
pts464036
 

Analysis of-quality-of-pkgs-in-packagist-univ-20171024

  • 1. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 1 An Analysis of the Quality of Libraries in the Packagist Universe Clark Everetts Sr. Professional Services Consultant 24 October 2017
  • 2. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 2 Slides, Joind.in, and Stuff ‱ Rate & comment: https://joind.in/talk/377ec ‱ Slides: https://www.slideshare.net/clarkphp ‱ Tweets: #zendcon2017 ‱ Twitter: @clarkphp
  • 3. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 3 INTRODUCTION/ BACKGROUND
  • 4. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 4 As you know, Composer is
 ‱ Knows what packages your application or library depends upon ‱ Obtains those packages, and all of their dependencies, and installs appropriate versions of them into your project (and local cache) ‱ When requested, checks for updates compatible with your project, and downloads them into your project (and local cache) ‱ Allows you to pin multiple applications/libraries to the same or different versions of the packages they use. Composer makes it easier to manage application dependencies. 
 a per-project PHP dependency manager
  • 5. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 5 Packagist.org Repository Package Archivist Just a Composer (“type”) Repository
 ‱ 
 but it is the primary repository for open source packages ‱ Best Practice for Open Source Projects: register it at packagist.org ‱ Searchable / Browsable ‱ Less work for people to find and use your package. ‱ Many, many, many packages available. There is duplication in functionality and – I suspect - a wide range of quality.
  • 6. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 6 https://packagist.org/
  • 7. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 7 Questions of interest to me ‱ Composer & Packagist are GREAT! ‱ Easy to consume 3rd party libraries ‱ Easy to publish libraries How to find libraries that are ‱ Reliable ‱ Well-built ‱ Tested ‱ Secure How many projects available via Packagist?
  • 8. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 8 Why so important? Your Project Project == Application == Library DEPENDENCIES, PACKAGES, LIBRARIES needs A, B, C, D A B C D needs E E F needs E You are responsible for the application, and for the choice in which dependencies you use. HG needs G, H
  • 9. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 9 No, really! Consumer-only. Why important? Higher quality libraries are ‱ Easier to understand ‱ Easier to maintain ‱ Easier to test ‱ Get updated sooner ‱ Bugfixes ‱ New features ‱ Don’t break as easily or as often ‱ Greater interest/participation from dev team and users *Campbell County Kentucky Public Library
  • 10. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 10 SECURITY
  • 11. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 11 Security – Known Vulnerabilities
  • 12. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 12 "conflict": { "adodb/adodb-php": "<5.20.6", "amphp/artax": ">=2,<2.0.6|<1.0.6", "aws/aws-sdk-php": ">=3,<3.2.1", "bugsnag/bugsnag-laravel": ">=2,<2.0.2", "cakephp/cakephp": ">=3,<3.0.15|>=2,<2.4.99|>=2.5,<2.5.99|>=2.6,<2.6.12|>=1.3,<1.3.18|>=2.7,<2.7.6|>=3.1,<3.1.4", "cart2quote/module-quotation": ">=4.1.6,<=4.4.5|>=5,<5.4.4", "cartalyst/sentry": "<2.1", "codeigniter/framework": "<=3.0.6", "composer/composer": "<=1.0.0-alpha11", "contao-components/mediaelement": ">=2.14.2,<2.21.1", "contao/core": ">=2,<3.5.28", "contao/core-bundle": ">=4,<4.4.1", "doctrine/annotations": ">=1,<1.2.7", "doctrine/cache": ">=1,<1.3.2|>=1.4,<1.4.2", "doctrine/common": ">=2,<2.4.3|>=2.5,<2.5.1", "doctrine/dbal": ">=2,<2.0.8|>=2.1,<2.1.2", "doctrine/doctrine-bundle": "<1.5.2", "doctrine/doctrine-module": "<=0.7.1", "doctrine/mongodb-odm": ">=1,<1.0.2", "doctrine/mongodb-odm-bundle": ">=2,<3.0.1", "doctrine/orm": ">=2,<2.4.8|>=2.5,<2.5.1", "dompdf/dompdf": ">=0.6,<0.6.2", "drupal/core": ">=8,<8.3.7", "drupal/drupal": ">=8,<8.3.7", etc., etc. Avoiding packages with known vulnerabilities
  • 13. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 13 Security – The Unknown Vulnerabilities ‱ OWASP Discussion on Source Code Analysis Tools – https://www.owasp.org/index.php/Source_Code_Analysis_Tools ‱ Static Application Security Testing – Security flaws are currently better than the tools – IDEs, catch potential vulnerable code during development ‱ Strengths – Scalability – Reliable for buffer overflow, SQL injection ‱ Weaknesses – Configuration affects security – False positives – Many flaws not amenable to automatic discovery ‱ Authentication/Access Control ‱ Poor use of crypto
  • 14. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 14 Security – Some Tools for PHP ‱ RIPS – http://sourceforge.net/projects/rips-scanner/ ‱ phpcs-security-audit – PHP_CodeSniffer sniffs for core PHP and Drupal 7 – https://github.com/FloeDesignTechnologies/phpcs-security-audit ‱ VisualCodeGrepper (VCG) – C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments indicative of insecure code – http://sourceforge.net/projects/visualcodegrepp/ ‱ (Plug for KlocWork from Rogue Wave, for C, C++, Java, C#) – https://www.roguewave.com/capabilities/static-code-analysis
  • 15. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 15 OTHER TOOLS
  • 16. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 16 Some static analysis tools for PHP ‱ PHPLOC – size metrics, complexity, dependencies, structure – https://github.com/sebastianbergmann/phploc ‱ Copy/Paste Detector – finds duplicated code (exact dups) – https://github.com/sebastianbergmann/phpcpd ‱ PHP Depend - – https://pdepend.org/ ‱ PHP Mess Detector – https://phpmd.org/ ‱ Phan – https://github.com/phan/phan ‱ Exakat – https://www.exakat.io/ ‱ PHPMetrics – http://www.phpmetrics.org/
  • 17. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 17 WHAT MEAN QUALI-TAY?
  • 18. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 18 What do we mean by “quality”? Concepts/words ‱ Reliable, well-built, tested, secure What “observables” indicate higher quality? Quantifiable vs. Non- Quantifiable Download totals indicate popularity, but download trend would be more indicative. Download totals also show developers trying out a package, not necessarily continued usage. Stars or Likes or ratings
  • 19. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 19 Identifying a quality package ‱ Does it do what we think it does/expect it do to, and at what level of confidence? ‱ How many defects are known to exist, or have existed, in it? – Relative to size of the package codebase (defect density)? ‱ How many defects are known to have been corrected, and at what level of confidence? ‱ What is the rate of defect discovery? Of defect removal? ‱ What are the densities of various severity levels of defects?
  • 20. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 20 Context! Critical Programming errors can lead to catastrophic outages or performance degradations that make a system unusable. Such programming errors at the system level are 90% of production issues. Such errors at the unit level, though far more numerous, are less than 10% of production issues.
  • 21. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 21 A SIMPLE ONE-STOP ANALYSIS SCRIPT
  • 22. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 22 A PHP script to run some QA tools $ cat analyse-codebase.php <?php // analyse-codebase.php // Runs several PHP static analysis tools; assumes *.phar names // This could be done with a shell/batch script, instead of php. // If running under Windows, use paths like 'C:desiredpathtofile' // If running under *nix, use paths like '/desired/path/to/file' // If PHP binary is not in your command path, you need to specify path to it. // const PHP_EXE = '/the/path/to/your/PHP/executable'; const PHP_EXE = 'php'; // the location where the QA tools reside (anywhere you like) // const QATOOLS = '/a/folder/containing/this/analyse-codebase.php/script'; const QATOOLS = 'C:Usersclark.ePHP-QA-Tools’;
  • 23. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 23 PHP QA tools script: setup continued // the directory (folder) ENCLOSING the code you wish to analyze // const CODEBASE_ROOT = '/path/to/parent/of/your/source/code/appname'; const CODEBASE_ROOT = 'C:Usersclark.escratchpadqasamples'; // where to write the analyses OUTPUT files const OUTPUT_DIR = 'C:Usersclark.escratchpadqa'; // used to give a name to the output files $appName = 'samples'; // this is a folder containing the code to analyze. // It is APPENDED to CODEBASE_ROOT, allowing for analysis of // a particular folder within CODEBASE_ROOT), if needed. $codebaseLocation = '';
  • 24. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 24 PHP QA tools script: setup continued // list of subfolders to exclude from counting (can be empty, but you often should exclude some folders) // PLEASE check this list and edit accordingly, BEFORE running the script. // Note, for PHPLOC, any directory in the tree with a name in this array is ignored, not just the top-level dirs. $excludeDirs = [ 'cache', 'config', 'documents', 'logs', 'public', 'sql_source', 'third_party', 'test', 'vendor', 'views', ];
  • 25. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 25 PHP QA tools script: calling the tools // becomes part of the output file names $timestamp = date('YmdHis'); $targetPhpVersion = '7.1'; print phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; //print phpCompatibility($appName, $timestamp, $excludeDirs, $codebaseLocation, $targetPhpVersion) . PHP_EOL;
  • 26. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 26 PHP QA tools script: Lines of Code function phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phploc'; $options = '--log-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --log-csv=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.csv" . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 27. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 27 PHP QA tools script: Copy/Paste Detector function copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpcpd'; $options = '--log-pmd=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --no-ansi' . ' --no-interaction' . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 28. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 28 PHP QA tools script: PHP Depend function pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'pdepend'; $options = '--dependency-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- dependencies.xml" . ' --jdepend-chart=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.svg" . ' --jdepend-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --overview-pyramid=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- overview-pyramid.svg" . ' --summary-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- summary.xml" . ' --coderank-mode=inheritance' // how is this used? . ' --coverage-report=' . OUTPUT_DIR . "$appName- $tool-$timestamp-coverage.xml" . ' --ignore=' . implode(',', $excludeDirs);
  • 29. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 29 PHP QA tools script: PHP Depend $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 30. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 30 PHP QA tools script: PHP Mess Detector function messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpmd'; $ruleSets = ['cleancode', 'codesize', 'controversial', 'design', 'naming', 'unusedcode']; foreach ($ruleSets as $ruleSet) { $options = "text $ruleSet --reportfile " . OUTPUT_DIR . "/$appName-$tool-$timestamp-$ruleSet.out" . ' --exclude ' . implode(',', $excludeDirs); // --strict: also report those nodes with a @SuppressWarnings annotation shell_exec('php ' . QATOOLS . "/$tool.phar " . CODEBASE_ROOT . "/$codebaseLocation $options"); print $ruleSet . PHP_EOL; } }
  • 31. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 31 Other Resources ‱ PHP The Right Way - http://www.phptherightway.com/ ‱ OWASP Tools - https://www.owasp.org/index.php/Source_Code_Analysis_Tools ‱ Survive the Deep End: PHP Security (2013, but relevant) - http://phpsecurity.readthedocs.io/en/latest/ ‱ SonarQube – “Water Leak” approach to code quality – https://www.sonarqube.org/
  • 32. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 32 So, who is this guy? Clark Everetts, ZCE ‱ Rogue Wave Software (acquired Zend October 2015) ‱ PHP since 2005 ‱ Professional Services Consultant – Architecture and Performance Audits – PHP, Zend Framework Training – Application Development, Best Practices, etc. – IBM i ‱ clark.everetts@roguewave.com @clarkphp +ClarkEveretts
  • 33. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 33 THANK-YOU clark.everetts@roguewave.com @clarkphp +ClarkEveretts Tweet: #zendcon2017 Rate, comment, get slides https://joind.in/talk/377ec Your feedback is invaluable!