Security Redefined - Prevention is the future!!

Preventative Security SDLC Application Testing
Automated SAST | DAST Scanning
By Daniel L. Cruz
EVP Chief Marketing Officer
NewSky Security positions itself and its AppRisk core technology as a security
industry change agent. Security has long been a network, system, desktop and endpoint
device centric practice with most enterprise IT, OEM, ISV, and DevOps and their security
vendors approaching security as a during deployment or post-deployment practice.
NewSky continues to observe most security approaches to be reactionary; wherein
several recurring practices are sighted on websites and data sheets all consistent with the
early 1990’s, those being; a) Capabilities: detection, identification, analysis and
remediation; and the scope of their practice b) Solution: network risk and threat
management. With little differentiation in capabilities, major DOS and DDOS product
suppliers, DDI product suppliers and others are all seemingly converging to these
reactionary security approaches. What NewSky does not see, except for a small minority,
are new technologies, and automated approaches to “preventative” or “proactive”
security. This paper endeavors to provide a snapshot on what NewSky Security’s new
technology and approach is leveraging a largely untapped practice of Security Software
Development Life Cycle (SSDLC)” and the dynamic preventative automation be applied to
Mobile and IOT application security. Comparative note: Think of SSDLC as you would
view the evolution of a medical practice. Until the mid-80’s or early 90’s primary care
physicians were a person’s life line for both diagnosis and treatment. As the late 90’s
arrived specialists of all sorts arrived on the scene to deal with different micro and macro
parts of the human anatomy be that on the exterior or interior of the body. Then as
pharmacology rose to address the mental well-being of patients even more specialization
arose. With diet, sleep, exercise identified as being additional key elements to overall
good health, we have also transitioned naming from medical industry to the healthcare
industry wherein preventative, or early detection medicine is now leading the way.
NewSky Security is and the security industry needs to be about a very similar change.
With IOT still a few years from becoming mainstream, the area of Mobility and
Applications are growing and is dominated by the Android Operating System. See Figure
1. As such, all one needs to do is read the research and industry publications to see
Android as a top contender as the OS of choice for IOT and why NewSky’s approach in
mobile is on the Android OS. Research reveals the millions of developers producing

2 | P a g e
Android applications across vertical and horizontal markets and industries continue to
trend upward and this supports our one OS decision. However, as is the case with
Microsoft’s Windows OS, Android will pose the greater security risk on the horizon.
Figure 1
NewSky and its AppRisk family of products designed around its ‘preventative’
SSDLC architecture (gateway-like functionality) that seamlessly interfaces with the
majority of MDM, EMM, MAM, enterprise IT end-customers, DevOps, ISVs, OEMs,
managed service provider offerings and their OSs. In addition, NewSky is partnering with
domestic and international testing labs providing them AppRisk Scanner as part of their
mobile and IOT practice in support of customers seeing great value in NewSky’s
preventative deep scanning and penetration testing of all Android - mobile and IOT
applications – and their support and contribution to the practice of preemptively
identifying and fixing vulnerabilities as part of app design or development and for sure,
well in advance of deployment.
With NewSky’s AppRisk Scanner solution not only is NewSky on a path toward
preventive “hacker proofing” of Android applications, we are on the cusp of reaching even
bigger dividends for the security industry as a whole, “Built-In Security” (more on this
topic in our next paper). For now however, AppRisk Scanner represents a strong new
technology possessing a high degree of automation, ease of use, accuracy, fast to scan
and produce reports. And of great importance to everyone’s bottom line, an inexpensive
preventative deterrent for risk, threat, and vulnerability “avoidance.” As you continue
reading this document, I hope you come to understand the value of our quality scoring
and reporting capabilities plus the fact that both simplicity and usability can lead to high
degrees of efficiency to all types of developers.

3 | P a g e
The notion and practice of preventative security within the context of an existing
or non-existent SDLC program as being either too much change or an unnecessary cost
has little to no merit, especially when one gains or is provided access to the true number
of serious breaches occurring. With the billions of dollars spent on security compliance,
fraud, assessments, audits, etc. combined with the insurance premiums associated with
outdated security practices and policies, CISO’s, their IT security specialists and the
software industry as a whole, need to view preemptive measures and practices with
preventative security as their smallest investment with the greatest return on their security
dollars.
Independent software developers worldwide should also take heed and endeavor
to make a stronger commitment to building security into their SDLC processes. Those
doing so will provide themselves greater opportunities as the will possess the unique and
differentiating skills in mobile and IOT application industries. DevOps groups and forums
have spent the past 10 plus years following the research performed by OWASP, BSIMM,
SANS, ITU, NIST, W3C, and the many independent scientists who agree the time has
arrived where security must be an SDLC component; and when and where feasible built -
into software to allow companies and consumers to have any chance of fending off
malicious attacks and breaches in advance.
Figure 2 below is a 2015 chart similarly published in differing formats but with
almost identical data by Gartner, Forrester, 451 Research, IDC, and several others. Of
course, the “Can you believe these number?” comes from NewSky. Every company that’s
experienced a serious breach and looks back in time at the IT professional who asked for
security budget without success and was terminated can see why.
Figure 2

4 | P a g e
For more than ten years, in addition to the previously mentioned groups, security
scientists working for or consultative contributors to the Dept. of Defense, Dept. of
Commerce, Dept. of Justice, CERT Security, ITU, IEEE Security as well as other noteworthy
industry forums both domestic and international have worked tirelessly to keep the “built-
in security software” vision alive. Yet, in spite of the thousands of reports, articles, surveys
and blogs there has been minimal change within enterprise and ISV development
practices that SDLC is not being used to the extent needed and worse yet, SSDLC is not
being practiced nor is funding making its way to either. As for its contributions, beyond
providing a very strong solution, NewSky launched an early adopter fee schedule aimed
at driving the much needed and collaborative sharing between customers and partners.
The pricing model in Figure 3 represents NewSky’s go-to-market investment (discount)
to promote and encourage customer and partner alike to use the tool as an evaluation
and as an enhancement to existing SDLC best practices. This type of change is not
difficult, nor painful and as you review the price sheet below to begin and execute the
change process required NewSky AppRisk is truly very inexpensive.
Figure 3

5 | P a g e
AppRisk Scanner, our SaaS SSDLC solution for Android applications also comes
with a smart device Agent and a well thought-out and highly functional API and SDK. The
product is designed and positioned as a Mobile Application Vulnerability Scanning
Solution. Designed ground up as a cloud-based offering, the AppRisk core technology
was developed based on patents and work related observations that have allowed for the
incorporation of 15 of the 19 primary functional and risk identification security software
security testing methodologies associated with SSDLC. This approach provides a
tremendous value proposition to AppRisk mobile customers today and even more value
to AppRisk for IOT as the code base remains the same except for dealing with more OS
and device types. We have modeled our core security technology approach around an
industry very concerned about the safety of their customer’s operational environment,
that being the automobile. The automotive industry has many similarities to Healthcare
and First Responders with each possessing a very real human injury or worse outcome
should software break or someone executes a breach of the software for malicious intent.
While one cannot place value on the human factor previously noted, Figure 3 does
provide the potential scale and magnitude of financial loss due to a malicious breach. In
order to minimize or eliminate attacks and successful breaches requires a minimal level
of change. However, the level within a company that approves budgets and spending
must support a radically different understanding of priorities and investment associated
with safeguarding, next to its personnel of course, the company’s proprietary
data/information.
While security is recognized as an important need, it is often compared to an
insurance policy, be it life insurance, medical insurance, vehicle insurance, errors and
omissions, where it is purchased as a safeguard in the event something bad occurs. The
actual number of successful beaches occurring worldwide is alarming and they are costly.
Figure 4

6 | P a g e
Hosted by Amazon and based on NewSky Security’s unique configuration needs,
the AppRisk system meets the highest levels of SLA and QoS assurance AWS has to offer.
With this system, one large telecom uses AppRisk Scanner with a number of custom add-
ons to scan 200,000 plus appstore applications with over 150 million subscribers as well
as several similarly large enterprises and government organizations. Our primary
customers worldwide are the SMB’s who do not necessarily have large application
volumes but every application is strategic to their business operations and success. A
customer’s ability to take advantage of a limitless number of scans of a powerful yet easy
to use system that performs deep analysis, with pinpoint accuracy locating vulnerabilities,
comprehensive and highly accurate vulnerability scoring, with easily read reports. All
performed and made available and beautifully formatted in as little as 5 minutes and
typically no more than 25 minutes.
NewSky’s long term mission is to provide a total SSDLC preventative assessment
solution based on our patented “True Contextual Behavioral Predictive Detection”
(TCBPD) technology. As a part of this solution, NewSky is complementing the Application
side scanning with a mobile/IOT agent that brings the full communication between the
client and application in view. While facilitating a more secure mobile and IOT client side
environment, the metadata that is gathered from both the scanning actions and client
side captured data can be shared with risk and vulnerability data gathered from the
enterprise ecosystem. This combination of data provides a far reaching pool of data to
help create a realistic risk assessment and defense posture of the enterprise. The guiding
principles behind AppRisk Scanner along with in industry-specific AppRisk for
Automotive, AppRisk for Healthcare and so forth, is to facilitate, promote and drive the
rapid adoption of AppRisk at part of your SSDLC. Customers, IT development groups and
independent software developers all share, by virtue of not incorporating in-depth
preventive security testing, the potential reality of exposing their client’s data. If not today
or not tomorrow, given adequate opportunity during a new version release or an update,
and when combined with the fact that as applications grow in functionality they grow in
size and complexity, they become bigger targets for potential hackers.
Striving to demonstrate and promote security as a realistic “built-in” possibility for
all forms of software will be achieved and AppRisk Scanner supporting automated
scanning and pentesting of Android application APK files is a strong beginning. By Q4
AppRisk Scanner will be joined by AppRisk Agent, AppRisk API and AppRisk SDK. While
during 2016, AppRisk will remain primarily focused on mobile, activities with clients in
automotive, healthcare and first responder verticals will be taking shape.

7 | P a g e
Our Capabilities Today and Tomorrow
1) Extensive Expertise on Vulnerabilities
The NewSky team comes from the software side of the industry and have worked
with; its investor and world class IT security solution supplier Venustech; OWASP; the five
organizations listed above; and have been security visionaries for McAfee/Intel, Symantec,
Microsoft, IBM, HP, AT&T, and others. These rich experiences afforded hands-on
development, use, and testing of security products, deep research in the security discipline
across the OSI model, machine learning, and AI and extensive expertise with customers in
the Healthcare/Medical, Government, Automotive, Telecom, Energy, and Retail industries.
2) One of the Largest Vulnerability Repositories
The NewSky maintains a malware, blacklist, pre-scanned and whitelist of a
combined total of approximately five (5) million Android APK files with nearly 6,000
identified vulnerabilities across six (6) industries and twenty (20) types of environments.
Both customers and partners can benefit greatly from the coverage of many important
and prevalent Android related applications interacting with in-house or cloud-based
critical systems and databases exposed to vulnerabilities unknown or worse yet have are
already breached the network and merely waiting for the right moment to strike. The
repository is a shared asset and adjusted and updated in line with changes driven through
NewSky or customer use of AppRisk Scanner to reveal the comprehensive and timely
vulnerabilities identification reports produced.
3) Scanning of Objects
AppRisk Scanner currently scans Android files, can perform iOS but there has been
few requests. Major companies who at one time were iOS are changing to Android and
other Linux versions of software. Our primary experience with Linux, server, desktop and
embedded begins Q4 2016 as NewSky enters the automotive industry.
Accuracy of the Scoring and Reporting
1) Accurate Identification of the Object Information
The AppRisk core systems use of patented “TCBPD” to concurrently and
progressively perform real-time SAST, DAST, and Decomposition has proven its ability to
produce findings with zero defects. Acting much like an operating system, AppRisk
applies a fingerprint-like identifier with intelligent service identification - contextual and
behavioral – producing an industry leading level of accuracy in scoring, reporting, and
eliminating false-positives of the scanned objects.

8 | P a g e
2) Verification of Vulnerabilities Information.
In addition to its powerful SAST and DAST scanning methods, NewSky can utilize,
on a case-by-case basis use the AppRisk platform to perform purpose-specific scanning
on a single vulnerability for verification purposes.
Time is Money - Scanning Needs to be Fast
1) Maximized scanning efficiency.
The discovery and scan processes can be finished rapidly with the help of various
technologies. The comprehensive scan efficiency holds leading position in contrast to
rival products.
2) Vulnerability Scans (SAST and DAST) of 5 Minutes to 25 Minutes – Easily the
Quickest in the Industry.
Updating, with each of the three price plans means the ability to perform
immediate updating (scanning) your in-house repository with potential or real
vulnerabilities on a 7x24x365 basis.
Conclusion
We have the right vision, right approach, and the right first product, please visit our
website, or have one of our test lab partners such as West Coast Labs, show you how they
are applying AppRisk Scanner for their customers. Take your Android APKs and reveal
what you never thought was hiding within them. For organizations where HIPPA or other
compliance is mandated, look to AppRisk for help in doing so.
Please Visit: http://www.newskysecurity.com
https://apprisk.newskysecurity.com
Email: info@newskysecurity.com

9 | P a g e
Listing of companies and organizations
providing reference materials for this paper.

10 | P a g e
Security Industry Glossary of Terms
Acceptance Testing
Formal testing conducted to enable a user, customer, or other
authorized entity to determine whether to accept a system or
component – from IEEE.
Access Control
"Access control ensures that resources are only granted to
those users who are entitled to them" [SANS 09].
Account Harvesting
"Account harvesting is the process of collecting all the
legitimate account names on a system" [SANS 09].
Ad hoc Testing
Testing carried out using no recognized test case design
technique.- from BS 7925.
Attack
"The act of trying to bypass security controls on a system. An
attack may be active, resulting in the alteration of data; or
passive, resulting in the release of data. Note: The fact that an
attack is made does not necessarily mean that it will succeed.
The degree of success depends on the vulnerability of the
system or activity and the effectiveness of existing
countermeasures" [NCSC 88].
Auditing
"Auditing is the information gathering and analysis of assets to
ensure such things as policy compliance and security from
vulnerabilities" [SANS 09].
Authentication
The process of confirming the correctness of the claimed
identity – from SANS 03
Authorization
"Authorization is the approval, permission, or empowerment
for someone or something to do something" [SANS 09].

11 | P a g e
Backdoor
"A backdoor is a tool installed after a compromise to give an
attacker easier access to the compromised system around any
security mechanisms that are in place" [SANS 09].
Black Box Testing
Testing that is based on an analysis of the specification of the
component without reference to its internal workings – from
BS-7925.
Brute Force
"A cryptanalysis technique or other kind of attack method
involving an exhaustive procedure that tries all possibilities, one
by one" [SANS 09].
Buffer Overflow
A buffer overflow occurs when a program or process tries to
store more data in a data storage area than it was intended to
hold. Since buffers are created to contain a finite amount of
data, the extra information—which has to go somewhere—can
overflow into the runtime stack, which contains control
information such as function return addresses and error
handlers.
"An exploitation technique that alters the flow of an application
by overwriting parts of memory. Buffer overflows are a common
cause of malfunctioning software. If the data written into a
buffer exceeds its size, adjacent memory space will be
corrupted and normally produce a fault. An attacker may be
able to utilize a buffer overflow situation to alter an
application’s process flow. Overfilling the buffer and rewriting
memory-stack pointers could be used to execute arbitrary
operating-system commands" [WASC 04].
Buffer Overflow
Attack See stack smashing.
Bug See fault.

12 | P a g e
Capture/Replay Tool
A test tool that records test input as it is sent to the software
under test. The input cases stored can then be used to
reproduce the test at a later time.
Compatibility Testing
Testing whether the system is compatible with other systems
with which it should communicate.
Component
A minimal software item for which a separate specification is
available.
Conformance Testing
The process of testing that an implementation conforms to the
specification on which it is based.
Control-flow Analysis
Any one of several techniques used to statically trace and
characterize the flow of control in software source code.
Cookie
Data exchanged between an HTTP server and a browser (a
client of the server) to store state information on the client side
and retrieve it later for server use. An HTTP server, when
sending data to a client, may send along a cookie, which the
client retains after the HTTP connection closes. A server can use
this mechanism to maintain persistent client-side state
information for HTTP-based applications, retrieving the state
information in later connections.
Correctness The degree to which software conforms to its specification.
Corruption
A threat action that undesirably alters system operation by
adversely modifying system functions or data. [SANS 09]
Cryptographic Attack
A technique for successfully undermining an encryption
scheme.
Cryptography
Cryptography garbles a message in such a way that anyone
who intercepts the message cannot understand it.

13 | P a g e
Data-flow Analysis
Any one of several techniques used to statically trace and
characterize the flow of data in software source code.
Defense in Depth
"Defense In-Depth is the approach of using multiple layers of
security to guard against failure of a single security component"
[SANS 09].
Denial of Service
"The prevention of authorized access to a system resource or
the delaying of system operations and functions" [SANS 09].
Domain The set from which values are selected.
Domain Testing
Testing with test cases based on the specification of input
values accepted by a software component.
Dynamic Analysis
The process of evaluating a system or component based on its
behavior during execution.
Dynamic Link Library
(DLL)
A collection of small programs, any of which can be called when
needed by a larger program that is running in the computer.
Small programs that enable larger programs to communicate
with a specific device such as a printer or scanner are often
packaged as DLL programs (usually referred to as DLL files).
Encryption
Cryptographic transformation of data (called ”plaintext”) into a
form (called ”cipher text”) that conceals the data’s original
meaning to prevent it from being known or used.
Failure
The inability of a system or component to perform its required
functions within specified performance requirements.
Fault
A manifestation of an error in software. A fault, if encountered,
may cause a failure.

14 | P a g e
File Descriptor
Spoofing
An attack where one or more of the three standard C file
descriptors, stdin, stdout, or stderr, are closed before executing
an application. The next file opened by the application will be
assigned one of the standard file descriptors, and output sent
to that standard file descriptor will also go to the newly opened
file.
Format String Attack
"An exploit technique that alters the flow of an application by
using string formatting library features to access other memory
space" [WASC].
Hypertext Transfer
Protocol (HTTP)
The protocol in the Internet Protocol (IP) family used to
transport hypertext documents across an internet.
Integration Testing
Testing performed to expose faults in the interfaces and in the
interaction between integrated components.
Interface Testing
Integration testing in which the interfaces between system
components are tested.
Isolation Testing
Component testing of individual components in isolation from
surrounding components, with surrounding components being
simulated by stubs.
Kernel
The essential center of a computer operating system, the core
that provides basic services for all other parts of the operating
system. A synonym is nucleus. A kernel can be contrasted with
a shell, the outermost part of an operating system that interacts
with user commands. Kernel and shell are terms used more
frequently in UNIX and some other operating systems than in
IBM mainframe systems.
"The essential center of a computer operating system, the core
that provides basic services for all other parts of the operating
system. A synonym is nucleus. A kernel can be contrasted with
a shell, the outermost part of an operating system that interacts
with user commands. Kernel and shell are terms used more

15 | P a g e
frequently in UNIX and some other operating systems than in
IBM mainframe systems" [SANS 09].
National Institute of
Standards and
Technology (NIST)
A unit of the U.S. Commerce Department. Formerly known as
the National Bureau of Standards, NIST promotes and
maintains measurement standards. It also has active programs
for encouraging and helping industry and science to develop
and use these standards.
Negative
Requirements Requirements that state what software should not do.
Operational Testing
Testing conducted to evaluate a system or component in its
operational environment.
Port
A port is nothing more than an integer that uniquely identifies
an endpoint of a communication stream. Only one process per
machine can listen on the same port number.
Precondition
Environmental and state conditions that must be fulfilled
before the component can be executed with a particular input
value.
Protocol
A formal specification for communicating; the special set of
rules that end points in a telecommunication connection use
when they communicate. Protocols exist at several levels in a
telecommunication connection.
Pseudorandom
Appearing to be random, when actually generated according to
a predictable algorithm or drawn from a prearranged sequence.
Race Condition
A race condition exploits the small window of time between a
security control being applied and the service being used.

16 | P a g e
"A race condition exploits the small window of time between a
security control being applied and when the service is used"
[SANS 09].
Registry
The registry in Windows operating systems is the central set of
settings and information required to run the Windows
computer.
Regression Testing
Retesting of a previously tested program following
modification to ensure that faults have not been introduced or
uncovered as a result of the changes made.
Requirement
A capability that must be met or possessed by the
system/software (requirements may be functional or non-
functional).
Requirements-Based
Testing
Designing tests based on objectives derived from requirements
for the software component (e.g., tests that exercise specific
functions or probe the non-functional constraints such as
performance or security).
Reverse Engineering
Acquiring sensitive data by disassembling and analyzing the
design of a system component; acquiring knowledge of a
binary program’s algorithms or data structures.
Risk Assessment
The process by which risks are identified and the impact of
those risks is determined.
Root
"Root is the name of the administrator account in UNIX
systems" [SANS 09].
Security Policy
A set of rules and practices that specify or regulate how a
system or organization provides security services to protect
sensitive and critical system resources.

17 | P a g e
Sensitive Information
"Sensitive information, as defined by the federal government, is
any unclassified information that, if compromised, could
adversely affect the national interest or conduct of federal
initiatives" [SANS 09].
Server
A system entity that provides a service in response to requests
from other system entities called clients.
Session
A virtual connection between two hosts by which network
traffic is passed.
Shell
"A UNIX term for the interactive user interface with an
operating system. The shell is the layer of programming that
understands and executes the commands a user enters. In
some systems, the shell is called a command interpreter. A shell
usually implies an interface with a command syntax (think of
the DOS operating system and its “C:>” prompts and user
commands such as “dir” and “edit”)" [SANS 09].
Socket
The socket tells a host’s IP stack where to plug in a data stream
so that it connects to the right application
Software
Computer programs (which are stored in and executed by
computer hardware) and associated data (which also is stored
in the hardware) that may be dynamically written or modified
during execution.
Specification A description, in any suitable form, of requirements
Specification Testing
An approach to testing wherein the testing is restricted to
verifying that the system/software meets the specification.
SQL Injection
SQL injection is a type of input validation attack specific to
database-driven applications where SQL code is inserted into
application queries to manipulate the database.

18 | P a g e
Stack Smashing
The technique of using a buffer overflow to trick a computer
into executing arbitrary code.
"Stack mashing is the technique of using a buffer overflow to
trick a computer into executing arbitrary code" [SANS 09].
State Transition
A transition between two allowable states of a system or
component.
State Transition
Testing
A test case design technique in which test cases are designed
to execute state transitions.
Static Analysis
Analysis of a program carried out without executing the
program.
Static Analyzer A tool that carries out static analysis.
Stress Testing
Testing conducted to evaluate a system or component at or
beyond the limits of its specified requirements.
Stub
A skeletal or special-purpose implementation of a software
module used to develop or test a component that calls or is
otherwise dependent on it. [IEEE 90].
Symbolic Links "Special files which point at another file" [SANS 09].
Syntax Testing
Test case design technique for a component or system in which
test case design is based on the syntax of the input
System Testing
The process of testing an integrated system to verify that it
meets specified requirements.
Tamper
"To deliberately alter a system’s logic, data, or control
information to cause the system to perform unauthorized
functions or services" [SANS 09].

19 | P a g e
Test Automation
The use of software to control the execution of tests, the
comparison of actual outcomes to predicted outcomes, the
setting up of test preconditions, and other test control and test
reporting functions
Test Case
A set of inputs, execution preconditions, and expected
outcomes developed for a particular objective, such as to
exercise a particular program path or to verify compliance with
a specific requirement.
Test Suite
A collection of one or more test cases for the software under
test.
Test Driver
A program or test tool used to execute software against a test
suite.
Test Environment
A description of the hardware and software environment in
which tests will be run and any other software with which the
software under test interacts when under test, including stubs
and test drivers.
Test Plan
A record of the test planning process detailing the degree of
tester independence, the test environment, the test case design
techniques and test measurement techniques to be used, and
the rationale for their choice
Testware
Software associated with carrying out tests, such as test drivers,
stubs, and software needed to set up and tear down test cases.
Vulnerability
A defect or weakness in a system’s design, implementation, or
operation and management that could be exploited to violate
the system’s security policy.

20 | P a g e
"A flaw or weakness in a system’s design, implementation, or
operation and management that could be exploited to violate
the system’s security policy" [SANS 09].
Web Server
A software process that runs on a host computer connected to
the Internet to respond to HTTP requests for documents from
client web browsers.
Industry Testing Techniques – Only 3 Not Applied By NewSky
Ad hoc Testing (Experience and Exploratory Testing)
Derive tests based on tester’s skill, intuition, and experience with similar programs. This
is also called ”exploratory testing.” This kind of testing is only effective when done by
trained or experienced testers to flesh out special tests not captured in more formal
techniques. Ad hoc testing can take advantage of the specialized instincts of security
analysts, and it also comes into play when a tester has discovered indirect evidence of a
vulnerability and decides to follow up. Penetration testing tends to have an exploratory
flavor.
Requirements-based Testing
Given a set of requirements, devise tests so that each requirement has an associated test
set. Trace test cases back to requirements to ensure that all requirements are covered. In
security testing it can also be useful to build test cases around ambiguities in the
requirements.
Specification-based Testing and Model-based Testing (API testing is a subset)
Given a specification (or even a definition of an interface), test cases can be derived
automatically and can even include an oracle. This sometimes requires a specification
created in a formal language (which is not often encountered). An alternative form is to
create a program model, especially based on interfaces, and derive tests from the interface
model. Test cases can also be created by hand based on a specification, but this is much
more of an art. In security testing, it can be useful to test situations that are not covered
in the specifications.
Equivalence Partitioning

21 | P a g e
Divide the input domain into a collection of subsets, or ”equivalence classes,” which are
deemed equivalent according to the specification. Pick representative tests (sometimes
only one) from within each class. Can also be done with output, path, and program
structure equivalence classes.
Boundary Value Analysis
Choose test cases on or near the boundaries of the input domain of variables, with the
rationale that many defects tend to concentrate near the extreme values of inputs. A
classic example of boundary-value analysis in security testing is to create long input
strings in order to probe potential buffer overflows. More generally, insecure behavior in
boundary cases is often unforeseen by developers, who tend to focus on nominal
situations instead.
Robustness and Fault Tolerance Testing
A variation on boundary value analysis where test cases are chosen outside the domain
in order to test program robustness to unexpected and erroneous inputs. Also useful for
probing fault tolerance and error handling. Errors can lead to insecure conditions, such
as the disclosure of sensitive information in debugging messages or core dumps. Error
handlers are also notorious for containing security bugs.
Decision Table - Logic-based Testing
Decision tables represent logical relationships between conditions (for example, inputs)
and actions (for example, outputs). Derive test cases systematically by considering every
possible combination of conditions and actions. Security testers often focus on conditions
that are not covered in the requirements or specifications.
State-based Testing
Model the program under test as a finite state machine, and then select tests that cover
states and transitions using diverse techniques. This is good for transaction processing,
reactive, and real-time systems. In security testing, it can often be useful to try to force
transitions that do not appear in higher level design artifacts, since vulnerabilities often
appear when software enters an unexpected state.
Control-Flow Testing
Control-flow based coverage criteria aim at covering all statements, classes, or blocks in
a program (or some specified combinations). Reduce the program to a directed graph
and analyze the graph. Decision/condition coverage is one example. The aim is to detect

22 | P a g e
poor and potentially incorrect program structures. This is often infeasible for all but trivial
programs in a white box test setting.
Data Flow-based Testing
Annotate a program control flow graph with information about how variables are defined
and used. Use definition-use pairs (often called d/u testing) such that where V is a
variable, d is a node where V is defined, and u is a node where V is used and there is a
path from d to u. The aim is to detect poor and potentially incorrect program structures.
Data flow testing is often used to test interfaces between subsystems.
Use Case based Testing
Database tests on use of the product in real operation by creating an operational profile
or creating a set of use cases. It is sometimes possible to infer future reliability from test
results (given a statistically correct operational profile). Do this by assigning inputs to a
probability distribution according to their occurrence in actual operation.
White Box / Code-based Testing)
Use the control structure, the data flow structure, decision control, and modularity to
design tests to cover the code. Use coverage analysis (e.g., white box) to assess test
completeness and goodness. This technique is a superset of control flow testing and data
flow testing. White box testing is covered in a separate module of the BSI portal.
Fault-based Testing
Intentionally introduce faults during testing to probe program robustness and reliability.
Determining which kind of faults to introduce and how to observe their effects is a
challenge. Experience with this method is necessary for it to be useful. Code-based fault
injection is discussed in the BSI module on white box testing.
Protocol Conformance Testing
Use a program’s communication protocol as a direct basis for testing the program. This
is useful when a program is supposed to accept a protocol. In combination with boundary-
value testing and equivalence-based testing, this method is useful for web-based
programs and other Internet-based code. Protocol-based testing is especially important
for security testing in web-based applications, since the easiest way for remote attackers
to access such applications is through web protocols (or their buggy implementations, as
the case may be). Protocol-based testing uses black box tools.
Load and Performance Testing

23 | P a g e
Testing specifically aimed at verifying that the subsystem meets specified performance
requirements (e.g., capacity and response time). Load and stress testing exercise a system
to the maximum design load and beyond it. Stressful conditions can expose
vulnerabilities that are otherwise hard to see, and vulnerabilities can also be caused by
the mechanisms that software uses to try to deal with extreme environments. Developers
are often focused on graceful degradation when they create these mechanisms, and they
overlook security.
Security Testing
The use of a variety of testing techniques specifically to probe security. There are two
major aspects of security testing: testing security functionality to ensure that it works and
testing the subsystem in light of malicious attack. Security testing is motivated by probing
undocumented assumptions and areas of particular complexity to determine how a
program can be broken.
Run-Time Verification
Run-time verification seeks to validate that an application conforms to its security
requirements and specifications by dynamically observing the application’s behavior in a
test environment. Requirements such as “all authentication credentials must be encrypted
while in transit” can thus be dynamically verified through observation.
Risk-based / Black Box Testing
Recall that in security testing, there is an increased emphasis on negative requirements,
which state what a software system should not do. Tests can be developed in a number
of ways for negative requirements. The tests should be derived from a risk analysis, which
should encompass not only the high-level risks identified during the design process but
also low-level risks derived from the software itself.
When negative requirements are tested, security testers typically look for common
mistakes and test suspected weaknesses in the application. The emphasis is often on
finding vulnerabilities, often by executing abuse and misuse tests that attempt to exploit
the weaknesses in the application. In addition to demonstrating the presence of
vulnerabilities, security tests can also assist in uncovering symptoms that suggest
vulnerabilities might exist.
It was stated earlier that requirements can be expected to contain mitigations for many
risks. Mitigations generally result in positive requirements, but the fact that some risk has
a mitigation does not imply that it should be ignored during risk-based testing. Even if a
mitigation is correctly implemented, there is still a need to ask whether it really does

24 | P a g e
mitigate the risk it is intended for. Each mitigation generates a positive requirement—
the correct implementation of the mitigation strategy—but it also generates a negative
requirement stating that the mitigation must not be circumventable. To put it another
way, the mitigation might not be sufficient for avoiding the underlying risk, and this
possibility constitutes a risk in and of itself.

Security Redefined - Prevention is the future!!

More Related Content

What's hot (19)

Similar to Security Redefined - Prevention is the future!! (20)

Recently uploaded (20)

Security Redefined - Prevention is the future!!