1	
	
Some	Unconventional	Security	Metrics		
	
Roger	G.	Johnston,	Ph.D.,	CPP	
Right	Brain	Sekurity	
http://rbsekurity.com	
	
				The	traditional	measure	of	security	effectiveness—has	anything	really	bad	happened	
recently?—is	not	a	very	effective	metric.		It	fails	in	at	least	3	ways:		It	ignores	changes	in	
assets,	resources,	technologies,	threats,	and	vulnerabilities.		It	does	not	encourage	a	
proactive	approach	to	security.		It	fails	to	prepare	us	for	rare	catastrophic	incidents.		This	
article	discusses	some	unconventional	metrics	for	security	that	might	be	worth	
considering,	especially	for	complex	security	programs.	
	
				Any	good	security	metric	should	have	certain	key	attributes.		It	should	measure	actual	
security,	not	just	security	management.		The	important	things	should	get	measured,	not	
just	the	things	easy	to	measure.		Quality,	not	just	quantity	needs	to	be	emphasized.		A	good	
metric	should	also	recognize	that	compliance	and	security	are	not	the	same	thing.			And	all	
metrics	should	avoid	driving	undesirable	employee	behaviors	and	attitudes.		
	
				Some	metrics	worth	considering:	
	
• Degree	of	transparency.		Somewhat	counter-intuitively,	security	is	usually	better	
when	it	is	transparent	because	this	allows	for	review,	criticism,	questions,	
improvements,	and	buy-in.		“Security	by	Obscurity”	is	not	a	viable	security	strategy.		
People	and	organizations	cannot	keep	long-term	secrets,	and	you	usually	have	to	
assume	the	adversaries	(insiders	or	outsiders)	understand	your	security.	
	
• Amount	of	thoughtful	pushback	on	auditors	and	high-level	security	rules	to	allow	
for	local	conditions.		The	key	test	for	local	security	practice	ought	to	be	whether	it	is	
good	security,	not	whether	it	follows	the	one-size-fits-all	rules	mandated	by	high-
level	bureaucrats	with	no	understanding	of	the	local	environment.		Pushback	
suggests	there	has	been	some	local	critical	thinking	about	security.	
	
• Frequency	of	sanity	checks	on	security	rules	by	talking	with	employees	affected	
by	them.
2	
	
• Disgruntlement	mitigation.		Percentage	of	the	time	when	managers	and	HR	are	
aware	of	allegations	of	an	unfair	or	hostile	work	environment	(bully	bosses,	
coercion,	sexual	or	racial	harassment,	etc.)	that	they	take	positive	actions,	and	do	
not	retaliate	against	the	alleged	victims.		While	disgruntlement	is	only	one	of	many	
motivators	for	inside	attacks,	it	is	one	of	the	easiest	to	counter.		A	related	metric	is	
the	number	of	times	that	the	organization’s	grievance	and	employee	assistance	
programs	are	used.		They	will	only	be	used	frequently	if	employees	view	them	as	
safe,	effective,	and	legitimate.		Perception	is	everything.	
	
• Employee	turnover	rates	for	both	security	and	non-security	personnel.		This	is	
closely	related	to	the	insider	threat.	
	
• Frequency	of	formal	and	informal	communication	between	security	personnel	
(including	low-level	personnel)	and	non-security	employees	and	contractors.		
Security	by	“walking	around”	is	an	effective	strategy.	
	
• Resiliency	preparation.		Prevention	is	difficult.		A	good	security	program	needs	to	
be	ready	in	advance	to	lead	recovery	after	a	serious	security	incident,	including	
tampering,	hacking,	and	counterfeiting.	
	
• Amount	of	“What	Ifs?”		How	often	do	employees	and	security	personnel	mentally	
or	physically	rehearse	possible	security	incidents,	and	how	often	are	novel	incidents	
considered?		Even	wildly	implausible	scenarios	get	people	thinking	creatively	about	
security!	
	
• Frequency	of	formal	and	informal	vulnerability	assessments,	number	of	
ongoing	vulnerabilities	and	potential	countermeasures	identified,	and	number	of	
suggestions	for	security	improvements,	including	from	low-level	personnel	and	
non-security	employees.		(It	is	important	to	realize	that	vulnerability	assessments	
are	not	the	same	thing	as	threat	assessments,	security	surveys,	performance	testing,	
“Red	Teaming”,	pen(etration)	testing,	or	compliance	auditing—though	these	things	
are	worth	doing	and	can	shed	some	light	on	vulnerabilities.)	
	
• Number	of	security	changes	recently	introduced.		This	leads	us	to	the	idea	of	
“Marginal	Analysis”.		(In	mathematics	and	economics,	“marginal”	means	rate	of	
change.)			
	
	
				Now	securing	even	a	medium-sized	enterprise	or	facility	is	a	very	complex	minimization	
problem.		Risk	needs	to	be	minimized	while	considering	hundreds	of	different	security	
parameters	(variables)	involving	security	personnel,	technologies,	spatial	and	temporal	
deployment	of	resources,	possible	security	strategies,	assets	to	be	protected,	threats,	
vulnerabilities,	training,	etc.		This	is	very	much	like	a	classic	minimization	problem	in	N-
dimensional	space,	where	N	is	quite	large.
3	
				Figure	1	shows	a	3-dimensional	schematic	of	risk	plotted	as	a	function	of	only	2	security	
parameters	(so	N=3).		The	risk	surface	has	peaks	and	valleys.		In	theory,	the	goal	is	to	find	
the	values	for	the	two	security	parameters	that	corresponds	to	the	lowest	valley	in	the	risk	
surface;		this	is	the	point	of	minimum	risk.	
	
				The	idea	with	Marginal	Analysis	is	to	introduce	changes—real	or	theoretical—in	your	
security	program,	and	then	determine	if	the	risk	is	lowered	as	a	result.		If	it	is,	try	
additional	similar	changes	to	see	if	you	can	get	the	risk	even	lower.		If	the	risk	instead	
increases,	try	changes	in	approximately	the	opposite	direction.		The	goal	is	to	travel	down	
the	red	line	shown	in	figure	1	by	adjusting	parameters	in	an	attempt	to	find	the	minimum	
in	the	risk	surface,	and	more	importantly,	the	values	of	the	various	parameters	that	gives	
this	minimum	risk.	
	
	
	
Figure	1		-		Risk	as	a	function	of	two	security	parameters.		Traveling	“down	hill”	to	a	valley		
(minimum)	in	the	risk	surface	allows	us	to	find	the	optimum	settings	for	the	2	parameters.	
	
	
				We	can	conclude	that	we	have	“pretty	good	security”	if	no	changes	significantly	lower	the	
risk.		This	may	be	more	practical	than	an	absolute	determination	of	security	effectiveness.	
	
				Somewhat	counter-intuitively,	the	changes	in	your	security	should	involve	variations	in	
more	than	just	1	parameter	at	a	time.		The	changes	will	usually	be	minor.		Every	once	in	a	
while,	however,	it	is	important	to	consider	large	changes.		This	is	because	you	may	
currently	be	in	a	local	valley	in	the	risk	surface.		There	might	be	a	lower	valley	over	the	next	
hill	or	mountain,	and	a	large	change	could	allow	you	to	locate	this	lower	risk.		(It	is,	
however,	important	not	to	let	the	best	be	the	enemy	of	the	good.		Often	a	good	security	
solution	is	acceptable	rather	than	demanding	the	absolute	best,	i.e.,	the	absolute	lowest	
valley.)	
	
				Now	the	risk	surface	in	figure	1	is	constantly	morphing	and	fluctuating	over	time	with	
changes	in	threats,	assets,	personnel,	technologies,	etc.		So	this	process	of	introducing	
changes	to	see	if	the	risk	gets	lowered	is	ongoing	and	not	a	one-time	activity.		It	is	also
4	
important	to	recognize	that	the	mathematical	N-dimensional	minimization	problem	is	only	
an	analogy.		The	process	is	not	really	mathematical.		We	do	not	yet	understand	enough	
about	security	to	do	this	complete	process	mathematically	in	any	realistic	way.		Perhaps	
someday	we	will.	
	
				So	how	do	you	know	if	a	given	set	of	changes	improves	or	degrades	your	security,	i.e.,	
lowers	or	raises	the	risk	(and	takes	you	“downhill”	in	N-dimensional	space)?		There	are	
several	possible	answers:	
	
				Possible	Answer	1:		It	doesn’t	matter.		The	main	goal	of	Marginal	Analysis	is	to	
encourage	change,	flexibility,	and	critical	thinking	about	your	security.		Whether	the	change	
is	actually	implemented	or	merely	contemplated,	a	subjective	estimate	of	whether	there	is	
security	improvement	or	not	may	be	adequate.		Often,	it	is	easier	to	judge	incremental	
improvement	in	security	than	absolute	effectiveness.	
	
					Possible	Answer	2:		For	a	more	nuanced	approach,	the	changes	can	be	implemented	for	
real,	then	the	security	system	studied	for	evidence	of	improvement	or	degradation.	
	
					Possible	Answer	3:		Perhaps	the	best	approach	is	to	let	vulnerability	assessors,	threat	
assessors,	and	risk	analysts	help	you	determine	whether	the	change	(implemented	or	
contemplated)	actually	improves	your	security.	
	
				The	ultimate	question	worth	considering	with	Marginal	Analysis	is	the	following:		Can	
continually	focusing	on	changes	help	us	be	flexible,	adaptable,	and	proactive	about	security,	
rather	than	being	stuck	with	inertia,	reactive	approaches,	wishful	thinking,	and	
groupthink?		Give	it	a	try!	
	
	
	
About	the	Author	
					Roger	G.	Johnston,	Ph.D,	CPP	is	CEO	and	Chief	Vulnerability	Wrangler	at	Right	Brain	
Sekurity,	a	company	devoted	to	security	consulting	and	vulnerability	assessments.		He	
previously	was	head	of	the	Vulnerability	Assessment	Teams	at	Los	Alamos	and	Argonne	
National	Laboratories	(1992-2007	and	2007-2015).

More Related Content

PDF
SMB270: Security Essentials for ITSM
PDF
Risk Based Security Management
PPT
Security Maturity Assessment
PDF
when minutes counts
PPTX
Enterprise security management II
PDF
Priming your digital immune system: Cybersecurity in the cognitive era
PPTX
Information Security Metrics - Practical Security Metrics
PDF
Security Maturity Model
SMB270: Security Essentials for ITSM
Risk Based Security Management
Security Maturity Assessment
when minutes counts
Enterprise security management II
Priming your digital immune system: Cybersecurity in the cognitive era
Information Security Metrics - Practical Security Metrics
Security Maturity Model

What's hot (20)

PPTX
Uncovering the key to a successful psim installation
DOC
Adversarial Safety Analysis
PDF
Risk Assessments
PDF
7 Lessons Learned From BSIMM
PDF
Information Security Maturity Model
PPTX
Enterprise incident response 2017
PPTX
Selling security to the C-level
PDF
Vendor Cybersecurity Governance: Scaling the risk
PPTX
Emerging Need of a Chief Information Security Officer (CISO)
PDF
Sensible defence
PDF
u10a1-Risk Assessment Report-Beji Jacob
PPTX
Connecting the Dots Between Your Threat Tntelligence Tradecraft and Business ...
PDF
Enabling Embedded Business Continuity
PDF
Connection can help keep your business secure!
PDF
Practical approach to security risk management
PDF
Security-Brochure
PDF
Security-Brochure
PPTX
Risk Management and Security in Strategic Planning
PDF
Chief Information Security Officer - A Critical Leadership Role
PDF
A Self-Report Measure of End-User Security Attitudes (SA-6)
Uncovering the key to a successful psim installation
Adversarial Safety Analysis
Risk Assessments
7 Lessons Learned From BSIMM
Information Security Maturity Model
Enterprise incident response 2017
Selling security to the C-level
Vendor Cybersecurity Governance: Scaling the risk
Emerging Need of a Chief Information Security Officer (CISO)
Sensible defence
u10a1-Risk Assessment Report-Beji Jacob
Connecting the Dots Between Your Threat Tntelligence Tradecraft and Business ...
Enabling Embedded Business Continuity
Connection can help keep your business secure!
Practical approach to security risk management
Security-Brochure
Security-Brochure
Risk Management and Security in Strategic Planning
Chief Information Security Officer - A Critical Leadership Role
A Self-Report Measure of End-User Security Attitudes (SA-6)
Ad

Similar to Unconventional Security Metrics & Marginal Analysis (20)

PDF
Security Assessments and Vulnerability Scanning_ A Critical Component of Cybe...
PPTX
Managing Enterprise Risk: Why U No Haz Metrics?
PDF
Making the Business Case for Security Investment
PDF
Rogers eBook Security
PDF
Vskills Certified Network Security Professional Sample Material
PPSX
Cyber Security Awareness Month 2017-Nugget 3
PPTX
Information System Audit and Control
DOCX
The Significance of IT Security Management & Risk Assessment
PDF
It Security Audit Process
PDF
Proactive information security michael
PPTX
Nabil Malik - Security performance metrics
PDF
Security_by_Design.pdf
PPTX
Security_by_Design.pptx
PDF
Cybersecurity - Whose responsibility is it?
PDF
Taubenberger
PDF
ISACA Cybersecurity Audit course brochure
PDF
Superior Glove Definitive Guide To Hand Protection
PPT
Why Security Awareness Assessments Are Critical to Your Cybersecurity Strategy
DOC
The security risk management guide
Security Assessments and Vulnerability Scanning_ A Critical Component of Cybe...
Managing Enterprise Risk: Why U No Haz Metrics?
Making the Business Case for Security Investment
Rogers eBook Security
Vskills Certified Network Security Professional Sample Material
Cyber Security Awareness Month 2017-Nugget 3
Information System Audit and Control
The Significance of IT Security Management & Risk Assessment
It Security Audit Process
Proactive information security michael
Nabil Malik - Security performance metrics
Security_by_Design.pdf
Security_by_Design.pptx
Cybersecurity - Whose responsibility is it?
Taubenberger
ISACA Cybersecurity Audit course brochure
Superior Glove Definitive Guide To Hand Protection
Why Security Awareness Assessments Are Critical to Your Cybersecurity Strategy
The security risk management guide
Ad

More from Roger Johnston (20)

PDF
In Risu Veritas: Humor & Security
PDF
Journal of Physical Security 15(1)
PDF
Security Audits.pdf
PDF
Camera Obscura and Security/Privacy
PDF
Vulnerability Assessment: The Missing Manual for the Missing Link
PDF
Journal of Physical Security 14(1)
PDF
Want seals with that?
PDF
Journal of Physical Security 13(1)
DOCX
Election Security 2020
DOCX
Security Assurance
PDF
A New Approach to Vulnerability Assessment
PDF
Understanding Vulnerability Assessments
PDF
Devil's Dictionary of Security Terms
PDF
Vulnerability Assessments
PDF
Design Reviews Versus Vulnerability Assessments for Physical Security
PDF
Journal of Physical Security 12(3)
PDF
Journal of Physical Security 12(2)
PDF
Unconventional Security Devices
PDF
Journal of Physical Security 11(1)
PDF
Journal of Physical Security 10(1)
In Risu Veritas: Humor & Security
Journal of Physical Security 15(1)
Security Audits.pdf
Camera Obscura and Security/Privacy
Vulnerability Assessment: The Missing Manual for the Missing Link
Journal of Physical Security 14(1)
Want seals with that?
Journal of Physical Security 13(1)
Election Security 2020
Security Assurance
A New Approach to Vulnerability Assessment
Understanding Vulnerability Assessments
Devil's Dictionary of Security Terms
Vulnerability Assessments
Design Reviews Versus Vulnerability Assessments for Physical Security
Journal of Physical Security 12(3)
Journal of Physical Security 12(2)
Unconventional Security Devices
Journal of Physical Security 11(1)
Journal of Physical Security 10(1)

Recently uploaded (20)

PDF
Satish NS: Fostering Innovation and Sustainability: Haier India’s Customer-Ce...
PPTX
IITM - FINAL Option - 01 - 12.08.25.pptx
PPTX
BUSINESS CYCLE_INFLATION AND UNEMPLOYMENT.pptx
PDF
THE COMPLETE GUIDE TO BUILDING PASSIVE INCOME ONLINE
PPTX
Board-Reporting-Package-by-Umbrex-5-23-23.pptx
PDF
#1 Safe and Secure Verified Cash App Accounts for Purchase.pdf
PPTX
interschool scomp.pptxzdkjhdjvdjvdjdhjhieij
PDF
Booking.com The Global AI Sentiment Report 2025
PPTX
chapter 2 entrepreneurship full lecture ppt
PDF
NEW - FEES STRUCTURES (01-july-2024).pdf
PPT
Lecture notes on Business Research Methods
PDF
Tortilla Mexican Grill 发射点犯得上发射点发生发射点犯得上发生
PPTX
TRAINNING, DEVELOPMENT AND APPRAISAL.pptx
PDF
ICv2 White Paper - Gen Con Trade Day 2025
PDF
TyAnn Osborn: A Visionary Leader Shaping Corporate Workforce Dynamics
DOCX
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
PDF
Environmental Law Communication: Strategies for Advocacy (www.kiu.ac.ug)
DOCX
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
PDF
Susan Semmelmann: Enriching the Lives of others through her Talents and Bless...
PPTX
2 - Self & Personality 587689213yiuedhwejbmansbeakjrk
Satish NS: Fostering Innovation and Sustainability: Haier India’s Customer-Ce...
IITM - FINAL Option - 01 - 12.08.25.pptx
BUSINESS CYCLE_INFLATION AND UNEMPLOYMENT.pptx
THE COMPLETE GUIDE TO BUILDING PASSIVE INCOME ONLINE
Board-Reporting-Package-by-Umbrex-5-23-23.pptx
#1 Safe and Secure Verified Cash App Accounts for Purchase.pdf
interschool scomp.pptxzdkjhdjvdjvdjdhjhieij
Booking.com The Global AI Sentiment Report 2025
chapter 2 entrepreneurship full lecture ppt
NEW - FEES STRUCTURES (01-july-2024).pdf
Lecture notes on Business Research Methods
Tortilla Mexican Grill 发射点犯得上发射点发生发射点犯得上发生
TRAINNING, DEVELOPMENT AND APPRAISAL.pptx
ICv2 White Paper - Gen Con Trade Day 2025
TyAnn Osborn: A Visionary Leader Shaping Corporate Workforce Dynamics
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
Environmental Law Communication: Strategies for Advocacy (www.kiu.ac.ug)
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
Susan Semmelmann: Enriching the Lives of others through her Talents and Bless...
2 - Self & Personality 587689213yiuedhwejbmansbeakjrk

Unconventional Security Metrics & Marginal Analysis