Dissertation_Austin_Kana_FINAL

Enabling Decision Insight by Applying Monte
Carlo Simulations and Eigenvalue Spectral
Analysis to the Ship-Centric Markov Decision
Process Framework
by
Austin A. Kana
A dissertation submitted in partial fulﬁllment
of the requirements for the degree of
Doctor of Philosophy
(Naval Architecture and Marine Engineering)
in The University of Michigan
2016
Doctoral Committee:
Assistant Professor David J. Singer, Chair
Research Investigator Laura K. Alford
Assistant Professor Matthew D. Collette
Professor Lawrence M. Seiford
Professor Armin W. Troesch

“Believe me, my young friend,
there is nothing - absolutely nothing -
half so much worth doing
as simply messing about in boats.”
- Kenneth Grahame, The Wind in the Willows

c Austin A. Kana 2016
All Rights Reserved

ACKNOWLEDGEMENTS
First and foremost I would like to thank my adviser, Dr. David Singer. Without him,
it may not have been possible to even pursue a PhD at the University of Michigan,
let alone complete one. His guidance and support throughout the years have been in-
valuable in not only this research but helping me grow personally and as an academic.
Several fellow students provided invaluable help and insight throughout the process,
as well as guidance when it was needed. Specifically, I would like to thank: Dylan
Temple, Dorian Brefort, Tom Devine, Brandon Harrison, Josh Knight, Doug Rig-
terink, Harleigh Seyffert, Colin Shields, Jason Strickland, and Mike Sypniewski.
Thank you to my committee members, who so generously lent their time and exper-
tise to the project I undertook. It was with their help that this dissertation is possible.
Thank you to my parents Dr. Todd Kana and Dr. Patricia Glibert, for not only be-
lieving in me from the moment I told them I was going to pursue Naval Architecture
at the University of Michigan, but also for continuing to push me every step of the
way. I could not have done this without you.
And finally, I would like to thank my fiancé, Hannah, for her love and support through-
out every step and turn of the process. Without her, I would not be where I am today.
iii

TABLE OF CONTENTS
DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . iii
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
LIST OF APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
CHAPTER
I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Overview of Dissertation . . . . . . . . . . . . . . . . . . . . . 4
II. The Why Behind the What: Identifying Ship Design’s Next
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1 The US Navy Littoral Combat Ship / Frigate Program . . . . 7
2.2 The Importance of Decision Making: The Why not the What 11
2.2.1 Practical Reasons Engineering Decision Making is
Diﬃcult . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Psychological Reasons Engineering Decision Making
is Diﬃcult . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.3 Practical Reasons Engineers Do Not Focus on the Why 16
2.2.4 Psychological Reasons Engineers Do Not Focus on
the Why . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 What Makes Marine Design Unique? . . . . . . . . . . . . . . 19
2.4 The Ship-Centric Markov Decision Process . . . . . . . . . . 21
iv

2.4.1 Monte Carlo Simulations: Need, Background, and
Related Work . . . . . . . . . . . . . . . . . . . . . 23
2.4.2 Eigenvalue Spectral Analysis: Need, Background, and
Related Work . . . . . . . . . . . . . . . . . . . . . 26
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
III. New Methods and Metrics for the Ship-Centric Markov De-
cision Process Framework . . . . . . . . . . . . . . . . . . . . . . 29
3.1 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 The Markov Decision Process . . . . . . . . . . . . . . . . . . 32
3.3 A Monte Carlo Approach to the SC-MDP Framework . . . . 36
3.4 An Eigenvalue Spectral Analysis Approach to the SC-MDP
Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.1 Spectral Analysis Methodology Overview . . . . . . 39
3.4.2 Forming the Representative Markov Chains . . . . . 40
3.4.3 Spectral Analysis . . . . . . . . . . . . . . . . . . . 42
3.4.4 The Damping Ratio . . . . . . . . . . . . . . . . . . 42
3.4.5 The Eigenvector as a Metric for Steady State Behavior 45
3.4.6 The relationship between the damping ratio and the
eigenvector . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.7 The signiﬁcance of repeated dominant eigenvalues . 48
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
IV. Monte Carlo Case Study: Design for Evolving Emission Con-
trol Area Regulations . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1.1 Regulatory Framework . . . . . . . . . . . . . . . . 54
4.1.2 Compliance Mechanisms . . . . . . . . . . . . . . . 58
4.2 Case Study: Designing for Evolving Emission Control Area
Regulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.2.1 Fixed model parameters . . . . . . . . . . . . . . . 61
4.2.2 Markov decision process framework . . . . . . . . . 62
4.2.3 Monte Carlo Parameters . . . . . . . . . . . . . . . 65
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.3.1 Decisions . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3.2 Economic Costs . . . . . . . . . . . . . . . . . . . . 71
4.3.3 Decision Drivers . . . . . . . . . . . . . . . . . . . . 74
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
V. Spectral Analysis Case Study 1: Ship Egress Analysis and
General Arrangements Design . . . . . . . . . . . . . . . . . . . 82
v

5.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2 Case Study: Ship Egress Analysis and Ship General Arrange-
ments Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.3.1 Algorithm Validation . . . . . . . . . . . . . . . . . 87
5.3.2 Markov Chain Validation . . . . . . . . . . . . . . . 88
5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.4.1 Variations in the Decision Paths . . . . . . . . . . . 91
5.4.2 Spectral Analysis . . . . . . . . . . . . . . . . . . . 93
5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.7 Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
VI. Spectral Analysis Case Study 2: Lifecycle Planning for Bal-
last Water Treatment Compliance . . . . . . . . . . . . . . . . . 111
6.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.1.1 Regulatory Framework . . . . . . . . . . . . . . . . 113
6.1.2 Compliance Mechanisms . . . . . . . . . . . . . . . 115
6.2 Case Study: Lifecycle Planning for Ballast Water Treatment
Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.2.1 Markov decision process framework . . . . . . . . . 116
6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.3.1 Optimal States Accessed . . . . . . . . . . . . . . . 123
6.3.2 Principal Eigenvector Analysis . . . . . . . . . . . . 124
6.3.3 Relationship between Optimal States and Principal
Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 131
6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
VII. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.1 Dissertation Conclusion . . . . . . . . . . . . . . . . . . . . . 134
7.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.3.1 Spectral Analysis Future Work . . . . . . . . . . . . 137
7.3.2 Generalized SC-MDP Future Work . . . . . . . . . 138
APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
vi

LIST OF FIGURES
Figure
2.1 The two LCS variants. The top is the Freedom class steel-hulled
monohull LCS built by Marinette Marine in Wisconsin, while the
bottom is the Independence class aluminum hulled trimaran LCS
built by Austal USA in Alabama. (US Navy, 2010) . . . . . . . . . 8
2.2 External factors affecting the LCS upgrade program. . . . . . . . . 9
2.3 Estimates of change costs during different stages of design for naval
vessels (adapted from Keane and Tibbitts (1996)). . . . . . . . . . . 17
2.4 Learning styles and learning cycle based on Kolb’s model (adapted
from (Montgomery and Groat, 1997) . . . . . . . . . . . . . . . . . 18
3.1 Sample Markov chain with three states for a single epoch. pi,j denotes
the probability of transitioning from State i to State j. . . . . . . . 30
3.2 Sample three state MDP with two actions. Note that transition prob-
abilities and rewards may vary between actions. . . . . . . . . . . . 32
3.3 Visual representation of the Monte Carlo approach to the ship-centric
Markov decision process framework. . . . . . . . . . . . . . . . . . . 37
3.4 Visualization of how a change in decisions affects the representative
Markov chain transition matrix, M. . . . . . . . . . . . . . . . . . . 44
3.5 Visual representation of the eigenvectors as a leading indicator for
the impact of design decisions. wi are independent eigenvectors as-
sociated with a given set of decisions which are identical to the future
steady state of the system or design, s∞
= s. . . . . . . . . . . . . . 47
4.1 IMO regulated Emission Control Areas (Blikom, 2011) . . . . . . . 55
4.2 MARPOL Annex VI NOx Emission Limits (DieselNet, 2011) . . . . 56
4.3 MARPOL Annex VI SOx Emission Limits (DieselNet, 2011) . . . . 57
4.4 Fuel consumption curves for three different fuels and two drafts. The
curves were developed using both the Holtrop and Mennen method
as well as estimated from MAN B&W and Wartsila. . . . . . . . . . 63
4.5 Historical average of freight rates (UNCTAD, 2014). The Monte
Carlo simulations assumed a normal distribution of freight rates from
Asia to Europe with a mean of US$1,500 and standard deviation of
US$285. From Europe to Asia the mean was set to US$800 with a
standard deviation of US$125. . . . . . . . . . . . . . . . . . . . . . 66
vii

4.6 Average vessel load factors from 2010 to 2015 (adapted from Alpha-
liner (2015). Vessel load factors have averaged 88% with a standard
deviation of 7.5%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.7 Convergence of the Monte Carlo simulations. The close up shows
that after 1,000 simulations, the model has consistently converged to
a value less than 10e-4, which is deemed an acceptable range. . . . . 69
4.8 The expected net present cost (US$) for the range of speeds. 4.8.a
shows the expected range of costs for the slowest speed tested of 12
knots. 4.8.b shows the range of potential costs for a high speed of 22
knots, where there is a small possibility that is it preferable to keep
the single fuel engine. 4.8.c displays the costs for the highest speed
of 24 knots where is always preferable to switch to a dual fuel engine
as soon as possible. Note the variations in the y-axis between figures
to show specifics within each speed. . . . . . . . . . . . . . . . . . 72
4.9 The accumulated lifecycle cost varies greatly both between speeds
and within individual speeds. The spread of possible costs grows
both in magnitude and in percent as speeds increase. Outliers tend
to fall on the upper end of costs for the faster speeds. . . . . . . . . 74
4.10 The sensitivity study on the ECA regulation showed almost no vari-
ation in the accumulated lifecycle costs for individual speeds. The
median cost for each speed, however, remained unchanged as com-
pared to the original analysis. . . . . . . . . . . . . . . . . . . . . . 76
4.11 The sensitivity study on the LNG supply chain risk shows a slightly
smaller spread than the original analysis. The median cost for each
speed also remained unchanged as compared to the original analysis. 78
4.12 The sensitivity study on the fuel prices shows a slightly smaller spread
than the original analysis. The median cost for each speed remained
unchanged as compared to the original analysis. . . . . . . . . . . . 79
5.1 Utility estimate validation. Note: Russell and Norvig (2003) labeled
their states with the convention of (column, row). For instance, state
(1,4) in the research code matches with state (4,1) in the Russell and
Norvig (2003) result. . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.2 Building the representative Markov chain transition matrix, M, for
p = 0.8 and r = −0.04. The colors are representative of the specific
actions. Note the action “down” is never optimal for any of the states. 89
5.3 Damping ratio, ρ, for a range of rewards and uncertainties. Note the
precipitous drop in ρ when the rewards become positive, as well as
the rapidly increasing nature of ρ for 0.5 < p. . . . . . . . . . . . . . 95
5.4 Damping ratio, ρ, over a range of rewards for p = 0.8. The points
indicate locations where there are transitions in the decision paths.
Note the step function behavior and the precipitous drop at r =
−0.028 (between h and i). . . . . . . . . . . . . . . . . . . . . . . . 96
5.5 Percent of individuals that make it to the safe state (state (3,4)) in
the long run based on a variation of uncertainties and rewards. . . . 101
viii

5.6 Visual display of the magnitude of the angles between the eigen-
vectors for given rewards and p = 0.8. The rewards indicate the
transition regions where the steady state distribution changes. . . . 104
5.7 Visual display of the magnitude of the angles between the eigenvec-
tors for given probability of moving in the desired direction, p, and
r = −0.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.1 Ballast water treatment technology compliance schedule. (Lloyd’s
Register, 2012) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.2 Ballast water treatment technology process (Lloyd’s Register, 2012) 115
6.3 Ballast water system commercial availability and regulatory compli-
ance for the 1-4-4-9 schedule. Shading represents the percent likeli-
hood a given system will be located in that state. . . . . . . . . . . 121
6.4 Optimal states accessed for 1-4-4-9 regulatory schedule and two treat-
ment strengths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6.5 The number of initial condition dependent absorbing paths for the
1-4-4-9 schedule and Tier 1 strength. . . . . . . . . . . . . . . . . . 126
6.6 The number of initial condition dependent absorbing paths for the
1-4-4-9 schedule and Tier 3 strength. . . . . . . . . . . . . . . . . . 126
6.7 The set of principle eigenvectors representing the various initial con-
dition dependent absorbing states: Tier 1 regulatory strength. Notice
that the number of possible unique paths increases through time. . . 128
6.8 The set of principle eigenvectors representing the various initial con-
dition dependent absorbing states: Tier 3 regulatory strength. Notice
the number of unique paths for year 20 is less than it is for the Tier
1 policy, due to many ballast systems not being regulatory compliant. 129
ment strengths: Tier 1 strength. . . . . . . . . . . . . . . . . . . . . 130
ment strengths: Tier 3 strength. . . . . . . . . . . . . . . . . . . . . 131
6.11 Decision path as determined by two methods for the 1-4-4-9 regula-
tory schedule and Tier 1 compliance. . . . . . . . . . . . . . . . . . 132
A.1 Results from validation between the SC-MDP framework and a more
traditional lifecycle analysis performed by a commercial client. . . . 143
ix

LIST OF TABLES
Table
2.1 Perry’s model for the development of college students. . . . . . . . . 14
3.1 Markov chain transition matrix for Figure 3.1. The entries indicate
the probability of transitioning from the previous state, si, to the
new state, sj. Note, each row must sum to unity. . . . . . . . . . . . 30
3.2 Sample transition matrices for the system in Figure 3.2. Each action
is represented by its own transition matrix. The specific transition
probabilities in this case have been chosen simply for demonstration
purposes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 The reward matrix associated with Figure 3.2. The entries indicate
the reward received after taking a given action, ai, and landing a
given state, sj. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Example non-stationary decision matrix developed from solving the
Bellman equation for the system in Figure 3.2. For example, Action
2 is preferred when the system is in State 2 and Epoch 2. Note the
decisions may change through time, as is the case between Epoch 4
and Epoch 5 for State 1. . . . . . . . . . . . . . . . . . . . . . . . . 35
3.5 Sample transition matrices for the system given in Figure 3.2. The
colors denote the specific transition probabilities that are transferred
to the representative Markov chain transition matrix according to the
decision matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.6 The decision matrix for the first decision epoch from Table 3.4. This
is generated by solving the Bellman Equation (Equation 3.4). . . . . 41
3.7 Sample representative Markov chain transition matrix, M, for Epoch
1 (Table 3.6). Rows are determined from the action transition ma-
trices (Table 3.5) and the decision matrix (Table 3.6). . . . . . . . . 41
4.1 MARPOL Annex VI NOx Emission Limits . . . . . . . . . . . . . . 55
4.2 MARPOL Annex VI SOx Emission Limits . . . . . . . . . . . . . . 56
4.3 The vessel principal characteristics for the notional 13,000 TEU con-
tainership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.4 Percent of time a given action is optimal. The “convert engines even-
tually” action means that it is best to convert to an LNG engine at
a time after the first two voyages. . . . . . . . . . . . . . . . . . . . 70
x

4.5 Average savings and payback periods for all speeds. Savings are in-
creased and payback periods are reduced as speeds are increased. . . 74
4.6 Parameters used for the sensitivity studies. The variables for the reg-
ulation and LNG supply chain sensitivity were uniformly distributed,
while the fuel price sensitivity used a normal distribution. . . . . . . 75
4.7 Sensitivity due to uncertainty in the ECA regulation implementation.
For 12 knots it is always best to keep the original engine, while above
12 knots it is always best to convert to a dual fuel LNG engine as
soon as possible. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.8 Sensitivity due to LNG supply chain risk. The probability of it being
best to never convert engines is reduced between 3-13%, compared
to the original analysis. . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.9 Fuel Price sensitivity. Fuel price variability may be one of the causing
leading to delaying engine retrofits beyond the first two voyages. . . 79
5.1 Visual representation of the eleven room general arrangement. The
entries include the labeling convention of the states for the following
discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2 The initial rewards the individuals receive for landing in a given state.
The following analyses vary the -0.04 rewards, while the +1 and -1
rewards remain fixed. . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.3 The expected utility and decision paths for p = 0.8 and r = −0.04. . 87
5.4 Decision path validation. The decision paths for the research code
match identically with that of the published ones from Russell and
Norvig (2003). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.5 Initial placement of individuals. 25% of the individuals were placed
in the each of rooms identified. . . . . . . . . . . . . . . . . . . . . . 89
5.6 Expected utility and decision paths for p = 0.8 and r = −0.04. . . . 91
5.7 Expected utility and decision paths for p = 0.7 and r = −0.04. Note
the reduction in expected utility due to increased uncertainty and
changes in decision paths highlighted by the double arrows. . . . . . 92
5.8 Expected utility and decision paths for p = 0.8 and r = −0.1. Note
the reduction in expected utility due to more painful rewards and
changes in decision paths highlighted by the double arrows. . . . . . 92
5.9 Expected utility and decision paths for p = 0.8 and r = 0.1. Note the
large expected utility due to positive rewards and non uniqueness of
the decision paths highlighted by the multiple arrows signifying any
decision is optimal. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.10 Damping ratio, ρ, for the four previous scenarios. . . . . . . . . . . 94
5.11 Variations in the decision paths for p = 0.8. Moving from lowest
rewards to highest rewards, the specific state that changed optimal
actions is identified by a double arrow. The states that changed
optimal actions with an associated change in the damping ratio are
identified by a double arrow and yellow background. . . . . . . . . . 97
xi

5.12 Steady state distributions as calculated by the dominant eigenvector
for p = 0.8. The values indicate the long term probabilistic location
of the individuals in the vessel. . . . . . . . . . . . . . . . . . . . . . 102
5.13 The magnitude of the angles between the eigenvectors for given re-
wards and p = 0.8. The rewards indicate the transition regions where
the steady state distribution changes. . . . . . . . . . . . . . . . . . 104
5.14 Steady state distributions as calculated by the dominant eigenvector
for r = −0.1. The values indicate the long term probabilistic location
of the individuals in the general arrangement. . . . . . . . . . . . . 105
5.15 The magnitude of the angles between the eigenvectors for given prob-
ability of moving in the desired direction, p, and r = −0.1. . . . . . 107
6.1 Discharge limits for ballast water as prescribed by IMO ballast water
convention (Lloyd’s Register, 2015). . . . . . . . . . . . . . . . . . . 114
6.2 The ballast water technology availability schedule and compliance
level. The mean availability details the number of years after the
convention that particular technology is expected to be both com-
mercially and regulatory compliant. . . . . . . . . . . . . . . . . . . 119
6.3 Ballast water technology costs. The Capex #/# corresponds to costs
before/after Basic approval. The Install #/# corresponds to costs
for newbuild/retroﬁt. . . . . . . . . . . . . . . . . . . . . . . . . . . 122
A.1 Model parameters used to August validation study. . . . . . . . . . 143
xii

LIST OF APPENDICES
Appendix
A. Validation of University of Michigan’s SC-MDP code with a commercial
client’s internal analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 140
B. Preliminary Work on Chapter IV presented at the 2015 International
Marine Design Conference . . . . . . . . . . . . . . . . . . . . . . . . 144
xiii

ABSTRACT
Enabling Decision Insight by Applying Monte Carlo Simulations and Eigenvalue
Spectral Analysis to the Ship-Centric Markov Decision Process Framework
by
Austin A. Kana
Chair: David J. Singer
One of the major problems facing ship design today is that engineers often focus most
of their efforts on the What of the design as opposed to understanding the Why. The
What is defined as the solution itself, while the Why is defined as the decisions that
drive how the set of solutions change through time. Decision making through time,
especially in the face of uncertainty, has consistently been difficult for engineers. This
is due to both uncertainty and the interconnected nature of complex decision making
problems. There are no standard definitions or metrics that quantify the impact of
engineering design decisions. This dissertation aims to address that need. This re-
search extends the ship-centric Markov decision process (SC-MDP) framework which
involves applying Markov decision processes to ship design and decision making. The
SC-MDP framework is useful for analyzing decision making in the maritime domain
due to its inherent temporal structure and ability to handle uncertainty. However,
the framework is limited in its ability to clearly show how uncertainty affects deci-
sions, and its inability to quantify the changes and long term implications of decisions.
xiv

Two methods unique to this research are developed and explored. First, applying
Monte Carlo simulations to the SC-MDP framework is proposed to give insight into
the impacts of uncertainty on the decisions and set of results. Second, a method
to perform eigenvalue spectral analysis within the framework was developed to un-
derstand the behavior of the decisions themselves. Three metrics are developed in
regards to eigenvalue analysis. To quantify changes in decisions, the damping ratio is
proposed, deﬁned as the ratio of the largest eigenvalue to the magnitude of the second
largest. To understand the long term implications of a set of decisions the principal
eigenvector is presented. For eliciting relationships and inter-dependencies of deci-
sions, analyzing repeated dominant eigenvalues and the set of principal eigenvectors
are used. Three maritime case studies are presented that demonstrate the utility of
these methods and metrics involving designing for evolving Emission Control Area
regulations, ship egress analysis and general arrangements design, and lifecycle plan-
ning for ballast water treatment regulations.
xv

CHAPTER I
Introduction
One of the major problems facing ship design today is that all too often engineers
focus most of their efforts on the What of the design as opposed to understanding
the Why. The What is defined as the solution itself; whether it be a specific product,
the vessel itself, or a given technology. The Why, on the other hand, is defined as
the decisions that drive how the set of solutions may change through time. Decision
making through time, especially in the face of uncertainty, has consistently been dif-
ficult for engineers. The U.S. Office of Naval Research has identified the problem of
understanding the impact of design decisions due to the lack of standard definitions
and metrics that measure these implications (ONR, 2011). One example of where a
failure in understanding the impacts of the decisions has led to serious ramifications
is with the U.S. Navy LCS (now frigate) upgrade program. This program has been
plagued with technical issues, cost overruns, and schedule delays (O’Rourke, 2015).
Specifics of this program are discussed in more detail in Chapter II: The Why Behind
the What: Identifying Ship Design’s Next Challenge and how it relates to the overar-
ching goals of this dissertation.
This goal of this dissertation is to develop new techniques that enable decision making
insight by changing the focus away from the What of design towards one that focuses
1

on understanding the Why. A focus of the decisions themselves, their relationships
and implications, as well as the impact of uncertainty on decisions are the focus of
this research. Specifically, the objectives of this research are:
1. Develop a method that elicits decision making insight in the face of multiple
layers of uncertainty that may change and evolve through time.
2. Develop new leading indicator methods and metrics to quantify the impact of
design decisions.
3. Develop a means to understand the inter-dependencies and relationships be-
tween various decision paths.
1.1 Motivation
The motivation of this dissertation came from a project with a commercial client.
The client requested a comparison of economic results from University of Michigan’s
ship-centric Markov decision process (SC-MDP) tool with their own internal lifecycle
cost analysis tools. The SC-MDP tool was developed by Niese (2012) to help generate
and analyze time domain design decisions and costs. It is stochastic in nature, while
the client’s internal lifecycle cost analysis was a static model. The results appeared
different at first, but it soon became clear that the differences were a function of
the model setup and the assumptions. Many of these assumptions came from how
externalities within the case study itself were modeled. The client requested a valida-
tion study to clarify these discrepancies. A description of that study and the results
are presented in Appendix A. The Emission Control Area case study presented in
Chapter IV: Monte Carlo Case Study: Design for Evolving Emission Control Area
Regulations provides an extension of this study to include some of the techniques
developed in this dissertation.
2

After the validation study, new questions arose. There was more interest in under-
standing sensitivities of the inputs and the structure of the model than there was in
the actual answer itself. There was a recognition that this understanding was neces-
sary for the results to be meaningful. Initially, traditional interrogation methods were
used to gain this information. Due to the structure of the case study, this process soon
proved to be intractable. There was a need for new methods and metrics that could
elicit this information in an intelligent way. This method would help gain insight into
the implications of the decisions on the set of results through time. It would also
provide insight into the underlying relationships of the decisions to highlight interde-
pendencies within the model itself. The goal was to develop leading indicator metrics
for design decision making. The methods and metrics presented in this dissertation
have been developed with that goal in mind.
1.2 Contributions
Several contributions have been identified stemming from this research. Those contri-
butions are briefly mentioned here, while a more in depth discussion of them follows
in the conclusion. They are highlighted in the order they are presented in this dis-
sertation.
1. Applying Monte Carlo simulations to the SC-MDP framework to understand
the effect of uncertainty on a temporal non-stationary decision process. This
dissertation presents a new method for handling multiple layers of uncertainty
in a system by applying Monte Carlo simulations to the SC-MDP framework.
This method shows the impact of multiple decision scenarios on a suite of results.
2. Applying eigenvalue spectral analysis to a stationary SC-MDP case study to
3

examine the future impact of decisions. A new method was developed that
enables the ability to perform eigenvalue spectral analysis on Markov decision
processes. This method was designed to examine and quantify the impact of
decision making. New metrics involving both the eigenvalues and eigenvectors
are discussed.
3. Applying eigenvalue spectral analysis to a non-stationary temporal SC-MDP
case study to examine relationships within the decision making process. This
research introduced the concept of applying eigenvalue spectral analysis to a
non-stationary temporal Markov decision process. Metrics that handle repeated
eigenvalues are used to study initial condition dependence of design absorbing
paths.
1.3 Overview of Dissertation
This dissertation consists of seven chapters that outline the identiﬁed problem, the
methods proposed to address that problem, and three case studies examining various
applications of the methodology applied to ship design and decision making.
Chapter 2 lays out the problem formulation that this research aims to address. This
research is about understanding the need for a design methodology that moves beyond
the What of design to understanding the Why. A overview of existing methodologies
is discussed and a new framework is proposed. The new framework includes applying
Monte Carlo simulations and eigenvalue spectral analysis to the ship-centric Markov
decision process.
Chapter 3 presents the new methods and metrics proposed in this dissertation to ad-
dress the research problem outlined in Chapter 2. Background on Markov chains and
4

Markov decision processes is discussed first, followed by a discussion of Monte Carlo
simulations. The methods and metrics involved with applying eigenvalue spectral
analysis are discussed at the end of the chapter.
Chapter 4 presents the Monte Carlo case study, involving design considerations in the
face of evolving Emission Control Area Regulations. The focus of this chapter is on
the implications of uncertainty on decision making, and how stochastic analysis can
be used to help synthesize this uncertainty.
Chapter 5 presents the first case study involving applying eigenvalue spectral analysis
to the SC-MDP framework. The focus of this chapter is on quantifying the impact of
specific decisions and examining their long term implications. A stationary decision
process is presented, and metrics involving both the eigenvalues and eigenvectors are
explored. This case study involves studying egress patterns and general arrangements
design.
Chapter 6 presents a second case study on eigenvalue spectral analysis using the SC-
MDP framework. This chapter differs from Chapter 5 by presenting a non-stationary,
temporal decision process. This chapter focuses on identifying relationships and inter-
dependencies in the decision process and how those relationships may change through
time. Metrics involving repeated eigenvalues and eigenvectors are presented. The case
study involves lifecycle considerations for ballast water treatment compliance.
Chapter 7 consists of the conclusion of the dissertation, specific contributions of this
research, and areas of potential future work.
5

CHAPTER II
The Why Behind the What: Identifying Ship
Design’s Next Challenge
“Without a clear capabilities-based assessment, it is not clear what oper-
ational requirements the upgraded LCS is designed to meet. The Navy
must demonstrate what problem the upgraded LCS is trying to solve [em-
phasis added]. We must not make this mistake again.” U.S. Senator John
McCain (Freedberg Jr., 2015)
“No one in the Navy seems to have ever figured out quite what to do with
all that expensive speed in a real-world tactical situation. It’s a solution
searching for a problem [emphasis added].” (Freedberg Jr., 2015)
“In short, you figure out what problem you’re trying to solve, then how
to solve it, then how best to implement that solution. The upgraded LCS
skipped the first two steps.”(Freedberg Jr., 2015)
The quotes above regarding the upgraded US Navy Littoral Combat Ship (LCS)
program (now re-branded Frigate (FF)) adequately set the stage for the dissertation
that follows. These quotes lay out the common perceptions as to why the LCS upgrade
program has been considered a failure. Essentially, the common thought is the Navy
6

failed to understand the problem, failed to understand the implications of specific
requirements, and that there was a breakdown in the structured decision process.
This dissertation argues that the real problems came from the external pressures
facing the program, and the inability for designers to fully comprehend their impacts
on the design realizations through time. The LCS upgrade program is discussed as
simply an example of the problems that can arise with a failure to understand the
Why. The Why is defined as the decisions that drive how the set of solutions may
change through time, while the What is defined as the solution itself; whether it be a
specific product, the vessel itself, or a given technology.
2.1 The US Navy Littoral Combat Ship / Frigate Program
Originally funded in FY2005, the US Navy LCS program was a program to procure
52 LCSs and frigates. The ships were designed to be a relatively inexpensive surface
combatant equipped with modular mission packages to enable the ship to be adapt-
able to different missions and operational scenarios. Two variants of the vessel were
designed and built: a steel-hulled monohull designed by Lockheed Martin and built
by Marinette Marine shipyard in Marinette, WI, and an aluminum hulled trimaran
designed by General Dynamics built by Austal USA in Mobile, AL (Figure 2.1).
According to O’Rourke (2015) the program has been controversial from the outset,
due to past cost growth, design and construction issues with the lead ships
built to each design, concerns of the ships’ survivability (i.e. ability to
withstand battle damage), concerns over whether the ships are sufficiently
armed and would be able to perform their stated missions effectively, and
concerns over the development and testing of the ships’ modular mission
packages.
7

Figure 2.1: The two LCS variants. The top is the Freedom class steel-hulled monohull
LCS built by Marinette Marine in Wisconsin, while the bottom is the Independence
class aluminum hulled trimaran LCS built by Austal USA in Alabama. (US Navy,
2010)
Due to these reasons, in 2014 under the direction of the Secretary of Defense Chuck
Hagel, the program was restructured significantly. The program has been reclassified
as a Frigate program which entails significant modifications to both the remaining
vessels in the program and well as certain modifications to some of the existing vessels
already in service.
While this restructuring was intended to breath new life into a faltering program, the
decision making process the Navy performed has come under serious criticism. Some
of the most damning criticism has been outlined in the opening three quotes and by
Ronald O’Rourke, a specialist in Naval Affairs (Freedberg Jr., 2015; O’Rourke, 2015).
The Navy did not have a clear understanding of the root problem the original LCS
was intended to address, or if and how that problem had changed in the nearly 10
8

years since the program’s inception. They also never examined whether a modified
LCS was the best option. Instead, the US Navy moved forward with a solution that
was a modified LCS without a formal rigorous assessment of not only whether the
upgraded LCS was the best option, but if it even addressed a known gap in the Navy
(O’Rourke, 2015).
However, this dissertation argues that the problems that plagued the LCS upgrade
program run deeper than those previously discussed. That problem involves properly
accounting for all the uncertain external factors affecting both the design and decision
making throughout the upgrade process. Figure 2.2 lists many of the pressures that
had to be accounted for in the program. Not only is there uncertainty surrounding the
actual affect of these factors, but the nature of which changed and evolved through
time.
Figure 2.2: External factors affecting the LCS upgrade program.
9

Due to the multi-mission capabilities required for this vessel, new technology was
required. This technology development had a long lead time, and in the meantime
requirements changed, and, as discussed above, some requirements remained through-
out the process that eventually proved to have no utility as the design progressed.
What are the implications of these requirements both throughout the design phase
and throughout the lifecycle of the vessel? Throughout this, the management of the
program changed, as well as the needs of the Navy as whole. How does the upgraded
LCS ﬁt into the big picture needs of the Navy both today and into the future? As
this is a federal government program, there was congressional oversight. Due to the
two design realizations, how does Congress and the Navy balance the needs of the
two diﬀerent shipyards contracted to build the two variants? All of these issues have
led to cost overruns plaguing the original program and the restructuring as well.
From a design perspective, how do the designers make sense of all these external
pressures through time? How do they understand the implications and impacts they
have on how the solutions evolve through time? To achieve this, this dissertation ar-
gues, requires changing the focus away from designing for the What towards a focus
of understanding the Why. The LCS case study is just one example of how focusing
on the Why can help decision makers navigate through evolving uncertain external
pressures. Other examples that will be be discussed later in this dissertation include:
designing for evolving Emission Control Areas, ship egress analysis and general ar-
rangements design, and lifecycle planning for ballast water treatment compliance.
10

2.2 The Importance of Decision Making: The Why not the
What
One of the real benefits of focusing on the Why is gaining an understanding of the im-
pacts of design decisions. According to the American Board for Engineering and Tech-
nology, “engineering design...is a decision-making process (often iterative) (ABET,
2015).” Thus, engineering decision making has been recognized as one of the funda-
mental constructs in engineering design for decades (Le Masson et al., 2013). These
decisions vary in importance (Seram, 2013), and the US Navy Office of Naval Research
has specifically identified the following problem involving decision making, stating,
It is often difficult to measure the impact of design decisions, as there are
no standard definitions, metrics, and measurements that define, let alone
calculate, the return on investment of any design decision that impacts
multiple aspects of the Navy enterprise (ONR, 2011).
This distinction between the What and the Why is important. At its core, engineering
and ship design are disciplines designed around making decisions to solve problems.
However, as individuals, all too often engineers and ship designers tend to become
too singularly focused on generating solutions. Some engineers are only able to de-
scribe the design itself and not the decisions that went into its development. This
dissertation argues that understanding the Why first can lead to designing a better
What.
This dissertations aims to explore new methods and metrics that measure the impact
of decision making. Thus moving the focus away from the What of design towards
one that examines the Why. However, developing sound decision making methods
and processes has traditionally been difficult. This is due to both practical and psy-
chological reasons, as described in the following sections.
11

2.2.1 Practical Reasons Engineering Decision Making is Difficult
Practically, decision making for large scale engineering projects is difficult due to
the temporal, sometimes fragmented, and uncertain nature that is inherent with the
design process (Hastings and McManus, 2004; Seram, 2013). This dissertation decom-
poses the problem of engineering decision making into two parts: first the problem
of understanding the uncertainty and how it affects decision making, and second
understanding the implications of the decisions themselves and the inherent interre-
lationships that may be hard discern.
2.2.1.1 Understanding Uncertainty and Decision Making
Uncertainty arises in decision making from not only endogenous factors, such as tech-
nological or engineering uncertainty, but also arises from exogenous factors, such as
regulatory, economic (Niese and Singer, 2013), or weather and climate uncertainty
(Vanem, 2015). Due to the complex and sometimes intractable nature of large scale
strategic planning and decision making problems, practitioners typically omit the un-
certain or stochastic elements when modeling the problem (Fagerholt et al., 2010).
However, Zayed et al. (2002) showed that differing and sometimes conflicting results
may arise when comparing deterministic and stochastic methods for the same prob-
lem. Zayed et al. (2002) studied the economics of maintenance and scheduling of
bridge painting by comparing a deterministic economic analysis using net present
value to a stochastic model using a Markov decision process. They concluded that
while the deterministic method may show more promising results at times, its advan-
tages are “offset by the MDP’s ability to incorporate the inherent stochastic nature of
12

the phenomenon being modeled” (Zayed et al., 2002). Thus, properly accounting for
the uncertainty in any decision aiding model is necessary when attempting to elicit
decision making insight.
Uncertainty is also not uniform, as many types of uncertainty affect lifecycle design
and operation. While numerous types of design aim to mitigate the impact of uncer-
tainty - for example: robust, versatile, and flexible designs - developing a coherent,
comprehensive strategy for decision making in the face of this remains difficult (Hast-
ings and McManus, 2004). Decisions are also expected to be made earlier in the
design process to help gain strategic advantages over competitors (Seram, 2013).
2.2.1.2 Understanding Complexity and Interrelationships of Decisions
Decisions are also sequential and have inter-relationships and dependencies that may
not be immediately obvious to the decision maker. To understand these relation-
ships, Klein et al. (2009) notes that decisions must be contextualized in terms of the
environment in which they are being made, stating:
For each [decision] there is a distribution of possible consequences. Each
distribution is a function of uncertainty of elements in the decision situa-
tion and uncertainty regarding executing the course of actions defined in
the decision option. Decision makers must choose among the options for
action that are at their disposal. Decision makers therefore must be able
to compare these options ... and choose among them, given an analysis of
the facts of the situation, which maps from the facts to the consequences
of each option.
Failure to recognize these consequences and implications increases risk and may im-
pede future opportunities. These relationships and dependencies can also have a sig-
13

nificant effect on lifecycle cost estimates. The total complexity of the design problem
makes it difficult to develop tools to handle everything from a lifecycle cost perspec-
tive (Hoff, 2007). This is problematic since so many engineering design decisions are
based on cost implications. When there are multiple decision paths available, select-
ing the one optimal strategy may prove problematic (Klein et al., 2009). Providing
insight into the Why may prove to be beneficial in understanding how the What will
behave through time.
2.2.2 Psychological Reasons Engineering Decision Making is Difficult
Psychologically, decision making is difficult because many engineers are not adept at
handling the multifaceted and uncertain nature that is inherent with many engineer-
ing projects. Wankat and Oreovicz (2015) claim this struggle has roots in education,
stating that decision making is consistently difficult for engineering students. To dis-
cuss this, an understanding of Perry’s model of intellectual development is necessary
(Table 2.1). Perry’s model shows different levels of intellectual development a student
may achieve while in college (Felder and Brent, 2005). This understanding is more
conceptual, as opposed to related to a specific subject.
Table 2.1: Perry’s model for the development of college students.
1 and 2 Dualism All knowledge known, right and wrong answers exist
for everything.
3 Early multiplicity Knowledge includes methods for solving problems,
OK to have more than one right answer.
4 Late multiplicity Uncertainty with respect to knowledge is OK,
all opinions equally valid.
5 Relativism All knowledge must be viewed in context.
6 - 9 Commitment For life to have meaning commitments must be
made, taking into account that the world is a
relativistic place.
14

Students at various levels of their collegiate career were studied to see where they fell
on this intellectual development scale. Felder and Brent (2004) describes the results
of those studies,
On average, freshmen enter college at the level of prereflective thinking
(dualism), basing their judgments on unsubstantiated beliefs and the pro-
nouncements of authorities, and leave at the quasi-reflective thinking level
(multiplicity), beginning to seek and use evidence to support their judg-
ments. Very few graduates reach the level of reflective thinking (contex-
tual relativism), however...only advanced doctoral students were consis-
tently found to reason reflectively.
The level of intellectual development relates to decision making because, “reflective
thinkers accept the inevitability of uncertainty in decision making but they are not
immobilized by it. They make judgments and decisions on the basis of a careful
weighing of all available evidence, the practicality of the solution, and the pragmatic
need for action (Felder and Brent, 2004).” Wankat and Oreovicz (2015) claim that
engineering students may not have to confront issues of multiplicity and uncertainty
until graduate level studies. This is limiting because students and practicing engi-
neers in the first two positions in the model will have significant difficulty practicing
engineering in our multiplistic society. Wankat and Oreovicz (2015) states, “these
engineers cannot see the big picture, and without further growth they are unlikely to
advance significantly in their careers.”
Wankat and Oreovicz (2015) summarize how evolutions in the problems that engineers
face may affect the functionality a engineer at a particular development level,
“Fifty years ago [1920], our researches suggest, a college senior might
achieve a world view such as that of Position 3 or Position 4 on our
15

scheme and count himself a mature man [(Perry Jr., 1970)].” However,
the world has changed and current practice of engineering involves solving
open-ended problems that are complicated by lack of data, interactions
with various stakeholders, and rapidly changing conditions. To function
as a seasoned engineer in this environment requires a person who is at
level 5 or higher (Pavelich and Moore, 1996).
2.2.3 Practical Reasons Engineers Do Not Focus on the Why
There are other reasons engineers do not focus on the Why. One of the most com-
mon practical reasons designers do not is because designers want to know What to
design so that it can actually be built. It is natural for engineers to focus on the
What because it is what they are trying to build. Eventually the product needs to be
built, so spending adequate and sufficient time designing the details of the What is
important. However, due to tight schedules and budgets, many feel there is no time
to focus on the Why. Properly defining the problem up front and understanding how
decisions can impact how the solutions evolve can save time and effort on the back
end. This attention to the Why may even reduce re-work at later stages of design
if it is determined the design does not meet desired goals (Sommez, 2012). Figure
2.3 outlines how change costs grow by a factor of ten for each step in the design and
construction process, highlighting the importance of making good decisions upfront
to mitigate changes and high costs later.
2.2.4 Psychological Reasons Engineers Do Not Focus on the Why
On top of the practical reasons, there is a deeper, more psychological reason as to why
many engineers do not focus on the Why. This is explained by Kolb learning cycle,
and the significant body of social research surrounding it. The Kolb learning cycle is
16

Figure 2.3: Estimates of change costs during diﬀerent stages of design for naval vessels
(adapted from Keane and Tibbitts (1996)).
presented graphically in Figure 2.4. This process explains the cycle students need to
complete in order to fully master a topic, such as engineering. At each step there is a
characteristic question. The question associated with the ﬁrst step is why, meaning a
focus on the problem and the decisions. This is where students study the importance
of identifying and characterizing the problem correctly and understanding how to
make the decisions to approach the problem. The next two steps are described by the
questions what and how, meaning a focus on possible solutions methods, and how to
implement them. This is where students learn ways of approaching the problem as
well as ways to actually deconstruct and solve the problem. The cycle ends with the
question what if, which is where students start exploring hypothetical situations to
gain a full understanding of the topic. This last step is where the boundaries of the
methods studied are explored to test new problems. Achieving this last step is when
full understanding of a subject happens.
Multiple studies have been performed on thousands of engineering students to de-
termine the breakdown of where most students fall between the Type 1 to Type 4
learners. The results of those studies have been remarkably consistent showing that
on average, roughly 40% of undergraduate engineering students are Type 2, 30% Type
17

Figure 2.4: Learning styles and learning cycle based on Kolb’s model (adapted from
(Montgomery and Groat, 1997)
3, 20% Type 4, and only 10% Type 1 (Harb et al., 1993; Sharp, 2001; Spurlin et al.,
2003). The grade level of the student also has an effect on both their Type as well
as their potential ability to succeed in their engineering studies. Spurlin et al. (2003)
have studied freshman engineering students and have shown that Type 2 and Type
3 students typically perform better academically than Type 1 and Type 4 students.
On top of the individual student breakdown, the classroom instruction has also been
structured to reflect a stronger focus on the What as opposed to the Why. According
to Felder and Brent (2005), “traditional science and engineering instruction focuses
almost exclusively on lecturing, a style comfortable for Type 2 learners.” Bernold et al.
(2000) studied the effect of teaching styles on students with various preferred learn-
ing styles. Their results were not conclusive but suggested that Type 1 and Type 4
learners perform worse when taught using traditional methods, as opposed to teach-
ing using the full Kolb cycle. This may lead to an associated higher risk of attrition
for Type 1 and Type 4 students (Felder and Brent, 2005). Thus, the structure of
the lecture based classroom alone may have a significant effect on the type of person
18

who pursues engineering as a career. The current education structure favors those
individuals more interested in the What and How as opposed to those interested in
the Why.
Many graduate students in engineering develop an understanding and appreciation
for Type 1 and Type 4 learning during their highly specialized study (Harb et al.,
1993). However, most practicing engineers have only a bachelors level education, thus
the problem of failing to focus on the Why still persists in the engineering community.
Even though each individual engineer will have a preference to a given learning Type,
Felder and Brent (2005) note that, “to function eﬀectively as engineers or members of
any other profession, students will need skills characteristic of each type of learner.”
Harb et al. (1993) also discussed the importance for students to complete the full
cycle during their education,
Failure to consistently traverse the full cycle is likely to produce deﬁ-
ciencies in the abilities of those whom we teach. For example, we have
all observed students who were very good at the mechanics of problem
solving, but lacked the vision and perspective necessary to recognize the
problem.
As discussed previously, this failure to recognize the problem and understand the
implications of certain decisions was one of the key reasons behind the problems that
have plagued the US Navy LCS upgrade program.
2.3 What Makes Marine Design Unique?
The maritime design problem is unique due to its physically large, often fragmented
structure of complex systems with long lifespans involving multiple stakeholders (Fet
19

et al., 2013). Ships also face a myriad of disturbances throughout their lifespan that
engineers and designers need to account for during all stages of design. While the
maritime domain has always faced some form of disturbance, the nature of which is
different today. Evolutions in the global marketplace, competition, technology, regu-
latory constraints, and societal systems (Frickle and Schulz, 2005) are necessitating
a change in the way ship design is approached. These disturbances arise from both
internal and external factors. Internal disturbances come from such factors as techno-
logical performance drift (Son and Savage, 2007; Styblinski, 1991; Niese, 2012), asset
depreciation (Stopford, 2009) and even evolving design requirements (Mouravieff,
2014). External disturbances come in the form of federal or international regulations,
such as environmental policies (IMO, 2011; Whitefoot, 2011), economic or budgetary
changes (Frickle and Schulz, 2005; LaGrone, 2014), supply chain risks (El-Gohary,
2012), and posturing of economic competitors (Tan and Park, 2014) or other navies
(D’eon, 2014). These disturbances can drastically change the vessel’s mission, use-
fulness, functionality and operating profile (Almeida, 2014; Niese and Singer, 2013),
and can vary in terms of sources, strengths, and time scales (Niese, 2012).
It is not only the presence of these disturbances themselves, but the uncertainty
associated with them through time that complicates maritime design and decision
making. While uncertainty in system design and operation has been studied in var-
ious engineering fields for decades (Pistikopoulos, 1995), the maritime domain faces
unique challenges. The complicated, and sometimes complex, maritime regulatory
environment is not precisely defined (Princaud et al., 2010) due to the uncertainty
associated with the policy instrument, geographical extent, and implementation date
of certain upcoming regulations. This is on top of flag states, coastal states, or in-
dividual ports that may implement their own individual regulations (Balland et al.,
2013; Stopford, 2009).
20

Numerous techniques have been proposed to handle the topic of maritime life cycle
design and decision making under uncertainty. The following literature is intended to
show a breadth of the techniques used to approach this problem. Yang et al. (2009)
approached the problem of vessel selection by relating it to multiple criterion decision
making (MCDM) under uncertainty. Their technique involved a decomposition of
the criteria using hierarchical decision trees and fuzzy reasoning to help guide the
decision making. Balland et al. (2013) looked into the effects of uncertainty over the
actual emission reductions of certain emission abatement technologies. They employ
a two stage optimization technique to address the uncertainty in the decision mak-
ing. Finally, Fet et al. (2013) proposed systems engineering as a means for holistic life
cycle design to increase sustainability performance. They describe various process,
product, and environmental conscious management tools that can be used within the
systems engineering framework; however, they note that these efforts are often frag-
mented and that there is a lack of a holistic approach in the literature.
2.4 The Ship-Centric Markov Decision Process
To approach this problem, this research proposes using two novel methods applied to
the ship-centric Markov decision process (SC-MDP) framework. Those methods are
performing Monte Carlo simulations to the inputs of the SC-MDP framework, and
applying eigenvalue spectral analysis to the structure of the SC-MDP. The theory,
need, and literature background of the SC-MDP framework and the two novel meth-
ods are presented here, while specifics of the methods and applications are found in
later chapters.
Markov decision processes (MDPs) were first applied to ship design and decision
21

making by Niese (2012) as a means to generate and analyze predictive time domain
design data. The SC-MDP framework is defined as applying MDPs to ship design
and decision making. MDPs are a mathematical model developed in the 1950’s to
solve dynamic decision-making problems under current and future uncertainty (Put-
erman, 2005). The SC-MDP framework has been shown to be a beneficial means
of analyzing decision making in the maritime domain due to its inherent temporal
structure and ability to handle uncertainty. The benefits of this model include a
state-based representation of both the system attributes and the environment in the
face of uncertainty, the ability to differentiate actions and sequences of decisions, and
the ability to incorporate temporal disturbances (Niese, 2012). Prior to their use in
ship design and decision making, MDPs have traditionally been used across a wide
variety of other disciplines, including: robot navigation (Russell and Norvig, 2003), fi-
nancial, economic, or portfolio management (Sheskin, 2011), inventory management,
scheduling of industrial system maintenance and replacement, and even behavioral
ecology (Puterman, 2005).
From a ship design perspective, MDPs allow for analysis of the physical product
itself, the sequential life-cycle decisions associated with that product, and the fu-
ture expected value of these products and decisions throughout their life-cycle. The
advantages of the SC-MDP framework over conventional design and analysis meth-
ods are numerous, including: an explicit model of the uncertainties associated with
the system itself and any environmental risks that may be present throughout the
system’s life-cycle, the ability to analyze dynamic operating profiles and external en-
vironments, and the ability to enable active management and decision making. The
additional use of simulations in the SC-MDP framework has enabled the ability to
examine how and why certain decisions may constrain future opportunities and to
discern differences in seemingly similar solutions. Previous research using the SC-
22

MDP framework include analysis of ballast water treatment methods, designing for
the Energy Efficiency Design Index (Niese, 2012), and design considerations in the
face of evolving Emission Control Area regulations (Kana et al., 2015).
Despite these advantages, the current SC-MDP framework does have several limita-
tions. First, the sheer size and scope of the results of the model can become vast,
and in some cases can become overwhelming for the decision maker. Second, through
the use of simulation, the SC-MDP is unable to be used as a leading indicator for
predicting when and where changes in the system or operating environment may
lead to changes in the set of available decisions. The decision maker must manually
backtrack through the model to find where these decisions may change and is unable
to quantify the magnitude of these changes in the decisions. Finally, the SC-MDP
framework is limited in its ability to clearly show the sensitivities of the uncertainties
on specific decisions and the long term implications of those decisions. This research
proposes to address these limitations by introducing both Monte Carlo simulations
and eigenvalue spectral analysis techniques to the SC-MDP framework itself. In re-
gards to understanding the Why Monte Carlo simulations are proposed first, and give
high level, preliminary understanding of the decision process. To fully understand the
decisions themselves, and thus the Why, eigenvalue spectral methods are developed
and presented as the primary contribution of this research.
2.4.1 Monte Carlo Simulations: Need, Background, and Related Work
Applying Monte Carlo simulations to the SC-MDP framework distinguishes itself
from previous work by proposing a different method for handling situations with
deep uncertainty. Previous work using the SC-MDP framework has assumed discrete
probabilistic values for the uncertainty in the model. These assumptions can be lim-
23

iting when applied to ship design and decision-making because of the difficulty in
precisely defining the specific stochastic values. For example, what is the exact prob-
ability that a vessel is able to obtain LNG as a bunker fuel at a given port during early
stages of infrastructure development? The exact value of this is not known. Monte
Carlo simulation account for uncertainty within the probabilistic transitions and re-
wards within the SC-MDP framework itself. Monte Carlo simulations are introduced
in this thesis as a means to properly handle this type of stochastic uncertainty in the
marine design decision making problem (Fagerholt et al., 2010).
These Monte Carlo simulations run through a range of uncertainties and input pa-
rameters to determine their respective effect on the overall solution. Monte Carlo
simulations have been used by others studying ship design, including Coraddu et al.
(2014) who used this technique to examine ship energy efficiency measures to meet
both the Energy Efficiency Design Index (EEDI) and the Energy Efficiency Opera-
tional Indicator (EEOI). Their technique differs from this work as this research uses
the MDP as the underlying framework with which to run the Monte Carlo simulations.
Nevertheless, Monte Carlo simulations are not the only means for handling uncer-
tainty regarding the probabilistic parameters of the MDP. Other techniques have
been developed, such as reinforcement learning (RL) (Otterlo and Wiering, 2012)
and partially observable Markov decision processes (Amato et al., 2013); however,
they both struggle when applying to life-cycle marine design decision making. Most
notably, both techniques involve virtually interacting and exploring the environment
in order to resolve and approximate these probabilistic values (Russell and Norvig,
2003). Physical or simulated exploration of the environment, as applied to marine
design, is difficult because there is little feedback for the model without physically
performing the actions in real life. For instance, what is the probability that the IMO
24

will designate a certain body of water an ECA zone at a given time during the life-
cycle of the vessel? This answer cannot be resolved using RL or partial observability
because the only feedback to the model would come from the regulation actually
changing, and at which point the model may not be necessary as the regulation has
already changed. For uncertainties with shorter time frames that do have feedback
mechanisms, such as system maintenance and replacement scheduling, RL or partial
observability may be applicable methods. The case study presented in Chapter IV:
Design for Evolving Emission Control Area Regulations, however, focuses on the un-
certainties with longer time frames and no viable feedback mechanisms, where these
methods are not appropriate.
RL and partial observability methods are inherently different than the Monte Carlo
methods used in this paper because the Monte Carlo methods do not attempt to learn
or approximate what the specific uncertainty may be. Instead, Monte Carlo methods
run through a wide range of parameters and the objective is to understand how those
variances may affect the solution. The goal of this research is to understand how
changes in the uncertainty affect how the solutions behave through time, rather than
identifying what the specific uncertainty level may be. The objective of the overall
method is to draw from the strengths of MDPs in handling uncertain temporal deci-
sion making, and the strengths of Monte Carlo simulations in enabling true stochastic
analysis. Again, in terms of fully understanding the Why, Monte Carlo simulations
only provide some of the insight. To understand the decisions themselves requires
eigenvalue spectral analysis.
25

2.4.2 Eigenvalue Spectral Analysis: Need, Background, and Related Work
Understanding the effect of the uncertainty is only part of the problem with decision
making in the maritime domain. There is also a need to understand decision making
when there is a vast number of possible decisions paths, and those decision paths are
interconnected in ways not immediately clear to the decision maker. For complex
scenarios, simply obtaining the final result does not provide sufficient insight, espe-
cially if it not clear how those results were obtained. In these cases, understanding
the implications of the decisions themselves may be just as important as obtaining
the results.
Spectral analysis applied to the SC-MDP framework is introduced as a specific con-
tribution of this thesis as a means to elicit decision making insight by parsing out the
implications and inter-relationships between decisions to help understand the Why.
In general, spectral analysis is a method for identifying and analyzing base properties
of a system, and has been applied to a wide variety of disciplines, such as: physics,
engineering, Earth sciences, social sciences and medical sciences (Stoica and Moses,
2005). This concept has even been applied to individual engineering disciplines in
the naval architecture community, such as wave mechanics and seakeeping (Dean and
Dalrymple, 1991; Faltinsen, 1990). In all cases spectral analysis is used to understand
the generalized response of the processes as opposed to specific results. For instance,
frequency analyses for dynamic systems provide information on the generalized am-
plitude and frequency across a wide variety of forcing frequencies, as opposed to
detailing the time history response given a specific forcing function and initial condi-
tions. Thus, spectral analysis applied to the SC-MDP framework has been selected as
a means to gain a deeper understanding into holistic ship design and decision making
through time.
26

The applicability of spectral methods to analyze ship design and decision making in
the SC-MDP process framework is vast. Specifically, spectral analysis can be used
to identify the specific system attributes that affect the set of actions and decisions
available to the designer or operator through time. It can also be used to help identify
those secondary, tertiary, and and even weaker inter-dependencies in the system that
may not be noticeable by other methods. This can be done by identifying the minor
spectral modes of the system. The identification of the underlying system attributes
driving the system behavior will be greatly beneficial to designers and decision mak-
ers by allowing them to focus their efforts systematically on the important factors on
the design. Spectral analysis applied to the SC-MDP framework is a truly leading
indicator by enabling the identification of these design drivers without the need for
costly computer simulation.
While no single framework can capture all aspects of design decision making (Reich,
1995; Seram, 2013), the objective of this research is to provide a unique perspective on
engineering decision making processes to help elicit new insight that may improve un-
derstanding and design. Two chapters showing the application of eigenvalue spectral
analysis are presented in Chapter V: Ship Egress Analysis and General Arrangements
Design, and Chapter VI: Lifecycle Planning for Ballast Water Treatment Compliance.
2.5 Summary
This chapter outlined the primary themes and engineering problems addressed in
this dissertation. First, the importance of understanding the Why was discussed and
reasons why it has not been fully appreciated were detailed. Second, the importance
and difficulty of design decision making under uncertainty and through time was
presented. Many engineering tools do not fully address this problem, necessitating
the need for new methods and metrics. The ship-centric Markov decision process
27

is proposed as the framework. Two new methods to the SC-MDP framework are
discussed that are unique to this thesis aimed at addressing the problems outlined
above.
28

CHAPTER III
New Methods and Metrics for the Ship-Centric
Markov Decision Process Framework
The underlying mathematical model behind the SC-MDP framework is the Markov
decision process (MDP). This chapter covers the background methodology behind
Markov decision processes, Markov chains, as well as the two techniques unique to this
thesis: 1) applying Monte Carlo simulations to the inputs of the SC-MDP framework,
and 2) applying eigenvalue spectral analysis to the the SC-MDP framework.
3.1 Markov Chains
An understanding of basic Markov chains is a necessary precursor to Markov decision
processes. Markov chains are a linear ﬁrst-order model used for tracking uncertain
system movement through time. They model sequential events under the assumption
that future behavior is independent of past events. They are characterized by a ﬁnite
set of states, S, and a set of probabilities, T, denoting the uncertainty of transitioning
from one state to another through time (Sheskin, 2011).
The probability of transitioning between states is presented mathematically using the
Markov chain transition matrix, M. An example Markov chain with three states is
29

presented graphically in Figure 3.1. The probability of transitioning between states is
presented mathematically using the transition matrix, M, given in Table 3.1, where si
denotes the ith
state of the previous time step and sj denotes the jth
state in the next
time step. For example, the probability of transitioning from State 1 to State 2 in the
next time step is 0.7. The cumulative probability of transitioning from one state to
any other state is always one. Thus, the rows of the Markov chain transition matrix
always sums to one. Matrices with this property are known as stochastic matrices
(Anton and Rorres, 2005).
Figure 3.1: Sample Markov chain with three states for a single epoch. pi,j denotes
the probability of transitioning from State i to State j.
Table 3.1: Markov chain transition matrix for Figure 3.1. The entries indicate the
probability of transitioning from the previous state, si, to the new state, sj. Note,
each row must sum to unity.
New State
s1 s2 s3
Previous State
s1 0 0.7 0.3
s2 0.2 0.8 0
s3 1 0 0
These transitions may vary through time, and thus M = f(t) = Mt
. When the
transitions vary through time the process is known as non-stationary, otherwise, the
process is stationary.
30

The probability of being located in a given state at a given time is denoted by the
state vector. The state vector is deﬁned in Equation 3.1. The sum of all elements in
the state vector must be one.
s = (s1 s2 ... sn) (3.1)
An example state vector for the system above is given in Equation 3.2. This state
vector shows that there is a 50% probability of being located in State 1, a 30%
probability of being located in State 2, and a 20% probability of being located in
State 3.
s = (0.5 0.3 0.2) (3.2)
The probability of being located in a given state may vary through time, and this
evolution is calculated by multiplication of the state vector with the transition matrix,
as given in Equation 3.3.
st+1
= st
Mt
(3.3)
While Markov chains are a useful method for modeling system evolution through
time, they do have limitations. They are unable to model various decision making
scenarios, nor are they able to calculate the expected value of landing in a given
state at a given time. Extending Markov chains to handle both decision making and
expected values is known as a Markov decision process.
31

3.2 The Markov Decision Process
Markov decision processes are an extension of Markov chains designed to handle
dynamic sequential decision-making problems under uncertainty. They represent un-
certain systems, can differentiate actions, and can handle temporal system variations.
The classic MDP is defined as a four-tuple < S, A, T, R >, where S is a set of finite
states where the agent can exist, A is the set of actions which the agent can take, T
is the probability the agent will transition from one state to another after taking a
given action, and R is the reward the agent receives by executing a given action, a,
and transitioning to a new state, s . Essentially, MDPs can be thought of as a series
of action dependent Markov chains with rewards (Sheskin, 2011), where each action
can be represented by its own transition matrix.
(a) Action 1 (b) Action 2
Figure 3.2: Sample three state MDP with two actions. Note that transition proba-
bilities and rewards may vary between actions.
An example three state, two action Markov decision process is given in Figure 3.2.
Note the transition probabilities and rewards may vary between different actions. As
with the Markov chains, the transition probabilities are represented in a transition
matrix. However, there is now a different transition matrix for each action. This is
32

shown in Table 3.2. Table 3.3 is the associated reward matrix, which outlines the
reward obtained after taking a given action, a, and landing a given state, s .
Table 3.2: Sample transition matrices for the system in Figure 3.2. Each action is
represented by its own transition matrix. The specific transition probabilities in this
case have been chosen simply for demonstration purposes.
Action 1
s1 s2 s3
s1 0 0.7 0.3
s2 0.2 0.8 0
s3 1 0 0
Action 2
s1 s2 s3
s1 0 0 1
s2 0 1 0
s3 0.1 0.1 0.8
Table 3.3: The reward matrix associated with Figure 3.2. The entries indicate the
reward received after taking a given action, ai, and landing a given state, sj.
a1 a2
s1 5 -1
s2 1 3
s3 0 6
The objective of an MDP is to identify the sequence of actions that maximizes the
cumulative, long term expected utility of the system. This sequence of actions iden-
tifies the set of recommended decisions the agent should take during each decision
epoch. An epoch is defined as an instance when the agent must make a decision.
Epochs can represent any such decision making period, such as a time step, or phys-
ical movement by an individual. The set of decisions is known as the policy of the
MDP (Russell and Norvig, 2003). This policy takes into account both the outcomes
of current decisions and future opportunities. That is, MDPs are memory-less, which
means the preferred actions do not rely on the history of how the agent arrived at a
given state (Puterman, 2005).
The expected utility of the MDP can be obtained via Equation 3.4, known as the
Bellman equation, where U is the expected utility, γ is the discount factor, and the
33

other variables are defined previously.
U(s) = R(s) + γ max
a
s
T(s, a, s )U(s ) (3.4)
The decision policy, π, is found by taking the argument of Equation 3.4, as defined
in Equation 3.5 (Russell and Norvig, 2003).
π(s) = arg maxa
s
T(s, a, s )U(s ) (3.5)
MDPs are commonly solved via backward induction (i.e. dynamic programming)
to evaluate the expected utilities. That is, the model is solved backward in time, by
starting at the desired end state, and then moving backwards to find the optimal route
and expected value. This method is used to ensure that the sequence of decisions
prescribed is optimal (Puterman, 2005; Sheskin, 2011). If the transition probabilities,
rewards, or decision policy do not change with time, the process is known as station-
ary, otherwise the process is non-stationary (Niese, 2012; Puterman, 2005).
The common output of solving a non-stationary MDP for displaying the decision pol-
icy is a decision matrix. Decision matrices provide the recommended actions for each
state for each decision epoch. A generic non-stationary decision matrix adapted from
Niese and Singer (2014) is given in Table 3.4. Niese and Singer (2014) used Markov
decision processes to assess changeability of engineering designs under uncertainty by
performing simulations through the decision matrix. To read the decision matrix, the
decision maker would identify the preferred action in the table that corresponds to
the system state and epoch. For instance, if the system were in State 2 during Epoch
2, then Action 2 would be preferred.
The decision matrix can be a beneficial way of displaying information for several
34

Table 3.4: Example non-stationary decision matrix developed from solving the Bell-
man equation for the system in Figure 3.2. For example, Action 2 is preferred when
the system is in State 2 and Epoch 2. Note the decisions may change through time,
as is the case between Epoch 4 and Epoch 5 for State 1.
State 1 State 2 State 3
Epoch 1 Action 1 Action 2 Action 2
reasons. First, it provides a road-map of preferred actions for the decision maker.
Second, it displays the optimal decision for each state for all time. Lastly, it can be
used to simulate different decision scenarios to discern slight differences in seemingly
similar situations as well as communicating how decisions are constrained and which
states deserve greater focus (Niese, 2012).
However, there are clear limitations with the decision matrix, including: size, inability
to predict changes in decision making behavior, and inability to show long term impli-
cations and relationships between set of decisions. The decision matrix grows linearly
with number of states and can quickly become overwhelming or even intractable for
a decision maker. This problem is compounded by the uncertainty that may exist
associated with knowing the specific state the system is in at any given time (Amato
et al., 2013). Despite the model’s ability to predict the set of recommended decisions
in the face of uncertainty, the use of simulations through the decision matrix prevents
it from being used as a leading indicator to identify areas where the decisions may
change. In order to understand the impact of decision making on the full range of
possible outcomes, simulations and exhaustive sensitivity studies are required. The
decision matrix alone does not clearly show the impacts of uncertainty or sensitivity
on the decision paths. Also, the decision matrix is unable to identify the temporal
implications of specific decisions or the relationships between various decision paths.
35

In order to overcome these limitations to understand the Why for large scale decision
problems, the research presented in this thesis investigates two methods:
1. To understand the impacts the uncertainty within a decision making process
can have on the suite of results, this thesis advocates for applying Monte Carlo
simulations to the inputs of the SC-MDP framework.
2. To understand the implications and relationships between specific decisions
within a given process, this research proposes applying eigenvalue spectral anal-
ysis.
These techniques are described in more detail in the following sections.
3.3 A Monte Carlo Approach to the SC-MDP Framework
Monte Carlo simulations are used to handle the uncertainty associated with defining
the rewards and transition probabilities. As discussed in the previous chapter, these
Monte Carlo simulations are well suited for systems with uncertainties with long time
frames and no viable feedback mechanisms, such as future environmental policies and
regulations.
To perform the Monte Carlo simulations, value ranges are determined for each param-
eter and the simulations iteratively selects values at random from each input variable
distribution. This process is shown visually in Figure 3.3. Thousands of simulations
may be run to ensure convergence to a stable distribution. The cumulative incremen-
tal change in both the states and actions is calculated after each run. The maximum
incremental change is used to show that the system has stabilized and that additional
simulations do not affect the solution in any significant manner.
36

Figure 3.3: Visual representation of the Monte Carlo approach to the ship-centric
Markov decision process framework.
This method enables unique analysis on the sets of decisions. The decision maker
is able to determine the percentage of time a given action may be optimal, given a
large suite of potential scenarios, as opposed to looking at a given optimal decision
path for one given scenario. This helps decision makers understand which actions
are most likely to be preferable, even in the face of large uncertainty. Understanding
the effect of uncertainty on the expected value is also very important. Monte Carlo
methods enable the decision maker to calculate the range of expected values both
through time and through various system scenarios. Understanding the probability
that a given value expectation will be met through time is possible by applying Monte
Carlo methods to the SC-MDP framework.
Sensitivities on specific variables are also performed to gain an understanding of why
the system may behaving the way that it is and what system parameters may be
driving its behavior. A case study involving designing for evolving Emission Control
Area regulations is given to detail the specifics of this framework in Chapter IV: De-
sign for Evolving Emission Control Area Regulations.
37

In regards to understanding the Why Monte Carlo simulations come close, but do not
provide a full understanding. There are several unanswered questions, specifically:
• Which decisions change the set of solutions over time?
• How do you quantify the difference in specific decisions?
• How do you understand the long term implications of specific decisions?
• How do you understand the inter-dependencies and relationships of various
decisions within the process?
There remains a need for a method that provides a generalized response to the system
that shows the implications of the various parameters and decisions, as opposed to
one that is focused on specific correlations and the results themselves. Eigenvalue
spectral methods are proposed as a means to address these issues.
3.4 An Eigenvalue Spectral Analysis Approach to the SC-
MDP Framework
Eigenvalue spectral methods applied to the SC-MDP framework are considered the
major contribution of this thesis. Spectral methods are introduced as a means to
analyze and quantify specific decisions, examine the long term implications of those
decisions, as well as eliciting the inter-dependencies and relationships between var-
ious decision paths. In order to perform spectral analysis, a different formulation
from the decision matrix described above is necessary. This formulation involves us-
ing the decision matrix to develop a series of Markov chain transition matrices to
represent the underlying dynamics of the system. Eigenvalue spectral analysis is then
38

performed in these transition matrices. The eigenvalues and eigenvectors provide key
information of the system that may not be able to be found using traditional methods.
Two case studies are presented demonstrating the utility of spectral analysis and
the metrics described in the following sections. A case study involving ship egress
analysis and general arrangements design is given in Chapter V to show the utility
of the damping ratio, using the eigenvector as a metric for steady state behavior,
and the importance of the relationship between the eigenvector and the damping
ratio. A final case study discussing lifecycle planning for ballast water treatment
compliance is presented in Chapter VI to demonstrate the significance of repeated
dominant eigenvalues and how their associated eigenvectors can be used to identify
and characterize various initial condition dependent decision absorbing paths.
3.4.1 Spectral Analysis Methodology Overview
The following steps outline how to apply eigenvalue spectral analysis methods to the
SC-MDP framework. While each step individually can be found in the literature,
it is the combination of the steps, the new algorithm development, and the new
applications presented in this dissertation, that makes this research unique. The
method is summarized first, with the details of each step following.
1. Obtain the decision policy and associated expected utilities by solving the stan-
dard ship-centric Markov decision process (Puterman, 2005).
2. From the set of decisions, develop a series of representative Markov chain tran-
sition matrices, M, for each decision epoch (Sheskin, 2011).
3. Perform eigenvalue spectral analysis on the transition matrices to generate the
spectrum of the MDP (Caswell, 2001).
39

3.4.2 Forming the Representative Markov Chains
Instead of displaying the policy using the decision matrix, the proposed methodol-
ogy uses a series of Markov chain transition matrics, M, to represent each decision
epoch. Using this method has several advantages. First, it preserves state transition
probabilities, as opposed to simply stating the optimal action. Second, and more
importantly for this research, formation of a series of transition matrices enables the
ability to perform eigenvalue spectral analysis.
These transition matrices are developed from the decision matrix (Sheskin, 2011).
This is done by selecting the state transitions for each state from its respective opti-
mal action and placing it in its respective row in the representative Markov chain. For
example, if Action 1 is optimal for State 1 according to the decision matrix, then the
first row for the representative Markov chain is identical to the first row of the Action
1 transition matrix. Likewise for State 2, if Action 2 is optimal, then the second row
of the representative Markov chain will be identical to the second row of the Action 2
transition matrix. This logic is followed for all states. This new transition matrix has
all the same properties of a standard Markov chain, namely it is square stochastic.
The only difference is that this new chain is able to represent the various optimal
actions for all states. It is essentially an amalgamation of the set of action transition
matrices displaying only the optimal action for each state. The result is a different
representative Markov chain for each decision epoch.
An explicit example of this formation of the representative transition matrix is given
below. The individual action transition matrices from the system in Figure 3.2 are
presented again in Table 3.5. The colors denote the specific transition probabilities
that are transferred to the representative transition matrix according to the decision
matrix. The decision matrix for the first decision epoch is presented in Table 3.6, and
40

the resulting representative transition matrix is developed in Table 3.7. Eigenvalue
analysis is then performed on this transition matrix given in Table 3.7.
Table 3.5: Sample transition matrices for the system given in Figure 3.2. The colors
denote the specific transition probabilities that are transferred to the representative
Markov chain transition matrix according to the decision matrix.
Action 1
s1 s2 s3
s1 0 0.7 0.3
s2 0.2 0.8 0
s3 1 0 0
Action 2
s1 s2 s3
s1 0 0 1
s2 0 1 0
s3 0.1 0.1 0.8
Table 3.6: The decision matrix for the first decision epoch from Table 3.4. This is
generated by solving the Bellman Equation (Equation 3.4).
State 1 State 2 State 3
Table 3.7: Sample representative Markov chain transition matrix, M, for Epoch 1
(Table 3.6). Rows are determined from the action transition matrices (Table 3.5) and
the decision matrix (Table 3.6).
s1 s2 s3
s1 0 0.7 0.3 (from Action 1)
s2 0 1 0 (from Action 2)
s3 0.1 0.1 0.8 (from Action 2)
This formulation of the decision process in this manner has the added benefit of
clearly differentiating the set of decisions from that of the design. The representative
Markov chain transition matrix can be thought of as the the set of decisions, while the
state vector described previously can be thought of as the physical system or design.
Thus, using the techniques described in this thesis, one can clearly show the impact
specific decisions have on specific areas of the design.
41

3.4.3 Spectral Analysis
Once the representative Markov chains are formed for each decision epoch, eigenvalue
spectral analysis can then be performed. The eigenvalues, λ, and eigenvectors, w, are
defined according to Equation 3.6. The spectrum of a Markov chain, M, is defined
as the set of its eigenvalues, λ (Cressie and Wikle, 2011). For this research, Equation
3.6 was solved numerically using a built-in MATLAB function.
wiM = λiwi (3.6)
The eigenvalues are key to understanding the underlying dynamics of the system
(Salzman, 2007). They represent the analytic solution to each Markov chain, and
can also be used to examine the system attributes driving the behavior through time.
Identification of global oscillations, system stability, and bifurcation regions (Cressie
and Wikle, 2011) in the decision pathways is also possible with eigenvalue spectral
analysis. This type of analysis is common practice for Markov chains, and its appli-
cability has proven to be far-reaching (Caswell, 2001). However, applying eigenvalue
spectral analysis on Markov chains developed from a Markov decision process to an-
alyze various design decision making behavior is a unique contribution of this thesis.
By enabling spectral analysis of MDPs, this methodology has opened up vast new
areas of research and potential metrics for analyzing sequential decision making. Two
overarching metrics are developed and explored in this thesis: one is the damping
ratio, and the other is the eigenvector associated with the dominant eigenvalue.
3.4.4 The Damping Ratio
The damping ratio has traditionally been used to study the transient behavior of a
system and its rate of convergence to a steady state behavior. The damping ratio, as
42

defined here for linear models, is defined in Equation 3.7, where λ1 is the dominant
eigenvalue, meaning it has the largest magnitude. |λ2| is the magnitude of the primary
sub-dominant eigenvalue, meaning it is second largest (Caswell, 2001). For stochastic
matrices, such as M, the dominant eigenvalue, λ1, is always one. Thus, for Markov
chains, the damping ratio can be uniquely defined by just λ2. All eigenvalues lie
within the unit circle in the complex plane (Kirkland, 2009).
ρ =
λ1
|λ2|
(3.7)
This definition of the damping ratio has been used to analyze general state space
models in biological population modeling, such as the Leslie matrix model and the
Horn model (Caswell, 2001; Tanner and Hughes, 1994). For these more general state
space models which are not defined as stochastic, the constraint on λ1 does not apply,
and the damping ratio is defined by both λ1 and |λ2|. For Markov chains specifically,
the use of λ2 as a metric for convergence rates is more common (Pryde, 2009). Appli-
cations of the damping ratio as defined here, while available in the literature, is less
common and appears to be limited to only a few specialized topics and researchers in
biological population modeling (Hill et al., 2002, 2004) and theoretical mathematics
(Hartfiel and Meyer, 1998; Kirkland, 2009).
The damping ratio will be used in two unique ways in this dissertation. First it will
be used to identify and quantify changes in sets of decisions. Changes in the damping
ratio are coincident with changes in the Markov chain transition matrix, and thus
represent changes in the sets of decisions. When the decision of an individual state
changes, that state’s row in the representative Markov chain changes as well. The
row that consisted of the transition probabilities from the action transition matrix
from the previous set of decisions is replaced by the row consisting of the transition
probabilities of the new action transition matrix associated with the new decisions.
43

This process is shown visually in Figure 3.4.
Figure 3.4: Visualization of how a change in decisions affects the representative
Markov chain transition matrix, M.
Since eigenvalue analysis is performed on these transition matrices, changes in the
representative Markov chain transition matrix change the eigenvalue spectrum of the
system. When this change affects the sub-dominant eigenvalue, λ2, it is concluded
that a major change in the system has occurred because the primary spectral mode
has changed. Thus, changes in the damping ratio signify major changes in the deci-
sion paths that have a significant effect on the overall system. When the damping
ratio is unaffected by changes in the sets of decisions, there is no significant change
to the overall system. This is similar to only affecting the minor spectral modes of
the system.
The second new application for the damping ratio is that it will be used to help
identify the significant state/action combinations that affect the process as a whole.
By associating the changes in the sets of decisions with the changes in the damping
ratio, the decision maker can identify important state/action combinations. While
the physical meaning of the eigenvalues themselves is hard to define, this research
instead focuses on how the eigenvalues change and then relates those changes to deci-
sion making behavior. These applications of the damping ratio to a Markov decision
process is a unique contribution of this thesis.
44

3.4.5 The Eigenvector as a Metric for Steady State Behavior
This section outlines a brief derivation of how the eigenvector associated with the
dominant eigenvalue for stochastic matrices can be used as a metric for steady state
behavior. This eigenvector is known as the principal eigenvector. The derivation
is presented, followed by a discussion of its application to ship design and decision
making. First, assume a stationary system defined by the stochastic transition matrix,
M, and the initial state vector s0
. As the system evolves through time, the state vector
will eventually converge to a steady state vector, that is st
→ s∞
= s. This process
is shown in Equation 3.8.
s0
M = s1
s1
M = s2
s2
M = s3
...
sM = s
(3.8)
Notice that the final step in Process 3.8 can be defined as a eigenvalue equation with
s representing the eigenvectors and the assumption that λ = 1 (Anton and Rorres,
2005). The final line in Equation 3.8 is identical to Equation 3.6. In this case, the left
eigenvector is used because it is located on the left hand side of M in the left hand side
of Equation 3.8. As discussed previously, for stochastic matrices, not only does λ = 1
exist, but it is also the largest eigenvalue in magnitude. Thus, the eigenvector asso-
ciated with the dominant eigenvalue can be used as a metric for steady state behavior.
Even though most decision making is inherently non-stationary, this metric is still
significant to decision makers. This is because for every instant that is defined by M,
45

the steady state distribution can be calculated without having to run the simulation
through time. Thus, the system converges based solely on the set of decisions, defined
by M, and not on the initial state vector s0
. In these situations the projected system
is entirely independent of where the system starts.
As decisions change through time, and thus as M changes, the steady state distri-
bution may change as well. Calculating the magnitude of this change will be used
as a means to quantify the effect of a given decision on the future effect of the sys-
tem. Calculating this change between eigenvectors will be defined as determining
the magnitude of the angle between the vectors. This angle is calculated using the
identity presented in Equation 3.9 (Anton and Rorres, 2005), where w0 is the original
eigenvector and w1 is the eigenvector for the system with the updated set of decisions.
θ = cos−1
(
w0 · w1
||w0|| ||w1||
) (3.9)
Thinking of the eigenvector as a vector pointing in the direction of how the system
will evolve given a set of decisions it becomes clear that this metric is truly a leading
indicator for analyzing design decisions. For instance, given the current set of deci-
sions, the specific design, as characterized by the state vector, will eventually progress
to a design characterized by the eigenvector. This is represented graphically in Figure
3.5. Here, wi are independent eigenvectors associated with a given set of decisions.
As stated, these eigenvectors are identical to the future steady state of the system
s∞
= s. Using the eigenvector in this fashion for forecasting the impact of design
decisions is a unique contribution of this thesis.
46

Visualization techniques are especially helpful for when the state-space is large. For
example, the ballast water case study presented in Chapter VI has 240 states, while
the EEDI case study developed by Niese et al. (2015) has over 3,500 states.
1 1.5 2 2.5
0
0.5
1
1.5
Direction design is heading
after small change in decisions
Original direction
design is heading
Direction design is heading
after significant change
in decisions
θ
w2
w
0
w1
Figure 3.5: Visual representation of the eigenvectors as a leading indicator for the
impact of design decisions. wi are independent eigenvectors associated with a given
set of decisions which are identical to the future steady state of the system or design,
s∞
= s.
3.4.6 The relationship between the damping ratio and the eigenvector
There are instances when using the eigenvector as a metric for ﬁnding a single steady
state behavior will not work. For example, when ρ = 1. This means that the sub-
dominant eigenvalue, λ2 = 1; that is λ2 = λ1 = 1. Thus, the sub-dominant eigenvalue
is actually a repeat of the dominant eigenvalue. When the dominant eigenvalue is
repeated there are now multiple eigenvectors associated with the set of dominant
eigenvalues. This relationship between λ1, λ2, and s is why the damping ratio metric
is deﬁned by both λ1 and λ2 in this thesis in order to highlight the importance of the
relationship between these two eigenvalues, despite λ1 = 1 in all cases.
47

Two types of behavior may occur when the dominant eigenvalue is repeated. First,
the state vector may fail to converge to a single steady state distribution. This may
happen if the steady state vector oscillates between more than one distribution, or if
multiple distributions exist simultaneously, that is more than one s exists. Second,
the convergence of the state vector may be initial condition dependent (Kirkland,
2009). This means the system will converge based on both the set of decisions and
on where the system starts. In situations such as these the designer needs to be very
careful in how they select their starting state.
This relationship between the damping ratio and the principal eigenvector is logical.
As the damping ratio approaches 1, that means λ1 and λ2 approach each other. Two
things happen here. First, the system approaches repeated roots for the primary
spectral mode, which cause different phenomena then when the roots are distinct.
Second, when ρ = 1 there are multiple dominant eigenvalues, and thus there are
multiple principal eigenvectors.
3.4.7 The significance of repeated dominant eigenvalues
When the dominant eigenvalue is repeated, different analysis techniques are war-
ranted to gain insight. Two analysis techniques are examined in this dissertation.
The first involves the technique of reducible matrices to group specific aspects of the
design and decision process that align with each other (Gebali, 2008). This technique
helps identify relationships and inter-dependencies between specific decisions within
the whole decision process. The physical meaning of this is that this technique de-
composes the one decision process into multiple independent decision processes. The
second technique uses the set of principal eigenvectors to estimate the behavior of the
48

state vector through time. These two techniques are the forefront of the understand-
ing of applying spectral methods to the SC-MDP framework. Results are presented
in Chapter VI: Lifecycle Planning for Ballast Water Treatment Compliance, while
further studies remain as future work.
3.4.7.1 Reducible Markov processes
There may be situations when the dominant eigenvalue is repeated multiple times. In
these situations, the Markov process is known as reducible. Reducible Markov pro-
cesses are those in which not every state is reachable from every other state. Thus, by
starting in a specific set of states, it is not possible to reach certain other states. This
aspect is important because it highlights the initial condition dependence of reducible
processes. Also, it may inform the decision maker that certain design or decision paths
may not be reachable given a specific initial condition. Reducible Markov processes
are defined by closed and transient states. Transient states are those in which the
process may pass into and through, but will not remain in for the long term. Closed
states are those in which that once the process enters those sets of states, it will never
leave. They are states the system will converge to in the long term (Gebali, 2008).
The set of closed and transient states can be identified by examining the set of princi-
ple eigenvectors (Theorem 1). The proof of the theorem can be found in Gebali (2008).
Theorem 1: Let M be the transition matrix of a reducible Markov chain whose
eigenvalue λ = 1 corresponds to an eigenvector w. The closed states of the chain cor-
respond to the nonzero elements of w and the transient states of the chain correspond
to the zero elements of w.
Essentially, the nonzero elements of each principle eigenvector represents one partic-
ular steady state distribution. These distributions depend on the initial conditions of
49

the system (Gebali, 2008). By associating the absorbing paths with the closed states
of the Markov chain, Theorem 1 can be used to justify the use of the principle eigen-
vectors in identifying long term absorbing paths of the decision process. The concept
of absorbing paths is analogous to the steady state analysis described above. An ab-
sorbing path represents the long term behavior of a non-stationary decision process.
There may be more than one absorbing path for the whole decision process, each
one being dependent on the initial conditions of the system. Niese et al. (2015) dis-
cussed the importance of identifying the presence of multiple absorbing paths. They
discussed that differing absorbing paths may mean that differing decision sequences
may be viewed as only locally optimal. They were able to identify the multiple paths
via simulation through the decision matrix. This dissertation, on the other hand,
claims that these differing paths are in fact dependent on the initial conditions of the
system. Also, this dissertation uses spectral analysis as a leading indicator metric to
identify these multiple absorbing paths without the need for costly simulations and
recursive investigation.
Traditional analysis using these techniques involve stationary Markov processes. This
dissertation extends those studies to examine their applicability to non-stationary de-
cision processes. By studying non-stationary processes with this method, the decision
maker is able to gain insight into not only the instantaneous impact of their decisions
on future absorbing paths, but they can also gain an understanding of how those
absorbing paths may change and evolve over time.
3.4.7.2 Using the Principle Eigenvectors to Estimate the State Vector
The other technique proposed to handle repeated dominant eigenvalues is by using the
set of principle eigenvectors to estimate the behavior of the state vector. This method
50

is used to highlight the relationship between the set of principle eigenvectors and the
state vector. This method is a unique contribution of this thesis. The objective
is to combine the set of principle eigenvectors in such a manner that it estimates
the behavior of the state vector as close as possible. Mathematically, the goal is to
minimize the diﬀerence between the state vector, st
, and set of principle eigenvectors,
[w]. This is given in Equation 3.10, where α is a set of scaling factors for the principle
eigenvectors.
min||s − [w] ∗ α|| (3.10)
To solve for α, set the parameters equal to each other (Equation 3.11) and solve for
α using the pseudo-inverse of [w] (Equation 3.12). Here [w]†
is the Moore-Penrose
pseudo-inverse of [w]. This is a generalized form of the inverse that can handle non-
square matrices. Details on how to calculate the Moore-Penrose pseudo-inverse can
be found in Bishop (2009).
s = [w] ∗ α (3.11)
α = [w]†
∗ s (3.12)
This method ﬁnds the α that minimize this distance. The pseudo-inverse is necessary
because [w] is not square. If [w] were square, the actual inverse could be used and
α would be able to solve this equation exactly. However, since the pseudo-inverse is
used, this technique is only an estimation.
51

3.5 Summary
This chapter has highlighted three key aspects of this thesis. First, it provided the
mathematical background for Markov chains and Markov decision processes, the back-
ground model to the SC-MDP framework. Second, it outlined the methods behind
applying Monte Carlo simulations to the SC-MDP framework and how they be be
greatly beneficial in situations of multi-layered, complex uncertainty. Third, and most
significant to this thesis, eigenvalue spectral methods were explained as they relate to
the SC-MDP framework. New methods and metrics involving both eigenvalues and
eigenvectors were explained, as well as how those different aspects of the spectrum
relate to each other. Methods were described that handle both unique and repeated
eigenvalues, as well as methods that describe the relationship between the spectrum
of the decision process and the physical system itself.
52

CHAPTER IV
Monte Carlo Case Study: Design for Evolving
Emission Control Area Regulations1
This chapter presents a case study showing the utility of applying Monte Carlo simu-
lations to the SC-MDP framework. The insight gained is in terms of how uncertainty
affects decision making behavior, how those decisions affect the set of possible out-
comes, as well as the economic costs associated with making those decisions. As
discussed in Chapter II, understanding how multiple layers of uncertainty affect de-
cision making and lifecycle costs is difficult. In response to this, the objectives of this
chapter are:
1. Demonstrate the benefit of applying Monte Carlo simulations to the SC-MDP
framework for a temporal, non-stationary process.
2. Demonstrate how Monte Carlo simulations can be used to analyze the impact
multiple decision scenarios on a suite of results.
3. Demonstrate how Monte Carlo simulations provide unique insight compared to
static probabilistic values traditionally used in Markov decision processes.
1
Early work on this chapter was presented at the 2015 International Marine Design Conference
(IMDC) in Tokyo, Japan with the assistance of Dr. Joshua Knight, Michael Sypniewski, and Dr.
David Singer (Kana et al., 2015). See Appendix B.
53

4.1 Background
Designing and adapting to evolving international emissions regulations is one area
that has, and will continue to have, drastic effects on ship design (ABS, 2010, 2013;
Bengtsson et al., 2011; Goh, 2014; Rynbach, 2014). Even though it can be argued
that maritime shipping is one of the most environmentally friendly modes of cargo
transportation due to its low CO2 emissions per ton-kilometer, it has been estimated
that before 2020 international shipping will overtake all land-based transport as the
largest emitter of NOx and SOx in Europe (Ma, 2010). To tackle this problem, the
International Maritime Organization (IMO) regulates the emissions of nitrogen oxide
(NOx) and sulfur oxide (SOx) via MARPOL Annex VI regulations as part of their
overall strategy of limiting maritime pollution (IMO, 2008). Originally adopted in
1997 under the “1997 Protocol”, these regulations were updated in 2008 with a more
stringent emissions limit schedule (DieselNet, 2011). These emission pollutants cause
respiratory problems in humans while increasing ambient ozone concentrations, acid
rain, smog, and particulate matter (EPA, 2014a,b; Pinkerton, 2007). These environ-
mental and health concerns have caused the IMO to designate certain environmentally
sensitive areas as Emission Control Areas (ECAs) where more stringent emissions
standards apply. The current ECAs lie in either densely populated or environmen-
tally sensitive areas, while proposed areas are still under consideration (Figure 4.1).
4.1.1 Regulatory Framework
The limits of NOx emissions are set based on the engine’s rated speed. The original
emissions were set in 1997 and are designated Tier I. These limits affect ships built
after January 1, 2000. Tier II limits were implemented during the 2008 amendments
and affect ships built after January 1, 2011. Tier III limits apply only in NOx ECAs
and affect ships built after January 1, 2016 (IMO, 2008). Tier II limits can be met
54

Figure 4.1: IMO regulated Emission Control Areas (Blikom, 2011)
using combustion process optimization, while Tier III emissions compliance will likely
require speciﬁc NOx emission control technologies, such as exhaust gas re-circulation,
or selective catalytic reduction (SCR) (DieselNet, 2011). The NOx emissions limits
are summarized in Table 4.1 and Figure 4.2 (IMO, 2008).
Table 4.1: MARPOL Annex VI NOx Emission Limits
NOx Limit, g/kWh
n = engine’s rating (RPM)
Tier Ship Construction Date n < 130 130 ≤ n < 2000 2000 ≤ n
(January 1 or after)
Tier I 2000 17.0 45 ∗ n−0.2
9.8
Tier II 2011 14.4 44 ∗ n−0.23
7.7
Tier III 2016* 3.4 9 ∗ n−0.2
1.96
* In NOx ECAs (Tier II standards apply outside ECAs)
SOx emissions are regulated slightly diﬀerent than NOx emissions. For SOx emis-
sions, there are two levels of compliance, one for ECA zones, and one for elsewhere.
Both limits become more stringent through time. The date for implementing the
global 0.5% SOx emission limit is set for 2020, but may be extended to 2025 if the
55

0 500 1000 1500 2000
0
5
10
15
20
Rated Engine Speed, RPM
NOx
Limit,g/kWh
Tier I
Tier II (global)
Tier III (NO
x
Emission Control Areas)
Figure 4.2: MARPOL Annex VI NOx Emission Limits (DieselNet, 2011)
IMO concludes that there is not enough available fuel. This will be decided in 2018,
which if the IMO decides to stick to the 2020 deadline, would only give vessel own-
ers two years to comply (IMO, 2008). The limits are measured as a percent by mass
emitted of SOx (%m/m) and are summarized in Table 4.2 and Figure 4.3 (IMO, 2008).
Table 4.2: MARPOL Annex VI SOx Emission Limits
Sulfur Limit in Fuel (%m/m)
Date SOx ECA Global
prior to July 1, 2000 1.5%
4.5%
prior to July 1, 2010
1.0%
prior to July 1, 2012
3.5%
after January 1, 2015
0.1%
after January 1, 2020∗
0.5%
* date could be delayed to 2025 subject to
a review of available fuel to be completed
in 2018
Despite the illusion of clarity, these regulations are not precisely defined (Princaud
et al., 2010), and the uncertainty that vessel owners, operators, and designers face
remains large. The uncertainty associated with the geographical extent, implementa-
tion date, and policy instrument will significantly affect how vessels operate and do
business in the coming years. Uncertainty also exists in flag states, coastal states, or
56

2000 2005 2010 2015 2020 2025
0
1
2
3
4
5
year
Sulfur,%
Global
Subject to 2018 review
SO
x
Emission Control Areas
Figure 4.3: MARPOL Annex VI SOx Emission Limits (DieselNet, 2011)
individual ports who may decide to set their own regulatory emission limits (Balland
et al., 2013; Stopford, 2009). In the Northern European ECA a cap-and-trade market
has been discussed as an alternative approach as opposed to the top-down command
and control mechanism that currently regulates each individual ship (Nikopoulou
et al., 2013).
These regulations can, in many cases, hinder the proﬁtability of the shipping compa-
nies (Stopford, 2009). In some cases, vessel owners have applied for temporary exten-
sions, and when not granted, have been forced to change their compliance strategy,
costing millions of dollars (Schuler, 2014a). Some have feared that these regulations
are so costly that some companies may intentionally skirt the rules, leaving those in
compliance at a serious competitive disadvantage (George, 2014). These factors add
to the risk that owners and operators must manage in order to remain proﬁtable.
Other forms of risk that need to be accounted for include: freight rate risk, operating
cost-risk, or interest rate risk (Alizadeh and Nomikos, 2009; Psaraftis et al., 2012).
This risk may be compounded by imprecise or incomplete information regarding the
fuel or even the vessel itself (Buckley, 2008; Yang et al., 2009). Thus, identifying the
57

optimal decision for compliance in the face of these risks and uncertainties is not only
challenging but highly important to remain economically competitive.
4.1.2 Compliance Mechanisms
The MARPOL Annex VI regulation is a top down regulation that directly limits the
amount of pollutants a vessel is allowed to emit; however, the regulation does not
explicitly state how each vessel may meet these standards. Currently there are four
compliance pathways available to vessels: (1) reduce vessel transit speed (Ship and
Bunker, 2013), (2) burn distillate fuel, such as marine gas oil (MGO) or marine diesel
oil (MDO) (Bengtsson et al., 2011), (3) use liquefied natural gas (LNG) as a bunker-
ing fuel, or (4) install SOx scrubbers or selective catalytic converters (SCRs) for NOx
(Santala, 2012; Andersson and Winnes, 2011). While all four are potential avenues
for compliance, they each face technological and economic challenges. Fathom Ship-
ping (2014) summarizes many of the issues with compliance, including: the rising
cost of bunker fuel and transportation, the practicality and costs of retrofitting ves-
sels, the mechanical problems arising from fuel switching, fuel availability issues, the
probability of losing vessel power, competitive disadvantage with making the wrong
compliance choice, the changes to bunker delivery notes, and the economic issues with
supplying abundant and adequate lubricant. Thus, deciding on the best solution for
a given vessel is a challenging process due to the vast number of possible compliance
strategies (Balland et al., 2013).
The cheapest solution that requires no additional technology upgrades is reducing
vessel transit speeds, known as slow steaming. Slow steaming is only viable in the
short term, however, as the emissions reductions obtained are not sufficient to meet
future limits. Slow steaming also has the additional problem of affecting transport
58

schedules and thus costs (Ship and Bunker, 2013). Vessels that choose to slow steam
will need to either de-rate their engine or adjust their engine lubrication in order to
prevent premature degradation and wear on the engine (Lack et al., 2012).
Instead of slow steaming, distillate fuels are one popular option being discussed.
Distillate fuels are a more refined fuel than the standard residual crude oils, such
as heavy fuel oil (HFO), that have been used by many vessels for decades. There
are many distillate fuels available, the more common of which are MDO and MGO
(MDO is just a mixture of HFO and MGO) (El-Gohary, 2012). For the most part,
these fuels are able to run on standard engines without expensive upgrades. How-
ever, extended use of distillate fuels on standard engines can cause problems from
the lower viscosity compared to heavier crude oils, such as HFO (ABS, 2010). These
distillate fuels also tend to be more limited in supply, more expensive, and may not
meet impending 0.1% sulfur content without some form of SO2 abatement technology
(Bengtsson et al., 2011). Since, in general, over half of a ships operating costs are
fuel, these more expensive fuels struggle to be economically viable (Lin and Lin, 2006).
Despite the fact that roughly 95% of the world’s shipping fleet has traditionally run
on diesel fuel (Nikopoulou et al., 2013), many have looked to switching to LNG as a
logical choice from both an environmental and economic perspective (Banawan et al.,
2010). LNG is able to comply with both 2015 SOx regulations and Tier III NOx
regulations without the need for secondary emission treatment systems (Bengtsson
et al., 2011). LNG fuel is not only cheaper now, but the estimated maintenance costs
for a LNG engine can be as much as 1/3 that of traditional diesel engines, offering
the potential for significant economic savings (Banawan et al., 2010).
However, switching to LNG as a primary fuel can have drastic implications on the
59

ship as a whole. The required volume for LNG fuel tanks can be as much as 3-4
times that of standard bunker oil, plus the ship still needs the ability to carry the
required amount of bunker oil in cases where LNG may not be available (Rynbach,
2014). This is on top of the auxiliary equipment that is necessary, such as gas supply
piping, gas detection and exhaust ventilation systems, and other components. When
changing from diesel mode to LNG, the engine must be running at greater than 20%
maximum load to avoid misfiring, which would cause a loss of power. It has also been
estimated that the specific fuel consumption for a converted LNG engines is roughly
23% higher than that of traditional diesel engines (Banawan et al., 2010). Switching
to LNG can drastically affect the number of TEUs a given containership may be able
to carry, which causes lost potential revenue to the ship owner or operator. This
lost revenue is only potential, as most vessels do not necessarily leave port at full
capacity due to market conditions (Almeida, 2014) or port draft restrictions, as may
be the case for the very large cargo ships (Schuler, 2014b). These technical reasons
have caused estimations of shipbuilding costs to be 20-25% higher than ships with
conventional engines (Nikopoulou et al., 2013). There are also supply chain issues,
as the regulatory environment and infrastructure for storage and bunkering of LNG
fuel is still under development (Bengtsson et al., 2011; Nikopoulou et al., 2013).
Another option for limiting the emissions of these pollutants is by using some form
of exhaust gas scrubber or selective catalytic converter (SCR) system. Scrubbers
are primarily used to reduce SO2, while SCRs work well in limiting NOx emissions
(ABS, 2013; Andersson and Winnes, 2011). Scrubbers have been developed and
matured over the years on land based factories, and have recently been introduced
into the maritime sector. The use of scrubbers allows the vessel to operate using
cheaper bunker fuel, such as HFO, without any required changes to the prime mover.
However, scrubbers may have high purchase, installation, and operation costs (ABS,
60

2013). While SCRs are versatile in the type of vessel in which they can be installed,
technical problems arise when operating at low engine loads or with lower exhaust
gas temperature (Andersson and Winnes, 2011; Bengtsson et al., 2011).
4.2 Case Study: Designing for Evolving Emission Control
Area Regulations
This case study is designed to show the utility of the Monte-Carlo approach to the
SC-MDP framework in a maritime example that involves design and operating con-
siderations in the face of uncertain evolving ECA regulations. Nielsen and Schack
(2012) have also examined compliance strategies for vessels facing ECA regulations.
Their work included a deterministic economic analysis with sensitivity studies, as
opposed to the stochastic, temporal approach presented here. The specific case study
that follows is a modification and extension of the one originally developed by Kana
et al. (2015) and presented at the Twelfth International Marine Design Conference.
This paper proposes the use of Monte Carlo simulations to the SC-MDP framework
to this case study to gain a deeper understanding of the effects of uncertainty and
how they may change optimal decision making behavior.
4.2.1 Fixed model parameters
A notional 13,000 TEU containership routed between Rotterdam and China is ex-
amined. The route covers 22,000 nm, of which 1,100 nm is a designated ECA zone
(IMO, 2008). This ECA coverage increases to 6,800 nm of the total route in a single
year. The specifics behind when exactly the regulation changes is described in Sec-
tion 4.2.3 Monte Carlo Parameters. Two drafts are studied to simulate a full load
traveling to Rotterdam, and a partial load (or back-hauling) back to China (Table
61

4.3). The vessel is at sea for a total of 290 days per year, to account for lost time in
port and dry-docking.
Table 4.3: The vessel principal characteristics for the notional 13,000 TEU contain-
ership
Length Greater than 300 m
Beam Less than 45 m
Draft - full load 13.0 m
Draft - partial load 11.5 m
Block coefficient (Cb) 0.61
Displacement (∆) - full load 112,000 MT
Displacement (∆) - partial load 99,000 MT
Ship brake power Greater than 67,000 kW
The Holtrop and Mennen (1982) method was used to estimate the required brake
power for speeds between 12 and 24 knots, while estimates from MAN B&W and
Wartsila were used to estimate base specific consumption (MAN B&W, 2012; Wart-
sila, 2014). Combining both, the fuel consumption was calculated for all three fuels
and for both drafts (Figure 4.4). When operating in dual fuel mode, the engine burns
95% LNG and 5% HFO as a pilot fuel, which is in line with estimates made by both
MAN B&W and Wartsila (MAN B&W, 2012; Wartsila, 2014).
4.2.2 Markov decision process framework
The details of how the states, actions, transition probabilities, and rewards are defined
is presented in the following section.
4.2.2.1 States
There are eight possible states, split between three state variables. The three state
variables are:
62

12 14 16 18 20 22 24
50
100
150
200
250
300
Speed (kts)
Fuelconsumption(tons/day)
T = 13m, HFO
T = 11.5m, HFO
T = 13m, MDO
T = 11.5m, MDO
T = 13m, LNG
T = 11.5m, LNG
Figure 4.4: Fuel consumption curves for three diﬀerent fuels and two drafts. The
curves were developed using both the Holtrop and Mennen method as well as esti-
mated from MAN B&W and Wartsila.
1. The amount of ECA coverage. The two possibilities for ECA coverage are 1,100
nm and 6,800 nm.
2. The engine installed. The two types of possible engines are single fuel and dual
fuel.
3. The bunker fuel type. The two fuel options are: 1) a combination of LNG and
HFO, and 2) a combination of MDO and HFO. The LNG and HFO option is only
valid when the dual fuel engine is installed. When LNG is not available, the dual
fuel engine will burn MDO and HFO instead (El-Gohary, 2012). The MDO and
HFO option is valid for either engine. When running this fuel combination, the
engine alternates between burning MDO in the ECA zones, and HFO elsewhere.
4.2.2.2 Starting State
The simulation begins with an ECA coverage of 1,100 nm and a single fuel engine
installed that burns MDO and HFO.
63

4.2.2.3 Actions
Four possible actions are available to the vessel operator when the vessel arrives in
port:
1. Do not switch engines, and try to purchase LNG fuel
2. Do not switch engines, and purchase MDO fuel
3. Switch to a dual fuel engine, and try to purchase LNG fuel
4. Switch to a dual fuel engine, and purchase MDO fuel
The action “Do not switch engines, try to purchase LNG fuel” is only available once
a dual fuel engine is installed. The action “Switch to a dual fuel engine, and purchase
MDO fuel” is included to account for possible situations where the preferred decision
is to retroﬁt engines immediately in preparation for future lower LNG prices. The
preferred decision is the one that minimizes cumulative lifecycle cost.
4.2.2.4 Transition Probabilities
The state transition probabilities are deﬁned as follows:
• The probability of transitioning between an ECA coverage of 1,100 nm and an
ECA coverage of 6,800 nm happens according to the inputs selected from the
Monte Carlo simulations, as described in Section 4.2.3 Monte Carlo Parameters.
• The probability of transitioning from the single fuel engine to the dual fuel
engine is deterministic based on the preferred action.
• The preferred fuel type is chosen according to both the preferred decision and
the supply chain risk. When the vessel wishes to purchase LNG fuel, but it is
unavailable, it will purchase MDO instead.
64

4.2.2.5 Rewards
The rewards are defined by the cost function given in Equation 4.1. The costs are
calculated after each leg and are accumulated across the lifecycle of the vessel.
min(fuel cost + opportunity cost + retrofit cost) (4.1)
• The fuel cost is calculated via Equation 4.2. The given percentage accounts for
either the size of the ECA coverage or the dual fuel mixture.
fuel cost = fuel consumption ∗ # of sea days ∗ fuel price ∗ given % (4.2)
• The opportunity cost accounts for the lost potential revenue from the LNG fuel
tanks.
• The retrofit cost is the cost of converting to a dual fuel engine.
4.2.3 Monte Carlo Parameters
Various economic, regulatory, and supply chain scenarios were modeled as part of
the Monte Carlo simulations. The economic parameters that were varied include:
engine conversion costs, fuel prices, freight rates, and interest rates. The cost of
converting to a dual fuel engine was estimated between $220-$340/kW (Banawan
et al., 2010). This estimate includes all the auxiliary equipment necessary to fully
install and operate on LNG fuel. With an engine power greater than 67,000 kW, total
engine retrofit cost was modeled with a uniform distribution between $14-$23 million.
The fuel prices for HFO, MDO, and LNG were assigned normal distributions with
means of US $650/ton, $950/ton, $500/ton respectively, and standard deviations of
US$50/ton. While more advanced fuel projection models exist, for the purposes of
65

this case study, this fuel cost model is suﬃcient in showing both the utility of Monte
Carlo simulations as well as conclusions regarding sensitivity of the fuel prices.
Freight rates were developed from historical data from UNCTAD (2014) shown in
Figure 4.5. Rates from China to Rotterdam were modeled as a normal distribution
with a mean of US$1,500 and a standard deviation of US$285. Likewise, the rates
from Rotterdam to China were also set as a normal distribution, however, with a
mean of US$800 and a standard deviation of US$125.
2000 2002 2004 2006 2008 2010
500
1000
1500
2000
Year
FreightRate($/TEU)
Europe to Asia
Asia to Europe
Figure 4.5: Historical average of freight rates (UNCTAD, 2014). The Monte Carlo
simulations assumed a normal distribution of freight rates from Asia to Europe with
a mean of US$1,500 and standard deviation of US$285. From Europe to Asia the
mean was set to US$800 with a standard deviation of US$125.
In addition to the freight rate uncertainty, there is also uncertainty associated with
the lost revenue stemming from installation of the LNG fuel tanks that reduce cargo
capacity. To model this, the capacity for 244 TEUs is assumed to be lost to accom-
modate the required LNG fuel tanks and equipment. This lost capacity, however,
may not necessarily lead to lost revenue potential. Ships are rarely fully laden due to
market conditions or port draft restrictions (Almeida, 2014; Schuler, 2014b). For this
case, 244 TEUs represents less than 2% of total TEU capacity. According to Alpha-
liner (2015), the average vessel capacity for traveling from China to Northern Europe
66

is 88% with a standard deviation of 7.5% (Figure 4.6). Back haul load capacities
are typically much less in the range of 50 to 70% (Søndergaard et al., 2012). Thus,
lost revenue only comes into play when market conditions dictate that vessel load
conditions are above 98%. The Monte Carlo simulations were structured to match
this.
2010 2011 2012 2013 2014 2015
60
70
80
90
100
Percentfullyloaded
Figure 4.6: Average vessel load factors from 2010 to 2015 (adapted from Alphaliner
(2015). Vessel load factors have averaged 88% with a standard deviation of 7.5%.
Interest rates were modeled as a normal distribution with a mean of 7% and a standard
deviation of 1%. The discount factor used in the MDP is related to the interest rate
by Equation 4.3 (Puterman, 2005) where i is the interest rate and γ is the discount
factor.
i = (i/γ) − 1 (4.3)
Modeling the regulatory uncertainty was more diﬃcult due to the challenge in quanti-
fying the probability of when the ECA regulation will actually change. The attempt
at quantifying this uncertainty stems from the desire to examine its sensitivity on
the recommended decisions, as opposed to claiming that this particular uncertainty
model is actually how the regulations will behave. At the start of the simulation,
67

ECA covers 1,100 nm of the total route. The specific year in which the ECA coverage
increases from 1,100 nm to 6,800 nm varies depending on the simulation run. The
range is uniformly distributed between 3 years and 10 years. There is also uncertainty
associated with whether the regulation actually changes at that given year. This un-
certainty is uniformly distributed between 0% and 100%. For example, one scenario
may be that there is a 75% probability that the ECA regulation will increase 5 years
from now.
While infrastructure and regulations are developed for LNG bunkering facilities, there
is great uncertainty whether the fuel will be available should a ship go into port and
want to purchase LNG. While other literature aims to quantify this development
(Danish Maritime Authority, 2012; Lee, 2014), this research is instead focused on
the implications of supply chain risk on the decisions. To simulate supply chain risk
associated with uncertainty of LNG availability, the probability of obtaining LNG in
Rotterdam is modeled uniformly between 50% and 100%, while the probability of
obtaining LNG in China is uniformly distributed between 0% and 100%.
4.3 Results
Three sets of results were explored, covering an examination of the decisions, the
economic costs, and the specific design drivers leading to both the decisions and eco-
nomic costs. Before examining the results, the system was tested for convergence. For
each simulation run, there is some uncertainty that at any given time the system may
be in a given state or that a given action may be selected. This uncertainty is in the
set [0, 1], and a running average of this uncertainty is calculated for each successive
simulation run. The cumulative incremental difference between the ith
simulation
run and the ith
− 1 run is then calculated. For Figure 4.7, the maximum cumulative
68

difference between all actions, all states, and all speeds is plotted. As is shown, the
simulation converges visually in less than 400 simulations; however, 1,000 simulations
were performed to ensure confidence of convergence. The inset of the figure shows
that after 1,000 simulations the model has consistently converged to a value less than
10e-4, which was considered acceptable convergence.
200 400 600 800 1000
0
0.1
0.2
0.3
0.4
0.5
Simulation Number
CumulativeIncrementalDifference
900 950 1000
4
6
8
10
x 10
−4
Figure 4.7: Convergence of the Monte Carlo simulations. The close up shows that
after 1,000 simulations, the model has consistently converged to a value less than
10e-4, which is deemed an acceptable range.
4.3.1 Decisions
The key decision defined in this problem is not just whether the ship owner should
convert to an LNG engine, but also when it may be best to perform the conversion.
The SC-MDP is able to identify when specific actions are preferred throughout the
lifecycle of the vessel, while the Monte Carlo simulations provide the likelihood that a
given operating environment may be in place to yield such actions. Thus, the frame-
work presented in this paper enables the ability to identify the percent of time a given
action is optimal, and when throughout the lifecycle of the vessel it may be optimal.
To show this, Table 4.4 presents the percent of time it is optimal for the vessel to a)
69

never convert engines, b) convert engines as soon as possible, and c) convert engines
eventually. Converting engines as soon as possible is defined as converting the en-
gines within the first two voyages, while converting engines eventually is defined as
converting engines at some point after that. “Convert engines eventually” is included
to account for those situations where it may be best to hold off on converting the
vessel until the ECA regulation has increased.
Table 4.4: Percent of time a given action is optimal. The “convert engines eventually”
action means that it is best to convert to an LNG engine at a time after the first two
voyages.
Speed Never Convert engines Convert
(kts) convert as soon as engines
engines possible eventually
12 67% 33% 0%
14 30% 70% < 1%
16 8% 92% < 1%
18 3% 96% < 1%
20 < 1% 99% < 1%
22 < 1% 100% 0%
24 0% 100% 0%
As speeds are increased, the probability that it is best to convert engines increases.
For 12 knots it is best to keep the single fuel engine for 67% of the simulations, and
this percent drops significantly with only a small increase in speed. For 16 knots
and faster the percent of time it is best to keep the single fuel engine is less than
8%, and at the highest speed of 24 knots, it is never optimal to keep the single fuel
engine. The probability that it is best to convert to a dual fuel LNG engine as soon
as possible follows nearly the exact opposite trend, with the probability increasing
with increasing speed. Rarely is it preferable to delay converting the engines. Kana
et al. (2015) discussed in detail the situations where it is preferred to convert engines
later in the lifecycle, such as when the regulation changes. This analysis, however,
shows that those situations are rare, occurring less than 1% of the time in only four of
70

the speeds tested. No matter the speed, there is always a possibility that converting
to an LNG engine is preferred; however, as the speed increases eventually there is a
point where it is never preferred to keep the single fuel engine.
4.3.2 Economic Costs
Understanding what decisions will likely be made is only part of the problem; the
decision maker must also understand the range of costs that are likely to occur given
each decision scenario. The expected net present lifecycle costs are given in Figure
4.8. Figure 4.8.a shows the results for a speed of 12 knots, where it is clear there is
a large spread of potential costs, given differing starting scenarios. The large beige
region signifies the extreme limits, displaying the maximum and minimum, while the
blue region shows one standard deviation above and below the mean. The high costs
at year zero come from the conversion costs during those situations when it is best to
convert to an LNG engine as soon as possible. The solid black line is the mean cost,
while the dashed red line is the median cost of all simulations.
Figure 4.8.b shows the costs for a speed of 22 knots. Even at the higher speeds, there
is still a possibility that it is best to keep the single fuel engine installed. This is
shown by the small beige area around US$0 during the first year. This is also shown
in Table 4.4, where it is apparent that this situation occurs < 1% of the simulations.
The costs for speeds of 14, 16, 18, and 20 knots follow a similar trend as that of 22
knots, however, their specific values vary with the speed. Figure 4.8.c shows the costs
for 24 knots, where it is apparent that it is never beneficial to keep the single fuel
engine installed.
The final accumulated lifecycle costs after 20 years for all speeds is given in Figure
71

0 5 10 15 20
0
2
4
6
8
10
x 10
7 Expected Cost
Year
expectednetpresent
cost(US$)
12 knots
a
0 5 10 15 20
0
5
10
15
x 10
7
Year
expectednetpresent
cost(US$)
Expected Cost
14 knots
0 5 10 15 20
0
0.5
1
1.5
2
x 10
8
Year
expectednetpresent
cost(US$)
16 knots
0 5 10 15 20
0
0.5
1
1.5
2
2.5
x 10
8
Year
expectednetpresent
cost(US$)
18 knots
0 5 10 15 20
0
1
2
3
4
x 10
8
Year
expectednetpresent
cost(US$)
20 knots
0 5 10 15 20
0
1
2
3
4
5
x 10
8
Year
expectednetpresent
cost(US$)
22 knots
b
0 5 10 15 20
0
2
4
6
x 10
8
Year
expectednetpresent
cost(US$)
Extreme limits (min/max)
One STDEV spread
mean cost
median cost
24 knots
c
Figure 4.8: The expected net present cost (US$) for the range of speeds. 4.8.a shows
the expected range of costs for the slowest speed tested of 12 knots. 4.8.b shows the
range of potential costs for a high speed of 22 knots, where there is a small possibility
that is it preferable to keep the single fuel engine. 4.8.c displays the costs for the
highest speed of 24 knots where is always preferable to switch to a dual fuel engine as
soon as possible. Note the variations in the y-axis between ﬁgures to show speciﬁcs
within each speed.
72

4.9. The edges of the box represent the 25th and 75th percentile respectively, while
the centerline is the median. The red marks that extend beyond the whiskers are
labeled as outliers that fall outside 2.7 standard deviations of the data. As seen, the
costs do not grow linearly with speed, instead they increase similar to the speed power
consumption curves given in Figure 4.4. The variation of costs within each speed is
large, and increases with increasing speed. That is, for a speed of 12 knots, there is
just over a 45% variation between the lowest possible cost and the highest cost, while
at 24 knots, that variation grows to nearly 68%. Thus, both the percent variation
and the gross magnitude of the variation grows with increasing speed. For 12 knots,
the variation is between US$52 million for the lowest cost and US$83 million on the
high end with a median of US$68 million. For 24 knots, that variation increases to
a minimum of US$265 million for the lowest cost and US$536 million for the highest
cost with a median of US$400 million. Finally, the outliers for the faster speeds all
lay on the high end of the costs. Since these costs were calculated via the MDP
framework, each result is considered the best scenario given the set of inputs. Thus,
should a decision maker not follow the best decision pathway, they can expect their
costs to be higher than what is displayed here.
For each speed the average savings and time to pay back the engine conversion costs
was calculated against a baseline scenario where the vessel continues to operate on a
single fuel engine throughout its lifecycle and alternates between operating on MDO
and HFO fuel. This calculation only accounts for those simulations where it is best
to convert to an LNG engine. Thus, for situations where converting engines is only
preferable a small portion of the time, the savings only account for those times when
it is preferable to convert engines. As shown in Table 4.5, for speeds less than 16
knots the average savings are less than US$16 million with a payback time of longer
than 10 years. For the highest speed of 24 knots, the savings are over US$100 million
73

12 14 16 18 20 22 24
1
2
3
4
5
6
x 10
8
Speed (kts)
AccumulatedLifecycle
Cost(US$)
Figure 4.9: The accumulated lifecycle cost varies greatly both between speeds and
within individual speeds. The spread of possible costs grows both in magnitude and
in percent as speeds increase. Outliers tend to fall on the upper end of costs for the
faster speeds.
with a payback of less than 3 years. Due to the large variation in costs, the actual
savings and payback period may vary from this average.
Table 4.5: Average savings and payback periods for all speeds. Savings are increased
and payback periods are reduced as speeds are increased.
Speed Average savings Average payback
(kts) (US$MM) period (years)
12 2 18.1
14 7 14.4
16 16 10.6
18 29 7.5
20 49 5.1
22 76 3.5
24 108 2.7
4.3.3 Decision Drivers
Sensitivity studies were performed on the ECA regulation, the supply chain risk,
and the fuel prices to determine the drivers behind the decisions. The analyses were
74

performed by holding constant each parameter except the variables of interest. The
constant parameters were fixed near their designated mean value, as given in Table
4.6. 1,000 simulations were run for each sensitivity test.
Table 4.6: Parameters used for the sensitivity studies. The variables for the regulation
and LNG supply chain sensitivity were uniformly distributed, while the fuel price
sensitivity used a normal distribution.
Parameter ECA LNG Supply Fuel Price
Regulation Chain Sensitivity
Sensitivity Sensitivity
Engine conversion cost $18.8M $18.8M $18.8M
Interest rate 7% 7% 7%
Lost TEUs to fit LNG equipment 18 18 18
(to Rotterdam)
TEU freight rate to Rotterdam $1,500 $1,500 $1,500
TEU freight rate to China $800 $800 $800
Probability of obtaining LNG 0.5 [0,1] 0.5
in China
Probability of obtaining LNG 0.75 [0.5,1] 0.75
in Europe
HFO price $650 $650 µ = $650, σ = $50
MDO price $950 $950 µ = $950, σ = $50
LNG price $500 $500 µ = $500, σ = $50
Year ECA coverage may increase [3,10] 5 5
Probability that ECA will increase [0,1] 0.5 0.5
at given year
4.3.3.1 ECA Regulation Sensitivity Study
The ECA regulation sensitivity showed clear results. First, there is no variation in
the individual speeds in regards to the best decision. A clear bifurcation becomes
apparent at 14 knots (Table 4.7). Below 14 knots it is always best to maintain the
single fuel engine, while at and above 14 knots it is always best to convert to a dual
fuel LNG engine. There were no instances during this study when it is best to delay
converting engines beyond the first two voyages. This study also showed almost no
variation in the cost both through the lifecycle, and as a cumulative amount (Figure
75

4.10). The median cost for each speed remained unchanged in this study. Thus, this
study shows that the variation in the results, both in the decisions and the costs, is
not due to the uncertainty in the ECA regulation implementation. There were no
signiﬁcant changes in the average savings or payback period as compared to the full
analysis.
Table 4.7: Sensitivity due to uncertainty in the ECA regulation implementation. For
12 knots it is always best to keep the original engine, while above 12 knots it is always
best to convert to a dual fuel LNG engine as soon as possible.
12 100% 0% 0%
14 0% 100% 0%
16 0% 100% 0%
18 0% 100% 0%
20 0% 100% 0%
22 0% 100% 0%
24 0% 100% 0%
12 14 16 18 20 22 24
1
2
3
4
x 10
8
Speed
Cost(US$)
Figure 4.10: The sensitivity study on the ECA regulation showed almost no variation
in the accumulated lifecycle costs for individual speeds. The median cost for each
speed, however, remained unchanged as compared to the original analysis.
76

4.3.3.2 LNG Supply Chain Risk Sensitivity Study
The effect of the LNG supply chain risk was tested as to its impact on the results.
The decisions are similar to that from the full simulation (Table 4.8). For the slow
speeds of 12, 14, and 16 knots, the probability of it being best to never convert en-
gines is reduced between 3-13%, thus increasing the probability it is best to convert
engines as soon as possible. For 18 knots and faster, the results were very similar to
the original results, varying at most 3% from the original simulation. No instances
arose where it is preferable to delay retrofitting the engine until later.
Table 4.8: Sensitivity due to LNG supply chain risk. The probability of it being best
to never convert engines is reduced between 3-13%, compared to the original analysis.
12 64% 36% 0%
14 17% 83% 0%
16 2% 98% 0%
18 0% 100% 0%
20 0% 100% 0%
22 0% 100% 0%
24 0% 100% 0%
Varying the availability of LNG in port causes a large spread in lifecycle costs (Figure
4.11). This variation, however, does not account for the full variation that is present
in the original simulation. The percent variation for 12 knots is just over 10%, while
for 24 knots the variation is only 36%. Across all speeds, the cost variation only
accounts for just over 60% of the total variation shown in the full simulation. There
are also very few outliers. Lastly, as with the ECA regulation sensitivity study, there
were no significant changes in the average savings or payback period as compared to
the full simulation.
77

12 14 16 18 20 22 24
1
2
3
4
x 10
8
Speed
Cost(US$)
Figure 4.11: The sensitivity study on the LNG supply chain risk shows a slightly
smaller spread than the original analysis. The median cost for each speed also re-
mained unchanged as compared to the original analysis.
4.3.3.3 Fuel Price Sensitivity Study
The fuel prices were tested as to their effect on both the decisions and the life-cycle
costs. This study revealed that fuel price variation is one of the reasons it may be best
to delay retrofitting the engine until after the first two voyages (Table 4.9). Sensitivity
studies on the ECA regulation and LNG supply chain did not reveal any instances
when it would be best to delay retrofit, while this studied showed the opposite. Vari-
ations in the fuel prices displayed a similar trend to that of the original analysis, in
that for speeds between 14 knots and 20 knots, there are instances-albeit rare-that
delaying the engine retrofit is the best option.
This study also showed large variation in regards to the lifecycle costs (Figure 4.12).
Unlike the previous analyses, the variation here was consistent across all speeds, aver-
aging between 25-29% between lowest and highest quartile. Combined, this variation
accounts for roughly 70% of the total variation in the model. This sensitivity study
also had outliers present, meaning that while most of the data is spread through
78

Table 4.9: Fuel Price sensitivity. Fuel price variability may be one of the causing
leading to delaying engine retroﬁts beyond the ﬁrst two voyages.
12 72% 28% 0%
14 11% 89% < 1%
16 < 1% 99% < 1%
18 < 1% 99% < 1%
20 < 1% < 100% < 1%
22 0% 100% 0%
24 0% 100% 0%
a consistent distribution, variable fuel prices can lead to lifecycle costs that are far
outside what is expected.
12 14 16 18 20 22 24
1
2
3
4
x 10
8
Speed
Cost(US$)
Figure 4.12: The sensitivity study on the fuel prices shows a slightly smaller spread
than the original analysis. The median cost for each speed remained unchanged as
compared to the original analysis.
4.4 Discussion
There are several points worthy of discussion following the results of the model.
79

1. This framework is intended to provide insight into decision making, not the
decision itself. It is still up to the individual decision maker to decide whether
or not to follow the results of the model. For instance, it is not expected that the
decision maker will actually follow the results in situations where the savings
are small and the payback time is long, as is the case for the slower speeds. As
the savings increase and payback time decreases with higher speeds, it is at the
discretion of the decision maker to decide for themselves whether they wish to
convert engines or not.
2. There are still inherent risks associated with making decisions. Despite giving
the best decision pathway for each scenario, this framework does not remove
all risks that vessel owners face. As with all probabilistic models, there is still
a chance that the actual situation may vary from the normal bounds of the
results, possibly causing great economic harm.
3. The framework is dynamic. It can be adapted to include any sub models for
the given scenario. It would be beneficial to include a more advanced fuel
cost and freight rate model, supply chain risk model, and measured vessel fuel
consumption curves for commercial use. While these underlying models appear
simplistic, the overarching theory and methods still hold.
4. The specific case study provides key insights despite not being fully inclusive.
The case study did not account for the potential profit loss from slow steaming.
Also, this chapter only discussed decisions related to LNG fuel; however, other
ways of meeting upcoming ECA regulations include the use of distillate fuels, or
installation of scrubbers. Decisions are also severely impacted by whether the
vessel is under charter, and the type of charter. These points do not, however,
negate the applicability of the insights gained from the framework.
80

The objective of this framework was to provide the quantitative information neces-
sary for the decision maker to make sound decisions under deep uncertainty.
4.5 Conclusion
This chapter demonstrated how Monte Carlo simulations applied to the ship-centric
Markov decision process can be used for eliciting design and decision making in-
sight under uncertainty for vessel designers, owners, and operators. The SC-MDP
framework was used to identify when certain decisions are preferred throughout the
lifecycle of the vessel as well as the lifecycle costs associated with making those de-
cisions. Monte Carlo simulations were used to move beyond individual probabilistic
values that are common in many MDP applications in order to enable true stochastic
analysis. These simulations were also used to develop probabilistic distributions of
not only the decisions but also the lifecycle costs.
New insights were gained regarding lifecycle decision making for containerships facing
upcoming emissions regulations. Uncertainty regarding the regulation showed to have
little eﬀect on when certain decisions should be made as well as contributing little to
the uncertainty in the lifecycle costs. Uncertainty over the availability of LNG as a
bunker fuel and fuel prices showed to be more signiﬁcant drivers of the results.
This chapter provided a viable analysis technique to handle decision making problems
under a multitude of uncertain parameters. However, to address the limitations
outlined in Section 3.3, eigenvalue spectral methods are warranted.
81

CHAPTER V
Spectral Analysis Case Study 1: Ship Egress
Analysis and General Arrangements Design
This chapter proposes a novel means of performing a ship egress analysis by applying
eigenvalue spectral analysis to the ship-centric Markov decision process framework.
Chapter II outlined the need for a method that provides insight into the implications
of decisions and the need to understand their impact on the system. These implica-
tions will provide the Why that is currently lacking in large scale engineering decision
making processes. To that end, the objectives of this chapter are:
1. Demonstrate the applicability of eigenvalue and eigenvector spectral analysis to
the SC-MDP framework in general.
2. Demonstrate how the damping ratio can be used to identify and quantify
changes in sets of decisions as well as be used to identify the system attributes
leading to those changes.
3. Demonstrate how the principal eigenvector can be used as a leading indicator
metric for steady state analysis in quantifying the impact of decision making.
4. Highlight the relationship between the damping ratio and the the principal
eigenvector.
82

5.1 Background
Understanding evacuation patterns and egress routes is arguably one of the most im-
portant aspects of ship design affecting safety of those on board (Guarin et al., 2014).
Emergency situations such as fires, compartmental flooding, damage due to collisions,
or even ballistic damage, happen on vessels. Understanding how individuals on the
vessel react and move about to safety zones, muster points, or life boats during these
situations is important to save lives and minimize injuries. The IMO has recognized
evacuation and egress analysis as an important aspect that needs to be regulated for
the safety of mariners (IMO, 2007).
Many models have been developed to approach this problem. Two types of methods
have been developed: those focused on generating and analyzing the physical layout
of the vessel (Andrews et al., 2012), and those focused on evacuation patterns of the
individuals on board (Guarin et al., 2014; Qiao et al., 2014; Vanem and Skjong, 2006).
Methods focused on the layout of the vessel are defined as solution-centric by this
dissertation, meaning they are focused on the rooms themselves and their physical
layout throughout the vessel. While methods focused on evacuation routes are typ-
ically computationally expensive, and can involve multi-agent simulations (Guarin
et al., 2014), or optimization methods (Qiao et al., 2014).
Both type of analyses require a full general arrangement of the layout of the vessel
as well as the physical distribution of the crew throughout the vessel in order to run
detailed discrete event simulations to study evacuation routes (Rigterink, 2014). Two
major problems arise with this. First, during early stages of design little information
is known about the details of the general arrangement, and second, a new discrete
event simulation is required for each simulation involving a different distribution of
individuals throughout the ship.
83

The complexity of the egress problem grows with vessel size and complexity. The
problem becomes more difficult due to various passenger populations (such as able
bodied seamen, children, or handicapped individuals), the number of decks and pas-
sageways, as well as the multitude of ways emergency situations may start or percolate
through the vessel. The behavior of crew versus passengers may also be an issue. Paid
crew on a vessel who know the inner layout of the vessel well will likely react differ-
ently than passengers on a cruise ship who have been on board for only a matter
of days. Thus, for a full detailed analysis, advanced methods are necessary. While
some propose using more advanced, computationally expensive evacuation models in
earlier stages of design (Vanem and Skjong, 2006), the IMO (2007) has recognized the
importance of more simplified methods to be used in the concept stage. This has led
to the increased prominence of analyses aimed at evacuation routes and egress pat-
terns in preliminary stages of design (Casarosa, 2011; Guarin et al., 2014; Rigterink,
2014). However, finding the proper balance between computational expense, analysis
time, and model fidelity remains difficult.
This dissertation proposes a method that is inherently focused on the problem of
the decision making of individuals on the vessel as they maneuver to evacuate. This
method is focused on understanding the impact that uncertainty and external pain
may have on the decision making of individuals in the vessel. This is opposed to a
method that is more focused on generating a solution of a specific layout that best
meets the safety needs. The problem is defined as how people egress, understanding
the decisions they make under uncertainty, and the interaction between the individ-
uals themselves and the general arrangement of the layout of the vessel. To do this,
eigenvalue spectral analysis applied to the SC-MDP framework is proposed to under-
stand the implications of evacuation decision making as it pertains to ship design.
84

5.2 Case Study: Ship Egress Analysis and Ship General Ar-
rangements Design
This case study is designed to examine personnel movement inside a ship. The as-
sumption is that a fire has broken out in one of the rooms in the ship, and that
individuals need to move about the ship to find the exit. Their movement is prob-
abilistic to simulate the uncertainty associated with smoke that may be percolating
throughout the ship or uncertainty with hot door handles blocking passageways. A
utility function is used to simulate the pain the person experiences the longer they are
in the ship looking for the exit. The rooms themselves have not been designated for
a specific use for this analysis, as it is the attempt of this case study to understand
their interaction prior to designating their use. That is, these rooms have not yet
been classed as berthing, hallway, mess hall, engine room, etc.
To set up the problem, individuals are located somewhere in the eleven state envi-
ronment shown in Table 5.1. The solid black state represents an inaccessible area.
Their probabilistic position at any time is denoted by the generic state vector de-
fined in Equation 5.1. The person’s objective is to minimize their pain while heading
towards the exit. To do this they aim to maximize their cumulative reward after a
given number of steps. The SC-MDP framework will be used to determine their best
sequence of decisions, as well as their expected value. The states are defined as the
individual rooms in the ship. The individuals may move one step at each decision
epoch, and may choose from the set of actions: up, down, left, or right. However,
there is uncertainty in the individual’s movement, and the probability of actually
moving in the desired direction is only 0.8. There is a 0.1 probability of moving in
85

either direction laterally to the desired one. The transition probability in this case is
defined as p = 0.8.
Table 5.1: Visual representation of the eleven room general arrangement. The entries
include the labeling convention of the states for the following discussion.
(3,1) (3,2) (3,3) (3,4)
(2,1) (2,3) (2,4)
(1,1) (1,2) (1,3) (1,4)
s = (s1,1 s1,2 s1,3 s1,4 s2,1 s2,2 s2,3 s2,4 s3,1 s3,2 s3,3 s3,4) (5.1)
Individuals receive a reward for landing in a given state, and those rewards are given
in Table 5.2. The room with the r = −1 reward is designed to simulate the room
with a fire, while the room with the r = +1 reward is designed to simulate the safe
exit or muster point. The incremental rewards of r = −0.04 will be varied in the
following analyses, however, the r = +1 and r = −1 rewards will remain fixed.
Table 5.2: The initial rewards the individuals receive for landing in a given state. The
following analyses vary the -0.04 rewards, while the +1 and -1 rewards remain fixed.
-0.04 -0.04 -0.04 +1
-0.04 -0.04 -1
-0.04 -0.04 -0.04 -0.04
For this case study, the transition probabilities and rewards are stationary, meaning
they do not change with time. The MDP is run for 30 decision epochs, essentially
allowing individuals to take up to 30 steps to maximize their expected utility. That
expected utility, as well as the best decision paths are given in Table 5.3. The deci-
sion paths display the preferred action a person should take for each state in order to
maximize their expected utility.
86

Table 5.3: The expected utility and decision paths for p = 0.8 and r = −0.04.
Expected value
0.81 0.87 0.92 +1
0.76 0.66 -1
0.71 0.66 0.61 0.39
Decision Paths
→ → → +1
↑ ↑ -1
↑ ← ← ←
5.3 Validation
Two validation studies were performed before examining the results. First, the deci-
sion paths and expected utilities were validated against published results in Russell
and Norvig (2003), and second, the representative Markov chain was validated against
the decision matrix to ensure that it tracked the decision paths correctly.
5.3.1 Algorithm Validation
A validation study was performed on the expected utilities and decision paths against
published results in Russell and Norvig (2003). As shown in Figure 5.1 and Table 5.4
the research code matches with published results for both the expected utilities and
decision paths after 30 decision epochs.
Table 5.4: Decision path validation. The decision paths for the research code match
identically with that of the published ones from Russell and Norvig (2003).
→ → → +1
↑ ↑ -1
↑ ← ← ←
(a) Research code
(b) Results from Russell and Norvig (2003)
87

0 10 20 30
−0.2
0
0.2
0.4
0.6
0.8
1
Number of Epochs
Expectedvalue
(3,4)
(3,3)
(1,1)
(1,3)
(1,4)
(a) Research code
(b) Results from Russell and Norvig
(2003)
Figure 5.1: Utility estimate validation. Note: Russell and Norvig (2003) labeled their
states with the convention of (column, row). For instance, state (1,4) in the research
code matches with state (4,1) in the Russell and Norvig (2003) result.
5.3.2 Markov Chain Validation
The Markov chain representing the decision paths was also validated. This validation
was performed to ensure that the Markov chain does, in fact, track the system be-
havior through time correctly, as described by Sheskin (2011). To build M, the rows
from the individual action transition matrices are selected based on actions for each
state given in Table 5.3. These individual rows are placed in the same order in the
new representative Markov chain transition matrix. For instance, for state (1,1), the
row from action transition “up” is selected, while for state (1,2), the row from “left”
is selected. Note that the action transition matrix for “down” is never used because
it is never optimal for any state. This process is shown graphically in Figure 5.2. The
colors have been selected to remain consistent between various actions.
To validate this, individuals were placed at speciﬁc locations in the vessel. The initial
distribution broke the individuals up into four groups, with 25% of the total popula-
tion located in the states below in Table 5.5 (also presented in the state vector given
88

Figure 5.2: Building the representative Markov chain transition matrix, M, for p =
0.8 and r = −0.04. The colors are representative of the specific actions. Note the
action “down” is never optimal for any of the states.
in Equation 5.2).
Table 5.5: Initial placement of individuals. 25% of the individuals were placed in the
each of rooms identified.
0.00 0.00 0.00 0.00
0.25 0.00 0.00
0.25 0.25 0.25 0.00
s = (0.25 0.25 0.25 0 0.25 0 0 0 0 0 0 0) (5.2)
The location of the individuals was measured after 5, 10, and 30 decision epochs
following the decision paths as defined by the decision matrix and the representative
89

Markov chain. Their probabilistic locations are given below. For example, after 5
epochs, roughly 5% of the individuals have reached the top left state, located at (3,1),
and roughly 25% of them have reached the exit located at state (3,4). Note that due
to the combination of the uncertainty and the rewards, there is a chance that an
individual may step into the room with the fire as they find their way to the exit
room. There is no variation between final location between these two methods, and
thus the representative Markov chains properly tracks the system behavior through
time.
After 5 decision epochs:
Following decision matrix
0.049 0.130 0.354 0.248
0.021 0.088 0.022
0.004 0.032 0.008 0.045
Following representative Markov chain
0.049 0.130 0.354 0.248
0.021 0.088 0.022
0.004 0.032 0.008 0.045
0.001 0.003 0.125 0.844
0.000 0.019 0.004
0.000 0.001 0.002 0.001
0.001 0.003 0.125 0.844
0.000 0.019 0.004
0.000 0.001 0.002 0.001
0.000 0.000 0.108 0.878
0.000 0.012 0.001
0.000 0.000 0.000 0.000
0.000 0.000 0.108 0.878
0.000 0.012 0.001
0.000 0.000 0.000 0.000
5.4 Results
Three independent sets of results were explored in this case study. First, a perturba-
tion analysis was performed on the uncertainty and rewards to examine their effect
90

on both the decisions and the expected utility. This was done without the use of
spectral analysis. Second, the ﬁrst spectral analysis metric was explored: the damp-
ing ratio. The third analysis explores the utility of the principal eigenvectors as a
leading indicator metric for analyzing the impact of design decisions.
5.4.1 Variations in the Decision Paths
The expected utilities and decision paths presented above in Table 5.3 are unique
only to the given rewards and uncertainty. Variations in either the rewards or the
uncertainty will change both the expected value and the decisions. The expected
value and decisions for the original case setup are presented again below in Table 5.6.
By increasing the uncertainty in the individual’s movement to p = 0.7, thus reducing
the probability of moving in the desired direction, there are noticeable changes. The
changes in the decisions are evident in states (1,4) and (2,3), which are highlighted by
the double arrow in Table 5.7. Due to the increased uncertainty, the best decision is
no longer to take the risk of stepping near the room with the ﬁre (the r = −1 state).
Instead, it is best to take a sequence of actions that will always avoid the -1 state.
This higher uncertainty also reduces the expected utility for all states.
Table 5.6: Expected utility and decision paths for p = 0.8 and r = −0.04.
Expected value
0.81 0.87 0.92 +1
0.76 0.66 -1
0.71 0.66 0.61 0.39
Decisions Paths
→ → → +1
↑ ↑ -1
↑ ← ← ←
By reducing the incremental reward to r = −0.1, it becomes more painful for indi-
viduals to remain in the ship longer. Here there are changes in states (1,2) and (1,3),
again highlighted by the double arrows in Table 5.8. Due to the increased penalties
for each step, the best decision pathway is no longer to take the long way around
91

Table 5.7: Expected utility and decision paths for p = 0.7 and r = −0.04. Note the
reduction in expected utility due to increased uncertainty and changes in decision
paths highlighted by the double arrows.
Expected value
0.75 0.82 0.87 +1
0.69 0.56 -1
0.62 0.56 0.51 0.23
Decisions Paths
→ → → +1
↑ ⇐ -1
↑ ← ← ⇓
state (2,2), but instead it is now optimal to risk the chance of taking a misstep to the
-1 state by going counter-clockwise around state (2,2).
Table 5.8: Expected utility and decision paths for p = 0.8 and r = −0.1. Note the
reduction in expected utility due to more painful rewards and changes in decision
paths highlighted by the double arrows.
Expected value
0.57 0.71 0.84 +1
0.45 0.52 -1
0.31 0.22 0.35 0.09
Decision Paths
→ → → +1
↑ ↑ -1
↑ ⇒ ⇑ ←
Finally, these decision paths may not be unique. When the incremental reward be-
comes positive, there is no penalty for individuals to move about in the ship, and
therefore no desire to proceed to either the exit (r = +1 state) or the room with the
ﬁre (r = −1 state). This may simulate a situation where the ﬁre has been completely
contained in a single room and there is no desire to either enter that room, nor pro-
ceed to the exit. The expected values will increase will each step, and presented below
in Table 5.9 is the result after 30 decision epochs when the reward is set to r = 0.1.
The decision paths only requires that states (1,4), (2,3), and (3,3) are as shown, while
all actions are optimal for the other states.
These results were developed without the use of spectral methods. Certain advan-
tages of the MDP are evident even without spectral analysis. First, the MDP is
able to quantify the expected values and identify the optimal decision paths for the
92

Table 5.9: Expected utility and decision paths for p = 0.8 and r = 0.1. Note the
large expected utility due to positive rewards and non uniqueness of the decision
paths highlighted by the multiple arrows signifying any decision is optimal.
Expected value
3.68 3.68 3.68 +1
3.68 3.68 -1
3.68 3.68 3.68 3.68
Decision Paths
↔ ↔ ← +1
↔ ← -1
↔ ↔ ↔ ↓
individuals in the face of uncertainty. The method is also able to identify variations
in the decision paths due to changes in rewards and uncertainty, as well as identify
when the system may not converge to a unique set of decisions. However, there are
also limitations to the analysis presented. Manual search or simulation is required to
find the specific regions in the uncertainties or reward values that lead to changes in
the decisions. This method is also unable to identify which attributes of the process
are driving the dynamics and which cause the changes in the set of decisions.
5.4.2 Spectral Analysis
Spectral methods are introduced as a unique contribution of this thesis as a means
to gain a deeper understanding into these variations in the decision paths, and the
driving characteristics of the system that may be causing them. Both the damping
ratio and the principal eigenvector are explored as metrics to quantify changes in
decisions and the long term implications of specific decisions.
5.4.2.1 Damping Ratio Results
The representative Markov chain was formed after the MDP was run for 30 decision
epochs. The eigenvalues were generated from this representative Markov chain, and
the damping ratio was calculated. As such, these results only represent one single
time step. The objective is to obtain the relationship between the damping ratio
93

and the system for a single time step before examining its behavior through time.
The results of the damping ratio for all four system inputs described previously are
presented in Table 5.10. As seen, small perturbations in both the uncertainty and
the rewards lead to damping ratios that vary between 1.111 and 2.288. In order to
fully understand these variations in the damping ratio, a broader examination of the
system is necessary.
Table 5.10: Damping ratio, ρ, for the four previous scenarios.
p r λ1 |λ2| ρ = λ1
|λ2|
0.8 -0.04 1 0.437 2.288
0.7 -0.04 1 0.850 1.765
0.8 -0.1 1 0.486 2.057
0.8 0.1 1 0.900 1.111
A sweep of the uncertainty and the rewards was performed to examine the relation-
ship of the damping ratio across a broad environment. The uncertainty was varied
from complete uncertainty of p = 0 to complete certainty of p = 1, while the rewards
were varied between −2.0 < r < 0.5. The results are presented in Figure 5.3.
Two major trends are apparent. First, there is a signiﬁcant drop in the damping ratio
around where the reward transitions from negative to positive. As the rewards move
close to and into the positive region, the set of decisions changes drastically, and may
even become non-unique for areas where the rewards are fully positive. Second, as
p increases, the damping ratio grows rapidly for 0.5 < p. For the region p < 0.5,
the uncertainty is so great that the individuals take a misstep more than 50% of
the time, and the damping ratio is roughly consistent across all rewards. Figure 5.3
is only intended to show the high level trends of the damping ratio across a broad
environment. A more detailed analysis of the underlying shape and behavior follows.
94

0
0.2
0.4
0.6
0.8
1−2
−1.5
−1
−0.5
0
0.5
2
4
6
8
10
Probability of moving
in desired direction (p)
Reward (r)
DampingRatio(ρ)
drop in ρ
for r~=0
rapidly increasing
ρ for 0.5<p
Figure 5.3: Damping ratio, ρ, for a range of rewards and uncertainties. Note the pre-
cipitous drop in ρ when the rewards become positive, as well as the rapidly increasing
nature of ρ for 0.5 < p.
In order to examine the genesis of the underlying shape of Figure 5.3, a closer exam-
ination of the data was performed with the uncertainty fixed at p = 0.8, while the
rewards were varied from −2.0 < r < 0.5. The results are shown in Figure 5.4. Five
step changes are apparent, located at points b, e, f, h, and i, which are associated
with r = −1.650, r = −0.453, r = −0.085, r = −0.028, and r = −0.022 respectively,
with the major drop occurring at r = −0.028.
The step changes in Figure 5.4 occur when there are changes in the decision paths,
and thus identify the specific changes in the environment that affect the set of deci-
sions. However, not all transitions in the decisions are identified as changes in the
damping ratio. There are in fact ten different sets of decision paths for an uncertainty
of p = 0.8. These sets are given in Table 5.11, and are arranged moving from left to
right in the damping ratio plot. The specific change in the decision path is identified
95

−2 −1.5 −1 −0.5 0 0.5
1
1.5
2
2.5
3
reward (r)
dampingratio(ρ)
a b
c d e
f
g h
i j k
Figure 5.4: Damping ratio, ρ, over a range of rewards for p = 0.8. The points indicate
locations where there are transitions in the decision paths. Note the step function
behavior and the precipitous drop at r = −0.028 (between h and i).
by a double arrow in the state that was affected. Those changes in the decision paths
that occurred simultaneously with changes in the damping ratio are denoted with
both double arrows as well as highlighted in yellow.
Starting from the most negative rewards, the first change in the decision path occurs
at r = −1.650 (point b), where state (2,3) changes from action “right” to “up”. At
the same time the damping ratio increases through a small step change. Due to the
painful rewards in the environment for r ≤ −1.650 the best decision is to take take
the shortest path to the -1 state. Essentially this is a situation where it is actually
less painful to be in the room with the fire than outside of it. However, when the
penalty is changed to −1.650 < r, the the best path is no longer to step directly at
the -1 state from the (2,3) state. This change in the decision paths is considered a
major change to the system because of the effect it has on the damping ratio.
This is in contrast to the change in decision paths that occurs at r = −1.565 (point
c). Here, state (1,3) changes from “right” to “up”; however, there is no change in the
damping ratio with this change in actions. This change is considered minor because
96

Table 5.11: Variations in the decision paths for p = 0.8. Moving from lowest rewards
to highest rewards, the specific state that changed optimal actions is identified by a
double arrow. The states that changed optimal actions with an associated change in
the damping ratio are identified by a double arrow and yellow background.
r ≤ −1.650
(a-b)
→ → → +1
↑ → -1
→ → → ↑
−1.650 < r ≤ −1.565
(b-c)
→ → → +1
↑ ⇑ -1
→ → → ↑
−1.565 < r ≤ −0.732
(c-d)
→ → → +1
↑ ↑ -1
→ → ⇑ ↑
−0.732 < r ≤ −0.453
(d-e)
→ → → +1
↑ ↑ -1
⇑ → ↑ ↑
−0.453 < r ≤ −0.085
(e-f)
→ → → +1
↑ ↑ -1
↑ → ↑ ⇐
−0.085 < r ≤ −0.045
(f-g)
→ → → +1
↑ ↑ -1
↑ ⇐ ↑ ←
−0.045 < r ≤ −0.028
(g-h)
→ → → +1
↑ ↑ -1
↑ ← ⇐ ←
−0.028 < r ≤ −0.022
(h-i)
→ → → +1
↑ ⇐ -1
↑ ← ← ←
−0.022 < r ≤ 0.000
(i-j)
→ → → +1
↑ ← -1
↑ ← ← ⇓
0.000 < r
(j-k)
↔ ↔ ⇐ +1
↔ ← -1
↔ ↔ ↔ ↓
the individuals are still two steps away form the -1 state no matter the policy. The
new set of decisions recognizes the +1 state is preferable, but due to the high painful
incremental rewards, may only be slightly more preferable than stepping into the -1
state. A similar trend is apparent at r = −0.732 (point d), where state (1,1) changes
from “right” to “up” with no change in the damping ratio. This change in decisions
is minor as the individuals are still 5 steps away from the safe exit state no matter
the decision path.
For the other transitions this relationship is consistent, both qualitatively and quanti-
tatively. When the decision paths change for a given state, the change is considered a
97

significant change to the overall system if there is an associated change in the damp-
ing ratio. These changes typically affect states directly adjacent to the room with
the fire (r = −1 state). State (1,2), while not directly adjacent to the -1 state does
have a significant change because the best decision is now to go the long way around
state (2,2) as opposed to taking the shorter way. On the other hand, states that are
farther away, in general, have less of an effect on the overall system as changes in
their decisions do not change the damping ratio.
Of particular note is the significant drop in the damping ratio that occurs when state
(2,3) changes from action “up” to action “left”, located at r = −0.028 (point h). Two
changes occur in the overall system with this set of decisions. First, by wanting to
move “left”, individuals will never take an uncertain misstep into the room with the
fire (r = −1 state) from the (2,3) state, and second, this set of actions effectively
blocks the passage between state (1,3) and (3,3). People must now travel clockwise
around state (2,2), as the shorter route is now blocked off. This result is, in fact,
consistent for all 0.6 < p ≤ 1. The significant drop in the damping ratio identified in
Figure 5.3 always occurs when the best decision for state (2,3) changes from “up” to
“left”. It is also noted that this drop happens at various rewards for various uncertain-
ties. For p = 0.6 this change happens at r = −0.068, while it occurs at r = −0.003
for p = 0.9. Accordingly, as more uncertainty is added into the system, this major
change in the damping ratio occurs further from 0. For p ≤ 0.5 there is no change in
the damping ratio when the decisions changes for this state.
5.4.2.2 Damping Ratio Discussion
The analysis of the damping ratio in this manner is significant because it shows where
the important transition regions in the decision paths are, as well as which states and
98

action combinations have the greatest effect on the system. Without the use of the
spectral methods, the designer would have to examine all nine different transition
regions in the recommended decisions; however, the damping ratio reduces that num-
ber to five areas, with one of significant importance. State (2,3) showed to be a very
important state in the system, especially when the decision transitions from “up” to
“left”. By highlighting the importance of this state, as well as the relative insignifi-
cance of other states (such as state (1,1)), the damping ratio has been able to show
which states deserve greater focus and which rewards are likely to cause these changes.
5.4.2.3 Eigenvector Analysis
The principal eigenvector’s effect on the steady state distribution of the crew was
also examined. Applying the methods discussed in Chapter III Section 3.4.5: The
Eigenvector as a Metric for Steady State Behavior, the principal eigenvector is used
to represent the steady state distribution of the crew following a consistent set of
prescribed actions. Similar to the damping ratio analysis, the MDP was run for 30
decision epochs and the principal eigenvector associated with λ1 was calculated for
the final epoch. Again, the aim was to show the relationship between the eigenvector
and the system for a single time step before performing temporal analyses.
Three analyses were performed using the eigenvector to examine the relationship the
uncertainty and rewards have on the stable distribution of the individuals in the ves-
sel. First, a sweep of both the uncertainty and rewards was performed simultaneously
to see the percentage of individuals that will make it to the exit state (state (3,4)) in
the long run. The next two studies looked at a sweep of the rewards and a sweep of
the uncertainty individually to examine both the total distribution of individuals in
vessel as well as the magnitude of the changes in distributions based on changes in
99

the uncertainty and rewards.
A Study Varying both the Uncertainty and Rewards
A sweep of the uncertainty and rewards was performed to test the impact these pa-
rameters have on the percentage of individuals that make it to the final exit state in
the long run. The uncertainty was varied from 0 < p < 1 and the rewards were varied
from −2.0 < r < 0. This range was selected to eliminate the r > 0 region where there
is no unique set of decisions, as discussed previously. For instances where r < 0.6,
ρ ≈ 1 however it never actually equals one, thus the dominant and sub-dominant
eigenvalue are distinct. Thus, the methods and metrics discussed in this chapter are
still applicable.
Figure 5.5 shows the percentage of individuals that successfully make it to the exit
state (state (3,4)) in the long term. Several items are worthy of note from this figure.
First, it is clear that the percentage of individuals that make it to the safe exit state
is more sensitive to the uncertainty than it is to the rewards. As the probability
of moving in the desires direction increases, the percentage of individuals gradually
increases as well. Below p = 0.3 the percentage of individuals that make it to the
exit remains steady at 33% for the majority of the rewards. When the r → 0, a
step change occurs and the percent of individuals drops drastically to 25%. Above
p = 0.3 the percent of individuals gradually increases until p = 1, in which case 100%
of individuals make it safely to the safe exit state.
Varying the rewards has little effect on the results. For most of the range of the
rewards tested, the percent of individuals remains mostly unchanged as the rewards
change. The one noticeable sensitivity happens as a step function near r = 0. As
the rewards increase, at some point slightly before r = 0, the percent of individuals
100

Reward, r
Probabilityofmoving
indesireddirection,p
−2 −1.5 −1 −0.5 0
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Precipitous drop as
identified via the
damping ratio
Figure 5.5: Percent of individuals that make it to the safe state (state (3,4)) in the
long run based on a variation of uncertainties and rewards.
that will make it to the safe state reduces drastically. This change starts close to
r = 0 for p = 1 and slowly moves towards more negative rewards as p is reduced.
Below p = 0.5 the step begins to happen much farther away from r = 0. The step
is significant for 0.5 < p < 1 because it is the same bifurcation line that was iden-
tified as the precipitous drop in the damping ratio plot in Figure 5.3. As discussed
previously in Section 5.4.2.1: Damping Ratio Results, this bifurcation happens when
state (2,3) changes from “up” to “left”. Thus, spectral analysis is able to identify this
transition region as one of significant importance using both eigenvalue analysis with
the damping ratio, and eigenvector analysis through steady state analysis.
A Study Varying only the Rewards
A sweep of the rewards from −2 < r < 0 was performed for p = 0.8. This range was
chosen to compare the results directly to those in Section 5.4.2.1: Damping Ratio Re-
sults. As can be seen in Table 5.12, four different steady state distributions exist for
101

this range of rewards. For the most negative rewards, there is nearly an 88% chance
an individual will make it to the final exit state (state (3,4)), while there is near an
11% chance they end up in the room with the fire (state (2,4)). There is a 1% chance
an individual will end up congregating in the passage near in the room with the fire
(state (2,3)). As the reward is adjusted to −1.65 < r ≤ −1.44, the probability of
an individual ending up long term in the room with the fire decreases to under 10%,
while the probability of those that end up in the safe exit remains nearly the same at
nearly 88%.
Table 5.12: Steady state distributions as calculated by the dominant eigenvector for
p = 0.8. The values indicate the long term probabilistic location of the individuals in
the vessel.
r ≤ −1.65
0.000 0.000 0.001 0.877
0.000 0.011 0.108
0.000 0.000 0.001 0.001
−1.65 < r ≤ −1.44
0.000 0.000 0.011 0.878
0.000 0.012 0.099
0.000 0.000 0.000 0.000
−1.44 < r ≤ −0.03
0.000 0.000 0.108 0.878
0.000 0.012 0.001
0.000 0.000 0.000 0.000
−0.03 < r
0.006 0.006 0.101 0.808
0.007 0.054 0.000
0.006 0.007 0.006 0.000
As soon as the reward changes above -1.44 for the range −1.44 < r ≤ −0.03, there
are clear changes. The probability of the individuals congregating long term in the
room with the fire drops to nearly zero. However, now more than 10% will end up
in the room just outside the safe exit (state (3,3)). One of the assumptions in this
model is that an individual must make a decision to move in a given direction at
each decision epoch, even if they have reached the safe state. The change in the
steady state distribution here is due to the change in underlying decision at the (3,4)
state. That is, for −1.65 < r ≤ −1.44, the best decision while in the safe state is
to move “right”, meaning 10% of the time individuals will accidentally step into the
102

room with the fire. For −1.44 < r ≤ −0.03, the best decision changes to “up”, which
means that individuals take a misstep outside of the safe room to state (3,3). This
assumption could model situations where there is panic and individuals may move out
of the safe room even after they have already landed there. This causes the change
in distributions between state (2,4) and state (3,3) for a reward of r = −1.44. Again,
the percentage of those in the safe exit room remains just under 88%. The previ-
ous study that performed a sensitivity study on both the uncertainty and rewards
was unable to discern the two transition regions at r = −1.65 and r = −1.44 because
it focused only on the impact on state (3,4), and missed the impact on the other states.
The final change happens when the rewards are increased to greater than -0.03. The
percent in the safe exit is now less than 81%, while there is a 0% chance someone will
end up long term in the room with the fire. Roughly 5% will remain immediately
adjacent to the fire room (state (2,3)). A noticeable change also happens for the
states far away from the fire and safe exit states, where in each state there is just
under a 1% chance that someone will end up there. This change happens at the same
transition region identified by the major drop in the damping ratio (Figure 5.4). This
last region could simulate a situation where the pain from the smoke is not that great
and a small portion of the individuals would prefer just to stay far away from the fire
room, as opposed to heading for the exit and risking the chance of ending up in the
fire room instead.
This first metric measured only the distributions of individuals, while this next metric
examines the magnitude of the changes between decisions by looking at the magni-
tude of the angle formed between the eigenvector of the previous distribution with
that of the new distribution. Since these distributions come about from a set of de-
cisions, this metric can also be used as a metric quantifying the impact of changes
103

in decisions. The angles between the vectors are given in Table 5.13 and Figure 5.6.
Displaying the angles graphically has the benefit of visually showing a vast number
of angles that may get confusing in a table alone. This will become clear in the next
section where there are numerous sets of decisions and thus multiple angles to display.
Table 5.13: The magnitude of the angles between the eigenvectors for given rewards
and p = 0.8. The rewards indicate the transition regions where the steady state
distribution changes.
Reward, r |θdeg|
-1.65 0.9
-1.44 8.9
-0.03 3.2
0 0.5 1 1.5 2 2.5 3
−0.5
0
0.5
1
x position
yposition
0.98 1 1.02
−0.05
0
0.05
0.1
0.15
0.2
r = −1.44,
θ = 8.9
°
r = −0.03,
θ = 3.2
°
θ
neutral axis
r = −1.65,
θ =0.9
°
Figure 5.6: Visual display of the magnitude of the angles between the eigenvectors
for given rewards and p = 0.8. The rewards indicate the transition regions where the
steady state distribution changes.
By examining the angles, it is evident that the change in steady state distributions
that occurs at r = −1.65 is indeed small. The magnitude of the vector is only 0.9◦
different that the original one. This is opposed to the significant change at r = −1.44,
104

which is nearly 9◦
. This change is significant even though it does not appear in Fig-
ure 5.5, which examined only the final exit state. Thus, this is an instance when
examining all states is important, otherwise the decision maker may miss an impor-
tant transition region if there is too strong a focus on a single state. The change
at r = −0.03 is less significant that the one at r = −1.44, accounting for only a
magnitude of 3.2◦
.
A Study Varying only the Uncertainty
The last study examined a sweep of uncertainties between 0.6 < p < 1 for r = −0.1.
This range was selected to examine the “ramp” section of Figure 5.3 where the
damping ratio increases rapidly for changes in the uncertainty level. The results
are markedly different than those from varying the rewards. Here, with each small
change in the uncertainty comes a small change in the distribution of individuals
in the vessel. Thus, there are no clear significant bifurcation regions, but instead a
gradual change. The stable distributions for every p = 0.1 are presented in Table 5.14.
Table 5.14: Steady state distributions as calculated by the dominant eigenvector for
r = −0.1. The values indicate the long term probabilistic location of the individuals
in the general arrangement.
p = 0.6
0.000 0.000 0.221 0.706
0.000 0.059 0.015
0.000 0.000 0.000 0.000
p = 0.7
0.000 0.000 0.166 0.799
0.000 0.030 0.005
0.000 0.000 0.000 0.000
p = 0.8
0.000 0.000 0.108 0.878
0.000 0.012 0.001
0.000 0.000 0.000 0.000
p = 0.9
0.000 0.000 0.052 0.945
0.000 0.003 0.000
0.000 0.000 0.000 0.000
As the probability of moving in the desired direction, p, increases the probability of
landing in the safe exit state gradually increases as well. For p = 0.6, just over 70%
105

p = 1.0
0.000 0.000 0.000 1.000
0.000 0.000 0.000
0.000 0.000 0.000 0.000
of individuals make it to the safe state, while that percentage increases to a full 100%
for p = 1.0. The long term chance of landing in the room with the fire is 1.5% for
p = 0.6 and drops to 0% by p = 0.9. The states just outside the fire and safe exits
also have decreasing probabilities with increasing p.
This study also examined the magnitude of these incremental changes using the mag-
nitude of the angle formed between the eigenvectors. Table 5.15 shows the angles
formed incrementally for each p = 0.05 for 0.6 < p < 1. The angles are all small,
less than 0.7◦
, and are decreasing with increasing p. For p = 0.61 the angle is 0.69◦
,
while for p = 1.0, the angle is much less at 0.29◦
. Due to the more continuous nature
of this progression, visualizing these angles is beneficial. Figure 5.7 shows numerous
small changes from p = 0.61 to p = 1. It can be seen that not only does the angle
get smaller, but the rate at which the angle gets smaller is decreased. That is, the
distributions begin to approach p = 1 asymptotically due to their ever decreasing
incremental change.
In all cases this change is gradual, and follows a more continuous trend. This behavior
is different to the step function nature that was present when varying the rewards.
This is logical based on the shape of both the damping ratio plot in Figure 5.3 and
the state distributions of the safe exit in Figure 5.5 where the gradient was more
pronounced by varying the uncertainty as opposed to the rewards. Thus, it is clearly
shown that variations in the uncertainty interact with the decisions in a different
manner than variations in the rewards.
106

Table 5.15: The magnitude of the angles between the eigenvectors for given proba-
bility of moving in the desired direction, p, and r = −0.1.
Probability of moving |θdeg|
in desired direction, p
0.61 0.69
0.65 0.62
0.70 0.55
0.75 0.49
0.80 0.44
0.85 0.40
0.90 0.36
0.95 0.32
1.0 0.29
0 0.5 1 1.5 2 2.5 3
−0.5
0
0.5
1
x position
yposition
0.98 1 1.02
0
5
10
15
x 10
−3
Increasing p
neutral axis
θ
p = 0.61,
θ = 0.64°
p = 1,
θ = 0.29
°
Figure 5.7: Visual display of the magnitude of the angles between the eigenvectors
for given probability of moving in the desired direction, p, and r = −0.1.
This diﬀerent behavior can be explained by smoke interfering with the decision mak-
ing of the individuals. The smoke may interfere with the respiratory system, thus
causing pain and reducing the “rewards” of the individual. The smoke may also im-
pair sight, thus causing uncertainty in movement. However, in instances such as fog
where there is high uncertainty in movement, there may not be any pain, and thus
there is no need to go to the muster point. This is analogous to situations where the
107

rewards may be positive and there is a high uncertainty in movement.
5.4.2.4 Eigenvector Analysis Discussion
Performing eigenvector analysis for SC-MDP framework in this manner is significant
because it enables the ability to quantify changes in system decision making without
the need to know which specific state the individuals are in. Previously, knowledge
of the specific state, or some estimation of which was necessary to determine whether
or not a given change may be important. Eigenvector analysis removes this need as
it examines all states at once and is able to relay their relative significance during
system and decision changes. This method is also significant because it informs de-
signers and decisions makers where individuals may tend to congregate in the event
of an emergency. This may highlight key area in the vessel where special attention
should be paid at future stages of the design process.
5.5 Discussion
On top of highlighting the significance of the damping ratio and the principal eigen-
vector individually as valuable metrics for measuring the impact of design decisions,
this chapter also highlighted the relationship between these two different metrics. The
damping ratio and the principal eigenvector measure different aspects of the system’s
spectrum. The damping ratio measures the magnitude of the second largest eigen-
value, while the steady state analysis measures the eigenvector associated with the
largest magnitude eigenvalue. Yet, both metrics were able to identify the same bifur-
cations in the decision making. Primarily, the transition around the precipitous drop
in the damping ratio proved to also have a significant impact on the steady state dis-
tributions of where individuals will congregate long term in the event of an emergency.
108

Performing eigenvalue and eigenvector spectral analysis for SC-MDP framework in
this manner is significant for ship design because it is inherently a discussion of the
Why instead of the What. This chapter focused exclusively on the egress problem,
which is defined as how people egress, understanding the decisions they make under
uncertainty, and the interaction between the individuals themselves and the general
arrangement of the layout of the vessel. This is juxtaposed to traditional analy-
ses that focus on the solution, namely the general arrangement of the layout of the
vessel and the physical distribution of the crew throughout the ship. This analysis
also illuminates why certain decisions were made and the driving factors behind them.
5.6 Conclusion
A method for enabling decision making insight has been presented involving apply-
ing eigenvalue spectral analysis to the SC-MDP framework. A damping ratio for
Markov decision processes was introduced, defined as the ratio between the largest
eigenvalue and the magnitude of the second largest eigenvalue. This damping ratio
was applied in two new ways: first to identify and quantify changes in the sets of
decisions, and second, to identify the specific system attributes causing the major
changes in decisions. The eigenvector associated with the dominant eigenvalue was
also demonstrated as a leading indicator metric for measuring the impact of decisions.
The relationship between the damping ratio and the principal eigenvector is apparent
from the identification of many of the same bifurcation regions in the decision space
using both metrics. Use of eigenvalue spectral methods will be beneficial for ship de-
sign and decision making by eliciting new insight into the design and decision space
that may not be possible using traditional methods.
109

5.7 Next steps
The results in this chapter showed the utility of spectral analysis in identifying and
quantifying the magnitude in changes in the decision paths. However, this was done
for stationary rewards and uncertainties, and for a single time step. Ship design
and decision making is inherently non-stationary and temporal. For example, non-
stationary transitions could arise from varying regulations, policies, supply chain risks,
or performance drift, while non-stationary rewards could come from economic vari-
ability in fuel prices or evolving budgetary requirements. Non-stationary transitions
or rewards always lead to non-stationary decisions.
While the results presented were only for a single case study, applying these tech-
niques to non-stationary temporal systems presents no signiﬁcant challenges. These
metrics can still be used to determine when changes in the decision paths change the
spectral modes of the system, thus causing signiﬁcant changes to the overall system.
Use of spectral methods to analyze temporal changes in non-stationary decision sce-
narios is a truly novel concept, and is a unique contribution of this research. The next
chapter shows the utility of spectral analysis on a non-stationary temporal problem of
decision making and lifecycle planning involving ballast water treatment compliance.
110

CHAPTER VI
Spectral Analysis Case Study 2: Lifecycle Planning
for Ballast Water Treatment Compliance1
This chapter introduces a means to elicit decision making insight for lifecycle planning
for ballast water treatment compliance by applying a temporal spectral analysis to
the SC-MDP framework. The previous case study on egress patterns and general
arrangement design discussed situations where the process was independent of the
initial state vector. This chapter, on the other hand, examines a situation where the
system is initial condition dependent where there are multiple independent paths the
design may go. To study this, the objective of this chapter are:
1. Demonstrate the applicability of eigenvalue spectral analysis to the SC-MDP
framework for a temporal, non-stationary problem.
2. Demonstrate how to identify and group independent states and processes within
the SC-MDP framework when there are repeated dominant eigenvalues.
3. Demonstrate how the set of principal eigenvectors can be used to show various
initial condition dependent design absorbing paths.
1
Early work on this case study was developed by Niese (2012); Niese and Singer (2013, 2014).
This chapter extends those studies to include the spectral methods proposed in this thesis.
111

Previous work on applying the SC-MDP framework to the ballast water treatment
problem has been performed by Niese (2012); Niese and Singer (2013, 2014). These
studies applied simulations through the decision matrix to capture the interplay be-
tween internal and external forces. This was done in an effort to develop both a
design and a lifecycle decision making strategy that minimize life cycle cost while
maintaining compliance and performance.
This chapter extends this work by introducing temporal eigenvalue spectral methods
to gain a deeper understanding of the driving forces behind the different decision mak-
ing scenarios, as well as quantifying their differences. Specifically, this dissertation
uses both eigenvalue and eigenvector analysis to help identify and examine inter-
dependencies between decision paths and projected design scenarios. This chapter
introduces a temporal perspective on using spectral analysis applied to the SC-MDP
framework in understanding ballast water treatment decision making and lifecycle
planning.
6.1 Background
Ballast water treatment has become compulsory due to local and international envi-
ronmental regulations. Unfortunately for ship owners and operators, decision making
and lifecycle planning for ballast water treatment methods is difficult due to the
interplay of various factors, including: stochastic degradation, technology develop-
ment, and multiple levels of environmental policy-making. Regulating the discharge
of ballast water has been recognized as an important part in the fight against inva-
sive species. As vessels unload their cargo, they bring aboard ballast water to help
ensure the vessel remains stable and floats on its lines. The vessel then transits with
this ballast water to its new location, carrying with it all the microorganisms that
112

are in the ballast water. While most of the organisms die along the way, many do
not. These organisms are then introduced into a new environment and can cause a
great disturbance to the local ecosystem. To combat this, many governing bodies
have put into place strict guidelines that specify the quality of the water that is being
discharged.
6.1.1 Regulatory Framework
Ballast water is regulated by multiple governing bodies, including, the IMO, the U.S.
Coast Guard, various states and local governing bodies, and the European Union.
In 2004, the IMO adopted the Ballast Water Management Convention designed to
regulate global transport of ballast water. These regulations will apply to all vessels
required to carry ballast, including: submersibles, floating craft, floating platforms,
floating storage units (FSUs) and floating production storage and offloading (FPSO)
vessels. However, the regulation will not apply to ships that do not carry ballast,
domestically-only operated vessels, warships and other vessels owned and operated
by the State, non-commercial ships, or ships with permanently sealed ballast tanks
(Lloyd’s Register, 2015). Table 6.1 outlines the specific limits imposed by the IMO.
The ramifications for violating the regulations are significant, ranging from monetary
fines to criminal sanctions for willful noncompliance (Davis and Levy, 2012).
While the ballast water convention was held in 2004, by 2015 it had not come into
effect. The regulation will go into effect 12 months after it has been ratified by 30
member States representing 35% of the world merchant shipping tonnage. As of
November 2015, it has been ratified by 44 countries representing just under 33% of
global merchant shipping tonnage (IMO, 2015). Despite the fact the IMO regula-
tions are not in force, there are still significant reasons to study their potential future
113

Table 6.1: Discharge limits for ballast water as prescribed by IMO ballast water
convention (Lloyd’s Register, 2015).
Organism Category Discharge Limit
Plankton, > 50µm (min. dimen.) < 10 cells/m3
Plankton, 10 − 50µm < 10 cells/ml
Toxicogenic Vibrio cholera
< 1 cfu∗
/100ml or < 1 cfu/g (wet weight)
(01 and 0139)
Escherichia coli < 250 cfu/100ml
Intestinal Enterocicci < 100 cfu/100ml
* cfu: colony forming unit
impact on ship design and decision making. These regulations will likely come into
force soon, creating a necessity to have a strategic plan for them now. Also, vessels
already have to consider national and regional regulations that are in force.
Figure 6.1: Ballast water treatment technology compliance schedule. (Lloyd’s Regis-
ter, 2012)
Figure 6.1 outlines the compliance schedule according to the IMO Ballast Water
Convention. Acceptable ballast water technologies are dependent on the size of the
ballast capacity and the year the ship was constructed. Note that the type of ap-
proved technology changes in 2016. This date was selected to give engineers time
to develop applicable technologies. Even though the regulation has yet to come into
force, technology developers and vessel owners have had to prepare for this upcom-
114

ing change well in advance, despite the uncertainty surrounding the enforcement date.
6.1.2 Compliance Mechanisms
Many technologies already exist that meet some of the regulations, and others are
still in development to meet the most stringent of the policies. Ballast water exchange
systems will no longer be allowed once the regulation goes into force. The other op-
tion is ballast water treatment, which tries to kill the bacteria and living organisms
in the ballast water.
Figure 6.2: Ballast water treatment technology process (Lloyd’s Register, 2012)
Figure 6.2 outlines the process by which ballast water is treated, first by separating
the solids from the liquids, and then finished by disinfecting the liquid completely.
Currently there are over 70 different manufacturers of ballast water treatment tech-
nologies across the range of various treatment options (Lloyd’s Register, 2012). This
makes the planning and selection of proper technologies complicated and difficult.
115

6.2 Case Study: Lifecycle Planning for Ballast Water Treat-
ment Compliance
A notional 150,000 deadweight tonnage containership with a 30,000 metric ton ballast
water capacity routed along the trans-pacific route is used for this study. The ballast
water treatment system must have a capacity of at least 10, 000m3
/h. The vessel
has a 20 year lifespan and is put in service sometime prior to the 2004 IMO Ballast
Water Management Convention. Ten ballast water systems, labeled 1-10, are consid-
ered. System 1 is a commercially available ballast water exchange system. Systems
2-10 represent ballast water treatment systems that become commercially available
at some time during the lifespan of the vessel. Specifics of the ballast water sys-
tems, including performance, capital costs, operating expenditures, availability and
approval have been derived from Lloyd’s Register (2007, 2010); California State Lands
Commission (2010). These systems represent various treatment technologies, such as:
filtration, electrochlorination, cavitation, radiation, and de-oxygenation. The original
case setup, including inputs, stochastic variables, and economic parameters have all
been tested and validated by Niese (2012); Niese and Singer (2013, 2014).
6.2.1 Markov decision process framework
This section details how the states, actions, transition probabilities, and rewards are
defined in the MDP for this case study.
6.2.1.1 States
The MDP states are defined by the individual ballast water systems, their commer-
cial availability and regulatory approval, as well as their deterioration level. For each
ballast water system, there are six availability states, including: unavailable, commer-
116

cially available, basic approval, final approval-Tier 1, final approval-Tier 2, and final
approval-Tier 3. The Tier 1, Tier 2, and Tier 3 approval states correspond to meeting
IMO regulations, State of New York regulations, and State of California regulations
respectively. The State of New York regulation is roughly 10x more stringent than
the IMO’s policy, and the State of California’s policy is roughly 100x more stringent
than the IMO’s regulation. Each system also has four deterioration levels. The de-
terioration level is defined as a percentage of total deterioration. Thus, there are 240
states, accounting for ten systems, six approval states, and four deterioration levels.
6.2.1.2 Starting State
This analysis assumes that the initial state is unknown. That is, there is equal proba-
bility of being located in any of the states at the start of the model. Niese et al. (2015)
discussed the problems associated with, and importance of, selecting the correct start
state, and its implications on future decision making opportunities. Designs may be
dependent on the initial conditions, and thus selecting differing starting states may
lead the design down a different path. By assuming equal starting probabilities, this
analysis aims to find the most natural path the design would take as opposed to
pre-determining its trajectory.
6.2.1.3 Actions
There are twelve possible actions the decision maker can make:
1. No Action: This action occurs when the system continues to deteriorate, yet
no action is necessary to maintain it.
2. Maintain: Maintenance is performed and the ballast water system is restored
to a less deteriorated state.
117

3. Replace System (1-10): The ballast water system is replaced with one of the
10 possible systems. The specific system that is installed is identified by the
index 1-10. A system can only be installed after it becomes both commercially
available and meets regulatory requirements.
6.2.1.4 Transition Probabilities
The probability of transitioning between states is defined by the following models.
• The probability of transitioning between ballast water systems is deterministic
based on the best action selected by the MDP and its availability. A system
can only be selected once it is available and approved.
• Transitioning between approval states is based both on the regulatory environ-
ment and the commercial availability. The following schedule is used to model
various regulatory scenarios.
1. The ballast water convention is held which outlines the expected strength
of the legislation, as well as expected date of enforcement.
2. Laboratory testing procedures specific to ballast water treatment efficacy
are available.
3. The legislation is ratified by member States.
4. The legislation enters force.
The implementation schedule is defined as the number of years following the
convention a policy trigger occurs. For instance, the 1-4-4-9 schedule would sim-
ulate a convention being held one year after the ship enters service, and then
4 years later laboratory testing procedures become available and the legislation
is ratified. Nine years after the convention the legislation enters force.
118

Prior to the convention, there is little demand for development of the ballast
water treatment technologies, and thus it is assumed the treatment technologies
will not become available until after the convention is held. Also, each individ-
ual technology will meet a diﬀerent threshold of regulatory compliance and will
become available at diﬀerent times. The schedule outlining the expected year
each technology will be available is given in Table 6.2. The table outlines the
number of years following the convention that the technologies are expected to
be available, as well as their expected regulatory compliance level. This data
has been based on actual dates when the technologies became available, while
the deviation has been included to simulate uncertainty in the commercializa-
tion process (Niese and Singer, 2013).
Table 6.2: The ballast water technology availability schedule and compliance level.
The mean availability details the number of years after the convention that particular
technology is expected to be both commercially and regulatory compliant.
System Mean Availability Deviation σ Compliance Level
1 - - Exchange
2 3 0.5 Treatment-Tier 3
• Transitioning between deterioration levels is modeled by a special case of the
gamma distribution, known as the exponential distribution, given in Equation
6.1.
fj(x) = γjeγjx
(6.1)
119

Deterioration happens independently from other factors in the model, and fol-
lows an exponential distribution for γ, given in Equation 6.2. λj is a function
of the system’s treatment method. This is due to ballast water treatment sys-
tems using ﬁltration, electrochlorination, cavitation, radiation, de-oxygenation,
and/or ozone-generation degrade diﬀerently (Niese and Singer, 2013). A full
description of this model can be found in van Noortwijk (2007).
γj = aje−bj
+ cj (6.2)
Figure 6.3 shows the availability of the various system according to both commer-
cial availability and regulatory compliance for the 1-4-4-9 regulatory implementation
schedule. For visualization purposes the 240 states have been condensed to 60 rep-
resentative states. To do this, the four deterioration levels for each ballast water
system and for each approval status have been added together. This creates a single
representative state that accounts for all four deterioration levels (Niese, 2012).
6.2.1.5 Rewards
The rewards are based on the system capital costs, installation costs, and operating
and maintenance costs, as given in Table 6.3 (Lloyd’s Register, 2007, 2010; California
State Lands Commission, 2010; Rigby and Taylor, 2001). The cost function is given
in Equation 6.3.
cost = min(captial + install + operating + maintenance) (6.3)
The capital costs are dependent on whether the system meets basic approval. Capi-
tal costs tend to increase in price after achieving basic approval because the approval
status may warrant a cost increase, or supply and demand economics may dictate it
120

Time
1 2 3 4 5 6 7 8 9 1011121314151617181920
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System10 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
% of states
Figure 6.3: Ballast water system commercial availability and regulatory compliance
for the 1-4-4-9 schedule. Shading represents the percent likelihood a given system
will be located in that state.
(Niese and Singer, 2013). Installation costs vary depending on whether it is during
vessel new construction or a retrofit. In cases of a retrofit, it is assumed there is
sufficient space.
As equipment deteriorates, it becomes less efficient and more costly. Equation 6.4 and
6.5 model the increasing operating costs as a function of deterioration. For this study
121

Table 6.3: Ballast water technology costs. The Capex #/# corresponds to costs be-
fore/after Basic approval. The Install #/# corresponds to costs for newbuild/retroﬁt.
System Capex (2, 000m3
/h) Install (2, 000m3
/h) O&M ($/m3
/h)
1 50/50 0/0 0.06
2 800/820 40/55 0.08
3 950/1,200 5/15 0.07
4 950/1,500 50/65 0.06
5 690/670 60/60 0.13
6 800/450 80/100 0.32
7 500/975 65/125 0.013
8 1,600/1,600 5/15 0.06
9 559/600 100/150 0.03
10 1,800/1,200 25/40 0.01
g = 0.01, and x = [1, 2, 3, 4] depending on the deterioration level. λ = [0.72, 0.78]
and is a function of the system installed (Niese, 2012). A full description of this
deterioration cost function model can be found in (Nguyen et al., 2010).
φ(x) = φ0 + geγjx
(6.4)
O&M cost = Annual trips ∗ required ballast ∗ φ(x) (6.5)
6.3 Results
Three sets of results are discussed in this chapter:
1. The optimal states are analyzed to see the impact a given regulatory strength
and schedule has on the ballast water system of choice. This is done without
the use of spectral methods.
2. Spectral methods are used to examine interdependencies of the decision process
and how those dependencies change through time. This is done through analysis
of the set of eigenvectors associated with the set of dominant eigenvalues and
using them as a metric for identifying independent decision absorbing paths.
These speciﬁc eigenvectors are referred to as the principle eigenvectors.
122

3. The relationship between the set of principle eigenvectors and the optimal state
plot is discussed.
6.3.1 Optimal States Accessed
The model was run to determine what the best decisions are, when they should be
made, and what ballast system is best to install under given conditions. An optimal
states plot is given that displays the preferred ballast system that should be installed
at any given time. This plot accounts for uncertainty in technology availability, thus
there is no uncertainty between making the choice to install a particular ballast wa-
ter system and actually having it installed. When a given system is selected in the
optimal states plot, it is assumed that the optimal action is to select that particular
system. This metric was used extensively by Kana et al. (2015) to study temporal
decision making behavior in the face of evolving Emission Control Area regulations
(see Appendix B).
A sensitivity study was performed on the strength of the regulation as to its aﬀect
on the preferred ballast system. As shown in Figure 6.4, for the 1-4-4-9 regulatory
schedule, with a Tier 1 regulation strength, the best choice is to install ballast System
9 after year 9. System 1 becomes unavailable due to regulatory requirements at year
9, thus necessitating a change. System 9 is selected as the best option, which meets
Tier 2 requirements, despite the regulation only requiring Tier 1 compliance. When
the strength of the regulation is increased to requiring Tier 3 compliance, ballast
System 2 becomes preferred after year 9. Only 5 of the original 10 systems meet Tier
3 standards, and System 2 was selected due to it lower lifecycle costs.
While this metric shows what the best decisions are, it doesn’t show why a given
123

Time
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
(a) Regulation: Tier 1
Time
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
(b) Regulation: Tier 3
Figure 6.4: Optimal states accessed for 1-4-4-9 regulatory schedule and two treatment
strengths.
technology was selected. This metric does not show which other technologies may
also be desirable or how the initial conditions may be aﬀecting future decision oppor-
tunities. Also, this analysis is only able to display one particular absorbing path. The
following study on the set of principal eigenvectors aims to address these limitations.
6.3.2 Principal Eigenvector Analysis
Two metrics are presented using spectral analysis to identify the set of design ab-
sorbing paths. The ﬁrst metric uses the number of dominant eigenvalues to show
124

the number of possible absorbing paths and how they may evolve through time. The
second metric uses the set of principle eigenvectors to identify the specific absorbing
paths and how those paths may change and evolve through the non-stationary process.
As discussed in Section 3.4.7.1: Reducible Markov Processes the absorbing paths rep-
resent the various initial condition dependent steady state distributions of the system.
6.3.2.1 Identifying the Number of Unique Absorbing Paths
The number of unique absorbing paths was examined to show how the structure of
decision process evolves through time. As discussed in Gebali (2008), the number of
dominant eigenvalues, λi = 1, is equal to the number of unique absorbing paths of
the decision process. In a sense, the number of unique dominant eigenvalues signify
that the decision process is not a single connected process, but rather a collection of
independent decision processes.
Figures 6.5 and 6.6 show the number of unique absorbing paths for Tier 1 and Tier 3
regulatory strength respectively. Up to year 4 there is only one possible path, meaning
the process will always converge to a single set of states. Beginning at year 5, when
testing becomes available and when the regulation is ratified by the member States,
multiple paths become possible. The increasing number of absorbing paths with time
is representative of the number of ballast water systems that may be installed in the
long term. At year 10 the regulation enters force, thus removing ballast System 1
from compliance. This explains the drop in both figures at year 10. After year 10,
only those technologies that meet the regulation can become a possible absorbing
path. Thus, the number of unique paths for the Tier 3 schedule is only five (Figure
6.6), while there are nine unique paths for the Tier 1 regulation (Figure 6.5).
125

0 2 4 6 8 10 12 14 16 18 20
0
2
4
6
8
10
Time
Numberofunique
absorbingpaths
Figure 6.5: The number of initial condition dependent absorbing paths for the 1-4-4-9
schedule and Tier 1 strength.
0 2 4 6 8 10 12 14 16 18 20
0
2
4
6
8
10
Time
Numberofunique
absorbingpaths
Figure 6.6: The number of initial condition dependent absorbing paths for the 1-4-4-9
schedule and Tier 3 strength.
The number of absorbing paths represents more than just technology availability and
compliance. It is essentially a synthesis of technology availability, compliance, their
uncertainty, as well as lifecycle costs. For instance, four diﬀerent ballast systems are
potentially available at year 4, and yet there is only one absorbing path. This is
because, while those system may be technically feasible, there is no decision path
that will select them in the long run. All ballast systems become available by year 9,
yet it is not until year 12 that the number of absorbing paths becomes steady. Thus,
lifecycle time also aﬀects which systems may be selected in the long run.
6.3.2.2 Analysis of the Set of Absorbing Paths
The eigenvectors associated with the set of dominant eigenvalues was examined as
a leading indicator metric for projecting out all possible absorbing paths. For each
decision epoch, there are a given number of principle eigenvectors equal to the num-
126

ber of dominant eigenvalues for that decision epoch. Each eigenvector is analyzed
separately, as each one represents one independent absorbing path. For the following
figures, each column visually shows the values of one particular principle eigenvector.
These values represent the long term behavior of the decision process. For instance,
if eigenvector i for decision epoch t displays System j, that means that System j
represents the only long term design that the decision process will go towards for
that one given set of initial conditions.
Figure 6.7 shows the results for the 1-4-4-9 regulatory schedule with Tier 1 strength.
Years 1, 5, 8, and 20 are presented to highlight the temporal variations of the absorb-
ing paths. For year 1, the only path the design will follow involves installing System 1
in the long run. This trend continues until year 5 when two paths become apparent:
one for System 1 and one for System 3. Even though System 3 becomes commercially
available in year 3 (Figure 6.3), it does not become a viable path until year 5. As the
number of paths increase, the number of ballast systems that become viable options
increases. For year 8, six different paths are possible, representing Systems 1, 2, 3,
5, 9, and 10 being viable options in the long run. For year 20, all ballast systems
become a viable option except ballast System 1, which became unavailable due to
regulatory requirements in year 10.
The set of long term absorbing paths changes when the regulatory strength changes.
The results for a Tier 3 regulatory strength are given in Figure 6.8. Through year
5, the results are similar to that of the Tier 1 policy. However, for year 8, only five
paths are identified, one fewer than for Tier 1. Ballast System 5 is no longer a viable
option for the Tier 3 policy. Even though the Tier 3 regulation does not come into
force until year 10, this analysis projects two years prior that ballast System 5 will
not be viable in the long run. For year 20, the only ballast systems that are viable
127

indexed principle
eigenvector
Absorbing paths
for year 1
1
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 5
1 2
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 8
1 2 3 4 5 6
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 20
1 2 3 4 5 6 7 8 9
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
Figure 6.7: The set of principle eigenvectors representing the various initial condition
dependent absorbing states: Tier 1 regulatory strength. Notice that the number of
possible unique paths increases through time.
represent those that meet Tier 3 requirements.
A study was performed to show that the different absorbing paths identified using
the principle eigenvectors match the behavior of the decision process with different
initial conditions. To show this, the state vector was changed so that at a given year
there was equal probability of landing in any state. The model was then run to see
how the process evolves through time given this new set of conditions. Year 8 was
chosen for this validation study. Thus, at year 8, the system is run assuming that the
prior year there is equal probability of being in any state. This is different from the
original analysis where the process was started at year 1.
128

indexed principle
eigenvector
Absorbing paths
for year 1
1
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 5
1 2
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 8
1 2 3 4 5
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
indexed principle
eigenvector
Absorbing paths
for year 20
1 2 3 4 5
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
Figure 6.8: The set of principle eigenvectors representing the various initial condition
dependent absorbing states: Tier 3 regulatory strength. Notice the number of unique
paths for year 20 is less than it is for the Tier 1 policy, due to many ballast systems
not being regulatory compliant.
Figure 6.9 shows the results for Tier 1 regulatory strength. For this case there are
six different absorbing paths identified by the set of principle eigenvectors. When
the initial conditions for year 8 are changed so that there is equal probability of be-
ing in each state those same six paths can be identified using the state vector. The
probability of landing in one absorbing path over another is not equal. For example,
it is more likely that System 9 will be the preferred choice over System 2, 3, 5, or
10. System 1 appears as a long term absorbing path in the eigenvectors even though
System 1 is not viable for the whole lifespan of the vessel. Since these eigenvectors
represent an instantaneous snapshot of how the design may progress, it is unaware
that shortly there after the regulation will change, making System 1 not available.
129

Time
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
% of
states
(a) State vector
indexed principle
eigenvector
Absorbing states
for year 8
1 2 3 4 5 6
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
(b) Principle eigenvectors
Figure 6.9: Optimal states accessed for 1-4-4-9 regulatory schedule and two treatment
strengths: Tier 1 strength.
Figure 6.10 shows how the set of possible absorbing paths change when the regulatory
strength is changed to Tier 3. Only five different absorbing paths are identifiable for
this regulatory strength. Unlike the situation with Tier 1, System 5 is no longer
a long term possibility. System 9, while clearly evident using eigenvector analysis,
is barely visible using the state vector. There is a very small possibility it will be
selected in the long run. Also, similar to System 1, System 9 becomes unavailable
at year 9 when the regulation changes. This study showed that the eigenvectors do
represent the various possible absorbing paths the design may follow, and that these
130

Time
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
% of
states
(a) State vector
indexed principle
eigenvector
Absorbing states
for year 8
1 2 3 4 5
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
(b) Principle eigenvectors
Figure 6.10: Optimal states accessed for 1-4-4-9 regulatory schedule and two treat-
ment strengths: Tier 3 strength.
paths are dependent of the initial conditions of the system.
6.3.3 Relationship between Optimal States and Principal Eigenvectors
As discussed in Chapter III Section 3.4.7.2: Using the Principle Eigenvectors to Es-
timate the State Vector, the set of principle eigenvectors can be used to estimate
the non-stationary behavior of the state vector. Figure 6.11 displays the optimal
states accessed using both the state vector and the estimate from the set of principle
eigenvectors. The eigenvectors provide a close approximation of the decision paths.
Even though the state vector was used to help develop this estimation, this spectral
131

recomposition of the best decision path highlights the relationship between spectral
methods and the physical decision process.
Time
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
% of
states
(a) Non-stationary state vector.
Time
1 2 3 4 5 6 7 8 9 1011121314151617181920
System1 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System2 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System3 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System4 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System5 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System6 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System7 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System8 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
System9 Unavailable
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
Available
Basic Approval
Tier1 Approval
Tier2 Approval
Tier3 Approval
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
% of
states
(b) Estimation using principle eigenvectors.
Figure 6.11: Decision path as determined by two methods for the 1-4-4-9 regulatory
schedule and Tier 1 compliance.
6.4 Discussion
The results presented in this chapter are signiﬁcant for ship designers and decision
makers for several reasons. First, the spectral techniques presented gave a unique
perspective into the structure of the decision process. Understanding the interde-
pendencies of the decision making process and how those dependencies may change
132

and evolve throughout the lifecycle of the vessel provides ship designers great power
as they aim to understand the impact of their decisions. Both the number of dom-
inant eigenvalues and the structure of the principle eigenvectors clearly display the
evolution of these relationships and dependencies. Second, the spectral methods are
inherently a leading indicator highlighting the impact of decision making. Spectral
analysis has represented the long term absorbing paths the design may follow without
the need for simulation. Finally, this analysis was inherently focused on the Why as
opposed to the What. The focus was less on what the ﬁnal design is, but instead this
analysis has focused on why that ﬁnal design was selected, the paths that lead the
decision process to that point, and the underlying structure of the entire process.
6.5 Conclusion
A method for applying eigenvalue spectral analysis to the SC-MDP framework for a
temporal, non-stationary problem has been presented. Both the eigenvalues and the
eigenvectors were used to identify and group independent states and processes within
the SC-MDP framework. The set of principle eigenvectors was used to show vari-
ous independent design absorbing paths that are dependent on the initial conditions
of the design. These methods will provide much needed insight for ship designers
and decision makers by moving beyond understanding the What and instead moving
towards comprehending the Why.
133

CHAPTER VII
Conclusion
7.1 Dissertation Conclusion
This dissertation presented a new method for enabling decision insight by applying
Monte Carlo simulations and eigenvalue spectral analysis to the SC-MDP framework.
The problem moved beyond understanding What the design looks like, to one that
focused instead on Why it was selected, the decision process that led to that design,
and understanding the structure of the speciﬁc problem has been discussed. Multiple
layers of uncertainty and the inter-dependency of decisions inhibit sound decision
making processes. To handle the uncertainty of the problem, Monte Carlo simulations
were presented as a viable method, especially during situations when the time frames
are long and there are no viable feedback mechanisms. When trying to breakdown
the complexity of decision making to understand the implications of those decisions,
eigenvalue spectral analysis was presented. These two new methods are signiﬁcant
contributions to ship design and decision making by providing leading indicators for
design decisions. This gives the decision maker insight into not only the results, but
the implications, structure, and generalized response of the system.
134

7.2 Contributions
The contributions of this research are highlighted in the order they were discussed in
this dissertation.
1. Applying Monte Carlo simulations to the SC-MDP framework to understand the
effect of temporal non-stationary uncertainty on decision making. This research
presented a new method for handling multiple layers of uncertainty in a system
that has a long time frame and no feedback mechanisms by applying Monte
Carlo simulations to the SC-MDP framework. Metrics were derived that calcu-
lated the percent of time a given action may be optimal given a set of uncertain
parameters. The range of best-case scenario lifecycle costs was developed to
highlight the risks and variation caused by different inputs. Sensitivity studies
showed that Monte Carlo simulations can lead to differing results compared to
the classic SC-MDP framework which uses discrete probabilistic values.
2. Applying eigenvalue spectral analysis to a stationary SC-MDP case study to
examine the future impact of decisions. A new method was developed that
enables the ability to perform eigenvalue spectral analysis on Markov decision
processes. This method was designed to examine and quantify the impact of
decision making. Three new metrics were derived:
(a) A damping ratio comparing the dominant eigenvalue and sub-dominant
eigenvalue was defined for the first time for Markov decision processes.
The damping ratio was used to identify and quantify the transition regions
in the decisions. The damping ratio was also used to identify the specific
system attributes behind the large transition regions in the decisions.
(b) The eigenvector associated with the dominant eigenvalue was used for the
first time to project the influence of a set of decisions on the future steady
135

state behavior of the system. The ability to identify specific design at-
tributes driving system behavior and decision making through time was
possible using the damping ratio and the principle eigenvector.
(c) The angle between two principle eigenvectors was used as a new metric for
determining the significance of the change in decisions.
3. Applying eigenvalue spectral analysis to a non-stationary temporal SC-MDP
case study to examine the structure of the decision making process. This re-
search introduced the concept of applying eigenvalue spectral analysis to a non-
stationary Markov decision process. To study this, three new methods were
developed:
(a) The number of repeated dominant eigenvalues was used as a metric to
determine whether the process was dependent on the initial conditions.
This number was also used to identify the number of independent absorbing
paths the design may progress down depending on its initial conditions.
(b) The set of eigenvectors associated with the dominant eigenvalues was used
to identify the specific independent design absorbing paths, and how those
absorbing paths may change and evolve through time.
(c) A method was presented that showed how the set of principle eigenvectors
can be used to estimate the behavior of the specific design process (i.e. the
state vector) through time.
7.3 Future Work
Several areas have been identified and potential areas of future work. The topics
have been divided into two categories, first related to spectral analysis specifically,
and second involving the SC-MDP framework in general.
136

7.3.1 Spectral Analysis Future Work
1. Analyze the impact of specific initial conditions. Chapter VI: Lifecycle Planning
for Ballast Water Treatment Compliance analyzed various initial condition de-
pendent absorbing paths. However, it only showed that the absorbing paths are
initial condition dependent, but did not describe what those initial conditions
may be. Developing a mathematical way to clearly identify which sets of initial
conditions may lead to specific absorbing states is one promising area of future
work.
2. Examine the full range of eigenvalues. This dissertation focused primarily on
two eigenvalues, the dominant and the sub-dominant ones; however, for each
Markov process there are as many eigenvalues as there are states. Understand-
ing the relationship of these minor eigenvalues and their relationship to the
behavior of the system could prove beneficial in understanding weak or non-
obvious relationships and interdependencies.
3. Understand the meaning of complex eigenvalues. For biological systems, com-
plex eigenvalues are used to study oscillatory patterns in system behavior (Cressie
and Wikle, 2011). What analogies can be applied to ship design and decision
making? Are there underlying dynamics of the system that oscillate in some
fashion that may be affecting decision making behavior? These type of inter-
esting questions are an area of future work.
4. Examine stability of eigenvalues. The eigenvalues are used in physical systems
to understand the stability of the system. Is it possible to identify instabilities
in the system, or parts of the systems using stability methods derived from
eigenvalues and eigenvectors? Could this lead to a better understanding of what
areas of the design and decision space may be more sensitive to instabilities than
others? Future work could examine these types of questions.
137

5. Examine singular values and singular vectors. Singular values and singular
vectors provide a different spectral transformation compared to eigenvalues and
eigenvectors. Singular values may provide unique insight not possible through
eigenvalue analysis.
7.3.2 Generalized SC-MDP Future Work
1. Explore multi-agent MDPs or Markov games. One of the major assumptions of
this dissertation is that there is a single agent making the decisions. In reality,
multiple decision makers interact both in cooperation or adversarially. For
instance, multiple design teams make independent decisions that affect the given
vessel, while the actions of a economic competitor may affect the decisions of
the company in regards to technology selection. Applying multi-agent, or more
simple Markov games to these situations could prove to be very beneficial.
2. Explore reinforcement learning methods or partially observable MDPs. As is
discussed briefly in Chapter II, reinforcement learning or partial observability
methods may prove to be beneficial in exploring problems with high uncertainty.
Problems would include those with short time frames and feedback mechanisms,
such as system maintenance and replacement scheduling.
3. Explore non-economic reward functions. The ECA case study and the ballast
water case study both used economic measures for the reward function, while
the egress analysis used a generic utility function. Is it possible to tie the reward
function to an engineering metric? For instance, could one use roll accelerations
from a seakeeping analysis to understand various roll mitigation devices in the
face of uncertain wave and sea states? This would tie individual engineering
disciplines to holistic design. Another option would be to use measures of
effectiveness (MOEs) as defined by the US Navy as the reward parameter.
138

APPENDIX A
Validation of University of Michigan’s SC-MDP
code with a commercial client’s internal analysis
This Appendix outlines the procedure and results from a validation study of the
University of Michigan’s SC-MDP code with the results from a commercial client’s
internal methods performed in the summer of 2014. This validation study was done
on the Emission Control Area case study. The objective was to validate the lifecycle
costs calculated by the SC-MDP framework with that of a more traditional lifecycle
cost analysis. The accumulated savings by switching to LNG as calculated by the
client and by the University of Michigan are compared.
Validation Assumptions
In order to validate the results between the two economic models, the assumptions for
both models had to be consistent. Some of the assumptions diﬀered initially between
the two models and they were adjusted to make sure the results were focused on the
same parameters. Those assumptions were:
140

1. ECA Implementation. The commercial client had initially assumed a gradual
annual increase of ECA from 5% to 31% beginning from year 1 onwards. This
dissertation instead proposed that the ECA coverage increases from 5% to 31%
as a step function, as opposed to gradually. This follows the predictions as laid
out by MARPOL Annex VI (IMO, 2008). For example, ECA would be 5% at
year 5 and would jump to 31% immediately at year 6. For the validation study
the client tested two cases, one where ECA switched at year 5 and one where
it switched at year 10. The SC-MDP code had ECA switch at year 7 to split
the difference between the two.
2. Trips per Year. The commercial client had assumed the vessel will make three
round-trips per year, while the University of Michigan SC-MDP code based the
number of round-trips as a function of number of annual sailing days and the
ship speed. For validation purposes, for a ship speed of 19 knots, the number
of sea days was set to 144 to achieve 3 round-trips per year. The analyses
performed in the body of this dissertation, however, used 290 annual sea days,
which essentially doubles the number of trips per year.
3. Lost Revenue. The commercial client had assumed a lost revenue of $1,500 per
TEU per year, while this dissertation used a lost revenue of $1,500 per TEU
per leg. Thus, the client assumes a lost revenue of $732,000 per year; however,
by calculating lost revenue by leg, the annual lost revenue jumps to $4,392,000
per year. This additional lost revenue has a significant impact on the expected
rewards by switching to LNG. In fact, this lost revenue completely outweighs
all benefit to switching to LNG, and in all cases switching to LNG is more
expensive than sticking with single fuel MDO/HFO. This lost revenue is all
under the assumption that the ship is carrying a full load. If the lost revenue
assumption is relaxed, the SC-MDP code does advocate for switching to LNG
141

immediately, similar to the original report developed by the client.
4. Engine Switch Date. The major diﬀerence between the client’s analysis and
the SC-MDP code is that the client’s analysis assumed two scenarios: one, a
baseline where the vessel does not switch engines, and two, one where the vessel
switches engines as soon as possible. The SC-MDP code, on the other hand,
selects the best time for the vessel to switch engines. In order to be consistent,
the SC-MDP code had was adjusted to force it to a) not switch, and b) switch
as soon as possible. The SC-MDP code was able to handle this restraint, even
though the framework is not generally intended to be used in this fashion.
5. Vessel Draft. The client’s analysis used a single draft for their analysis, either
of 13m or 11.5m. The SC-MDP code, instead alternated between the two drafts
between legs of the voyage to account for full or partial loading conditions. For
validation purposes, the SC-MDP code was set to a draft of 13m always.
Validation Results
Two independent sets of results are presented. First, an initial validation study was
performed with the basic University of Michigan SC-MDP code. This was done as
soon as the model was up and running to make sure the model was delivering good
answers from the outset. The other study happened several months later after several
changes and additions to the SC-MDP code had occurred. The vessel parameters used
for the SC-MDP code are given in Table A.1 and the accumulated lifecycle costs are
given in Figure A.1. As seen for this study, the MDP code agrees very well with the
internal methods currently used by the client, for both the original and the update
SC-MDP code.
142

Table A.1: Model parameters used to August validation study.
Ship Inputs
Number of service years 11 years
Year of ﬁrst dry dock 0
Lowest speed tested 19 knots
Highest speed tested 19 knots
LNG fuel mixture percentage 0.9
Number of annual sea days 144 days
Economic Parameters
Engine retroﬁt cost US $0
Number of TEUs removed 488
TEU revenue US $1,500
Annual discount rate 7%
Supply Chain Risk Inputs
Probability of obtaining LNG in Asia 100%
Probability of obtaining LNG in Europe 100%
Emission Control Area Parameters
First year ECA may switch 7
Probability that ECA will switch at given year 100%
First year ECA guaranteed to switch 7
Figure A.1: Results from validation between the SC-MDP framework and a more
traditional lifecycle analysis performed by a commercial client.
143

APPENDIX B
Preliminary Work on Chapter IV presented at the
2015 International Marine Design Conference
144

A Markov Decision Process Framework for Analyzing
LNG as Fuel in the Face of Uncertainty
Austin A. Kana1
, Joshua T. Knight2
, Michael J. Sypniewski1
, and David J. Singer1
ABSTRACT
A Markov decision process (MDP) framework is presented for analyzing temporal design and decision
pathways involving the impact of evolving Emission Control Areas (ECAs) on the design and operation
considerations of a notional 13,000 TEU containership. The major decision is between converting to a
dual fuel liquefied natural gas (LNG) engine or continue alternating between marine diesel oil (MDO)
in the ECA zones and heavy fuel oil (HFO) otherwise. The current low cost of LNG makes converting an
attractive option; however, uncertainties with fuel prices, fuel supply chain risks, the regulatory
framework, conversion costs, and lost revenue, due to reduced TEU capacity, make the decision less
obvious. The effect of lost revenue due to LNG fuel tanks, variations in economic discount rates, and fuel
supply chain risks are examined in detail over a range of speeds.
KEY WORDS
Ship design; LNG; decision analysis; Emission Control Area; uncertainty analysis; Markov decision process
1. INTRODUCTION
The list of challenges facing the shipping industry in the 21st
century is long and continually evolving (Branch 2007). One
of these challenges is designing and adapting to evolving international emissions regulations. These regulations have had,
and will continue to have, drastic effects on ship design (Bengtsson et al. 2011; Goh 2014; Rynbach 2014). The
International Maritime Organization (IMO) currently regulates the emissions of nitrogen oxides (NOx) and sulphur oxides
(SOx) via MARPOL Annex VI regulations as part of their overall strategy of limiting maritime pollution (IMO 2008). Even
though it can be argued that maritime shipping is one of the most environmentally friendly modes of cargo transportation
due to its low CO2 emissions per ton-kilometer, it has been estimated that before 2020 international shipping will overtake
all land-based transport as the largest emitter of SOx and NOx in Europe (Ma 2010). This has caused the IMO to designate
certain environmentally sensitive areas as Emission Control Areas (ECAs) where more stringent emissions standards apply.
The current ECAs lie in either densely populated or environmentally sensitive areas, while proposed areas are still under
consideration (Figure 1).
Figure 1: IMO Regulated Emission Control Areas (Blikom, 2011)
1
Department of Naval Architecture and Marine Engineering, University of Michigan, Ann Arbor, MI, USA.
2
Navatek Ltd. Rhode Island, USA.
145

The NOx and SOx emission limits are set to tighten in the coming years for both the designated ECA zones as well as global
non-ECA zones. Despite the illusion of clarity, these regulations are not precisely defined (Princaud et al. 2010), and the
uncertainty that vessel owners, operators, and designers face remains large. The uncertainty associated with the
geographical extent and implementation date of these regulations will significantly affect how vessels operate and do
business in the coming years. For example, the date for implementing the global 0.5% SOx emission limit is set for 2020,
but may be extended to 2025 if the IMO concludes that there is not enough available fuel. This will be decided in 2018,
which if the IMO decides to stick to the 2020 deadline, would only give vessel owners two years to comply (IMO 2008).
Uncertainty also exists in flag states, coastal states, or individual ports who may decide to set their own regulatory emission
limits (Balland et al. 2013; Stopford 2009).
These regulations can, in many cases, hinder the profitability of the shipping companies (Stopford 2009). In some cases,
vessel owners have applied for temporary extensions, and when not granted, have been forced to change their compliance
strategy, costing millions of dollars (Schuler 2014a). Some have feared that these regulations are so costly that some
companies may intentionally skirt the rules, leaving those in compliance at a serious competitive disadvantage (George
2014). These factors add to the risk that owners and operators have to manage to remain profitable. Other forms of risk that
need to be accounted for include: freight rate risk, operating cost-risk, or interest rate risk (Alizadeh and Nomikos 2009,
Psaraftis et al. 2012). This risk may be compounded by imprecise or incomplete information regarding the fuel or even the
vessel itself (Buckley 2008; Yang et al. 2009). Thus, identifying the optimal decision for compliance in the face of these
risks and uncertainties is not only challenging but highly important to remain economically competitive.
Currently there are four compliance pathways available to vessels to satisfy upcoming NOx and SOx regulations: (1) burning
distillate fuel, such as marine gas oil (MGO) or marine diesel oil (MDO) (Bengtsson et al. 2011), (2) use liquefied natural
gas (LNG) as a bunkering fuel, (3) install SOx scrubbers or selective catalytic converters (SCRs) for NOx (Santala 2012;
Andersson and Winnes 2011), or (4) reduce vessel transit speed (Ship and Bunker 2013). While all four are potential
avenues for compliance, they each face technological and economic challenges. Fathom Shipping (2014) summarizes many
of the issues with compliance, including: the rising cost of bunker fuel and transportation, the practicality and costs of
retrofitting vessels, the mechanical problems arising from fuel switching, fuel availability issues, the probability of losing
vessel power, competitive disadvantage with making the wrong compliance choice, the changes to bunker delivery notes,
and the economic issues with supplying abundant and adequate lubricant. Thus, deciding on the best solution for a given
vessel is a challenging process due to the vast number of possible compliance strategies (Balland et al. 2013).
Even though roughly 95% of the world’s shipping fleet has traditionally run on diesel fuel (Nikopoulou et al. 2013), many
have looked to switching to LNG as a logical choice from both an environmental and economic perspective (Banawan et
al. 2010). However, switching to LNG as a primary fuel can have drastic implications on the ship as a whole. The required
volume for LNG fuel tanks can be as much as 3-4 times that of standard bunker oil, plus the ship still needs the ability to
carry the required amount of bunker oil in cases where LNG may not be available (Rynbach 2014). This is on top of the
auxiliary equipment that is necessary, such as gas supply piping, gas detection and exhaust ventilation systems, and other
components (Banawan et al. 2010). Switching to LNG can drastically affect the number of TEUs a given containership
may be able to carry, which causes lost potential revenue to the ship owner or operator. This lost revenue is only potential,
as most vessels do not necessarily leave port at full capacity due to market conditions (Almeida 2014) or port draught
restrictions, as may be the case for the very large cargo ships (Schuler 2014b). These technical reasons have caused
estimations of shipbuilding costs to be 20%-25% higher than ships with conventional engines (Nikopoulou et al. 2013).
With over half of vessel operating costs going towards fuel (Lin and Lin 2006), any variations in fuel prices will have a
drastic effect on the vessel’s bottom line. There are also supply chain issues, as the regulatory environment and
infrastructure for storage and bunkering of LNG fuel is still under development (Bengtsson et al. 2011, Nikopoulou et al.
2013).
The Markov decision process (MDP) framework detailed below is designed specifically to quantifiably handle these
uncertainties in determining optimal design and decision pathways. A case study is presented involving making a design
decision for a containership between converting to a dual fuel liquefied natural gas (LNG) engine or continue alternating
between marine diesel oil (MDO) in the ECA zones and heavy fuel oil (HFO) otherwise.
2. METHODS
2.1 Model Assumptions
The vessel under consideration is a notional 13,000 TEU containership routed between Rotterdam and China, a round trip
distance of 22,000 nm. The principal characteristics of the ship are given in Table 1. The initial ECA coverage is 1,100 nm,
or 5% of the total route. This coverage eventually increases to 6,800 nm, or 31%, in a single year. The draught of the vessel
is 13 m when carrying a full load from China to Rotterdam, and is 11.5 m when carrying a partial load (or back-hauling
empty containers) from Rotterdam to China. An assumed 60% total propulsive efficiency is used to calculate ship brake
power. This efficiency is an aggregation of hull efficiency, rotative efficiency, shaft and bearing efficiency, gearing
efficiency, open water efficiency, and a service margin.
146

Table 1: Vessel Principal Characteristics
Length between perpendiculars (Lbp) Greater than 300 m
Beam Greater than 44 m
Draught 13.0 m (full load); 11.5 m (partial load)
Block coefficient (Cb) 0.63
Displacement (∆) 113,000 MT (full load); 100,000 MT (partial load)
Required power at 19 knots 22,000 kW
Total propulsive efficiency 60%
Ship brake power 37,000 kW
The fuel consumption curves for speeds of 12 to 19 knots and the two draughts were developed (Figure 2) using an in-
house University of Michigan powering and resistance prediction program that uses the Holtrop and Mennen method.
Figure 2: Projected Fuel Prices and Specific Fuel Consumption Curves
To model the lost revenue stemming from installing LNG tanks, it is assumed that a set number of TEUs are removed to
fit the LNG fuel tanks. Initially, it is assumed that 244 TEUs need to be removed. Different scenarios are run varying this
parameter and the results of which are discussed below in Section 4.2.2 Test Scenario 2: Effects of Ignoring Lost Revenue.
2.2 Model Variables
The model variables cover a range of ship parameters, economics parameters, supply chain risks, and Emission Control
Areas scenarios. The specific list of variables is given below.
2.2.1 Ship Parameters
 Number of service years
 The first year the ship can go into dry dock
 The fuel mixture percentage (LNG to HFO)
 Number of annual sea days
2.2.2 Economic Parameters
 The cost to retrofit to a dual fuel engine
 Number of TEUs removed to fit LNG fuel tanks
 TEU freight rate
 Annual discount rate
2.2.3 Supply Chain Risk Parameters
 Probability of obtaining LNG in both China and Rotterdam
2.2.4 Emission Control Areas Parameters
 First year ECA coverage might increase from 5% to 31% of total route.
 The probability that the designated ECA coverage will increase any given year after the first year it may increase.
 Year by which ECA is guaranteed to have increased: This is the year in which it is guaranteed that the ECA
coverage has increased from 5% to 31% of the prescribed route.
These variables model the uncertainty associated with the implementation of new Emission Control Areas. For example,
to model a situation where there is a 50% belief that ECA may increase in five years, but is guaranteed to have switched
after ten years, the first year will be set to 5, the probability will be set to 50%, and the guaranteed year will be set to 10.
2.3 Markov Decision Process
Markov decision processes (MDPs) are designed to model and solve dynamic stochastic sequential decision-making
problems. They are state-based representations of systems that handle uncertainty, can differentiate actions, and can handle
non-stationary developments. The classic MDP is defined as a 4-tuple ‹S, A, T, R›, where S is a set of finite states where
0 5 10 15 20
400
600
800
1000
Year
FuelPrice(US$perton)
Projected Fuel Prices
MDO
HFO
LNG
12 14 16 18 20
20
40
60
80
100
speed (kts)Specificconsumption(tons/day)
Specific Fuel Consumption
T = 13m, HFO
T = 11.5m, HFO
T = 13m, MDO
T = 11.5m, MDO
T = 13m, LNG
T = 11.5m, LNG
147

the agent can exist, A is the set of actions which the agent can take, T is the probability the agent will transition from one
state to another after taking a given action, and R is the reward the agent receives by executing a given action, a, and
transitioning to a new state, s’. The objective is to identify an optimal policy that maximizes the cumulative, long term
utility. This policy identifies the best action the agent should take during each decision epoch by considering both the
outcomes of current decisions and future opportunities (Puterman 2005). As such, MDPs are memoryless, that is, the
optimal decisions do not rely on the action history of how the agent arrived at a given state (Puterman 2005). The optimal
policy can be obtained via equation 1, known as the Bellman equation (Russell and Norvig 2003).
max ∑ , , [1]
Where U is the expected utility, is the discount factor, and is the state in the next epoch in time. The optimal policy, ,
is found by taking the argument of the max operator above, as defined in equation 2 (Russell and Norvig 2003).
max , , [2]
When the transition probabilities, rewards, or the optimal policy do not change with time, the process is known as
stationary, otherwise the process is non-stationary (Niese 2012; Puterman 2005). From a ship design perspective, non-
stationary processes are common. For example, non-stationary transitions arise from varying regulations, policies, supply
chain risks, or performance drift, while non-stationary rewards come from economic variability in fuel prices or evolving
budgetary requirements. Non-stationary transitions or rewards always lead to non-stationary optimal policies.
The common output of solving a non-stationary MDP is a decision matrix, which provides the optimal actions for each
state for each decision epoch. The decision matrix can be thought of as a roadmap of optimal actions for the decision maker.
An example of a non-stationary decision matrix from Niese (2012) is given in Table 2.
Table 2: Sample non-stationary decision matrix
State 1 State 2 … State n
Epoch 1 Action A Action A … Action A
Epoch 2 Action B Action C … Action C
… … … … …
Epoch T Action D Action B … Action A
The advantages of using MDPs over conventional methods are numerous, including: quantitative inclusion of uncertainties
associated with policy implementation and supply chain risks, a temporal optimal decision matrix which shows the optimal
decision at each time epoch, the optimal state at each time epoch, and the net present value of following the optimal policy.
Previous research using MDPs applied to ship design include analysis of ballast water treatment methods and designing
for the Energy Efficiency Design Index (Niese 2012; Niese and Singer 2013, 2014).
2.4 LNG Case Study Framework
This section details how the states, actions, transition probabilities, and rewards are defined in the LNG conversion model.
2.4.1 States
Each epoch level consists of three different state variables:
 The percentage of the route that is designated ECA. There are two possibilities for ECA coverage: one where
1,100 nm (5%) of the route is designated ECA, and one where 6,800 nm (31%) is designated ECA. The specific
year that ECA will switch from 5% to 31% varies depending on the specifics of the user defined inputs.
 Type of engine. There are two types of possible engines: single fuel and dual fuel. The model will only allow the
vessel to switch to a dual fuel engine after it has gone into dry dock.
 Type of fuel. There are two fuel options: 1) a single fuel option that alternates between burning MDO in the
designated ECA zones and HFO elsewhere, and 2) a dual fuel option that burns a mixture of LNG and HFO. When
the single fuel engine is installed, MDO and HFO are the only viable fuels; however, a dual fuel engine may run
on MDO, HFO, or a dual fuel mixture of LNG and HFO. When LNG fuel is unavailable, the dual fuel engine runs
on MDO and HFO (El-Gohary 2012).
2.4.2 Starting State
The vessel initially has a single fuel engine installed, is running on MDO and HFO, and designated ECA zones cover 5%
of the prescribed route.
2.4.3 Actions
After each leg of the voyage, a decision is made. This simulates the vessel arriving in port and having to make a decision
on the bunkering fuel, conditionally dependent on its type of engine. There are four possible actions:
148

1. Do not switch engines, and try to purchase LNG fuel
2. Do not switch engines, and purchase MDO fuel
3. Switch to a dual fuel engine, and try to purchase LNG fuel
4. Switch to a dual fuel engine, and purchase MDO fuel
The action to switch engines is only available after the ship has gone into dry dock. The time spent in dry dock is accounted
for in the number of annual sea days. Thus, while it may appear that engine switching can occur on the average stop in
port, this is not the case, as time is lost due to dry docking. The action “Switch to a dual fuel engine, and purchase MDO
fuel” is included in case there is a situation where the optimal action is to switch engines now to plan for low LNG fuel
prices in the future. The chosen action is based on the accumulated highest net present value reward (i.e. lowest cost).
2.4.4 Transition Probabilities
The transition probabilities for each state are split between action independent states and action dependent states. The size
of the designated ECA zones is an action independent state, as the probability of increasing to a state with larger ECA
coverage does not depend on the ship owner’s previous actions. However, the type of engine installed and the type of fuel
constitute action dependent states, as the probability of entering in to one of those states depends on the previous action
taken. The probability of transitioning between the various states is defined as follows:
 The percentage of the route that is designated ECA. The probability of transitioning from an ECA coverage of
5% to 31% varies depending on user defined variables, described above in section 2.2.4 Emission Control Areas.
 Engine type. The switch from a single fuel to a dual fuel engine is deterministic depending on the optimal action.
 Type of fuel. The type of fuel is selected based on the optimal action and associated supply chain risk. When LNG
is unavailable, the vessel may try to purchase LNG, but will be forced to purchase MDO and HFO instead.
2.4.5 Rewards
The reward at each state is the accumulated net present cost. This cost is calculated at each decision epoch (i.e. each leg of
the voyage) according to the following reward function given in equation 3.
min ) [3]
 Fuel cost: The fuel cost function is given in equation 4, where the given percentage takes into account either the
designated ECA coverage percentage or the dual fuel mixture.
∗ ∗ ∗ [4]
 Opportunity cost: This is the lost revenue due to the installation of LNG fuel tanks that reduces TEU capacity.
 Retrofit cost: This is the cost of retrofitting to a dual fuel engine.
3. RESULTS
Four different scenarios were tested. The first was a baseline case designed to model reasonable assumptions for the
variables above. The other three scenarios investigate the impacts of lost revenue, discount rates, and supply chain risk on
the decision space through time over a range of speeds.
3.1 Test Scenario 1: Baseline
The first scenario was developed to create a baseline (Table 3). $10 million is used as an estimate for the LNG system
retrofit cost, which amounts to US $270/kW. This value agrees with estimates made by Banawan et al. (2010) of US $220-
340/kW for retrofitting a diesel engine to LNG.
Table 3: Model variables for baseline scenario
Number of service years 20 years
Year of first dry dock 3
Fuel mixture percentage (LNG to HFO) 90%
Number of annual sea days 290
LNG system retrofit cost $10 million
Number of TEUs removed for LNG tanks 244
TEU freight rate 1500
Annual discount rate 7%
Probability of obtaining LNG 100%
First year ECA coverage may increase 5
Probability that ECA will increase at given year 50%
First year ECA is guaranteed to have increased 10
149

Two metrics unique to this method are used to display the optimal decision pathway: the optimal action entry plot and the
optimal states accessed plot. The optimal action entry plot displays the optimal action at each decision epoch, given the
prescribed starting state. Shading in the figure represents the percentage of time the given action is optimal. Solid white
means the action is never optimal, while solid black means the action is optimal 100% of the time. The optimal states
accessed plot pairs well with the optimal action entry plot by displaying the optimal states that the vessel will be in after
performing the optimal actions detailed above. The optimal states accessed plot marks which states are accessible through
time given that the optimal policies are followed. Shading is used in the same way as the optimal action entry plot to denote
the probability that a given state is accessed at a given epoch.
Two additional economic metrics are developed and presented, including: the expected cost plot and the expected savings
plot. The expected cost plot illustrates the cumulative, net present value cost curves for two unique decision pathways: 1)
assuming the optimal decision pathway as calculated by the MDP is followed, and 2) the base case where the vessel does
not switch engines and continues to burn MDO and HFO throughout its lifecycle. The expected savings from following
the optimal policy plot displays the difference between the base case pathways and optimal decision pathway.
As seen in Figures 3, 4 and 6, a decision transition area is identified around a speed of 18 knots. When slow-steaming at
speeds of 17 knots or less, the optimal decision pathway is to keep the single fuel engine and always burn MDO. For these
speeds the expected costs and savings are not shown, as the costs for following the optimal policy is the same as the status
quo. Thus, there are no savings for following the optimal policy.
Figure 3: Optimal action entry and optimal states accessed plot for baseline scenario at 17 knots
For a speed of 18 knots, the optimal decision is to switch to a dual fuel engine as soon as the designated ECA coverage
increases from 5% to 31% over the sample route. The shading in Figure 4 represents the uncertainty of the exact year that
ECA will increase. According to the plot, the optimal action up to year 5 is to keep the current engine, and continue burning
MDO. At the beginning of year 5 there is a chance that the optimal action may be to switch engines and try to purchase
LNG. This is due to ECA coverage increasing from 5% to 31%. However, if ECA does not switch in that year, there is a
probability that the optimal action is to keep the same engine, and continue burning MDO. By year 9, the probability that
the vessel has switched engines between years 5 and 8 is nearly certain, and the optimal action is to keep the dual fuel
engine, and try burning LNG. When the optimal action is to “keep engine”, the optimal states accessed plot below displays
whether that is single fuel or dual fuel.
Year
Optimal States Accessed
(speed 17 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Single fuel engine: burn MDO
Dual fuel engine: burn MDO
Single fuel engine: burn LNG
Dual fuel engine: burn LNG
0
0.5
1
Year
Optimal Action Entry Plot
(speed 17 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
keep engine: try buying LNG
keep engine: buy MDO
switch to dual fuel: try buying LNG
switch to dual fuel: buy MDO
0
0.5
1
Year
(speed 18 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
Year
(speed 18 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
150

The optimal states accessed plot (Figure 4) shows that the vessel should burn MDO on a single fuel engine up to year 5.
Between years 5 and 8 the optimal state may be either single fuel or dual fuel, depending on when ECA actually increases
from 5% to 31%. After year 9, the optimal state is the dual fuel engine burning LNG nearly 100% of the time.
As shown in Figure 5, by following this optimal pathway, the accumulated savings is just under US $2 million, with a
payback of upwards of ten years. Since the optimal decision is to switch engines when the regulatory policy changes, and
since there is uncertainty as to when the policy changes, there is also associated uncertainty as to when to switch engines.
This is shown by the jumps in expected cost and expected optimal policy savings curves at years five through ten, which
represent the retrofit costs of switching to a dual fuel engine. By year 15, the cost of switching engines has been recouped
by the cheaper LNG fuel, and by year 20 there are significant savings. In the expected optimal policy savings plot, there
are three distinct areas: 1) prior to switching engines (year 5 in this case) where there are no savings, 2) when the vessel
has not recouped the cost of the LNG system retrofit and the savings are negative (between year 5 and 15), and 3) when
the cost of the retrofit has been fully recouped by the low cost LNG fuel and savings are positive (after year 15).
Figure 5: Expected cost and optimal policy savings for the baseline case at 18 knots
For a speed of 19 knots the optimal decision pathway is to switch to a dual fuel engine at the first dry dock opportunity
(year 3). By following this optimal policy, an accumulated savings of over US $7 million is expected, with a payback of
less than 11 years. Cases were not run for speeds greater than 19 knots due to lack of specific fuel consumption data, but it
is expected that the decision to switch to a dual fuel engine at the first dry dock opportunity will continue to be optimal.
Figure 7: Expected cost and optimal policy savings for the baseline scenario at 19 knots
0 5 10 15
0
5
10
15
x 10
7
Expected Cost
(speed 18 knots)
Year
expectednetpresent
cost(US$)
optimal policy
base case
0 5 10 15 20
-6
-4
-2
0
2
x 10
6
Expected optimal policy savings
(speed 18 knots)
Year
expectednetpresent
savings(US$)
Year
(speed 19 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
Year
(speed 19 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
0 5 10 15 20
0
0.5
1
1.5
2
x 10
8
Expected Cost
(speed 19 knots)
Year
expectednetpresent
cost(US$)
optimal policy
base case
0 5 10 15 20
-1
-0.5
0
0.5
1
x 10
7
(speed 19 knots)
Year
expectednetpresent
savings(US$)
151

Table 4 shows this transition in the optimal policy around a speed of 18 knots. Analyzing the optimal decision pathway
from a temporal perspective across varying speeds gives the ship owner or operator great control as they can have some
control over their payback period based on retrofit timing and vessel speeds. Greater savings are realized at higher speeds;
however, as are higher costs. Broader market conditions may dictate whether the vessel can afford those higher costs.
Table 4: Results summary for baseline scenario
Speed
(kts)
Accumulated
Cost (US$MM)
Expected Savings
(US$MM)
Expected
Payback Year
Optimal Policy
12 $ 56 $ 0 N/A Never switch engines
13 $ 65 $ 0 N/A Never switch engine
18 $ 157 $ 2 16 Invest in dual fuel engine as soon as policy changes
19 $ 185 $ 7 11 Invest in dual fuel engine as soon as possible
3.2 Test Scenario 2: Effects of Ignoring Lost Revenue
This test scenario examines the importance of lost potential revenue due to the presence of the LNG fuel tanks has on the
decision space. A reasonable lost revenue model is key to accurately determining the transition areas in the decision
pathways. Even though ships are rarely at full capacity due to market conditions or port draught restrictions (Almeida 2014;
Schuler 2014b), adequate lost revenue assumptions are key. The assumption that ship is losing TEU volume may be too
strict for most markets, and may only be applicable in times of very profitable markets.
This case assumes no containers are removed to fit the LNG tanks, and thus there is no lost revenue associated with
switching to LNG. This models a potential situation when the market demand for transporting containers does not meet the
available supply. This assumption drastically changes the optimal decision (Table 5). In this scenario, the optimal decision
is to always switch to a dual fuel engine and try to burn LNG at the first dry dock opportunity, no matter the vessel speed.
Table 5: Results summary when the effects of lost revenue are ignored
Speed
(kts)
Accumulated
Cost (US$MM)
Expected Savings
(US$MM)
Expected
Payback Year
Optimal Policy
3.3 Test Scenario 3: Effects of Lower Economic Discount Rates
This test scenario explores the dependence of the financial discount rate on the optimal decision. The annual discount rate
in this study was dropped from the initial baseline case of 7% to 3.5%. The relationship between the annual discount rate,
d, and the discount factor, , is shown in equation 5, while the relation between the discount factor and the interest rate, i,
is given in equation 6 (Puterman 2005).
1 [5]
1 [6]
Referring to Figures 8 and 9, the discount rate plays a pivotal role in not only the speeds at which it is optimal to switch,
but also the optimal policy when dealing with regulatory uncertainty. Compared to higher discount rates, the speed at which
it is optimal to switch engines is lower. Here, the break point is roughly 17 knots. For speeds of 16 knots or lower, the
optimal decision is to keep the single fuel engine and burn MDO. However, for speeds of 18 or higher, the optimal policy
is to switch engines at the first dry dock opportunity.
For 17 knots, the optimal policy is to switch to a dual fuel engine and try to purchase LNG only if the designated ECA
coverage increases from 5% to 31% at year 5. If ECA coverage increases at year 6, the optimal policy is to keep the single
fuel engine and burn MDO. The lower discount rate increases the payback time necessary for the LNG system retrofit. The
accumulated savings is roughly US $100,000 with a payback of nearly 20 years, or the entire lifespan of the ship. While
152

this pathway is optimal, it is not expected that the vessel owner will make this switch given the low savings and long
payback time. Not shown is the case where the discount rate is 3.5% and when ECA coverage increases at year 6. In this
instance, the optimal action is to keep the single fuel engine throughout the lifecycle of the vessel.
Figure 8: Optimal action entry and optimal states accessed plot for lower economic discount rates at 17 knots
Figure 9: Expected cost and optimal policy savings for lower economic discount rates at 17 knots
The results for all speeds are summarized in Table 6. The optimal policy bifurcation at a speed of 17 knots is apparent.
Table 6: Results summary with a lower economic discount rate
Speed
(kts)
Accumulated
Cost (US$MM)
Expected Savings
(US$MM)
Expected
Payback Year
Optimal Policy
17 $ 189 $ 0.1 20 Invest in dual fuel engine only if ECA coverage
increases at year 5, otherwise do not switch engines
3.4 Test Scenario 4: Effects of LNG Supply Chain Risk
This test scenario examines the effect of supply chain issues on the optimal decision space. In this scenario, the probability
of obtaining LNG in Asia is only 50%, while the probability of obtaining LNG in Europe remains at 100%. For speeds of
17 knots or less, the optimal policy is to do nothing, and continue running MDO on the single fuel engine. For 18 knots the
optimal policy is to switch engines and try buying LNG only if ECA coverage increases at year 5. If ECA increases at year
6 or later, the optimal policy is to keep the single fuel engine. Once the vessel has switched to dual fuel engine, there is a
50% chance that it will have to burn MDO when bunkering in Asia. This is shown by the alternating hatching in the optimal
states accessed plot on the bottom of Figure 10. The accumulated savings are less than US $80,000 with a payback of 20
years. Thus, similar to the case above, while this pathway is optimal, it is not expected that the owner will switch engines
due to the low savings and long payback period.
Year
(speed 17 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
Year
(speed 17 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
0 5 10 15 20
0
0.5
1
1.5
2
x 10
8
Expected Cost
(speed 17 knots)
Year
expectednetpresent
cost(US$)
optimal policy
base case
0 5 10 15 20
-5
-4
-3
-2
-1
0
1
x 10
6
(speed 17 knots)
Year
expectednetpresent
savings(US$)
153

Figure 10: Optimal action entry and optimal states accessed plot for a scenario with supply chain risk at 18 knots
Figure 11: Expected cost and optimal policy savings for a scenario with supply chain risk at 18 knots
For a speed of 19 knots, the optimal policy is to switch to a dual fuel engine and try burning LNG at the first dry dock
opportunity, despite the supply chain risk issues. However, there is still a 50% chance the vessel will have to purchase
MDO in Asia when it wishes to purchase LNG. Accumulated savings of roughly US $3.5 million are expected with a
payback of nearly 14 years. Supply chain issues clearly reduce the accumulated savings and extend the payback period.
Figure 12: Optimal action entry and optimal states accessed plot for a scenario with supply chain risk at 19 knots
Figure 13: Expected cost and optimal policy savings for a scenario with supply chain risk at 19 knots
Year
(speed 18 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
Year
(speed 18 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
0 5 10 15 20
0
0.5
1
1.5
2
x 10
8
Expected Cost
(speed 18 knots)
Year
expectednetpresent
cost(US$)
optimal policy
base case
0 5 10 15 20
-4
-3
-2
-1
0
1
x 10
6
(speed 18 knots)
Year
expectednetpresent
savings(US$)
Year
(speed 19 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
Year
(speed 19 knots)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
0.5
1
0 5 10 15 20
0
0.5
1
1.5
2
x 10
8
Expected Cost
(speed 19 knots)
Year
expectednetpresent
cost(US$)
optimal policy
base case
0 5 10 15 20
-10
-5
0
5
x 10
6
(speed 19 knots)
Year
expectednetpresent
savings(US$)
154

Table 7 shows the transition in the optimal policy around a speed of 18 knots.
Table 7: Results summary when there are LNG supply chain risks
Speed
(kts)
Accumulated
Cost (US$MM)
Expected Savings
(US$MM)
Expected
Payback Year
Optimal Policy
18 $ 159 $ 0.1 20 Invest in dual fuel engine only if ECA coverage
increases at year 5, otherwise do not switch engines
4. DISCUSSION
While this framework quantifies the optimal pathways in the face of uncertainty, it is still at the discretion of the decision
makers as to whether they choose to follow the optimal policy. In some cases the payback period may be too long or the
savings too little for a ship owner to feel comfortable investing. Decisions are also severely impacted by whether the vessel
is under charter. The type of charter may also play a key role in not only who is making the decisions, but the risk the
decision maker is willing to accept. This approach is a means of providing the necessary and unique quantitative
information to those faced with making these decisions in the face of uncertainties.
This paper only discussed the optimal decision pathways as it pertains to LNG as a fuel; however, there are other ways of
meeting the upcoming ECA regulations, such as use of distillate fuels, installation of scrubbers, or possibly slow steaming.
Future research could include analysis across all methods of compliance. Also, for this particular case study to be applicable
for commercial use, a more advanced fuel cost and freight rate model, supply chain risk model, and measured vessel fuel
consumption curves would be beneficial. While these underlying models appear simplistic, the overarching theory and
methods still hold.
5. CONCLUSIONS
This paper has demonstrated the utility of using Markov decision processes as a design and decision analysis framework
for ship designers, owners, and operators. Several advantages of this approach bear repeating, including: a quantitative
framework that systematically determines optimal temporal decision pathways in the face of uncertainty, identification of
temporal decision bifurcation areas, and elucidation of how regulatory uncertainty and supply chain risks affect not only
the ship itself, but also its operational plan. In regards to the LNG case study, new insight has been gained from the four
case studies on the optimal decision pathways, with regards to lost revenue assumptions, discount rate impacts, and supply
chain risks.
6. REFERENCES
ALIZADEH, A. H., and NOMIKOS, N. K., Shipping Derivatives and Risk Management. New York, NY: Palgrave
Macmillan, 2009.
ALMEIDA, R. “2013 Was a Tough Year for China Shipping,” gCaptain, April 29, 2014.
ANDERSSON, K., and WINNES, H., “Environmental Trade-offs in Nitrogen Oxide Removal from Ship Engine Exhausts.”
Journal of Engineering for the Maritime Environment. 225:1 (2011): 33-42.
BALLAND, O., ERIKSTAD, S. O., FAGERHOLT, K., WALLACE, S. W., “Planning Vessel Air Emission Regulations
Compliance under Uncertainty.” Journal of Marine Science and Technology. 18 (2013): 349-357.
BANAWAN, A. A., EL-GOHARY, M. M., and SADEK, I. S., “Environmental and Economical Benefits of Changing from
Marine Diesel Oil to Natural-gas Fuel for Short-voyage High-power Passenger Ships.” Journal of Engineering for the
Maritime Environment. 224:2 (2010): 103-113.
BENGTSSON, S., ANDERSSON, K., and FRIDELL, E., “A Comparative Life Cycle Assessment of Marine Fuels: Liquefied
Natural Gas and Three Other Fossil Fuels.” Journal of Engineering for the Maritime Environment, 225:2 (2011): 97-110.
BLIKOM, L. P. (2011). LNG for Greener Shipping in North America. blogs.dnvgl.com. Accessed: August, 2014.
155

BRANCH, A. E., Elements of Shipping. 8th
Edition. New York, NY: Routledge, 2007.
BUCKLEY, J. J., The Business of Shipping. 8th
Edition. Atglen, PA: Cornell Maritime Press, 2008.
EL-GOHARY, M. M., “The Future of Natural Gas as a Fuel in Marine Gas Turbine for LNG Carriers.” Journal of Engineering
for the Maritime Environment, 226:4 (2012): 371-377.
FATHOM SHIPPING. “10 Reasons Why Ship Operators Should be Nervous about the 2015 ECA Regulations.” gCaptain.
November 4, 2014.
GEORGE, L., “New EU Marine Fuel Rules Erode Shippers’ Gains from Cheap Oil.” Reuters. December 17, 2014.
GOH, B, “High Fuel Costs and New Emissions Regs are pushing the Boundaries of Naval Architecture,” gCaptain, November
7, 2014.
IMO. MARPOL Annex VI: Regulations for the Prevention of Air Pollution from Ships, 2008.
LIN, B., and LIN C., “Compliance with International Emission Regulations: Reducing the Air Pollution from Merchant
Vessels.” Marine Policy, 30 (2006): 220-225.
MA, S., “Using Economic Measures for Global Control of Air Pollution from Ships.” The Handbook of Maritime
Economics and Business. 2nd
Edition. Ed. Costas Th. Grammenos. London, UK: Lloyd’s List, 2010.
NIESE, N., “Life Cycle Evaluation under Uncertain Environmental Policies Using a Ship-Centric Markov Decision Process
Framework.” PhD thesis, University of Michigan (2012).
NIESE, N. and SINGER, D., “Strategic Life Cycle Decision-making for the Management of Complex Systems Subject to
Uncertain Environmental Policy.” Ocean Engineering, 72 (2013): 365-374.
NIESE, N. and SINGER, D., “Assessing Changeability under Uncertain Exogenous Disturbance.” Research in Engineering
Design, 25 (2014): 241-258.
NIKOPOULOU, Z., CULLINANE, K., and JENSEN, A., “The Role of a Cap-and-trade Market in Reducing NOx and SOx
Emissions: Prospects and Benefits for Ships within the Northern European ECA.” Journal of Engineering for the Maritime
Environment, 227:2 (2013): 136-154.
PRINCAUD, M., CORNIER, A., and FROELICH, D., “Developing a Tool for Environmental Impact Assessment and
Eco-design for Ships.” Journal of Engineering for the Maritime Environment. 223:3 (2010): 207-223.
PSARAFTIS, H. N., LYRIDIS, D. V., and KONTOVAS, C. A., “The Economics of Ships.” The Blackwell Companion to
Maritime Economics. Ed. Wayne K. Talley. West Sussex, UK: Wiley-Blackwell, 2012.
PUTERMAN, M. L., Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ: Wiley, 2005.
RUSSELL, S., and NORVIG, P., Artificial Intelligence: A Modern Approach. 2nd
Edition. Upper Saddle River, NJ: Prentice
Hall, 2003.
RYNBACH, E. V., “Key Impacts of LNG Fuel Storage on Commercial Ship Design.” Maritime Professional, 4Q (2014):
24-29.
SANTALA, M., “Realising the Competitive Potential of Sulphur ECA Compliance.” Wartsila. August, 2012.
SCHULER, M., Brittany Ferries Suspends LNG-powered Ferry Plans. gCaptain. October, 2014a.
SCHULER, M., Watch: Triple-E Leaves Port with World Record Load. gCaptain. August, 2014b.
STOPFORD, M., Maritime Economics. 3rd
Edition. New York, NY: Routledge, 2009.
SHIP and BUNKER, “Transport Canada: ECA Compliance Achievable but Challenging.” Ship and Bunker, May, 2013.
YANG, Z. L., MASTRALIS, L., BONSALL, S., and WANG, J., “Incorporating Uncertainty and Multiple Criteria in Vessel
Selection.” Journal of Engineering for the Maritime Environment. 223:2 (2009): 177-188.
156

BIBLIOGRAPHY
ABET (2015). Criteria for Accrediting Engineering Programs. Engineering Accredi-
tation Commission, Baltimore, MD.
ABS (2010). Fuel switching advisory notice. Technical report, American Bureau of
Shipping (ABS).
ABS (2013). Exhaust gas scrubber systems: Status and guidance. Technical report,
American Bureau of Shipping (ABS).
Alizadeh, A. H. and Nomikos, N. K. (2009). Shipping Derivatives and Risk Manage-
ment. Palgrave Macmillan, New York, NY.
Almeida, R. (2014). Saving the environment by using old tankers in new ways.
gCaptain. May 13.
Alphaliner (2015). Weekly newsletter 15.04.2015 to 21.04.2015. Technical Report 16,
Alphaliner.
Amato, C., Chowdhary, G., Geramifard, A., ¨Ure, N. K., and Kochenderfer, M. J.
(2013). Decentralized control of partially observable markov decision processes. In
52nd IEEE Conference on Decision and Control, pages 2398–2405, Florence, Italy.
IEEE.
Andersson, K. and Winnes, H. (2011). Environmental trade-oﬀs in nitrogen oxide
removal from ship engine exhausts. Journal of Engineering in the Maritime Envi-
ronment, 225(1):33–42.
Andrews, D., Duchateau, E., Gillespe, J., Hopman, H., Pawling, R., and Singer, D.
(2012). State of the art report: Design for layout. In 11th International Marine
Design Conference, pages 111–137, Glasgow.
Anton, H. and Rorres, C. (2005). Elementary Linear Algebra: Applications Version.
Wiley, Hobokoen, NJ, ninth edition.
Balland, O., Erikstad, S. O., Fagerholt, K., and Wallace, S. W. (2013). Planning
vessel air emission regulations compliance under uncertainty. Journal of Marine
Science and Technology, 18(3):349–357.
158

Banawan, A. A., El-Gohary, M. M., and Sadek, I. S. (2010). Environmental and eco-
nomic benefits of changing from marine diesel oil to natural-gas fuel for short-voyage
high-power passenger ships. Journal of Engineering for the Maritime Environment,
224(2):103–113.
Bengtsson, S., Andersson, K., and Fridell, E. (2011). A comparative life cycle assess-
ment of marine fuels: liquefied natural gas and three other fossil fuels. Journal of
Engineering in the Maritime Environment, 225(2):97–110.
Bernold, L., Bingham, W., McDonald, P., and Attia, T. (2000). Impact of holistic and
learning-oriented teaching on academic success. Journal of Engineering Education,
89(2):191–199.
Bishop, C. M. (2009). Pattern Recognition and Machine Learning. Springer, USA.
Blikom, L. P. (2011). LNG for greener shipping in North America. blogs.dnvgl.com.
February.
Buckley, J. J. (2008). The Business of Shipping. Cornell Maritime Press, Atglen, PA,
eight edition.
California State Lands Commission (2010). 2010 assessment of the efficacy, avail-
ability, and environmental impacts of ballast water treatment systems for use in
California waters. Technical report, California State Lands Commission.
Casarosa, L. (2011). The integration of human factors, operability and personnel
movement simulation into the preliminary design of ships utilising the Design Build-
ing Block approach. PhD thesis, University College London, London, UK.
Caswell, H. (2001). Matrix Population Models: Construction, Analysis, and Interpre-
tation. Sinauer Associates, Inc., Sunderland, MA, 2nd edition.
Coraddu, A., Figari, M., and Savio, S. (2014). Numerical investigation on ship energy
efficiency by Monte Carlo simulation. Journal of Engineering for the Maritime
Environment, 228(3):220–234.
Cressie, N. and Wikle, C. K. (2011). Statistics for Spatio-Temporal Data. Wiley,
Hoboken, NJ.
Danish Maritime Authority (2012). North European LNG infrastructure project: A
feasibility study for an LNG filling station infrastructure and test for recommen-
dations. Technical report, Danish Maritime Authority.
Davis, A. N. and Levy, A. D. (2012). U.S. Coast Guard ballast water discharge rule
establishes new requirements and enforcement risks for ship owners and operators.
Shipman and Goodwin LLP.
Dean, R. G. and Dalrymple, R. A. (1991). Water Wave Mechanics for Engineers and
Scientists. World Scientific, River Edge, NJ.
159

D’eon, R. (2014). Marine Technology, chapter Recapitalizing the Fleet: Canada’s
National Shipbuilding Procurement Strategy and what it means for the future of
the country’s marine industry, pages 28–35. SNAME.
DieselNet (2011). International: IMO marine engine regulations. www.dieselnet.com.
El-Gohary, M. M. (2012). The future of natural gas as a fuel in marine gas turbine for
LNG carriers. Journal of Engineering for the Maritime Environment, 226(4):371–
377.
EPA (2014a). Nitrogen oxide: Health. www.epa.gov. November, 2014.
EPA (2014b). Sulfur dioxide: Health. www.epa.gov. November, 2014.
Fagerholt, K., Christiansen, M., Hvattum, L. M., Johnsen, T. A., and Vabø, T. J.
(2010). A decision support methodology for strategic planning in maritime trans-
portation. Omega, 38:465–474.
Faltinsen, O. M. (1990). Sea Loads on Ships and Offshore Structures. Cambridge
University Press, Cambridge, UK.
Felder, R. M. and Brent, R. (2004). The intellectual development of science and engi-
neering students: Part 1. Models and challenges. Journal of Engineering Education,
93(4):269–277.
Felder, R. M. and Brent, R. (2005). Understanding student differences. Journal of
Engineering Education, 94(1):57–72.
Fet, A. M., Aspen, D. M., and Ellingsen, H. (2013). Systems engineering as a holistic
approach to life cycle designs. Ocean Engineering, 62:1–9.
Freedberg Jr., S. J. (2015). McCain warns Navy on LCS upgrade. breakingde-
fense.com. March 10.
Frickle, E. and Schulz, A. P. (2005). Design for changeability (DfC): Principles to
enable changes in systems throughout their entire lifecycle. Systems Engineering,
8(4):342–358.
Gebali, F. (2008). Analysis of Computer and Communication Networks, chapter
Reducible Markov Chains. Springer, Victoria, B.C.
George, L. (2014). New EU marine fuel rules erode shippers’ gains from cheap oil.
Reuters. December 17.
Goh, B. (2014). High fuel costs and new emissions regs are pushing the boundaries
of naval architecture. gCaptain. November 17.
Guarin, L., Hifi, Y., and Vassalos, D. (2014). Passenger ship evacuation - design and
verification. In Virtual, Augmented and Mixed Reality. Applications of Virtual and
Augmented Reality, volume 8526, pages 354–365, Heraklion, Crete, Greece.
160

Harb, J. N., Durrant, S. O., and Terry, R. E. (1993). Use of the Kolb Learning Cycle
and the 4MAT system in engineering education. Journal of Engineering Education,
82(2):70–77.
Hartfiel, D. J. and Meyer, C. D. (1998). On the structure of stochastic matrices with
a subdominant eigenvalue near 1. Linear Algebra and its Applications, 272:193–203.
Hastings, D. and McManus, H. (2004). A framework for understanding uncertainty
and its mitigation and exploitation in complex systems. In 2004 Engineering Sys-
tems Symposium, pages 1–19, Cambridge, MA, USA. MIT.
Hill, M. F., Witman, J. D., and Caswell, H. (2002). Spatio-temporal variation in
Markov chain models of subtidal community succession. Ecology Letters, 2:665–
675.
Hill, M. F., Witman, J. D., and Caswell, H. (2004). Markov chain analysis of succes-
sion in a rocky subtidal community. The American Naturalist, 164(2):E46–E61.
Hoff, J. (2007). Equivalent uniform annual cost: A new approach to roof life cycle
analysis. www.roofingcenter.org.
Holtrop, J. and Mennen, G. (1982). An approximate power prediction method. In-
ternational Shipbuidling Progress, 29(335):166–170.
IMO (2007). Guidelines for evacuation analysis for new and existing passenger ships.
www.imo.org.
IMO (2008). MARPOL Annex VI: Regulations for the prevention of air pollution
from ships. www.imo.org.
IMO (2011). IMO and the environment. www.imo.org.
IMO (2015). Ballast water management. www.imo.org.
Kana, A. A., Knight, J. T., Sypniewski, M. J., and Singer, D. J. (2015). A Markov
decision process framework for analyzing LNG as fuel in the face of uncertainty.
In 12th International Marine Design Conference, volume 2, pages 297–308, Tokyo,
Japan.
Keane, R. and Tibbitts, B. (1996). A revolution in warship design: Navy-industry
integrated product teams. Journal of Ship Production, 12(4):254–268.
Kirkland, S. (2009). Subdominant eigenvalues for stochastic matrices with given
column sums. Electronic Journal of Linear Algebra, 18:784–800.
Klein, G. L., Pfaff, M., and Drury, J. L. (2009). Supporting a robust decision space.
In AAAI Spring Symposium - Technical Report, volume SS-09-09, pages 66–71.
161

Lack, D. A., Thuesen, J., and Elliot, R. (2012). Investigation of appropriate control
measure (abatement technologies) to reduce black carbon emissions from interna-
tional shipping - study report. Technical report, Litehauz.
LaGrone, S. (2014). Navy altered destroyer upgrades due to budget pressure, demand
for ships. news.usni.org/. Accesed November, 2014.
Lee, M. (2014). Singapore gears up for LNG bunkering. Technical report, Asia One
Business. business.asiaone.com.
Le Masson, P., Dorst, K., and Subrahmanian, E. (2013). Design theory: history, state
of the art and advancements. Research in Engineering Design, 24:97–103.
Lin, B. and Lin, C.-Y. (2006). Compliance with international emission regulations:
Reducing the air pollution from merchant vessels. Marine Policy, 30:220–225.
Lloyd’s Register (2007). Ballast water treatement technology. Technical report,
Lloyd’s Register.
Lloyd’s Register (2010). Ballast water treatement technology. Technical report,
Lloyd’s Register.
Lloyd’s Register (2012). Ballast water treatement technologies and current system
availability. Technical report, Lloyd’s Register.
Lloyd’s Register (2015). Understanding ballast water management: Guidance for
shipowners and operators. Technical report, Lloyd’s Register.
Ma, S. (2010). The Handbook of Maritime Economics and Business, chapter Using
Economic Measures for Global Control of Air Pollution From Ships, pages 481–518.
Lloyd’s List, London, UK, 2nd edition.
MAN B&W (2012). ME-GI duel fuel MAN B&W engines: A technical, operational
and cost-eﬀective solution for ships fuelded by gas. Technical report, MAN B&W.
www.corporate.man.edu.
Montgomery, S. M. and Groat, L. N. (1997). Student learning styles and their impli-
caitons for teaching. The Center for Research on Learning and Teaching (CRLT)
Occasional Papers, The University of Michigan, 10:1–8.
Mouravieﬀ, A. (2014). Marine Technology, chapter Transition of Requirements:
How United States Navy combatant design requiremnets are evolving, pages 7–
8. SNAME.
Nguyen, T. P. K., Castanier, B., and Yeung, T. (2010). Optimal maintenance and re-
placement decisions under technological change. In European Safety and Reliability,
Rhodes, Greece.
162

Nielsen, C. K. and Schack, C. (2012). Vessel emission study: Comparrison of various
abatement technologies to meet emission levels for ECA’s. In 9th Annual Green
Ship Technology Conference, Copenhagen.
Niese, N. D. (2012). Life Cycle Evaluation under Uncertain Environmental Policies
Using a Ship-Centric Markov Decision Process Framework. PhD thesis, University
of Michigan.
Niese, N. D., Kana, A. A., and Singer, D. J. (2015). Ship design evaluation subject to
carbon emission policymaking using a Markov decision process framework. Ocean
Engineering, 106:371–385.
Niese, N. D. and Singer, D. J. (2013). Strategic life cycle decision-making for the
management of complex systems subject to uncertain environmental policy. Ocean
Engineering, 72:365–374.
Niese, N. D. and Singer, D. J. (2014). Assessing changeability under uncertain ex-
ogenous disturbance. Research in Engineering Design, 25:241–258.
Nikopoulou, Z., Cullinane, K., and Jensen, A. (2013). The role of a cap-and-trade
market in reducing NOx and SOx emissions: Prospects and benefits for ships within
the northern eurpean ECA. Journal of Engineering for the Maritime Environment,
227(2):136–154.
ONR (2011). ONR BAA 11-022 assessing total ownership cost. Technical report,
U.S. Department of Navy Office of Naval Research.
O’Rourke, R. (2015). Navy Littoral Combat Ship (LCS)/Frigate Program: Back-
ground and issues for Congress. Technical report, Congressional Research Service.
Otterlo, M. and Wiering, M., editors (2012). Reinforcement Learning: State of the
Art. Springer, New York, New York.
Pavelich, M. J. and Moore, W. S. (1996). Measuring the effect of experiential educa-
tion using the Perry model. Journal of Engineering Education, 85(4):287–292.
Perry Jr., W. G. (1970). Forms of intellectual and ethical development in the college
years: A scheme. Holt, Rinehart, and Winston, New York.
Pinkerton, J. E. (2007). Sulfur dioxide and nitrogen oxides emissions from U.S. pulp
and paper mills, 1980-2005. Journal of the Air and Waste Management Association,
57:901–906.
Pistikopoulos, E. N. (1995). Uncertainty in process design and operations. Computers
and Chemical Engineering, 19:S553–S563.
Princaud, M., Cornier, A., and Froelich, D. (2010). Developing a tool for environ-
mental impact assessment and eco-design for ships. Journal of Engineering for the
Maritime Environment, 224(3):207–224.
163

Pryde, A. (2009). Comparison of subdominant eigenvalues of some linear search
schemes. Linear Algebra and its Applications, 431:1439–1442.
Psaraftis, H. N., Lyridis, D. V., and Kontovas, C. A. (2012). The economics of ships.
In Talley, W. K., editor, The Blackwell Companion to Maritime Economics, pages
373–391. Wiley-Blackwell, West Sussex, UK.
Puterman, M. L. (2005). Markov Decision Processes: Discrete Stochastic Dynamic
Programming. Wiley, Hoboken, NJ.
Qiao, Y., Han, D., Shen, J., and Wang, G. (2014). A study on the route selection
problem for ship evacuation. In 2014 IEEE International Conference on Systems,
Man, and Cybernetics, pages 1958–1962, San Diego, CA.
Reich, Y. (1995). A critical review of general design theory. Research in Engineering
Design, 7:1–18.
Rigby, G. R. and Taylor, A. H. (2001). Progress in the management and treatment of
ship’s ballast water to minimize the risks of translocating harmful nonindigenous
aquatic organisms. Journal of Marine Environmental Engineering, 6:153–173.
Rigterink, D. T. (2014). Methods for Analyzing Early Stage Naval Distributed Sys-
tems Designs, Employing Simplex, Multislice, and Multiplex Networks. PhD thesis,
University of Michigan, Ann Arbor, MI.
Russell, S. and Norvig, P. (2003). Artiﬁcial Intelligence: A Modern Approach. Prentice
Hall, Upper Saddle River, NJ, 2nd edition.
Rynbach, E. V. (2014). Key impacts of LNG fuel storage on commercial ship design.
In Maritime Professional, volume 4, pages 24–29. New Wave Media, New York,
NY.
Salzman, J. (2007). Spectral Analysis with Markov Chains. PhD thesis, Stanford
University.
Santala, M.-M. (2012). Realising the competitive potential of sulfur ECA compliance.
www.warstilla.com. Accessed June, 2014.
Schuler, M. (2014a). Brittany ferries suspends LNG-powered ferry plans. gCaptain.
October 13.
Schuler, M. (2014b). Watch: Triple-E leaves port with world record load. gCaptain.
October 19.
Seram, N. (2013). Decision making in product development - a review of the literature.
International Journal of Engineering and Applied Sciences, 2(4):1–11.
Sharp, J. E. (2001). Teaching teamwork communication with Kolb learning style
theory. In Proceedings, 2001 Frontiers in Education Conference, pages F2C–1 –
F2C–2. ASEE/IEEE.
164

Sheskin, T. J. (2011). Markov Chains and Decision Processes for Engineers and
Managers. CRC Press, New York.
Ship and Bunker (2013). Transport Canada: ECA compliance achievable but chal-
lenging. www.shipandbunker.com. Accessed: November, 2014.
Sommez, J. (2012). The Why is more important that the What.
http://simpleprogrammer.com/. Accessed: November, 2015.
Son, Y. K. and Savage, G. J. (2007). Set theoretic formulation of performance re-
liability of multiple response time-variant systems due to degradations in system
components. Quality and Reliability Engineering International, 23:171–188.
Søndergaard, J., Eismark, L. R., and Bovermann, J. (2012). Balancing the imblances
in container shipping. Technical report, A.T. Kearney.
Spurlin, J. E., Bernold, L. E., Crossland, C. L., and Anson, C. M. (2003). Un-
derstanding how freshmen engineering students think they learn. In Proceedings,
2003 ASEE Conference and Exposition, pages 8.1221.1–8.1221.9, Washington, DC.
American Society for Engineering Education.
Stoica, P. and Moses, R. (2005). Spectral Analysis of Signals. Prentice Hall, Upper
Saddle River, NJ.
Stopford, M. (2009). Maritime Economics. Routledge, New York, 3rd edition.
Styblinski, M. A. (1991). Formulation of the drift reliability optimization problem.
Microelectronics Reliability, 31(1):159–171.
Tan, C. and Park, K. (2014). China set to claim lead for biggest sea freighter by a
meter. washpost.bloomberg.com. Accessed: November, 2014.
Tanner, J. E. and Hughes, T. P. (1994). Species coexistence, keystone species, and
succession: A sensitivity analysis. Ecology, 75(8):2204–2219.
UNCTAD (2014). Review of maritime transport. Technical report, United Nations
Conference on Trade and Development (UNCTAD), New York and Geneva.
US Navy (2010). US Navy. http : //www.navy.mil/view image.asp?id = 93659.
Vanem, E. (2015). Uncertainties in extreme value analysis of wave climate data and
wave climate projections. In Proceedings of the ASME 2015 34th International
Conference on Ocean, Oﬀshore and Arctic Engineering, pages 1–13, St. John’s,
Newfoundland. ASME.
Vanem, E. and Skjong, R. (2006). Designing for safety in passenger ships utilizing
advanced evacuation analyses - a risk based approach. Safety Science, 44:111–135.
van Noortwijk, J. M. (2007). A survey of the application of gamma processes in
maintenance. Reliability Engineering and System Safety, 94:2–21.
165

Wankat, P. C. and Oreovicz, F. S. (2015). Teaching Engineering. McGraw-Hill Inc.,
USA, 2nd edition.
Wartsila (2014). Wartsila 2-stroke low pressure dual-fuel engines: Wartsila ship power
business paper. Technical report, Wartsila. www.wartsila.com.
Whitefoot, K. S. (2011). Quantifying the Impact of Environmental Policy on Engi-
neering Design Decisions. PhD thesis, University of Michigan.
Yang, Z. L., Mastralis, L., Bonsall, S., and Wang, J. (2009). Incorporating uncertainty
and multiple criteria in vessel selection. Journal of Engineering for the Maritime
Environment, 223(2):177–188.
Zayed, T. M., Chang, L.-M., and Fricker, J. D. (2002). Life-cycle cost analysis using
deterministic and stochastic methods: Conﬂicting results. Journal of Performance
of Constructed Facilities, 16(2):63–74.
166

Dissertation_Austin_Kana_FINAL

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to Dissertation_Austin_Kana_FINAL (20)

Dissertation_Austin_Kana_FINAL