Development of Workshops for Handling Trans-Science Problems

Development of Workshops
for Handling Trans-Science
Problems
Gensei Ishimura, Miho Namba
Hokkaido University, Japan
2015/10/2

Participatory Technology Assessment
Several methods for participatory technology
assessment have been used to address trans-science
issues.

Problems in General
1. The method needs much resources.
→ It is difficult to hold programs frequently
2. The discussions there include many options and
evaluation criteria (=“amalgam”).
→ Participants find difficulty in visualizing and
sharing important agendas.
→ It is difficult to reflect on, analyze or reuse the
results.

Development of the New Workshop
Method
• To mitigate the problems, we try to develop novel
workshop method based on AHP (Analytic
Hierarchy Process) for collective decision making,
community building, and citizenship learning in
order to handle trans-science problems.

AHP (Analytic Hierarchy Process)
• AHP is the method by which (is claimed that) we
can determine the best option to achieve the
goal among many, while considering multiple
criteria.

Goal and Methods as Options
plan A
Goal
plan B plan C plan D
↑ the methods as options to achieve the goal

To Decide Where to Travel
↑ the methods as options to achieve the goal
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Goal (to decide where to travel)

Basic Concept of AHP
• We can choose the best option by scoring each
option and by comparing them.
• However, these scores vary according to what
kind of evaluation criterion is chosen.
• For example, interest, cost, or required
time are candidates for evaluation criteria.
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo

Considering “ Multiple Evaluation Criteria”
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Interest Cost Time
multiple
evaluation
criteria
options

Comparisons of the Options
in Terms of Each Evaluation Criterion
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Interest Cost Time
multiple
evaluation
criteria
options

(1) Comparison of the Options in terms of
“Interest” as Evaluation Criterion
80 9070 30Score
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Interest
evaluation
criterion
options

(2) Comparison of the Options in terms of
“Cost” as Evaluation Criterion
60 3050 90Score
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Cost
evaluation
criterion
options

(3) Comparison of the Options in
terms of “Time” as Evaluation Criterion
60 2060 90Score
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Time
evaluation
criterion
options

Summary of Comparisons in terms
of Each Evaluation Criterion
plan
D:
Toky
oInterest: 80 70 90 30
Cost： 60 50 30 90
Time： 60 60 20 90
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
Interest Cost Time
evaluation
criteria
options

Score the Importance of Each
Evaluation Criterion
Importance： 90 60 40
Interes
t
Cos
t
Tim
e
evaluation
criteria
Interest Cost Time

Introducing “Importance” of
Each Evaluation Criterion
Interest: 80 70 90 30 90
Cost： 60 50 30 90 60
Time： 60 60 20 90 40
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
evaluation
criteria →
Interest Cost Time
Importance

Multiplying Each Score by Each
Importance and Summing them to
Calculate “Integrated Evaluation Score”
Integrated
Evaluation：
34.2 40.2 29.3
Interest: 80 70 90 30 90
Cost： 60 50 30 90 60
Time： 60 60 20 90 40
Importance
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
evaluation
criteria →
Interest Cost Time
(* The scores are normalized.)
34.7

However, it’s difficult to score 3 or more
objects simultaneously for us human.
Interest: 80 70 90 30 90
Cost： 60 50 30 90 60
Time： 60 60 20 90 40
Importance
plan C:
Kaohsiung
plan B:
Hokkaido
plan A:
Kyoto
plan D:
Tokyo
evaluation
criteria →
Interest Cost Time
Integrated
Evaluation：
(* The scores are normalized.)
34.2 40.2 29.334.7

The Original AHP uses “paired
comparison method”
• Evaluators compare a pair
of objects and answer
which is important and
score how important with 9
step scale.
• The answers are integrated
with AHP algorithm to
calculate each object’s
score.
A B
A B C D E ・・・
A C
9
step
scale
・・
・
・
・
9
step
scale

9 Step Scale
Importance of A
compared with B
absolutely weaker -9
much weaker -7
weaker -5
a little weaker -3
the same 1
a little stronger 3
stronger 5
much stronger 7
absolutely stronger 9
A B
9 -9
A is absolutely important
compared with B

Two Preliminary Workshops Which
Were Designed on the Basis of the
AHP Method
1. Recovery of Kesennuma shark fishery after the
Great East Japan Earthquake Disaster while
considering sustainability
2. Introduction of Robot technology into the field
of nursery practice

1.Recovery of Kesennuma
shark fishery after the
Great East Japan
Earthquake Disaster while
considering sustainability

Purpose of the Workshop
• The purpose of the workshop is not merely to
determine the best option nor to make consensus,
decision making, and problem solving.
• The most essential purposes are,
• to make participants reflect the result of AHP
calculation and their experience during
answering questionnaire,
• and to make them realize their implicit and
multidimentional value systems by
themselves and their diversity, in order to
understand themselves and others better.

Expected Outcome of the Workshop
• The workshop should become effective learning
process for complicated decision making,
consensus building, or problem solving, and of
course, understanding the nature of the given
trans-science problem itself.

Expected “Learning Feedback Loop”
Decomposition of the problem
To realize their own
implicit value system
To share diverse value systems
of themselves among them
Reflection
Better understanding of the
problem
Understanding of themselvesUnderstanding of the problem
The given trans-science problem Possible options
Identifying multiple evaluation criteria
Decomposition of the problem
Answering AHP questionnaire

Theme of the Workshop:
Sustainable Fishery
• The participants were supposed to buy
– pacific cod with or without MSC certification
http://en.wikipedia.org/wiki/Pacific_cod

MSC (Marine Stewardship Council)
• Vision
– Our vision is of the world’s oceans teeming with
life, and seafood supplies safeguarded for this
and future generations.
• Certification
– When you see seafood bearing the blue MSC
ecolabel, you can be sure it comes from a
certified sustainable fishery.
– Our standards were developed through
consultation with the fishing industry, scientists,
conservation groups, experts and stakeholders.
These standards detail the requirements for
fisheries to be certified as sustainable and for
businesses to trade in certified seafood.
http://www.msc.org/

MSC Fisheries Standard
http://www.msc.org/about-us/standards/fisheries-
standard/msc-environmental-standard-for-sustainable-fishing

MSC Ecolabel
Eco label © Marine Stewardship Councilhttp://en.wikipedia.org/wiki/Ecolabel

Outline of the AHP questionnaire
• The participants were supposed to buy
– pacific cod with or without MSC certification
• One with MSC certification
– A little more expensive and a little difficult to
find at shops for consumers.
• One without MSC certification
– less expensive and easy to find, although it
might give bad effect to fishery resource
management and marine environment.

Options and Evaluation Criteria
• Options
– You will buy
1. Pacific cod without MSC absolutely
2. Pacific cod with MSC, if you can find one
easily
3. Pacific cod with MSC absolutely
• Evaluation criteria
1. Cost
2. Environmental conservation (from
scientific point of view)
3. Personal concern (“emotional burden”)

The Structure of the Workshop
To chose one of the options (1)
To Answer the AHP questionnaire
To participate in the off-line workshop
The result of
AHP is given
on the web
Basic information about the issue is given
off-line
(*omitted this time)

Outline of the Off-Line Workshop
• Issue: sustainable fishery and MSC certification
• Date: 29, Nov., 2014
• Time: 10:00-12:00
• Place: Hokkaido University
• The number of participants: 5

Program of the Off-Line Workshop
1. Introduction to explain the purpose and
program of the workshop
2. Icebreak and self-introduction
3. Explanation on basic knowledge needed
4. Explanation on the result of AHP
questionnaire
5. Discussion about the result
6. Wrap-up

Results
1. Importance of each evaluation criterion
2. Advantage of each option in terms of “cost” as
evaluation criterion
3. Advantage of each option in terms of
“environmental conservation” as evaluation
criterion
4. Advantage of each option in terms of “personal
concern” as evaluation criterion
5. Change in choice of options and confidence
factor
6. Free style comments

Collective Integrated Evaluation
0
0.1
0.2
0.3
0.4
0.5
0.6
Importance
Average Scores
No MSC
MSC if found
MSC

Integrated Evaluation
0.2
0.1
0
0.3
0.4
0.8
0.7
0.6
0.5
Importance
No MSC
MSC if found
MSC
A B
Participants
D EC

Importance of Each Evaluation Criterion
0.2
0.1
0
0.4
0.3
0.8
0.7
0.6
0.5
A B D E
Importance
cost
environment
concern
Participants
C

Advantage in Terms of “Cost” as
Evaluation Criterion
0.2
0.4
A B D E
Importance
No MSC
MSC if found
MSC
0.3
0.5
0.6
0.1
0
Participants
C

Advantage in Terms of “Environmental
Conservation” as Evaluation Criterion
A B
Participants
D E
No MSC
MSC if found
MSC0.2
0.4
Importance
0.3
0.5
0.6
0.1
0
C

Advantage in Terms of “Personal
Concern” as Evaluation Criterion
0.1
0.15
Importance
0.25
0.2
0.3
0
0.05
A B
Participants
D EC
No MSC
MSC if found
MSC

Change in Choice of Options and
Confidence Factor
• Choice of options
– No MSC
– MSC if found
– MSC
• Confidence factor
– very strongly confident about the answer = 6
– very slightly confident about the answer = 0
• Looking at the change of them in each
participants

Confidence Factor
1
3
before
information
after
information
after AHP
answer
confidencefactor
No MSC
MSC if found
MSC
2
0
4
5
6
participant A

Confidence Factor
1
3
before
information
after
information
after AHP
answer
confidencefactor
No MSC
MSC if found
MSC
2
0
4
5
6
participant B

Confidence Factor
1
3
before
information
after
information
after AHP
answer
confidencefactor
No MSC
MSC if found
MSC
2
0
4
5
6
participant C

Confidence Factor
1
3
before
information
after
information
after AHP
answer
confidencefactor
No MSC
MSC if found
MSC
2
0
4
5
6
participant D

Confidence Factor
1
3
before
information
after
information
after AHP
answer
confidencefactor
No MSC
MSC if found
MSC
2
0
4
5
6
participant E

Comments in the Questionnaire
• The questionnaire was designed that
respondents can put comments on each choice
and for the whole questionnaire.

Some Examples of the Comments
• Evaluation Criteria
– Even if my standpoint is directed, I tend to
change it depending on the given pairs of
criteria, which was interesting.
– “Personal concern” seemed not to be
independent of other criteria.

• Options
– In the beginning, I thought that I was going to
be led to agree with MSC certification but I
found objective evidence at last.
– Choice depends on the difference of costs
between “With MSC” and “Without MSC”
conditions. Its information was not given in the
workshop.

• About the questionnaire itself
– It was interesting because I could think the
same issue from various point of views and I
could suppose many situations.

Discussion
• Although it might be important to design
evaluation criteria as mutually independent as
possible, the finding that they are dependent
itself might be valuable.
• If we collect many data, we might find consistent
tendency or typical patterns in “change in choice
of opinions and confidence factor”.

2. Introduction of Robot
technology into the field
of nursery practice

Purpose of the Workshop
• To deeply understand the issue of “introduction
of nursery robots into the field of medical care”,
by thinking and discussing about it from the
viewpoint of various evaluation criteria.
• To acquire the method to construct evaluation
frame for deliberating the issue of nursery robots
by oneself.

The Given Issue
• In what way should we introduce nursery
robots into the field of medical care in the
future?

Summary of the Workshop
• Date and time: 29/8/2015 15:00～17:00
• The number of participants: 10
• Program:
1. Lectures given by two researchers
2. Group work
3. Presentation, sharing, reflection

Difference Between the Two Workshops
Sustainable
Fishery
Workshop
Nursery robots
workshop
Pros Cons
Options given proposed by
participants
able to collect diverse
opinions
difficult to compare among
different group
evaluation
criteria
given proposed by
participants
able to collect diverse
opinions
difficult to compare among
different group
scoring
method
paired
comparison
directly lower burden
save time
lower credibility of scores
scoring
style
individually group discussion lower burden impossible to detect
individual’s opinion
number of
options and
criteria
3, at most around 8 free from combinational
explosion, much more
items can be handled
polling with
confidence
factors
3 times none - unable to trace the chance of
choices and conficence
ranking none before and after
scoring
able to compare two
rankings and reflect
appropriateness of scoring
process
-
interaction on the web and
face to face
face to face lower burden unable to collect detailed
data

Conclusion
• The workshops have potential to decompose
complicated trans-science problems into elements
to the extent that we can handle with.
• Possibly, the workshop process not only affect
participants opinion but their confidence of their
choice.
• The workshops might become a good tool to give
citizens learning occasion to realize their own
implicit value systems, to visualize and share their
diversity, and to analyze the multi-dimensional
structure of given trans-science problems.

Conclusion
• By using the obtained data of such workshops, the
method enables us to do comparative research
among different generations, professions, cultural
backgrounds, nationalities, etc.
• Information of “composition” of multiple
evaluation criteria or multiple participants’
preferences within each option can be useful to
design “portfolio” of multiple options for policy
making.
• It can also let citizens to find better options, better
evaluation criteria, and, what’s more, “frame of
reference” for developing better methodology for
deliberating trans-science issues.

Acknowledgment
• This work was supported by JSPS KAKENHI Grant
Number 25350222.
• The workshop would have never been done
without great ideas and elaboration of CoSTEP
(Science and Technology Communicator Education
Program in Hokkaido University) students and my
colleagues.

Development of Workshops for Handling Trans-Science Problems

More Related Content

Viewers also liked (9)

Similar to Development of Workshops for Handling Trans-Science Problems (20)

More from Professional University of Information and Management for Innovation　（情報経営イノベーション専門職大学） (15)

Recently uploaded (20)