Warning Information
Development Process

       Eric F. Shaver, Ph.D.
           November 4, 2011




                               1
Benchmark Research & Safety, Inc.
 Founded in 2000

 Services & products we provide:
   • Product & occupational safety
        Warning information design, development, & evaluation
        Product liability litigation support
        Research design, development & analysis


   • Usability

   • Web application design & development

   • Program administration & contract monitoring

   • Training & education
                                                                 2
What is a Warning?
 Warnings provide information about risks
   • Notification (facts)
        Nature of the hazard
        Instructions on how to avoid the hazard
        Consequences for failing to do so


   • Advice (judgments)
        What people ought to do
        Recommended course of action




                                                   3
Warnings & Product Safety
 Safety hierarchy
   •   Design
   •   Guard
   •   Warn
   •   Train


 Warnings are intended to contribute to product safety
   • Decreasing residual risk
   • Increasing tolerable risk


 Reasonably safe = residual risk < tolerable risk



                                                          4
Warning Considerations
 Why does it seem so easy to write a warning, but in
  actuality it’s quite difficult?

 Developing warnings is a balancing act
   • Trying to achieve many objectives
   • Working with limited resources
   • Choosing and implementing warning features


 So what is the solution?




                                                        5
Warning Information Development Process

         Phase 1: Project Planning


         Phase 2: Hazard Identification & Analysis



         Phase 3: Warning Development



         Phase 4: Warning Evaluation



         Phase 5: Final Approval & Documentation


                                                     6
Benefits
 Structured v. ad hoc process
   •   Minimize time & money waste
   •   Increased confidence in the warning information quality
   •   Greater repeatability
   •   Reduce likelihood of product recalls


 Legal concerns
   • Consistency across product lines
   • Maintain record of decisions
   • Deal with legal issues more efficiently and effectively




                                                                 7
Project Planning
 Defining the scope

 Identify team members

 Resource allotment




                          8
Project Planning, cont.
 Defining the scope
   • Saves times and money

   • Identify affected products (including accessories)

   • “System of Information”
          On-product labels/markings
          Hang tags
          Instruction & service manuals
          Packaging
          MSDSs
          Signs
          Point-of-purchase materials
          Advertising & promotional materials



                                                          9
Project Planning, cont.
 Identify team members
   • Goal: identify & respond to hazards

   • Consider including:
         Design engineers (mechanical, electrical, chemical, etc.)
         Human factors engineers/psychologists
         Packaging engineers
         Technical writers
         Marketing
         Customer service
         Regulatory affairs
         Trade association representatives
         Legal counsel




                                                                      10
Project Planning, cont.
 Resource allotment
   • Provide sufficient time & money

   • Engage early in the design process

   • Warnings are often an afterthought

   • Avoids costly redesigns




                                          11
Hazard Identification & Analysis
 “Know what was not previously known”

 Two general ways – formal techniques & external sources

 Formal tools:
   •   Hazard Checklists
   •   Preliminary Hazard Analysis
   •   Fault Tree Analysis
   •   Failure Modes and Effects Analysis
   •   Event Tree Analysis
   •   Root Cause Analysis
   •   Job Safety Analysis
   •   Critical Incident Technique
   •   Risk Assessment Matrices
   •   Hierarchical Task Analysis
                                                            12
Hazard Identification & Analysis, cont.
 External sources of information
   •   Codes, regulations, and standards
   •   Competitor’s warnings information
   •   Technical literature
   •   Trade or industry groups & publications
   •   Safety organizations & associations
   •   Incident/accident data & reports
   •   Medical literature
   •   Warranty returns
   •   Customer service complaints
   •   Litigation claims
   •   Subject matter experts
   •   Product users

                                                 13
Hazard Identification & Analysis, cont.
 Unlikely any single method will address all needs

 Consider using multiple methods

 Formal techniques may not be necessary
   • Established products w/known histories


 Ultimately, decision lies with design team




                                                      14
Warning Development
 Three parts:
   • Identify potential topics

   • Determine delivery method

   • Prototype creation


 Two key questions
   • What do you want people to know?

   • How do you expect them to know it?




                                          15
Warning Development, cont.
 Identify potential topics
   • Transform knowledge about hazards into possible topics
        Some hazards generate multiple topics

        Some topics address multiple hazards

        Not every hazard will lead to a warning topic


   • Why not provide a message about every topic?
        Space limitations

        Overwarning issues




                                                              16
Warning Development, cont.
 Identify potential topics, cont.
   • Characteristics of the target audience
           Age
           Gender
           Literacy (e.g., reading level, non-native language users, etc.)
           Personality traits (e.g., self-efficacy, locus of control, risk taking, etc.)
           Product familiarity
           Hazard perception
   •   Hazard likelihood & severity
   •   “Open and obvious” hazards
   •   Regulations & standards
   •   Space limitations
   •   Cost of compliance
   •   Spatial & temporal placement


                                                                                            17
Warning Development, cont.
 Determine delivery method
   • Content
         Warning components
         Message length
         Message explicitness
         Space limitations
         Multilingual presentation
   • Location
       Temporal (user should see before hazard exposure)
       Spatial (locate where product user will see it)
       Durability
   • Format
         Signal word
         Text v. symbols
         Color v. black & white
         List v. prose
         Font type & size
         Layout
                                                            18
Warning Development, cont.
 Determine delivery method, cont.
   • Presentation methods
         On-product labels/markings
         Hang tags
         Instruction & service manuals
         Packaging
         MSDSs
         Signs
         Point-of-purchase materials
         Advertising & promotional materials




                                                19
Warning Development, cont.
 Prototype creation
   • Consult relevant regulations & standards

   • ANSI Z535 series:
         Z535.1 – Safety Color Code
         Z535.2 – Environmental and Facility Safety Signs
         Z535.3 – Criteria for Safety Symbols
         Z535.4 – Product Safety Signs and Labels
         Z535.5 – Safety Tags and Barricade Tapes (for Temporary Hazards)
         Z535.6 – Product Safety Information in Product Manuals, Instructions, and
                   Other Collateral Materials


   • Focuses on format w/ some content & location guidance




                                                                                      20
Warning Development, cont.
 Prototype creation, cont.
   • Reference documents
       CPSC (2003, October) – “Manufacturer’s guide to developing consumer
        product instructions”

       FMC Corporation (1993) – “Product safety signs and label system”

       Lehto (1992) – “Designing warnings signs and labels”

       Robinson (2009) – “Writing and designing manuals and warnings, 4th ed”


   • Consider using outside subject matter experts




                                                                                 21
Warning Evaluation
 Prototypes should receive some type of evaluation

 Two general types
   • Internal reviews
        Can help identify potential errors or problems


   • Empirical testing
        Some believe it’s a requirement
        No easy answer re: when empirical testing is needed




                                                               22
Warning Evaluation, cont.
 Internal reviews
   •   Compliance - regulations, standards, & guideline compliance?
   •   Comprehension - is it understandable?
   •   Consistency – across products & industry?
   •   Content - hazard, how to avoid it, & consequences
   •   Durability
   •   Legibility – can users see & read information?
   •   Readability – can users understand & process info?




                                                                      23
Warning Evaluation, cont.
 Empirical testing
   • When might a design team consider it:
        Marketed & sold to special populations

        Substantially different from other products on the market

        Symbols/pictorials used without text

        Warning information included in a standard


   • Provides no guarantee that user will notice, read, & comply




                                                                     24
Research Findings

 Table 1: Rates of noticing, reading and complying with warnings reported in
          the literature

 Study                             % Noticed       % Read       % Complied

 Frantz & Rhoades (1993)               57             42             28
 Frantz, et al. (2000)                 98             --             17
 Friedmann (1998)                      88             46             27
 Gomer (1986)                         100             --             21
 Hatem (1993)                          87             --             2
 Otsubo (1988)                         74             52             38
 Shaver, et al. (2006)                 90             47             10
 Smith-Jackson & Durak (2000)          0              0              0
 Strawbridge (1986)                    91             77             37


                                                                               25
Why is behavioral compliance so low?
 Several steps involved:
   •   See it
   •   Read it
   •   Understand it
   •   Accept it
   •   Follow it


 The warning is only part of the puzzle




                                           26
FAB Model



Situation / Environment       Product User



                          Processing   Decision   Behavior
    Product / Task                     Making




       Warning




                                                             27
Warning Evaluation, cont.
 Empirical testing, cont.
   • Several considerations when
          What type of test(s) will be used?
          What research methods will be employed?
          Will intended product users be tested?
          How will the company recruit participants?
          Will the warning be tested as a whole or certain components?
          What acceptance criteria will be used?


   • Enlisting an outside consultant
          Decrease wasted time and money
          Guide process completion
          Determine if empirical testing is necessary
          Perform the empirical testing




                                                                          28
Final Approval & Documentation
 Final approval
   • Company personnel
   • Regulatory agencies (if necessary)


 Documentation
   • Easier to replicate in the future
   • Demonstrates reasonable care

 May want to implement a monitoring program
   •   Changes in regulations & standards
   •   New scientific findings re: warning information
   •   Warranty returns
   •   Customer service complaints
   •   Litigation claims

                                                         29
Contact Information
 For additional details, contact:
   • Eric F. Shaver, Ph.D.
     Senior Consultant
     Benchmark Research & Safety, Inc.
     eshaver@benchmarkrs.com




                                         30

More Related Content

PPTX
UX Research
PPTX
Remote Testing Methods & Tools Webinar
PDF
Process Performance Baselines на простом примере
PDF
Occupational Health & Safety
PPTX
15 workshop safety rules
PDF
Warning signs of a heart attack
PPTX
Workshop safety
UX Research
Remote Testing Methods & Tools Webinar
Process Performance Baselines на простом примере
Occupational Health & Safety
15 workshop safety rules
Warning signs of a heart attack
Workshop safety

Similar to Warning Information Development Process (20)

PDF
Agile and Secure
PPT
It_Club_NCP_Risk_Management_26_03_2013
PPTX
Project risk management in new product development
PPTX
Project risk management_in_new_product_development
KEY
Apresentação Ybios
PDF
Robert Barnes Business Profile
PDF
DigCurV Practitioner Lens
PPTX
Decision framing in Project Management
PPT
Unit 8-risk manaegement (1) -
PPTX
PMG May 2012 When Good Products Go Bad - Product Liability
PDF
imc Research Introduction
PPTX
Reduce Third Party Developer Risks
PPT
Risky Business
PDF
Implementing Intelligent Content Solutions (Joe Gollner - Intelligent Content...
PPTX
Product safety & Recall Prevention Seminar Mumbai
PDF
WQD2011 – INNOVATION – BRONZE WINNER – Tawam Hospital - Innovative Approaches...
PDF
Application Assessment Techniques
PDF
Five Reasons to Choose Benchmark
PDF
NPD- Stage Gate Presentation
PPTX
Owasp Summit - Wednesday evening briefing master
Agile and Secure
It_Club_NCP_Risk_Management_26_03_2013
Project risk management in new product development
Project risk management_in_new_product_development
Apresentação Ybios
Robert Barnes Business Profile
DigCurV Practitioner Lens
Decision framing in Project Management
Unit 8-risk manaegement (1) -
PMG May 2012 When Good Products Go Bad - Product Liability
imc Research Introduction
Reduce Third Party Developer Risks
Risky Business
Implementing Intelligent Content Solutions (Joe Gollner - Intelligent Content...
Product safety & Recall Prevention Seminar Mumbai
WQD2011 – INNOVATION – BRONZE WINNER – Tawam Hospital - Innovative Approaches...
Application Assessment Techniques
Five Reasons to Choose Benchmark
NPD- Stage Gate Presentation
Owasp Summit - Wednesday evening briefing master
Ad

More from Benchmark Research & Safety, Inc. (9)

PDF
Introduction to Usability
PDF
Organizational Leadership in the Wildland Fire Service - Chapter 1 Slides
PDF
Warning Development Process Article
PDF
Healthcare Device Assessment Article
PDF
PDF
Wildland Fire Services
PDF
Ethnographic Services
PDF
What is Human Factors and Ergonomics?
PDF
Introduction to Usability
Organizational Leadership in the Wildland Fire Service - Chapter 1 Slides
Warning Development Process Article
Healthcare Device Assessment Article
Wildland Fire Services
Ethnographic Services
What is Human Factors and Ergonomics?
Ad

Recently uploaded (20)

PDF
Highest-Paid CEO in 2025_ You Won’t Believe Who Tops the List.pdf
PDF
533158074-Saudi-Arabia-Companies-List-Contact.pdf
PPTX
chapter 2 entrepreneurship full lecture ppt
DOCX
Hand book of Entrepreneurship 4 Chapters.docx
PDF
Kishore Vora - Best CFO in India to watch in 2025.pdf
DOCX
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
PPTX
33ABJFA6556B1ZP researhchzfrsdfasdfsadzd
DOCX
Handbook of Entrepreneurship- Chapter 5: Identifying business opportunity.docx
PDF
Business Communication for MBA Students.
PDF
Value-based IP Management at Siemens: A Cross-Divisional Analysis
PPTX
Chapter 2 strategic Presentation (6).pptx
PDF
Cross-Cultural Leadership Practices in Education (www.kiu.ac.ug)
PDF
Tortilla Mexican Grill 发射点犯得上发射点发生发射点犯得上发生
PDF
Consumer Behavior in the Digital Age (www.kiu.ac.ug)
DOCX
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
PPTX
operations management : demand supply ch
PPT
Retail Management and Retail Markets and Concepts
PDF
Sustainable Digital Finance in Asia_FINAL_22.pdf
PPTX
Project Management_ SMART Projects Class.pptx
PPTX
IMM.pptx marketing communication givguhfh thfyu
Highest-Paid CEO in 2025_ You Won’t Believe Who Tops the List.pdf
533158074-Saudi-Arabia-Companies-List-Contact.pdf
chapter 2 entrepreneurship full lecture ppt
Hand book of Entrepreneurship 4 Chapters.docx
Kishore Vora - Best CFO in India to watch in 2025.pdf
Center Enamel A Strategic Partner for the Modernization of Georgia's Chemical...
33ABJFA6556B1ZP researhchzfrsdfasdfsadzd
Handbook of Entrepreneurship- Chapter 5: Identifying business opportunity.docx
Business Communication for MBA Students.
Value-based IP Management at Siemens: A Cross-Divisional Analysis
Chapter 2 strategic Presentation (6).pptx
Cross-Cultural Leadership Practices in Education (www.kiu.ac.ug)
Tortilla Mexican Grill 发射点犯得上发射点发生发射点犯得上发生
Consumer Behavior in the Digital Age (www.kiu.ac.ug)
80 DE ÔN VÀO 10 NĂM 2023vhkkkjjhhhhjjjj
operations management : demand supply ch
Retail Management and Retail Markets and Concepts
Sustainable Digital Finance in Asia_FINAL_22.pdf
Project Management_ SMART Projects Class.pptx
IMM.pptx marketing communication givguhfh thfyu

Warning Information Development Process

  • 1. Warning Information Development Process Eric F. Shaver, Ph.D. November 4, 2011 1
  • 2. Benchmark Research & Safety, Inc.  Founded in 2000  Services & products we provide: • Product & occupational safety  Warning information design, development, & evaluation  Product liability litigation support  Research design, development & analysis • Usability • Web application design & development • Program administration & contract monitoring • Training & education 2
  • 3. What is a Warning?  Warnings provide information about risks • Notification (facts)  Nature of the hazard  Instructions on how to avoid the hazard  Consequences for failing to do so • Advice (judgments)  What people ought to do  Recommended course of action 3
  • 4. Warnings & Product Safety  Safety hierarchy • Design • Guard • Warn • Train  Warnings are intended to contribute to product safety • Decreasing residual risk • Increasing tolerable risk  Reasonably safe = residual risk < tolerable risk 4
  • 5. Warning Considerations  Why does it seem so easy to write a warning, but in actuality it’s quite difficult?  Developing warnings is a balancing act • Trying to achieve many objectives • Working with limited resources • Choosing and implementing warning features  So what is the solution? 5
  • 6. Warning Information Development Process Phase 1: Project Planning Phase 2: Hazard Identification & Analysis Phase 3: Warning Development Phase 4: Warning Evaluation Phase 5: Final Approval & Documentation 6
  • 7. Benefits  Structured v. ad hoc process • Minimize time & money waste • Increased confidence in the warning information quality • Greater repeatability • Reduce likelihood of product recalls  Legal concerns • Consistency across product lines • Maintain record of decisions • Deal with legal issues more efficiently and effectively 7
  • 8. Project Planning  Defining the scope  Identify team members  Resource allotment 8
  • 9. Project Planning, cont.  Defining the scope • Saves times and money • Identify affected products (including accessories) • “System of Information”  On-product labels/markings  Hang tags  Instruction & service manuals  Packaging  MSDSs  Signs  Point-of-purchase materials  Advertising & promotional materials 9
  • 10. Project Planning, cont.  Identify team members • Goal: identify & respond to hazards • Consider including:  Design engineers (mechanical, electrical, chemical, etc.)  Human factors engineers/psychologists  Packaging engineers  Technical writers  Marketing  Customer service  Regulatory affairs  Trade association representatives  Legal counsel 10
  • 11. Project Planning, cont.  Resource allotment • Provide sufficient time & money • Engage early in the design process • Warnings are often an afterthought • Avoids costly redesigns 11
  • 12. Hazard Identification & Analysis  “Know what was not previously known”  Two general ways – formal techniques & external sources  Formal tools: • Hazard Checklists • Preliminary Hazard Analysis • Fault Tree Analysis • Failure Modes and Effects Analysis • Event Tree Analysis • Root Cause Analysis • Job Safety Analysis • Critical Incident Technique • Risk Assessment Matrices • Hierarchical Task Analysis 12
  • 13. Hazard Identification & Analysis, cont.  External sources of information • Codes, regulations, and standards • Competitor’s warnings information • Technical literature • Trade or industry groups & publications • Safety organizations & associations • Incident/accident data & reports • Medical literature • Warranty returns • Customer service complaints • Litigation claims • Subject matter experts • Product users 13
  • 14. Hazard Identification & Analysis, cont.  Unlikely any single method will address all needs  Consider using multiple methods  Formal techniques may not be necessary • Established products w/known histories  Ultimately, decision lies with design team 14
  • 15. Warning Development  Three parts: • Identify potential topics • Determine delivery method • Prototype creation  Two key questions • What do you want people to know? • How do you expect them to know it? 15
  • 16. Warning Development, cont.  Identify potential topics • Transform knowledge about hazards into possible topics  Some hazards generate multiple topics  Some topics address multiple hazards  Not every hazard will lead to a warning topic • Why not provide a message about every topic?  Space limitations  Overwarning issues 16
  • 17. Warning Development, cont.  Identify potential topics, cont. • Characteristics of the target audience  Age  Gender  Literacy (e.g., reading level, non-native language users, etc.)  Personality traits (e.g., self-efficacy, locus of control, risk taking, etc.)  Product familiarity  Hazard perception • Hazard likelihood & severity • “Open and obvious” hazards • Regulations & standards • Space limitations • Cost of compliance • Spatial & temporal placement 17
  • 18. Warning Development, cont.  Determine delivery method • Content  Warning components  Message length  Message explicitness  Space limitations  Multilingual presentation • Location  Temporal (user should see before hazard exposure)  Spatial (locate where product user will see it)  Durability • Format  Signal word  Text v. symbols  Color v. black & white  List v. prose  Font type & size  Layout 18
  • 19. Warning Development, cont.  Determine delivery method, cont. • Presentation methods  On-product labels/markings  Hang tags  Instruction & service manuals  Packaging  MSDSs  Signs  Point-of-purchase materials  Advertising & promotional materials 19
  • 20. Warning Development, cont.  Prototype creation • Consult relevant regulations & standards • ANSI Z535 series:  Z535.1 – Safety Color Code  Z535.2 – Environmental and Facility Safety Signs  Z535.3 – Criteria for Safety Symbols  Z535.4 – Product Safety Signs and Labels  Z535.5 – Safety Tags and Barricade Tapes (for Temporary Hazards)  Z535.6 – Product Safety Information in Product Manuals, Instructions, and Other Collateral Materials • Focuses on format w/ some content & location guidance 20
  • 21. Warning Development, cont.  Prototype creation, cont. • Reference documents  CPSC (2003, October) – “Manufacturer’s guide to developing consumer product instructions”  FMC Corporation (1993) – “Product safety signs and label system”  Lehto (1992) – “Designing warnings signs and labels”  Robinson (2009) – “Writing and designing manuals and warnings, 4th ed” • Consider using outside subject matter experts 21
  • 22. Warning Evaluation  Prototypes should receive some type of evaluation  Two general types • Internal reviews  Can help identify potential errors or problems • Empirical testing  Some believe it’s a requirement  No easy answer re: when empirical testing is needed 22
  • 23. Warning Evaluation, cont.  Internal reviews • Compliance - regulations, standards, & guideline compliance? • Comprehension - is it understandable? • Consistency – across products & industry? • Content - hazard, how to avoid it, & consequences • Durability • Legibility – can users see & read information? • Readability – can users understand & process info? 23
  • 24. Warning Evaluation, cont.  Empirical testing • When might a design team consider it:  Marketed & sold to special populations  Substantially different from other products on the market  Symbols/pictorials used without text  Warning information included in a standard • Provides no guarantee that user will notice, read, & comply 24
  • 25. Research Findings Table 1: Rates of noticing, reading and complying with warnings reported in the literature Study % Noticed % Read % Complied Frantz & Rhoades (1993) 57 42 28 Frantz, et al. (2000) 98 -- 17 Friedmann (1998) 88 46 27 Gomer (1986) 100 -- 21 Hatem (1993) 87 -- 2 Otsubo (1988) 74 52 38 Shaver, et al. (2006) 90 47 10 Smith-Jackson & Durak (2000) 0 0 0 Strawbridge (1986) 91 77 37 25
  • 26. Why is behavioral compliance so low?  Several steps involved: • See it • Read it • Understand it • Accept it • Follow it  The warning is only part of the puzzle 26
  • 27. FAB Model Situation / Environment Product User Processing Decision Behavior Product / Task Making Warning 27
  • 28. Warning Evaluation, cont.  Empirical testing, cont. • Several considerations when  What type of test(s) will be used?  What research methods will be employed?  Will intended product users be tested?  How will the company recruit participants?  Will the warning be tested as a whole or certain components?  What acceptance criteria will be used? • Enlisting an outside consultant  Decrease wasted time and money  Guide process completion  Determine if empirical testing is necessary  Perform the empirical testing 28
  • 29. Final Approval & Documentation  Final approval • Company personnel • Regulatory agencies (if necessary)  Documentation • Easier to replicate in the future • Demonstrates reasonable care  May want to implement a monitoring program • Changes in regulations & standards • New scientific findings re: warning information • Warranty returns • Customer service complaints • Litigation claims 29
  • 30. Contact Information  For additional details, contact: • Eric F. Shaver, Ph.D. Senior Consultant Benchmark Research & Safety, Inc. eshaver@benchmarkrs.com 30