SlideShare a Scribd company logo
Case Studies in
Educational Informatics
Donna M. D’Alessandro, M.D.
Professor of Pediatrics
University of Iowa
March 7, 2017
What Will I Talk About?
 Discuss 7 cases to learn more about the broad scope of digital
librarianship and medical informatics and how developing
technologies can influence online libraries
 Digital libraries as:
 Digital textbook
 Digital press and institutional repository
 Knowledge management system
 Decision support tool
 Curriculum unfolding in practice
 Personal learning portfolio
 Curator of Social Media
 Time periods – before the web, browsable web, searchable web,
social web, post-web/ mobile
Who Am I?
 A General Pediatrician and Educator – I use computers as a tool to help me
teach
 Laboratory Goal
 To improve patients' care, outcomes and lives;
 By changing physician's knowledge, attitudes, and behaviors;
 Through the creation and evaluation of tools, techniques, and procedures that shift learning from the
classroom and lecture hall to the point-of-care.
 Laboratory Description
 Educational informatics is the application of computers to education.
 Since 1989 we have pioneered the development in medicine of digital textbooks, digital libraries,
institutional repositories and communities of practice.
 The Internet serves a force amplifier for us - the efforts of a few have a global impact.
 For many medical subjects, our digital libraries have the highest impact factors on the Internet and
are therefore amongst the top 10 authoritative Web sites on these subjects.
www.educationalinformatics.org
Before the Web
Prehistoric Googling
Case Study 1
HyperLung + HyperAirway - 1991-1992
Digital Library as digital textbook
Case Study 1
HyperLung + HyperAirway - 1991-1992
Digital Library as digital textbook
 Design
 Mission - Create multimedia textbooks that are easily
updateable
 Audience - Medical students, medical residents, practicing
physicians
 Metrics for success - Successfully distribute digital
textbooks
 Creation
 Technology
 The Annotator, a HyperCard stack on Macintosh that
served as a hypermedia authoring tool which created a
HyperCard stack containing a digital textbook
 Page Style
 User interface should look + act like a book to make it easy
for computer novices to use - do this by presenting
hypermedia in linear rather than non-linear form
 Metadata Style
 N/A
 Information Architecture
 Organized by category
 Tools to Gather Data
 Search engine - HyperCard's search engine
 Curation
 Content creation strategy
Author-generated content
Case Study 1
HyperLung + HyperAirway - 1991-1992
Digital Library as digital textbook
 Operation
 Publicity and Marketing
 Described in articles in journals, performed demos +
distributed flyers at meetings
 Regular Maintenance
 None - published once on CD-ROM + never updated
 Business model - Gave it away as technology demonstration
+ to share our vision
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage - Distributed 1,000 CD-ROMs containing digital
textbooks around the world via postal mail
 Reviews - Our work was finally respected once we were
reviewed in Reader's Digest
 Outcome
 First completely digital medical textbooks
 Instructional effectiveness greater than that of lecture +
same as that of printed textbook
 Became founding content of Virtual Hospital digital library
Big Lessons Learned
 Mission and planning come first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture,
learning objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
To learn more about the broad scope of
digital librarianship and medical informatics
and how developing technologies can
influence online libraries.
To learn more about the broad scope of
digital librarianship and medical informatics
and how developing technologies can
influence online libraries.
Browseable Web
Mosaic
August 1993
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
www.vh.org (via Internet Archive)
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Design
 Planning
 Mission - Be the premier source of medical
educational information on Internet ~ Task is to
present information to users rather than
eyeballs to advertisers
 Audience - Health care providers and patients
 Educational construct - Learning situated in
practice
 Metrics for success - Help people learn about
medicine
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Creation
 Technology
 Dynamic Web site ~ 19,000 pages, 15,000 images,
530 movies, 18 audio files ~ Hosted at University of
Iowa
 Content – digitized and distributed
 Global mirroring of library via Cobalt Qube + sync in
Australia, Iceland, Japan, Korea, Taiwan,
Venezuela
 Page Style
 Stanford Guidelines with minimalist graphics -
makes content easily accessible, makes digital
library feel quick + responsive to user, makes
navigation intuitive
 All contently clearly displays author name,
credentials, last review date and peer review status
 Metadata Style
 Search engine optimization at site level
 Information Architecture
 Organized by hierarchy, category and
alphabetically + Problem-based interface of 50
most common medical problems for patients +
providers
 Textbooks indexed alphabetically by title, organ
system, medical discipline, author, type of
information
 Tools to Gather Data
 Stats - Analog, Webalizer, Wusage
 Communication - By comment forms + email
~Patient cries for help given automated response
directing them to local + Web resources to help
them answer their questions
 Search engine - Glimpse -> ht://dig -> Google
Custom search engine
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Curation
 Content creation strategy
 Content Providers Cooperative
 350 faculty authors
 26 departments
 5 colleges recruited to digital press
 Produced 2,000 textbooks / booklets
 Content peer reviewed ~ Received yearly book reports on
their content usage + comments ~ Yearly newsletter on
whole project
 Content licensed from author through Author's
Agreement giving them ownership of content +
University a non-exclusive license to it
 Content peer review boards in Internal Medicine,
Pediatrics, Nursing
 Translation was a challenge - machine translation
was cheap but bad + human translation was
expensive and good so little was translated
 Overseen by Board of Directors with representation
from College of Medicine, University Hospitals,
University Legal, University Health Science colleges
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Operation
 Publicity and Marketing
 Standard guerilla marketing on Web
 News releases, lectures, emails, brochures/bookmarks,
word of mouth
 Regular Maintenance
 Daily - Broken links found + fixed, answer all emails,
handle rights management requests
 Business model - Funded initially by Department of
Radiology -> Funded by grant from National Library of
Medicine -> Funded by University of Iowa Health
Care because University is a knowledge factory and
this is part of the mission
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage - At peak: 13 million users read 59 million pages
per year, 33% usage outside USA
 Reviews – Multiple journals in health care claimed it to be
amongst best Web sites in their specific field
 "(The Virtual Hospital) is an example of the potential for the World Wide Web
in medical education...."- The Lancet, 1995
 "The Virtual Hospital is one of the best medical sites and a compelling
demonstration of the Internet's potential to impact medical training and
practice." - JAMA, 1996
 "The Virtual Hospital contains digital versions of hundreds of medical
textbooks...in essence, it opens the bookshelves of the staff to the world." -
New York Times, 1999
 Awards
 Exhibition entitled "The Networked Planet: Traveling the Information
Highway" featured Virtual Hospital - The Computer Museum, 1995
 One of the best on the Internet when reviewed for quality, accuracy of
content, presentation and usability - Britannica.com, 2000
 One of the top Science sites on the Internet - British Broadcasting
Corporation, 2000
 Sci/Tech Web Award - Scientific American, 2001
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Outcome
 First medical Web site, 250th Web site overall ~
Accounted for first mention of "World Wide Web" in
MEDLINE biomedical literature database in February
1994
 Result was a unique, author-owned, international
mirrored university digital library + press that served
as an authoritative medical reference + education tool
for users around the world
Case Study 2
Virtual Hospital - 1992-2005
Digital Library as digital press and institutional repository
 Little Lessons Learned
 University digital libraries have four key features to distinguish them
from for-profit digital libraries
 They are free + open to all thus lowering barriers to access + their usage is
anonymous guaranteeing patron privacy
 They emphasize primarily information for health care providers which patients
can graduate to
 They have commitment to publishing complete medical reference + education
textbooks as well as booklets
 They are free from pressures from commercial advertisers
 Build simple systems + let complex behaviors / problems emerge +
then find solutions (intellectual property, mirroring)
 Reason this model not widely replicated is that faculty + universities
undervalue their intellectual property. To get ahead at university,
university forces faculty to give away intellectual property to
publishers…thus publishers still prevail today
Big Lessons Learned
 Mission and planning come
first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals
and objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture,
learning objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users -
continuous quality
improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users -
patients and international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
www.vnh.org (via Internet Archive)
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Design
 Planning
 Mission - Maximize naval readiness by creating + curating medical
digital library that can be used as medical reference / health
promotion / knowledge management tool to deliver expert medical
information to providers + patients at the point-of-care in order to
help providers take better care of patients and help patients live
healthier lives
 Audience - Naval health care providers + patients
 Educational construct - Learning situated in practice
 Metrics for success
 Follow Naval Surgeon General’s desire to take healthcare to the deckplates
+ move information not people,
 Own the keywords "military medicine" + "naval medicine" + "humanitarian
medicine" and get a top 10 ranking for it in search engines
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Creation
 Technology
 Great challenge was to deliver digital library services to nomadic
patron population on the sea, under the sea, in the air + in the field
who have heterogeneous access to Internet bandwidth
 Static Web site ~ 2,300 pages ~ Hosted at University of Iowa
 ~ 5,000 CD-ROM mirrors distributed every other year to operational
corpsmen + physicians
 Page Style
 Stanford Guidelines with minimalist graphics
 Metadata Style
 Search engine optimization at site level + book level
 Information Architecture
 Organized by hierarch, category, alphabetically and by problem-
based interface with user-centered design
 Tools to Gather Data
 Stats - Analog, Webalizer, Wusage
 Communication - By comment forms + email
 Search engine - ht://dig -> Google Custom Search Engine
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Curation
 Content creation strategy
 Serve as depository library for Navy Medicine = 60 US government
medical textbooks
 Information on Internet poorly organized, growing rapidly, of
questionable authority so digital librarians acted as intelligent agents +
did surfing for users on most common medical problems + health
promotion tasks creating > 1,000 links to authoritative Web content on
80 common medical problems + 25 health promotion topics
 Content quickly + easily updated in response to changing world
situation (i.e. chemical + biological warfare content)
 Kosovo 1998-99
 Operations Other Than War
 Content overseen by clinical Specialty Advisory Board
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Operation
 Publicity and Marketing
 Standard guerilla marketing on Web
 Lectures to General Medical Officer + Independent Duty Corpsmen trainees ~
Email corpsman training program directors
 Regular Maintenance
 Daily - Broken links found + fixed, answer all emails
 Business model - Funded by grant from US Navy Bureau of Medicine
+ Surgery ~ From 2006-on self-funded at $50 / year
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Evaluation
 Evaluation and Continuous Quality
Improvement
 Usage - At peak: 1.6 million users read 7 million
pages per year ~ At end: 10,000 users read
15,000 pages per year
 Reviews
 Virtual Naval Hospital named as one of the Navy's five "Success
Stories" in the field of Knowledge Management - The Department
of the Navy's Chief Information Officer, 1999
 "Although there are a few books on shipboard medicine, the US
Navy's Virtual Naval Hospital constitutes a comprehensive digital
library of medical information tailored to the Sea Service..." -
Textbook of Military Medicine, Shipboard Medicine by CAPT
Terrence Riley, MC, USN (retired), 2003
 Awards
 Award for Knowledge Sharing / Outstanding Knowledge Expert
System - Department of the Navy, 2000
 Recognized for its contributions to telemedicine - American
Telemedicine Association, 2006
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Evaluation
 Outcome
 Used by US Navy, US Coast Guard, Canadian Navy,
Royal Navy, Greek Navy (and US Army + US Air
Force) ~ Unauthorized mirror site created by Chinese
military
 Economic analysis showed it was used 8.2 hours /
week/health care provider, 70% thought it improved
patient care + improved diagnosis + treatment,
decreased need for medical evacuations, 94% were
satisfied with it, saved $144,000 / year
Case Study 3
Virtual Naval Hospital - 1997-2005
Digital Library as knowledge management system
 Little Lessons Learned
 People, not technology, make the project work - key intermediaries are individuals with multidisciplinary
backgrounds who can interact with different constituencies + who translate + negotiate between them to overcome
obstacles + keep project on course + moving forward
 Maximized readiness - had a + impact on Navy health care by improving health promotion + patient care + being
economically cost-effective
 Consistently over delivered + came in under budget -> defunded ~ Our focus was in operating with Sailors and
Marines at the tip of the spear…not in operating within BUMED bureaucracy
 Successes: User-centric, well received, tri-service, international, military + civil usage, cost-effective
Big Lessons Learned
 Mission and planning come
first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals
and objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture, learning
objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users -
continuous quality
improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Searchable Web
1991
1994-1995
1998
Case Study 4
SearchingPediatrics.com - 2008-Present
Digital Library as decision support tool
 Design
 Planning
 Mission - Create a point-of-care
decision support tool for pediatricians
and pediatric health care providers
 Audience Healthcare providers at all
levels of training practicing pediatrics
 Educational construct - Learning
situated in practice
 Metrics for success- Own the keywords
”pediatric education” and get a top 10
ranking for it in search engines
Case Study 4
SearchingPediatrics.com - 2008-Present
Digital Library as decision support tool
 Creation
 Technology
 Static Web site ~ 1 page in size
 Page Style
 Stanford Guidelines with minimalist graphics
 Metadata Style
 Search engine optimization at site level
 Information Architecture
 Organized by location - most relevant information is at top of results
page
 Can further scope search to textbooks, differential diagnoses,
anatomy atlases, journals, drug references, guidelines + policies
 Tools to Gather Data
 Stats - SiteMeter, Google Analytics
 Communication - By email
 Search engine - Google Custom Search Engine
 Curation
 Content creation strategy
 Curate an index of most authoritative pediatric textbooks and
journals whose content is free to use
 Pediatric information must be peer-reviewed and open Access
 Can receive CME for its use - indirectly
Case Study 4
SearchingPediatrics.com - 2008-Present
Digital Library as decision support tool
 Operation
 Publicity and Marketing
 Standard guerilla marketing on Web
 Regular Maintenance
 Daily - Yearly link check with Big Brother / W3C Link
Checker + archive
 Business model - Self-funded at $50 /year
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage - At peak: 15,000 users performed 20,000
searches per year ~ Today: 6,000 users perform
7,000 searches per year
 Outcome
 Small budget, easy to use tool
Big Lessons Learned
 Mission and planning come first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture,
learning objects, process
 Use open standards
 Push the standards but be open to
new ones
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Social Web
1997
Bulletin Boards 1970-80s
2004
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
www.biodefenseeducation.org (via Internet Archive)
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
 Design
 Planning
 Mission - Address challenges of how to continuously educate about
biodefense / how to develop a sense of community in biodefense /
how to manage biodefense information explosion. Collaboratory to
discuss news stories or other biodefense topics, Reference Library is
biodefense knowledge management system
 Audience - Current + next generation of biodefense investigators +
trainees, medical personnel, first responders
 Educational construct - Curriculum unfolds in practice
 Metrics for success - Own the keyword "biodefense" and get a top 10
ranking for it in search engines
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
 Creation
 Technology
 Dynamic Web site (Blog) - EditThisPage / Manila ~ Hundreds of
pages in size
 Page Style
 Stanford Guidelines with minimalist graphics
 Metadata Style
 Search engine optimization at site level ~ Tagging
 Information Architecture
 Organized by time and by category
 Tools to Gather Data
 Stats - SiteMeter
 Communication - By email
 Search engine - Google Custom Search Engine f
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
 Curation
 Content creation strategy
 News Library - Daily - intelligent agents (Google Alerts) searching
world news for 10 terms (Category A agents + biodefense +
bioterrorism + biological warfare + bioweapon) to find unclassified
biodefense news stories ~ Content syndicated to RSS (really simple
syndication
 Collaboratory - User-generated content
 Reference Library - indexed key biodefense textbooks + made them
searchable via scoped search, automated MEDLINE search to keep
researchers abreast of current literature, curated reading list of
articles to bring new researcher up to speed in biodefense issues
 Publish Biodefense Year in Review in January to summarize last 12
months biodefense news
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
 Operation
 Publicity and Marketing
 Standard guerilla marketing on Web Regular Maintenance
 Daily - Yearly link check + archive
 Business model - Funded by National Institutes of Health
Case Study 5
BiodefenseEducation.org - 2004-2009
Digital Library as curriculum unfolding in practice
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage - At peak: 15,000 users read 32,000 pages per year
 Reviews - "In the fast moving modern world, biodefence is
becoming a new specialty that we may all have to know more
about. So check out BiodefenseEducation.org and its library of
information on this topic." - British Medical Journal, 2004
 Outcome
 News Library - thus an unstructured curriculum unfolds in
practice
 Over two years, 678 news stories from 178 news sources
covered all categories in the required + elective biodefense
graduate program courses
 Collaboratory - Unused, was failure ~ Reference Library - Little
used
 Little Lessons Learned
 Strengths - Addressed content preferences of health care
professionals for biodefense news stories over Web-based
textbooks or courses, broad number of authors + news sources
used for stories which are of high accuracy, key role played by
curators
 Limitations - Use only English-language news stories, short half-life
of free news stories, news stories may not be free in future,
 Successfully addressed challenges on how to continuously educate
individuals about biodefense, having a curriculum unfold in practice,
+ how to manage biodefense information explosion but failed to
address challenge of creating sense of community amongst
individuals
Big Lessons Learned
 Mission and planning come first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture, learning
objects, process
 Use open standards
 Push the standards but be
open to new standards
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Case Study 6
PediatricEducation.org - 2004-Present
Digital Library as learning portfolio & curriculum unfolding in practice
www.pediatriceducation.org ~ www.facebook.com/pediatriceducation ~ www.twitter.com/pedseducation
Case Study 6
PediatricEducation.org - 2004-Present
Digital Library as learning portfolio & curriculum unfolding in practice
 Design
 Planning
 Mission - Serve as a source of continuing pediatric education with a goal
of building a pediatric virtual learning community
 Audience - Health care providers at all levels of training practicing
pediatrics
 Educational construct - Curriculum unfolds in practice, Case-based
learning
 Metrics for success - Own the keywords "pediatric education" +
"paediatric education" and get a top 10 ranking for it in search engines
 Creation
 Technology
 Dynamic Web site (Blog) - WordPress ~ Hundreds of pages in size
 Mirrored to developing world via Widernet
 Page Style
 Stanford Guidelines with minimalist graphics
 Metadata Style
 Search engine optimization at site level ~ Tagging
 Information Architecture
 Organized by time and by category
 Tools to Gather Data
 Stats - SiteMeter, Google Analytics -> WordPress
 Communication - By email and monthly email newsletter
 Search engine - Google Custom Search Engine -> WordPress search
engine
Case Study 6
PediatricEducation.org - 2004-Present
Digital Library as learning portfolio & curriculum unfolding in practice
 Curation
 Content creation strategy
 Case Library - Author publishes 1 case / week from
Learning Portfolio
 Content syndicated to RSS / Facebook / Twitter ~
Cross-platform syndication + integration is non-
standards based and ugly
 Learning Collaboratory - User-generated content
 Operation
 Regular Maintenance
 Daily - Yearly link check with Big Brother / W3C Link
Checker
 Business model - Self-funded at $150 / year
Case Study 6
PediatricEducation.org - 2004-Present
Digital Library as learning portfolio & curriculum unfolding in practice
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage- At peak: 536,000 users read 630,000 pages per year at peak ~ Today: 305,000
users read 665,000 pages per year;
 Reviews
 “This website serves as a useful tool for continuing education in pediatrics. This resource would certainly be
useful for a pediatric resident or clinician to address a subject each week. The website’s usefulness lies in
reaching point-of-care clinicians with quick relevant information while also offering more in-depth links and
discussion.” - Journal of Electronic Resources in Medical Libraries
 Case Library - Over 5 years an unstructured curriculum unfolds in practice - covers
100% of age ranges, 100% of specialties, 98% of symptoms, 55% of disease in
MedlinePlus, 90% of topics in 3 national pediatric curricula
 Currently 11+ years of weekly cases = ~ 575 published cases
 Highly ranked for Internet search term “pediatric education”
Case Study 6
PediatricEducation.org - 2004-Present
Digital Library as learning portfolio & curriculum unfolding in practice
 Little Lessons Learned
 "My reading is now focused on my patients"
 Example of learner taking control of + assuming responsibility for their own
learning
 Learning portfolio documents what you have learned, assessment by
examination documents what you don't know
 Case archive is a database of pediatric cases that can be used to demonstrate
spectrum of a disease, multiple etiologies of a symptom, key concepts to master
within a discipline, variety of diseases within different age groups
 Improvements
 Continuously updated MEDLINE searching, Images and Video searchs, Information
Prescriptions for Patients
 Fads – Competencies – Entrustable Professional Activities
 Use of social media
Big Lessons Learned
 Mission and planning come
first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture, learning
objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users -
patients and international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to
register
 Ship product
Post Web / Mobile
Case Study 7
@pedseducation - 2014-Present
Digital Library as curator of social media
 Twtitter Design
 Planning
 Mission - Provide a curated stream of pediatric education information, new
education resources, provide thoughts same topics
 Audience - Physicians at all levels of training and practice
 Educational construct - Curriculum unfolds in practice
 Metrics for success - 1,000 followers on Twitter ~ Own the keywords
"pediatric education” and get a top 10 ranking for it in Twitter search engine
~ Be a Top 10 Influencer on Symplur for #MedEd + #FOAMPed
 Creation
 Technology
 Dynamic Web site - Twitter
 Page Style
 Stanford Guidelines (hard to do in 160 character profile)
 Metadata Style
 Symplur Healthcare Hashtag Ontology for #FOAMed #FOAMPed #MedEd
#pediatrics #paediatrics #pediatria
 Information Architecture
 Organized by time
 Tools to Gather Data
 Stats - Twitter analytics
 Communication - By Twitter
 Search engine - Twitter search engine
 Curation
 Content creation strategy
 Daily - A curated stream of pediatric education tweets + retweets
 Weekly - Post case from my learning portfolio
Case Study 7
@pedseducation - 2014-Present
Digital Library as curator of social media
 Operation
 Publicity and Marketing
 Standard guerilla marketing on Web tough to do with Twitter ~ Linked to it from my digital libraries + emailed users who had
praised my digital libraries in past ~ Use relevant hashtags, weekly #FF (follow Friday) posts ~ Post good content + pray
for retweets
 Regular Maintenance
 Daily - Yearly link check with Big Brother / W3C Link Checker + archive
 Business model - Self-funded at $0 / year
 Evaluation
 Evaluation and Continuous Quality Improvement
 Usage - At peak/current: 130,000 impressions per year and 300 followers on Twitter
 Outcome
 At least 1 tweets / day, 4 Twitter lists, tweets + retweets viewed ~ 10,000 times / month, community of practice has arisen,
 Twitter - 147,000 impressions per year and 270 followers
 Top 10 influencer for #FOAMPed, #PediatricEducation (FOAM- Free Open Access Medical Education)
 Facebook - 13,000 post views and 8500 unique users per year and 3300 fans
Case Study 7
@pedseducation - 2014-Present
Digital Library as curator of social media
 Facebook
 Design
 Mission – Build online community of practice
 Curation
 Content creation strategy
 Weekly - Post case from my learning portfolio, plus
other content
 Evaluation
 Hard to develop a following
 Lots of noise
 People don’t see it because of FB algorithm which
constantly changes
 Can develop a following
 Smaller group that are dedicated with a
common focus
Case Study 8?
2017 +
Digital Library as environmentally integrated,
personally wearable or neurally integrated
Big Lessons Learned
 Mission and planning come first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture, learning
objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Summary
Big Lessons Learned
 Mission and planning come first
 Set goals and objectives
 Formal needs assessment
 Revisit the mission, goals and
objectives often
 Gather data
 Meeting your objective?
 Innovate
 Keep it simple
 Design, architecture, learning
objects, process
 Use open standards
 Push the standards but be open to
new standards
 Focus on your users - continuous
quality improvement
 Meet their needs
 Integrate into their workflow
 Be open and adaptable
 Unrecognized users - patients and
international
 Use new technologies
 Unintended consequences
 Learn from all the mistakes
 Keep the system open
 Make it free, don’t have to register
 Ship product
Biggest Lesson Learned
The rewards are from the
people served
Pediatriceducation.org, @pedseducation

More Related Content

PPTX
What Can Happen when Genome Sciences Meets Data Sciences?
PDF
The changing landscape of scholarly communication: presentation to the NFAIS ...
PPT
Using Social Media to Communicate Your Work
PDF
Library Process Redesign: Renewing Services, Changing Workflows
PPT
Access and Ownership Issues of Electronic Resources in the Library
PPT
2010 SC Environment Presentation
PDF
Leading the library of the future: w(h)ither technical services?
PPTX
Publishing Data on the Web
What Can Happen when Genome Sciences Meets Data Sciences?
The changing landscape of scholarly communication: presentation to the NFAIS ...
Using Social Media to Communicate Your Work
Library Process Redesign: Renewing Services, Changing Workflows
Access and Ownership Issues of Electronic Resources in the Library
2010 SC Environment Presentation
Leading the library of the future: w(h)ither technical services?
Publishing Data on the Web

What's hot (20)

PPT
Day 3: Introduction to Information Literacy
PPTX
Intersections Big and Small: A Review of Three Projects that Merge Scholarly ...
PPTX
Understanding the Big Data Enterprise
PPTX
Keynote speech - Carole Goble - Jisc Digital Festival 2015
PDF
Data Science - Poster - Kirk Borne - RDAP12
PPT
Mary Joan Crowley-going mobile-london-online-2011.ppt-29nov
PPT
Innovations in E-Resources
PPT
eBooks in Health Sciences - The Good, the Bad, and the Ugly (the 11th Annual ...
PPTX
NISO Webinar: Taking Your Website Wherever You Go: Delivering Great User Expe...
PPTX
Are we there yet? : Moving to an e-only collection development policy for books
PPTX
Scholarly publishing – Jisc and CNI conference 10 July 2014
PPT
Using data effectively worskhop presentation
PDF
Poster: Very Open Data Project
PPTX
The wider environment of open scholarship – Jisc and CNI conference 10 July ...
PPTX
SPARC Repositories conference in Baltimore - Nov 2010
PPTX
From Where Have We Come & Where Are We Going
PPT
Kno.e.sis Review: late 2012 to mid 2013
PPT
Dissemination Of Information
PPTX
Ps rwebinar january2019final
PPTX
Convenience or credibility
Day 3: Introduction to Information Literacy
Intersections Big and Small: A Review of Three Projects that Merge Scholarly ...
Understanding the Big Data Enterprise
Keynote speech - Carole Goble - Jisc Digital Festival 2015
Data Science - Poster - Kirk Borne - RDAP12
Mary Joan Crowley-going mobile-london-online-2011.ppt-29nov
Innovations in E-Resources
eBooks in Health Sciences - The Good, the Bad, and the Ugly (the 11th Annual ...
NISO Webinar: Taking Your Website Wherever You Go: Delivering Great User Expe...
Are we there yet? : Moving to an e-only collection development policy for books
Scholarly publishing – Jisc and CNI conference 10 July 2014
Using data effectively worskhop presentation
Poster: Very Open Data Project
The wider environment of open scholarship – Jisc and CNI conference 10 July ...
SPARC Repositories conference in Baltimore - Nov 2010
From Where Have We Come & Where Are We Going
Kno.e.sis Review: late 2012 to mid 2013
Dissemination Of Information
Ps rwebinar january2019final
Convenience or credibility
Ad

Similar to Case Studies In Educational Informatics (20)

PDF
Getting Started with Institutional Repositories and Open Access
PDF
29 cc 2_b_all_speakers
PDF
The Value of the Scholarly-led, Non-profit Business Model to Achieve Open Acc...
PDF
The value of the scholarly-led, non-profit business model to achieve Open Acc...
PPTX
Finding knowledge from digital library
PPTX
Cloud Based Learning in Healthcare
PDF
Augmenting health professionals and empowering patients with new technologies
PPT
Emerging Technologies in the Workplace For Quality Service
PPTX
Social media in Libraries: Toolkit for Promotion, Productivity & Reference: a...
PPTX
PJL workshop
KEY
Engaging clinical staff and non-users
PPT
Space Up Your Library : The Social Net Works!
PPTX
Open Access and Research Communication: The Perspective of Force11
PPT
ProTube; ProCare Moves Towards Web 2.0
PPT
The Importance of Marketing Digital Collections
PPTX
Social Media Tools and Mobile Apps for Research and Publishing
PPT
Marketing of e resources
PPT
Marketing of e- resources
PPT
Digital Academic Content and the Future of Libraries: International Cooperati...
PPTX
Open access & cIRcle: Scholarly Publishing in Context
Getting Started with Institutional Repositories and Open Access
29 cc 2_b_all_speakers
The Value of the Scholarly-led, Non-profit Business Model to Achieve Open Acc...
The value of the scholarly-led, non-profit business model to achieve Open Acc...
Finding knowledge from digital library
Cloud Based Learning in Healthcare
Augmenting health professionals and empowering patients with new technologies
Emerging Technologies in the Workplace For Quality Service
Social media in Libraries: Toolkit for Promotion, Productivity & Reference: a...
PJL workshop
Engaging clinical staff and non-users
Space Up Your Library : The Social Net Works!
Open Access and Research Communication: The Perspective of Force11
ProTube; ProCare Moves Towards Web 2.0
The Importance of Marketing Digital Collections
Social Media Tools and Mobile Apps for Research and Publishing
Marketing of e resources
Marketing of e- resources
Digital Academic Content and the Future of Libraries: International Cooperati...
Open access & cIRcle: Scholarly Publishing in Context
Ad

Recently uploaded (20)

PDF
Approach and Philosophy of On baking technology
PDF
A comparative analysis of optical character recognition models for extracting...
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PDF
Enhancing emotion recognition model for a student engagement use case through...
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
PPTX
TechTalks-8-2019-Service-Management-ITIL-Refresh-ITIL-4-Framework-Supports-Ou...
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Web App vs Mobile App What Should You Build First.pdf
PPTX
A Presentation on Touch Screen Technology
PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
1 - Historical Antecedents, Social Consideration.pdf
PDF
A novel scalable deep ensemble learning framework for big data classification...
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDF
A comparative study of natural language inference in Swahili using monolingua...
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
PDF
Hybrid model detection and classification of lung cancer
Approach and Philosophy of On baking technology
A comparative analysis of optical character recognition models for extracting...
Group 1 Presentation -Planning and Decision Making .pptx
Enhancing emotion recognition model for a student engagement use case through...
Programs and apps: productivity, graphics, security and other tools
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
TechTalks-8-2019-Service-Management-ITIL-Refresh-ITIL-4-Framework-Supports-Ou...
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Web App vs Mobile App What Should You Build First.pdf
A Presentation on Touch Screen Technology
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
1 - Historical Antecedents, Social Consideration.pdf
A novel scalable deep ensemble learning framework for big data classification...
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
A comparative study of natural language inference in Swahili using monolingua...
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
Hybrid model detection and classification of lung cancer

Case Studies In Educational Informatics

  • 1. Case Studies in Educational Informatics Donna M. D’Alessandro, M.D. Professor of Pediatrics University of Iowa March 7, 2017
  • 2. What Will I Talk About?  Discuss 7 cases to learn more about the broad scope of digital librarianship and medical informatics and how developing technologies can influence online libraries  Digital libraries as:  Digital textbook  Digital press and institutional repository  Knowledge management system  Decision support tool  Curriculum unfolding in practice  Personal learning portfolio  Curator of Social Media  Time periods – before the web, browsable web, searchable web, social web, post-web/ mobile
  • 3. Who Am I?  A General Pediatrician and Educator – I use computers as a tool to help me teach  Laboratory Goal  To improve patients' care, outcomes and lives;  By changing physician's knowledge, attitudes, and behaviors;  Through the creation and evaluation of tools, techniques, and procedures that shift learning from the classroom and lecture hall to the point-of-care.  Laboratory Description  Educational informatics is the application of computers to education.  Since 1989 we have pioneered the development in medicine of digital textbooks, digital libraries, institutional repositories and communities of practice.  The Internet serves a force amplifier for us - the efforts of a few have a global impact.  For many medical subjects, our digital libraries have the highest impact factors on the Internet and are therefore amongst the top 10 authoritative Web sites on these subjects. www.educationalinformatics.org
  • 5. Case Study 1 HyperLung + HyperAirway - 1991-1992 Digital Library as digital textbook
  • 6. Case Study 1 HyperLung + HyperAirway - 1991-1992 Digital Library as digital textbook  Design  Mission - Create multimedia textbooks that are easily updateable  Audience - Medical students, medical residents, practicing physicians  Metrics for success - Successfully distribute digital textbooks  Creation  Technology  The Annotator, a HyperCard stack on Macintosh that served as a hypermedia authoring tool which created a HyperCard stack containing a digital textbook  Page Style  User interface should look + act like a book to make it easy for computer novices to use - do this by presenting hypermedia in linear rather than non-linear form  Metadata Style  N/A  Information Architecture  Organized by category  Tools to Gather Data  Search engine - HyperCard's search engine  Curation  Content creation strategy Author-generated content
  • 7. Case Study 1 HyperLung + HyperAirway - 1991-1992 Digital Library as digital textbook  Operation  Publicity and Marketing  Described in articles in journals, performed demos + distributed flyers at meetings  Regular Maintenance  None - published once on CD-ROM + never updated  Business model - Gave it away as technology demonstration + to share our vision  Evaluation  Evaluation and Continuous Quality Improvement  Usage - Distributed 1,000 CD-ROMs containing digital textbooks around the world via postal mail  Reviews - Our work was finally respected once we were reviewed in Reader's Digest  Outcome  First completely digital medical textbooks  Instructional effectiveness greater than that of lecture + same as that of printed textbook  Became founding content of Virtual Hospital digital library
  • 8. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product To learn more about the broad scope of digital librarianship and medical informatics and how developing technologies can influence online libraries. To learn more about the broad scope of digital librarianship and medical informatics and how developing technologies can influence online libraries.
  • 11. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository www.vh.org (via Internet Archive)
  • 12. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Design  Planning  Mission - Be the premier source of medical educational information on Internet ~ Task is to present information to users rather than eyeballs to advertisers  Audience - Health care providers and patients  Educational construct - Learning situated in practice  Metrics for success - Help people learn about medicine
  • 13. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Creation  Technology  Dynamic Web site ~ 19,000 pages, 15,000 images, 530 movies, 18 audio files ~ Hosted at University of Iowa  Content – digitized and distributed  Global mirroring of library via Cobalt Qube + sync in Australia, Iceland, Japan, Korea, Taiwan, Venezuela  Page Style  Stanford Guidelines with minimalist graphics - makes content easily accessible, makes digital library feel quick + responsive to user, makes navigation intuitive  All contently clearly displays author name, credentials, last review date and peer review status  Metadata Style  Search engine optimization at site level  Information Architecture  Organized by hierarchy, category and alphabetically + Problem-based interface of 50 most common medical problems for patients + providers  Textbooks indexed alphabetically by title, organ system, medical discipline, author, type of information  Tools to Gather Data  Stats - Analog, Webalizer, Wusage  Communication - By comment forms + email ~Patient cries for help given automated response directing them to local + Web resources to help them answer their questions  Search engine - Glimpse -> ht://dig -> Google Custom search engine
  • 14. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Curation  Content creation strategy  Content Providers Cooperative  350 faculty authors  26 departments  5 colleges recruited to digital press  Produced 2,000 textbooks / booklets  Content peer reviewed ~ Received yearly book reports on their content usage + comments ~ Yearly newsletter on whole project  Content licensed from author through Author's Agreement giving them ownership of content + University a non-exclusive license to it  Content peer review boards in Internal Medicine, Pediatrics, Nursing  Translation was a challenge - machine translation was cheap but bad + human translation was expensive and good so little was translated  Overseen by Board of Directors with representation from College of Medicine, University Hospitals, University Legal, University Health Science colleges
  • 15. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Operation  Publicity and Marketing  Standard guerilla marketing on Web  News releases, lectures, emails, brochures/bookmarks, word of mouth  Regular Maintenance  Daily - Broken links found + fixed, answer all emails, handle rights management requests  Business model - Funded initially by Department of Radiology -> Funded by grant from National Library of Medicine -> Funded by University of Iowa Health Care because University is a knowledge factory and this is part of the mission
  • 16. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Evaluation  Evaluation and Continuous Quality Improvement  Usage - At peak: 13 million users read 59 million pages per year, 33% usage outside USA  Reviews – Multiple journals in health care claimed it to be amongst best Web sites in their specific field  "(The Virtual Hospital) is an example of the potential for the World Wide Web in medical education...."- The Lancet, 1995  "The Virtual Hospital is one of the best medical sites and a compelling demonstration of the Internet's potential to impact medical training and practice." - JAMA, 1996  "The Virtual Hospital contains digital versions of hundreds of medical textbooks...in essence, it opens the bookshelves of the staff to the world." - New York Times, 1999  Awards  Exhibition entitled "The Networked Planet: Traveling the Information Highway" featured Virtual Hospital - The Computer Museum, 1995  One of the best on the Internet when reviewed for quality, accuracy of content, presentation and usability - Britannica.com, 2000  One of the top Science sites on the Internet - British Broadcasting Corporation, 2000  Sci/Tech Web Award - Scientific American, 2001
  • 17. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Outcome  First medical Web site, 250th Web site overall ~ Accounted for first mention of "World Wide Web" in MEDLINE biomedical literature database in February 1994  Result was a unique, author-owned, international mirrored university digital library + press that served as an authoritative medical reference + education tool for users around the world
  • 18. Case Study 2 Virtual Hospital - 1992-2005 Digital Library as digital press and institutional repository  Little Lessons Learned  University digital libraries have four key features to distinguish them from for-profit digital libraries  They are free + open to all thus lowering barriers to access + their usage is anonymous guaranteeing patron privacy  They emphasize primarily information for health care providers which patients can graduate to  They have commitment to publishing complete medical reference + education textbooks as well as booklets  They are free from pressures from commercial advertisers  Build simple systems + let complex behaviors / problems emerge + then find solutions (intellectual property, mirroring)  Reason this model not widely replicated is that faculty + universities undervalue their intellectual property. To get ahead at university, university forces faculty to give away intellectual property to publishers…thus publishers still prevail today
  • 19. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 20. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system www.vnh.org (via Internet Archive)
  • 21. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Design  Planning  Mission - Maximize naval readiness by creating + curating medical digital library that can be used as medical reference / health promotion / knowledge management tool to deliver expert medical information to providers + patients at the point-of-care in order to help providers take better care of patients and help patients live healthier lives  Audience - Naval health care providers + patients  Educational construct - Learning situated in practice  Metrics for success  Follow Naval Surgeon General’s desire to take healthcare to the deckplates + move information not people,  Own the keywords "military medicine" + "naval medicine" + "humanitarian medicine" and get a top 10 ranking for it in search engines
  • 22. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Creation  Technology  Great challenge was to deliver digital library services to nomadic patron population on the sea, under the sea, in the air + in the field who have heterogeneous access to Internet bandwidth  Static Web site ~ 2,300 pages ~ Hosted at University of Iowa  ~ 5,000 CD-ROM mirrors distributed every other year to operational corpsmen + physicians  Page Style  Stanford Guidelines with minimalist graphics  Metadata Style  Search engine optimization at site level + book level  Information Architecture  Organized by hierarch, category, alphabetically and by problem- based interface with user-centered design  Tools to Gather Data  Stats - Analog, Webalizer, Wusage  Communication - By comment forms + email  Search engine - ht://dig -> Google Custom Search Engine
  • 23. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Curation  Content creation strategy  Serve as depository library for Navy Medicine = 60 US government medical textbooks  Information on Internet poorly organized, growing rapidly, of questionable authority so digital librarians acted as intelligent agents + did surfing for users on most common medical problems + health promotion tasks creating > 1,000 links to authoritative Web content on 80 common medical problems + 25 health promotion topics  Content quickly + easily updated in response to changing world situation (i.e. chemical + biological warfare content)  Kosovo 1998-99  Operations Other Than War  Content overseen by clinical Specialty Advisory Board
  • 24. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Operation  Publicity and Marketing  Standard guerilla marketing on Web  Lectures to General Medical Officer + Independent Duty Corpsmen trainees ~ Email corpsman training program directors  Regular Maintenance  Daily - Broken links found + fixed, answer all emails  Business model - Funded by grant from US Navy Bureau of Medicine + Surgery ~ From 2006-on self-funded at $50 / year
  • 25. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Evaluation  Evaluation and Continuous Quality Improvement  Usage - At peak: 1.6 million users read 7 million pages per year ~ At end: 10,000 users read 15,000 pages per year  Reviews  Virtual Naval Hospital named as one of the Navy's five "Success Stories" in the field of Knowledge Management - The Department of the Navy's Chief Information Officer, 1999  "Although there are a few books on shipboard medicine, the US Navy's Virtual Naval Hospital constitutes a comprehensive digital library of medical information tailored to the Sea Service..." - Textbook of Military Medicine, Shipboard Medicine by CAPT Terrence Riley, MC, USN (retired), 2003  Awards  Award for Knowledge Sharing / Outstanding Knowledge Expert System - Department of the Navy, 2000  Recognized for its contributions to telemedicine - American Telemedicine Association, 2006
  • 26. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Evaluation  Outcome  Used by US Navy, US Coast Guard, Canadian Navy, Royal Navy, Greek Navy (and US Army + US Air Force) ~ Unauthorized mirror site created by Chinese military  Economic analysis showed it was used 8.2 hours / week/health care provider, 70% thought it improved patient care + improved diagnosis + treatment, decreased need for medical evacuations, 94% were satisfied with it, saved $144,000 / year
  • 27. Case Study 3 Virtual Naval Hospital - 1997-2005 Digital Library as knowledge management system  Little Lessons Learned  People, not technology, make the project work - key intermediaries are individuals with multidisciplinary backgrounds who can interact with different constituencies + who translate + negotiate between them to overcome obstacles + keep project on course + moving forward  Maximized readiness - had a + impact on Navy health care by improving health promotion + patient care + being economically cost-effective  Consistently over delivered + came in under budget -> defunded ~ Our focus was in operating with Sailors and Marines at the tip of the spear…not in operating within BUMED bureaucracy  Successes: User-centric, well received, tri-service, international, military + civil usage, cost-effective
  • 28. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 31. Case Study 4 SearchingPediatrics.com - 2008-Present Digital Library as decision support tool  Design  Planning  Mission - Create a point-of-care decision support tool for pediatricians and pediatric health care providers  Audience Healthcare providers at all levels of training practicing pediatrics  Educational construct - Learning situated in practice  Metrics for success- Own the keywords ”pediatric education” and get a top 10 ranking for it in search engines
  • 32. Case Study 4 SearchingPediatrics.com - 2008-Present Digital Library as decision support tool  Creation  Technology  Static Web site ~ 1 page in size  Page Style  Stanford Guidelines with minimalist graphics  Metadata Style  Search engine optimization at site level  Information Architecture  Organized by location - most relevant information is at top of results page  Can further scope search to textbooks, differential diagnoses, anatomy atlases, journals, drug references, guidelines + policies  Tools to Gather Data  Stats - SiteMeter, Google Analytics  Communication - By email  Search engine - Google Custom Search Engine  Curation  Content creation strategy  Curate an index of most authoritative pediatric textbooks and journals whose content is free to use  Pediatric information must be peer-reviewed and open Access  Can receive CME for its use - indirectly
  • 33. Case Study 4 SearchingPediatrics.com - 2008-Present Digital Library as decision support tool  Operation  Publicity and Marketing  Standard guerilla marketing on Web  Regular Maintenance  Daily - Yearly link check with Big Brother / W3C Link Checker + archive  Business model - Self-funded at $50 /year  Evaluation  Evaluation and Continuous Quality Improvement  Usage - At peak: 15,000 users performed 20,000 searches per year ~ Today: 6,000 users perform 7,000 searches per year  Outcome  Small budget, easy to use tool
  • 34. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new ones  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 37. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice www.biodefenseeducation.org (via Internet Archive)
  • 38. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice  Design  Planning  Mission - Address challenges of how to continuously educate about biodefense / how to develop a sense of community in biodefense / how to manage biodefense information explosion. Collaboratory to discuss news stories or other biodefense topics, Reference Library is biodefense knowledge management system  Audience - Current + next generation of biodefense investigators + trainees, medical personnel, first responders  Educational construct - Curriculum unfolds in practice  Metrics for success - Own the keyword "biodefense" and get a top 10 ranking for it in search engines
  • 39. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice  Creation  Technology  Dynamic Web site (Blog) - EditThisPage / Manila ~ Hundreds of pages in size  Page Style  Stanford Guidelines with minimalist graphics  Metadata Style  Search engine optimization at site level ~ Tagging  Information Architecture  Organized by time and by category  Tools to Gather Data  Stats - SiteMeter  Communication - By email  Search engine - Google Custom Search Engine f
  • 40. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice  Curation  Content creation strategy  News Library - Daily - intelligent agents (Google Alerts) searching world news for 10 terms (Category A agents + biodefense + bioterrorism + biological warfare + bioweapon) to find unclassified biodefense news stories ~ Content syndicated to RSS (really simple syndication  Collaboratory - User-generated content  Reference Library - indexed key biodefense textbooks + made them searchable via scoped search, automated MEDLINE search to keep researchers abreast of current literature, curated reading list of articles to bring new researcher up to speed in biodefense issues  Publish Biodefense Year in Review in January to summarize last 12 months biodefense news
  • 41. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice  Operation  Publicity and Marketing  Standard guerilla marketing on Web Regular Maintenance  Daily - Yearly link check + archive  Business model - Funded by National Institutes of Health
  • 42. Case Study 5 BiodefenseEducation.org - 2004-2009 Digital Library as curriculum unfolding in practice  Evaluation  Evaluation and Continuous Quality Improvement  Usage - At peak: 15,000 users read 32,000 pages per year  Reviews - "In the fast moving modern world, biodefence is becoming a new specialty that we may all have to know more about. So check out BiodefenseEducation.org and its library of information on this topic." - British Medical Journal, 2004  Outcome  News Library - thus an unstructured curriculum unfolds in practice  Over two years, 678 news stories from 178 news sources covered all categories in the required + elective biodefense graduate program courses  Collaboratory - Unused, was failure ~ Reference Library - Little used  Little Lessons Learned  Strengths - Addressed content preferences of health care professionals for biodefense news stories over Web-based textbooks or courses, broad number of authors + news sources used for stories which are of high accuracy, key role played by curators  Limitations - Use only English-language news stories, short half-life of free news stories, news stories may not be free in future,  Successfully addressed challenges on how to continuously educate individuals about biodefense, having a curriculum unfold in practice, + how to manage biodefense information explosion but failed to address challenge of creating sense of community amongst individuals
  • 43. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 44. Case Study 6 PediatricEducation.org - 2004-Present Digital Library as learning portfolio & curriculum unfolding in practice www.pediatriceducation.org ~ www.facebook.com/pediatriceducation ~ www.twitter.com/pedseducation
  • 45. Case Study 6 PediatricEducation.org - 2004-Present Digital Library as learning portfolio & curriculum unfolding in practice  Design  Planning  Mission - Serve as a source of continuing pediatric education with a goal of building a pediatric virtual learning community  Audience - Health care providers at all levels of training practicing pediatrics  Educational construct - Curriculum unfolds in practice, Case-based learning  Metrics for success - Own the keywords "pediatric education" + "paediatric education" and get a top 10 ranking for it in search engines  Creation  Technology  Dynamic Web site (Blog) - WordPress ~ Hundreds of pages in size  Mirrored to developing world via Widernet  Page Style  Stanford Guidelines with minimalist graphics  Metadata Style  Search engine optimization at site level ~ Tagging  Information Architecture  Organized by time and by category  Tools to Gather Data  Stats - SiteMeter, Google Analytics -> WordPress  Communication - By email and monthly email newsletter  Search engine - Google Custom Search Engine -> WordPress search engine
  • 46. Case Study 6 PediatricEducation.org - 2004-Present Digital Library as learning portfolio & curriculum unfolding in practice  Curation  Content creation strategy  Case Library - Author publishes 1 case / week from Learning Portfolio  Content syndicated to RSS / Facebook / Twitter ~ Cross-platform syndication + integration is non- standards based and ugly  Learning Collaboratory - User-generated content  Operation  Regular Maintenance  Daily - Yearly link check with Big Brother / W3C Link Checker  Business model - Self-funded at $150 / year
  • 47. Case Study 6 PediatricEducation.org - 2004-Present Digital Library as learning portfolio & curriculum unfolding in practice  Evaluation  Evaluation and Continuous Quality Improvement  Usage- At peak: 536,000 users read 630,000 pages per year at peak ~ Today: 305,000 users read 665,000 pages per year;  Reviews  “This website serves as a useful tool for continuing education in pediatrics. This resource would certainly be useful for a pediatric resident or clinician to address a subject each week. The website’s usefulness lies in reaching point-of-care clinicians with quick relevant information while also offering more in-depth links and discussion.” - Journal of Electronic Resources in Medical Libraries  Case Library - Over 5 years an unstructured curriculum unfolds in practice - covers 100% of age ranges, 100% of specialties, 98% of symptoms, 55% of disease in MedlinePlus, 90% of topics in 3 national pediatric curricula  Currently 11+ years of weekly cases = ~ 575 published cases  Highly ranked for Internet search term “pediatric education”
  • 48. Case Study 6 PediatricEducation.org - 2004-Present Digital Library as learning portfolio & curriculum unfolding in practice  Little Lessons Learned  "My reading is now focused on my patients"  Example of learner taking control of + assuming responsibility for their own learning  Learning portfolio documents what you have learned, assessment by examination documents what you don't know  Case archive is a database of pediatric cases that can be used to demonstrate spectrum of a disease, multiple etiologies of a symptom, key concepts to master within a discipline, variety of diseases within different age groups  Improvements  Continuously updated MEDLINE searching, Images and Video searchs, Information Prescriptions for Patients  Fads – Competencies – Entrustable Professional Activities  Use of social media
  • 49. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 50. Post Web / Mobile
  • 51. Case Study 7 @pedseducation - 2014-Present Digital Library as curator of social media  Twtitter Design  Planning  Mission - Provide a curated stream of pediatric education information, new education resources, provide thoughts same topics  Audience - Physicians at all levels of training and practice  Educational construct - Curriculum unfolds in practice  Metrics for success - 1,000 followers on Twitter ~ Own the keywords "pediatric education” and get a top 10 ranking for it in Twitter search engine ~ Be a Top 10 Influencer on Symplur for #MedEd + #FOAMPed  Creation  Technology  Dynamic Web site - Twitter  Page Style  Stanford Guidelines (hard to do in 160 character profile)  Metadata Style  Symplur Healthcare Hashtag Ontology for #FOAMed #FOAMPed #MedEd #pediatrics #paediatrics #pediatria  Information Architecture  Organized by time  Tools to Gather Data  Stats - Twitter analytics  Communication - By Twitter  Search engine - Twitter search engine  Curation  Content creation strategy  Daily - A curated stream of pediatric education tweets + retweets  Weekly - Post case from my learning portfolio
  • 52. Case Study 7 @pedseducation - 2014-Present Digital Library as curator of social media  Operation  Publicity and Marketing  Standard guerilla marketing on Web tough to do with Twitter ~ Linked to it from my digital libraries + emailed users who had praised my digital libraries in past ~ Use relevant hashtags, weekly #FF (follow Friday) posts ~ Post good content + pray for retweets  Regular Maintenance  Daily - Yearly link check with Big Brother / W3C Link Checker + archive  Business model - Self-funded at $0 / year  Evaluation  Evaluation and Continuous Quality Improvement  Usage - At peak/current: 130,000 impressions per year and 300 followers on Twitter  Outcome  At least 1 tweets / day, 4 Twitter lists, tweets + retweets viewed ~ 10,000 times / month, community of practice has arisen,  Twitter - 147,000 impressions per year and 270 followers  Top 10 influencer for #FOAMPed, #PediatricEducation (FOAM- Free Open Access Medical Education)  Facebook - 13,000 post views and 8500 unique users per year and 3300 fans
  • 53. Case Study 7 @pedseducation - 2014-Present Digital Library as curator of social media  Facebook  Design  Mission – Build online community of practice  Curation  Content creation strategy  Weekly - Post case from my learning portfolio, plus other content  Evaluation  Hard to develop a following  Lots of noise  People don’t see it because of FB algorithm which constantly changes  Can develop a following  Smaller group that are dedicated with a common focus
  • 54. Case Study 8? 2017 + Digital Library as environmentally integrated, personally wearable or neurally integrated
  • 55. Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 56. Summary Big Lessons Learned  Mission and planning come first  Set goals and objectives  Formal needs assessment  Revisit the mission, goals and objectives often  Gather data  Meeting your objective?  Innovate  Keep it simple  Design, architecture, learning objects, process  Use open standards  Push the standards but be open to new standards  Focus on your users - continuous quality improvement  Meet their needs  Integrate into their workflow  Be open and adaptable  Unrecognized users - patients and international  Use new technologies  Unintended consequences  Learn from all the mistakes  Keep the system open  Make it free, don’t have to register  Ship product
  • 57. Biggest Lesson Learned The rewards are from the people served Pediatriceducation.org, @pedseducation

Editor's Notes

  • #6: Tim Berners-Lee – August 1991
  • #21: Knowledge management (KM) is the process of creating, sharing, using and managing the knowledge and information of an organization