RWJF.org's Rollout of Topic Taxonomy
- 1. RWJF.org’s Rollout of Topic Taxonomy The Project from a Client View
- 2. In This Presentation About RWJF Why do we need a taxonomy? The project: development and validation The result Lessons and next steps
- 3. About RWJF To improve the health and health care of all Americans Focus on problems that demand attention Passion, perseverance and rigor Support programs with measurable impact Seek to create meaningful and lasting social change
- 4. RWJF By the Numbers Total Assets: $10 billion Grants and Contracts Awarded in 2006 —928 Total Active Grants: 2402 7 Priorities
- 5. Why Taxonomy? A Lot of Content 4,296 publications, articles, journal articles, webcasts, videos on RWJF.org today 400 peer-reviewed journal articles and reports added each year 100 end-of-grant reports added each year Does not include (but will soon): news releases, feature stories, speeches, more!
- 6. Why Taxonomy? Clarity Guiding Principles: Be objective, rigorous and transparent and communicate clearly and openly … 2007: 7 Program Areas 2005: 11 Interest Areas 2003: 4 Goals 2001: 8 Priorities Goals change!
- 7. Before The Project: Internal Taxonomy Creation Workgroup of grants managers, Web staff, information architect, research team, librarian Back-indexing completed in early 2005 and takes place weekly on an ongoing basis Approximately 600 terms used to index publications as well as active, pending and turned-down grants Categories cover: RWJF goals and program areas What type of work it was (demonstration, research, communications) What population benefits from the grant and where, if any The subject matter of the project Any medical problem or health condition underlying the program
- 8. The Project Part of a merger of multiple sections into a single section begun in early 2006 User-focused vocabulary covering conditions, treatment, people, places and particular government and social services Permanent home for content as programs change Prioritizes frequently searched content Must be clear that not all topics are funded
- 9. Discovery Reviewed consumer health taxonomies, relevant sections of NLM taxonomies Reviewed our “competitors” – not much structure to parallel Rapid prototype showing a year’s worth of information from the internal taxonomy “Wow, that’s a lot of terms” “Let’s find out how users want to see this”
- 10. Open Card Sort 11 RWJF grantees who also were members of highest priority target audience 3x5 index cards of 100 terms covering the spectrum of content 1 hour to work as individuals to create as many/as few groups as they wanted
- 11. Open Card Sort: Results 11 different hierarchies: some flat, some deep (4 levels) Not a high degree of overlap especially on particular policy issues Certain areas of agreement (conditions and diseases, health care quality, people and places)
- 12. Taxonomy Iteration 10+ iterations between consultant and RWJF First iteration had about 500 topics Last iteration had 370 topics Top level hovered between 6 and 11 choices Hierarchy was between 3 and 5 levels
- 13. Iteration Factors Remove jargon: “pharmaceutical services” v. “prescription drugs” The same word has different meanings in different contexts: “health care system” How do hierarchy and visual presentation match up? Launching a browse without browse functionality and as a result went deeper
- 14. Closed card sort: “Taxi” (Catalyst Group Design in NY) 2 days: 7 hourlong sessions per day User, interviewer, computer Question: Where would you look for an item (for example, asthma in children) in this hierarchy? Users saw terms, but not content below terms, and could not go backwards Users rated their confidence on a scale of 1-5 Items developed from search logs and presented in varying order Big changes from Day 1 to Day 2
- 15. Post-Study Observations Top level still does not satisfy a number of users Not showing content or allowing back-and-forth really stresses the taxonomy Current iteration misses a sizeable number of topics Easier to use when looking for health care items and less easy to use for public health items Several topics had multiple successful paths
- 16. Post-Report Activities Relate topics (relate a chronic condition, such as asthma, to chronic care improvement) Eliminate redundancy Focus on other elements of a relaunch with a hard deadline
- 17. The result: taxonomy at launch 6 top level categories (plus a category of Popular Topics) 366 terms Used in our Publications & Research section and in limited amounts elsewhere
- 19. Long-Term Care
- 21. How’d We Do? Policy-makers seem to like the usage in popular topics but want keywords used more throughout the site Twice the proportion of usage as the previous standalone sections (7.5 v. 13.5) (attributable to multiple factors) Single-page visits down a few points Still difficult to find things Need flexible browse interface to account for multiple starting thoughts (a care setting v. a population v. a disease)
- 22. Our Lessons Design hierarchy and interface together Incorporated more informal feedback with rapid prototyping Still uncertain as to how to engage internal stakeholders Assessing usability and user feedback is not enough; need additional content expertise Look for alternative forms of presentation to suit multiple audience