From Card Sort to Redesigned Intranet Site: A Success Story

Editor's Notes

• #2: A case study: describing how we moved an intranet that wasn’t working for its employees into a successful website
• #3: Measured Progress is a not-for-profit educational testing and assessment company. The company bids on contracts with different states in the U.S. to write tests and assessments that meet legislative requirements of No Child Left Behind Act: to develop academic standards in reading and math. So they work on such projects as MCAS in Massachusetts. They’re located in Dover N.H. have 400 employees – so a lot of internal people who need to access contracts, tests, and other documents for their different projects Objectives: lead with the need, to figure out how current employees want their information organized and what should be emphasized first Goals as stated by Pat Ross, Communications Manager at Measured Progress Redesign Insite, focusing on the navigation and organization (overall IA) Make the site usable: especially when it comes to finding things Make the site attractive and understandable to employees
• #4: Insite was: Launched in July 2004 2 parts to the site, both built with diff. software ON LEFT Search box is only on home page Site map is only on home page Everything is yellow is NOT clickable What’s emphasized the most is the photo, not the info ON RIGHT Built with diff. software than page on left Built on Windows Explorermodel: can only open and close one subdirectory at a time So employees have to use trial and error methods to find info
• #5: Insite mainly relies on bulleted lists. It lacks any secondary or third level navigation. So one employee and I were looking for the Accident & Injury Report form, because she’s responsible for filling them out at the company We had to go down 7 levels of the hierarchy to find the form: She finally said: “ A person could bleed to death before the form is even located!” So people need to use the site but it’s very difficult for them to find what they’re looking for
• #6: Insite is broad and shallow 10 primary navigation categories, 1-2 levels deep No secondary or 3 rd level navigation HR Policies Procedures is narrow & deep 3 primary navigation categories (mission, general administration, HR) 4-8 levels deep Insite Job Request Form: InForm > Job Request Forms > Form (2 clicks) Policies and Procedures Safety Form: InService > HR > Policy Guide > General Administration > Operational Policies > Safety > Accident Reporting and Investigation Plan > [Text] … Accident Report Form (7 clicks)
• #7: Mental model: the supposedly clever navigation scheme was the use of “In” to describe everything: Insite (the name of the site), InGeneral, InService, InForm, InPerson, and so on Take into account different types of users: employees like Mary who had tried to find documents on Insite but had either given up looking for them, or would like so many employees, call up Elke and ask where she should go on the site new hires who had to find things through trial and error
• #8: A card sort seemed the best solution b/c we had 2 sites that needed to be unified, both that had grown too small for the additional information that kept getting added year in and year out, an IA that wasn’t working, …and we wanted to discover how employees – the principal users of the site – wanted their information organized We chose a open card sort b/c we wanted participants to tell us how they wanted the main categories of the site named (as opposed to us naming the categories for them)
• #9: Users: we had users in house We recruited people who were very familiar with and frustrated with the site, as well as new employees who could compare Insite to their intranet at their previous company Recruited employees, b/c they were familiar with the language and terms used in education Analysis: you have more information to sort though with open card sorts b/c besides what we think of as the menu options in the navigation, you also have the names of the main navigation categories, and participants use different names to express the same concept (Our Company, About Us, About Our Company, and so on) tree diagram to see how frequently our participants put cards together in the same groups
• #10: HOW WE CONDUCTED IT Create and run an open card sort with 15 MP employees, including follow-up interviews Jakob Nielsen and Tom Tullis recommend 15 participants in a card sort for the best results Analyze quantitative and qualitative data quantitative: Donna Maurer’s (UX expert in IA; written a book about card sorting): used her spreadsheet to organize the categories and generate correlation statistics IBM’s EZCalc to generate dendrographs Make recommendations for a new navigational hierarchy and structure
• #11: Met with design team to determine the most important categories: which was based on analysis of the main categories of the website and what employees needed to get their jobs done We had 1 hour with each employee, including pre- and post-task interviews, so we also wanted to limit the # of cards; optimum numbers are between 30 and 75 PHYSICAL INSTEAD OF ONLINE CARD SORT We wrote the name of each of the main items on an index card. Shuffle the cards and give the deck to the participant. Have each participant read through all the cards and ask any questions about terms they don’t understand. Have participants sort the cards into piles, placing items that belong together in the same pile. Participants can make as many or as few piles as they want; some piles can be big, others small. Participants can use blank cards to write names of items they believe are missing. At the end, have participants write labels for each of the piles. OPEN CARD SORT with 36 cards Participants are creating IA and organization of the website for you
• #12: 3 criteria in order or priority Participants who were familiar with the website (no learning curve; know the nomenclature) Recruit people from as many diff. Depts. as possible Get an equal number of men and women (came up short but was the least important of our 3 main criteria)
• #13: Actionable items: info you need to access WHAT I WANT TO USE Home Page Contracts Forms Reference material: background info WHAT I WANT TO REFER TO Department Sites Employee Directory Working at Measured Progress Helpdesk
• #14: Card sort over 5 days We began to see trends after the first day of testing (5 participants) Users were grouping categories in similar groups but with different names
• #15: After all 15 card sorts were completed, we standardized the main categories. We gave categories “with similar names or concepts a consistent name” and combined “groups where participants used the same basic concept but a slightly different label.” —Maurer (2007), “Instructions for Use: Card Sort Analysis Spreadsheet” Then you standardize the categories by finding patterns in the labels people use: Facility Info/Where/Facilities/Logistics So you bubble up the info to provide a label that can apply to similar groupings, like HR Facilities and Locations: standardized category for 4 different labels
• #16: Example: “Facilities & Locations.” 8 participants used this category. They put 23 cards into the “Facilities & Location” category (an average of ~3 cards per participant), and used 6 unique cards. It has an agreement number of 0.48. This means that about half - 48% - of participants put the same 3 cards in this category. Higher agreement numbers are better and let you know that more participants used this category.
• #17: Used EZCalc and USort to generate dendrographs. Beta product from IBM that’s no longer supported. But wanted something I could use quickly and that was free. In this case, we had 36 cards per participant and it worked. (I just did another card sort with 74 cards and the program crashed). The green threshold line (30%) indicates lower thresholds (users were more likely to pair any two cards together): Like corporate travel and reimbursement; shipping and distribution, and printing and copying while the pink threshold line (70%) indicates higher thresholds (users were less likely to pair them together): like For New Employees and other groupings: see HR Policies So you’re trying to look at how frequently users put two cards together in the same group. Take the results with a grain of salt but the items seem to group together Most important take-away: participants wanted to divide the HR Policies from the HR Procedures. They were really interested in the Policies having to do with payback and pay outs: payroll, purchasing and requisitions, corporate travel, reimbursement Wanted that separated out from HR Policies: wellness programs, diversity, safety, work schedules, personnel records, etc. THIS MAKES SENSE: what forms you need to get your job done vs. what documentation you need to reference
• #18: 9 major categories, which allows for scalability Can put new pages under at least one of the major categories
• #19: ALSO RELIED ON QUALITATIVE DATA (USER COMMENTS) Example: many users of Amazon.com have probably formed a mental model of a department store; this is a case where conceptual and mental models match. QUALITATIVE DATA Mental model in the current IA of Insite: metaphor of including “In” at the front of each of the labels – InGeneral, InService, InPerson – a fabricated language metaphor that didn’t help users in their wayfinding NEW: they saw it as a reference library, with a big bookshelf of benefits and other information. This mental model helped us drive the new IA of the site
• #20: Participants wanted easier access to information using primary, secondary, and third-level navigation Participants wanted a consistent navigational structure, i.e., one that didn’t change. (Search, Site Map) Participants wanted “HR Policies and Procedures” broken into two separate categories: HR Forms (such as “Corporate Travel,” “Payroll,” “Requisitions”) HR Policies (such as “Diversity,” “Safety,” “Workplace Conduct”) HR Forms: reimbursement info requiring action HR Policies: reference material requiring comprehension or application Use a navigation hierarchy of medium depth and breadth Use a primary navigation hierarchy of no more than 8 or 9 categories Go 3 or 4 levels deep As recommended in the Polar Bear book on Information Architecture (Rosenfield and Morville)
• #21: A SECOND LEVEL PAGE Place the Insite logo in the upper left Add Measured Progress branding at the top: “It’s all about student learning. Period.” Add a Home category in the menu bar Add a menu bar with 8 categories: “Home,” “News & Events,” “Forms,” “Contracts,” “Departments,” “HR,” “Helpdesk,” “About Us” Add a “Search” box, which is attached to the menu bar Add “Contact Us” and “Site Map” links in the upper right Add a “Search Employee Directory” box, with a link to the “Employee Directory” Add a “Quick Links” box, which acts as a “Favorites” area Add secondary navigation on the left-hand side of the page, in this case HR Forms (such as “Benefits” and “Corporate Travel”) Add sub-menus for the secondary navigation, in this case HR Policies and Programs (such as “Compensation” and “Diversity”) Add new information and display it in the main area of the page, including updated “HR Forms” and “HR Policies and Programs” sections
• #22: Main navigation bar at the top of each page Sub-navigation options Quick links on home page and in the footer Expanded navigational hierarchy: before 5 main categories, now 7 categories Designed wider: before 760 pixels, now 920 pixels
• #23: Three-column design on second-level pages Third-level sub-navigation Home on main navigation bar Search at top right of page
• #25: We implemented a page for each department, so each department feels represented Employees can find what each department does Departments can post information they want to post
• #26: Formerly known as the Policy Guide run by separate software different look and feel different search engine and its own website. HR can still update the section on their own.
• #27: Formerly known as the Policy Guide run by separate software, with a different look and feel, a different search engine and its own website. HR can still update the section on their own.
• #28: The Help section is new and is organized by issues.
• #31: Screenshot of highest level, with folder structure in the CMS, and that it’s a mirror of the navigation Content editors can find things quickly in the CMS. The hierarchy translated to the editing level. What it can do in regards to users Generates the side bar automatically
• #32: ITERATIVE PROCESS The main issue with Insite: employees couldn’t find information, and couldn’t complete their work, because the site provides only first-level navigation. It’s missing secondary and contextual embedded navigation (Favorites). So this puts the burden on employees to find their own way thru the website (trial and error) The open card sort allowed users to group items together and create the labels they wanted – in a way that fit their mental model of a reference library…. And it allowed them to show how they wanted to navigate the website. Analysis showed that employees wanted simple, straightforward ways to get to information. MP: clearly sees benefits of their corporate intranet and wants to improve the site for its employees, putting users first BOTTOM LINE: this card sort enabled MP employees to create the IA for the site