Repertory Grid: a missing UX technique?

Editor's Notes

• #4: Today, I’m going to introduce repertory grid technique – rep grid for short. I first heard of rep grid as a tool for eliciting knowledge for expert systems. Since I started reading more about it, I’ve learned that rep grid is used widely across many disciplines. But it’s not used for UX, as far as I can tell. I like the look of rep grid. To me, it has its heart in the right place for UX. So, I’d like to know what you think. Would rep grid be a worthy addition to our UX toolkit?I’m certainly no expert and I confess I’ve never used rep grid. So this won’t be a tutorial-style, “how to” presentation. I’d just like to cover enough to start discussion. If you’re interested in reading up on the finer points of the technique, I can recommend: Jankowicz – an easy “how to” book Fransella, Bell and Banister – explains underlying theory and uses(Full references at the end of the presentation.)
• #6: The “grid” part of rep grid simply means it’s a tabular rating technique, much like any other rating grid in appearance.The big difference is that rep grid doesn’t start with criteria defined by the researcher. It’s a form of structured interview with a scope that includes how subjects rate things as well as what their ratings are. We’re talking with subjects in order to understand how they think. It’s worth remembering that we often gain the best insights from the process, not the product. In UX we already know this. The process of user testing often yields more valuable information than the test results. Quick user research with 5 people can be just as useful as a formal evaluations with valid statistical samples. And so on.Card sorting, which is the closest UX technique to rep grid in intent and approach, is similar.Donna Spencer (www.maadmob.com.au) observed in her IA workshop at Edge of the Web 2008 that it’s a big mistake to take the results of a card sort and use it “raw” as the site architecture. The value of card sorting comes from the insights we gain into user terminology and searching behaviour, not the card sort results.Same goes for rep grid.
• #7: Rep grid technique is based on personal construct theory, which was pioneered by psychologist George Kelly in the 1950s. The theory is that we each make sense of our world by comparing and contrasting things using our own constructs – “classifying rules”, loosely. More about constructs in a minute.The “repertory” part of rep grid conveys the idea that everyone has a repertoire of constructs for making sense of a particular topic. Rep grid technique aims to tease out these constructs.
• #9: Construct a grid with the examples relevant to the topicLeave a column to the left and the right for the constructs we elicit.
• #10: Choose 3 of the examples.Ask the subject to name them if names are relevant to the study.Otherwise follow the protocol for eliciting constructs.Constructs come in pairs. However, avoid construct pairs that are simple negations, such as citrus / non-citrus. More on that later.
• #11: Write one pole of the construct pair in the left column, and the other in the right column.Then ask the subject to rate all the examples on a scale (1-5, 1-7, etc) between the two ends.
• #12: Repeat these steps until you’ve discovered and rated as many constructs as you can with the first group of 3 fruits.Then choose another 3 fruits and apply the same protocol. It’s OK to reuse some from the first group.
• #13: Do the rating for each pair of constructs before you move on to elicit the next constructs.(I believe there are complicated methodological reasons for doing this. I’m just told it works better.)
• #14: Eventually, you will have a completed grid, something like this.At this stage, you can work with what you’ve learned during the process. You can also use statistical methods to discover relationships in the data.
• #17: This time, let’s choose a more likely UX topic than fruit!To illustrate how the components build into a grid, let’s say we’re interested in understanding user impressions of various online bookstores.Always a good idea to include the topic on the grid to clarify its context and objectives.
• #19: Elements go across the top of the grid.Any number is OK, but studies suggest that 5-12 works best.In this example, we’ve selected Amazon, Dymocks, Booktopia and several others.
• #21: Constructs go down the page, in pairs. Each member of the pair is called a pole.Let’s assume we’ve elicited constructs like:Prices easy to find / need to commit before you find out how much you have to payEvidence of fast, reliable service / no way to assess whether service will be goodLooks funky and modern / looks corporate and conservativePublisher-centred / reader-centredRange covers quirky reader interest / mainstream and bestsellers only.
• #23: Ratings go in the middle of the grid, between the poles of each grid.You can use any rating scale you like – a scale of 1-5 is commonly used. On this scale, you would assign one end of the poles as 1 and the other as 5.Ask subjects to rate each element according to how close they are to the 1 pole or the 5 pole.The order of the poles only matters if you intend to use ratings to compare elements numerically. In that case, you can ask the subject which pole is preferred, and place all the preferred poles on the same side of the grid.Another approach is to include an element that represents “the ideal” – either the subject’s selected ideal, or some externally defined ideal, such as usability heuristics. Ratings can then be compared statistically with the ideal.
• #25: This is the basic method for eliciting constructs:1. Pick any three elements and ask the subject how one differs from the others.2. Then ask the subject what the difference is. Remember to elicit both ends of the construct.It reminds me of the old Sesame Street song – “which of these things is not like the other ones?”In the fruit example, we might give John a triad of apple, banana, and orange. If we ask John how one differs from the others, he might use the construct fleshy – [as opposed to] – juicy.Assess whether this is a useful construct for your purposes. Use laddering or pyramiding (coming up next) to get to the right level of understanding.Ask the subject to rate all the elements between the poles of the construct.Pick another three elements and repeat to elicit another construct.In the fruit example, John might now use the construct robust – [as opposed to] – spoils easily.
• #26: Sometimes, the subject comes up with constructs that are not all that useful for our objectives as researchers. For example, they might be too obvious, too vague, too similar to other constructs, or they might be simple opposites that don’t tell us much about how the subject thinks. Of course, we can and should talk to the subject to ensure we understand what their constructs mean. But we need to avoid imposing our own views as we try to elicit more useful constructs.Laddering is a restricted form of questioning to get to more useful constructs, without imposing our own views.Ask “how” or “in what way” questions to ladder down to a lower-level or more specific construct. For example, if we start with the construct reliable – unreliable, we can get to something more specific by asking the subject questions about each pole, such as “how is this element reliable?”, “in what way is this element unreliable?”, and so on.Ask “why” questions to ladder up to a higher-level or value-laden construct.For example, “why is this element reliable?”
• #27: If your purpose is to elicit as many constructs as possible, pyramiding can help to tease out more constructs after the subject has “run out” of the obvious ones.Start by laddering down from a construct with a “how” or “in what way” question about one pole. When the subject comes up with something more specific, then ask what its contrasting pole would be.Go back to the original construct and repeat, using the other pole.Continue this process with the new constructs, until you’ve exhausted the whole pyramid of possibilities.
• #28: Remember, the aim is to avoid imposing our own view.To avoid temptation, it’s a good idea to prepare scripts to use when respondents hesitate to give their own views or look to the researcher for hints and instructions.Uxers who do usability testing will be familiar with this approach.
• #30: Constructs   measure central tendency to find where the subject focuses ratings between the two poles understand how constructs are used overall, eg even or lopsided towards one pole (distribution statistics), dispersed or concentrated (standard deviation) discover relationships between constructs – correlations, cluster analysis Elements   measure distance between elements (eg cluster analysis) relationship between elements or between elements and ideal Multiple grids work with weightings (how important is this to you) determine how similar to each other, how different compare overall ratings Dependencies   identify level of dependency between constructs in the grid determine which constructs best predict future ratings
• #31:  In my opinion, rep grid has the characteristics of a technique that will translate well into UX. I think it has the right philosophy, meets our practicality needs, and could be applied to several aspects of UX practice.Here’s why . . .
• #32: In UX, we know we’re not anything like users. When we create things for others to use, we can’t use our own opinions and behaviour as a yardstick.Rep grid’s philosophy is to try to understand how others see the world, and to avoid imposing our own views as we go about it. This seems to be a good match for the philosophy of UX.
• #33: Rep grid acknowledges and respects the idea that each person sees the world his or her own way. This sits well with a user-centred approach to user research and design.
• #34: Rep Grid uses a structured approach to avoid suggesting that one answer is correct or better than another. It also avoids making assumptions that restrict the range of possible answers.This makes it useful for understanding the mental models users employ to evaluate or use a website, for example. With rep grid, we can apply a small amount of rigour to help us come to grips with an otherwise slippery aspect of user research and UX.
• #35: Unlike evaluations that use pre-defined criteria such as usability heuristics, an evaluation using rep grid can yield a rich insight into the way real users really think. We can get past the banal to learn much more than we could guess.Rep grid can also take us beyond our notions of what’s reasonable or logical, to understand the part played by human emotions and values.This seems well-suited for understanding user experience as a whole.
• #36: When we talk to people about what they think, there can sometimes be a moose on the table – something obviously wrong, but too awkward to acknowledge.As UX practitioners, we know that people don’t always do what they say they do. That’s why we prefer to observe users in action, or to combine observation and discussion.If we use rep grid to talk to people about how they evaluate their user experience, could we be misled? Well, yes. People tend to mention only their polite and socially acceptable thoughts, values and beliefs. When psychologists use rep grid, they call this the “social desirability effect”. Psychologists use two methods to minimise the social desirability effect.The easiest is to shade the expression – to find a socially acceptable name for a thought. It’s like calling a spade a digging device rather than a ruddy shovel.The second approach is to acknowledge and work with the social desirability effect. Ask respondents to identify the most socially desirable responses, and then identify how far or close their own reactions are to the “ideal”.Fortunately, we seldom deal with the touchy subject areas that psychologists work with. For UX user research, it may be sufficient simply to remove obstacles to frankness at the start of the session. You can assure participants of anonymity, explain the process, walk through an example . . . anything to put people at ease.
• #37: Rep grid has practical advantages that would make it an easy addition to the UXer’s toolbox.
• #38: Rep grid compares well with other UX user research techniques for speed and ease of use.It’s quick to administer, cheap and simple.
• #39: You can use rep grid with one subject, a handful of subjects, or a statistically significant sample size.
• #40: Sometimes, you are forced to don your suit and use facts and figures to convince stakeholders that your insights are worthy.Rep grid is amendable to statistical analysis and there are many applications available to help you do it.
• #41: To accept a technique into our UX bag of tricks, we don’t expect scientific precision and repeatability. What we want is a quick, cheap but legitimate way to gain better insights into the experience of our users.In fact, it’s just about essential for a UX technique to be lightweight when required.When there’s little or no budget for user research, we need to gain as much understanding of users as we can – on the cheap, on the run, and often on the sly. Ruth Ellison (www.ruthellison.com) is a great exponent of these “guerrilla” tactics.Rep grid is flexible and well suited for using with a small number of subjects. Psychologists often use rep grid to understand a single person’s view of the world.And, as we saw earlier, we can use rep grid process to gain qualitative insights informally. Formality and statistics are entirely optional.
• #42: UX is a young and eclectic discipline – still in the first blush of youth, full of energy and enthusiasm. It’s incredibly open to new ideas. The arrogance of our discipline’s youth also makes us unapologetic about pillaging methods and techniques from other disciplines. Typically, we make them our own, distilling key ideas and discarding methodological niceties to create a lightweight, practical UX approach. Rep grid has already been appropriated by a wide variety of disciplines. Why not UX?
• #43: I could see rep grid being useful for any UX activity in which we want to know what users really think. Examples might be: competitor analysis evaluation of an existing website comparison of prototypes – eg alternative visuals, interactions, IA, navigation eliciting decision processes for expert systems, AI, decision support, etc evaluating the meaningfulness of names and labels understanding user tasks, priorities, scenarios understanding what users want in reports developing realistic user personas evaluating draft content