You Call That Intuitive? Principles for Integrating User Experience Into Training

Editor's Notes

• #3: CHRIS: Welcome to “You Call That Intuitive?” or should it really say “You call that Intuitive?”. Today Jeff and I will be talking about User Experience or “UX” and how it can contribute to training design. We are going to walk you through the key principles and practices to successfully implement US to improve your training courses. In the second half of our session we will walk you through several examples of both good and bad design. Disclaimer: the examples are from my work, Jeff was kind enough to include them here, names have been changed to protect the innocent! Introductions are in order! I’m Chris King… JEFF: and I’m Jeff Barnes, I’ve been applying User Experience Principles to training, websites, and online tools for the Federal Government for 6 years, including the Department of Veteran’s Affairs “MyCareer@VA” career development website. I have a graduate degree in Human Factors Psychology from George Mason University.
• #4: JEFF: User experience (UX) is a broad term used to explain all aspects of a person’s interaction with a system, application, or tool. This includes the user interface, information architecture, graphics, interaction design, content strategy, communication, and even the guides and helpdesk support. The goal of UX Design is to create products that give users a positive experience through the measurement and improvement of the products efficiency, effectiveness, and satisfaction.
• #5: JEFF: So what do we mean here? Efficiency helps us to understand how quickly a user can accomplish a task or how long it takes them to learn an interaction. When we think of good design we often use words like “usable” or “intuitive”. These words are describing how easy it is for a user to adapt to a new system or to become proficient in its use. To demonstrate this point several recent studies have indicated the magical “three click” rule in web design is not necessarily hard-and-fast for user satisfaction, frequently, a user’s satisfaction can be more closely attributed to the difficulty of decisions they make along the way, rather than the number of clicks it takes. Effectiveness is a measure of how many errors the user makes when interacting with a system. This can be anything from clicking on the wrong the wrong link, to not being able to locate the desired information. Effectiveness is interesting, because the impact of errors can vary so greatly. If a user accidently presses “Exit” in your training course, the impact of this error depends on whether you designed a verification step in your course, if you did then the impact is minimized and the user selects “cancel”, if you did not, the user exits the course and loses their progress. Which leads me to, user satisfaction. User satisfaction is how we measure the user’s perspective on their interaction, this can be done through observation or surveys and it gives us insight into what the user’s attitude is toward the product. It is heavily dependent on the users perception of the system and their expectations for the interaction. Sometimes, especially with web-based training courses. A measure of success can be as simple as hearing a user say “That was less painful then I thought it would be!”
• #6: CHRIS: So, how do these principles apply to training design? What is there to be gained from integrating user experience practices into a training design lifecycle? These are some of the outcomes you can expect from more focusing on the user’s experience in training design. Since most folks probably aren't familiar with the term “conversions”, Jeff can you explain that bullet? JEFF: Sure, conversion rates one way we measure the success in our design. We identify key performance indicators like how many users click on the “learn more” button, then measure the number or percentage of “conversions” based on the entire user population. This tells us how successful that button was in the wild. I would also like to call special attention to the last two points on the left side. The goal of training is provide learning outcomes for the users. The best way to do that is to minimize the impact of the technology and the delivery method so the users can just focus on the learning content. Users don’t typically notice good UX. When the experience is fluid and intuitive, the user is so focused on the content that they don’t pay attention to the design.
• #7: CHRIS ALL: our good friend ADDIE, its iterative, very similar to the UX process and makes it easier to apply the principles to the different phases of ADDIE In the ADDIE instructional design methodology, an iterative process of analysis through evaluation allows the designers to understand their audience, design the training to their needs, and evaluate whether those needs are met. This process is very similar to user experience design, and the principles and tools of UX can easily be integrated into this model. Next slides shows us an overlay of the UX stuff on ADDIE, how you can apply them to learning design
• #8: JEFF: As you can see, at each step in the instructional design methodology, user experience can be incorporated to improve the user outcomes. These tools require varying levels of time and resources to apply, and can serve as a toolbox for instructional designers to choose from. I will go into detail about a few of these methods, and how they can be applied effectively in the training design lifecycle.
• #9: JEFF: Knowing your users is critical to designing a product that will provide a great experience. We use research methods such as interviewing, contextual observations, focus groups, web analytics, and helpdesk or user support data to gain a better understanding of the user groups before moving forward with development. Some key distinctions in our data collection process that affect user experience include, the users age, level of comfort with technology, frequency of system interaction, and expectations or goals. Frequently, ISDs don’t look at these factors when evaluating the audience of learners CHRIS: We can add in these data points to our existing audience analysis to give the UX team something to work with JEFF: We can use this data to create personas, which are profiles of representative users from each user group. They include demographic information, goals, frustrations, and other relevant information. These personas are based on data we’ve already collected about system users and help us to make tough design decisions. We refer to them throughout the design lifecycle by asking ourselves “Would this make sense to Mathew?” “How could this information best be conveyed to him?” Reap the benefits of the audience analysis you’ve performed by applying personas to the design process for your course. CHRIS: How many people in the audience have used personas in a training design? What was your experience like?
• #10: JEFF: Now we move on to the design phase… When most people think of user experience or usability, what they are envisioning is a heuristic review, where a UX designer or a UX team reviews a product and makes recommended changes to the layout, color, and organization of information on the interface. UX professionals use heuristics as our best practices for ensuring the usability of a product. Heuristic design principles are guidelines based on scientific research on how users perceive, process, and remember information. By observing your training’s conformance to these heuristics, you can predict areas where a user may have difficulty, or how the training will perform in user testing and in the wild. These guidelines are foundational to the field of Usability and there are many Heuristic review templates and checklists available to help guide you through your product review. CHRIS: In training we call this the User Interface Review… how many people have integrated these into their design process, frequency? Budget? Hours?
• #11: JEFF: After you have collected all the necessary information about your user groups and designed a product that you think will work, now its time to iterate! Test out your ideas and find out if the design you have so heavily invested in really works for the users you intended it for. Iterative design is critical to good User Experience and will save you lots of time, frustration, and dollars when you catch issues before your product moves into development. -slide- I would like to give two examples of iterative design to demonstrate this point. Frequently, in the training world, the bulk of testing comes after deployment. The designer creates the course based on their understanding of the learners, then asks subject matter experts to review the training course for any issues prior to distributing the training to the target audience. CHRIS: does this sound familiar? JEFF: While this method works and often creates a high-quality product, incorporating UX methods even earlier in the design lifecycle can lead to better results. CHRIS: This is the process my team would use we would design and deploy… we would end up deploying 3 or 4 times -slide- In this second example, you can see that evaluation, identification of issues and re-design are all taking place before the training has been deployed, and in some cases before certain portions are even developed. By conducting heuristic reviews, quick user tests, and getting feedback earlier in the design lifecycle, there is a potential to save time and reduce waste in both the development and deployment phases. CHRIS: So lets take 90 seconds and talk with your neighbor about which of these methods you use, and how you could improve that process in your next training How many people have used SAM? Megan Torrance LAMA. alternative to ADDIE…. JEFF: Now that we are ready to deploy our well designed product. Lets talk about implementation!
• #12: JEFF: After you have collected all the necessary information about your user groups and designed a product that you think will work, now its time to iterate! Test out your ideas and find out if the design you have so heavily invested in really works for the users you intended it for. Iterative design is critical to good User Experience and will save you lots of time, frustration, and dollars when you catch issues before your product moves into development. -slide- I would like to give two examples of iterative design to demonstrate this point. Frequently, in the training world, the bulk of testing comes after deployment. The designer creates the course based on their understanding of the learners, then asks subject matter experts to review the training course for any issues prior to distributing the training to the target audience. CHRIS: does this sound familiar? JEFF: While this method works and often creates a high-quality product, incorporating UX methods even earlier in the design lifecycle can lead to better results. CHRIS: This is the process my team would use we would design and deploy… we would end up deploying 3 or 4 times -slide- In this second example, you can see that evaluation, identification of issues and re-design are all taking place before the training has been deployed, and in some cases before certain portions are even developed. By conducting heuristic reviews, quick user tests, and getting feedback earlier in the design lifecycle, there is a potential to save time and reduce waste in both the development and deployment phases. CHRIS: So lets take 90 seconds and talk with your neighbor about which of these methods you use, and how you could improve that process in your next training How many people have used SAM? Megan Torrance LAMA. alternative to ADDIE…. JEFF: Now that we are ready to deploy our well designed product. Lets talk about implementation!
• #13: JEFF: After you have collected all the necessary information about your user groups and designed a product that you think will work, now its time to iterate! Test out your ideas and find out if the design you have so heavily invested in really works for the users you intended it for. Iterative design is critical to good User Experience and will save you lots of time, frustration, and dollars when you catch issues before your product moves into development. -slide- I would like to give two examples of iterative design to demonstrate this point. Frequently, in the training world, the bulk of testing comes after deployment. The designer creates the course based on their understanding of the learners, then asks subject matter experts to review the training course for any issues prior to distributing the training to the target audience. CHRIS: does this sound familiar? JEFF: While this method works and often creates a high-quality product, incorporating UX methods even earlier in the design lifecycle can lead to better results. CHRIS: This is the process my team would use we would design and deploy… we would end up deploying 3 or 4 times -slide- In this second example, you can see that evaluation, identification of issues and re-design are all taking place before the training has been deployed, and in some cases before certain portions are even developed. By conducting heuristic reviews, quick user tests, and getting feedback earlier in the design lifecycle, there is a potential to save time and reduce waste in both the development and deployment phases. CHRIS: So lets take 90 seconds and talk with your neighbor about which of these methods you use, and how you could improve that process in your next training How many people have used SAM? Megan Torrance LAMA. alternative to ADDIE…. JEFF: Now that we are ready to deploy our well designed product. Lets talk about implementation!
• #14: JEFF: In the implementation phase, UX can still provide value for training outcomes. UX does not stop after deployment. This is when the rubber meets the road. Thinking about how to support the users in the implementation phase is just as important as it is in the design phase. Supplemental documentation like email communication, instructions, and training aids are integral to the user experience and deserve the same amount of consideration as you course itself. Communication about your course is key, even more important is heading off any known issues with the technology. So If your player is bad and you cant fix it, put the instructions in the email. Head off design issues, and set your users expectations for the experience. We will get back to completion certificates in a bit, but I want to make a point about training aids. Make sure that they provide value to the user and solicit feedback to find out what they should include! CHRIS: How do I help users get through the interface? I lead them so they don’t have to make decisions unless it’s part of the training. Something about Gagne (Conditions of Learning)
• #15: JEFF: When it comes to evaluating user experience, the best method is to ask users directly! In the case of online trainings, many folks already do some sort of survey feedback whether it is to assess the quality of the training course itself, the learning outcomes, or both. What you are probably not currently assessing is the usability of your training course. When you follow-up with a user, by asking the right questions, you might be surprised what you can learn! CHRIS: Hey Jeff, how many questions would you add to a survey??? JEFF: You can get a lot of valuable UX information by adding just one or two questions to a Kirkpatrick level 1 survey, or in some cases by simply rephrasing questions to incorporate issues related to User Experience In one example, I worked with a training team who was issuing web-based compliance training to a large group of learners. They were consistently getting low scores on the question “How satisfied were you with this training course?”. After a few quick follow-up interviews with users, we decided to ask the question “What did you find most frustrating about this training?” and as it turned out, it was not a learning content issue that was leading to the low satisfaction. The majority of users who rated the training low, identified issues related to three categories 1. Accessing the training course, 2. Loading the videos during the training, and 3. Downloading the training certificate. This is just one example of how user experience issues related to the technology solution interfered with the leaners overall experience with the training course. CHRIS: How many of you use the Kirkpatrick level 1 survey as your main metric for evaluating training outcomes? Imagine the drag on those outcomes that could come from bad UX! JEFF: By asking the right questions, questions which expand feedback from just learning outcomes can identify a larger pool of issues that may be effecting the quality of the final product. CHRIS: As you can see on this slide we have some examples of survey questions that can give you UX survey data and fit easily with your existing evaluations!
• #16: JEFF: User testing is another method of evaluating User Experience by collecting information directly from the source. These tests can be as simple as a paper prototype test, where the designer prints out some screenshots and gets quick feedback about the layout, navigation, content, or structure of the course. User testing can vary in complexity, including structured observations and even eye tracking of implicit feedback like emotional reactions! This is infrequently necessary (or cost effective) in the training development lifecycle, but the more information you can collect from real users before or after the launch, the better off in the end. CHRIS: Here is where I take minute to ask how many of you use a peer review process? How many people have used SMEs to test their courses? How many have used actual users!?! So what's the difference between these reviews and user tests? SUMMARY CHRIS: These are the 5 phases of ADDIE and the tools that we borrow from the UX community. Jeff is going to take us through some examples of actual eLearning courses to show UX in action.
• #17: JEFF: In these first few examples I want to talk about my biggest pet-peeve in training UX, consistency! On the screen you can see that the user is prompted with a choice of two response options, and the navigation button in the bottom left has disappeared. Is it clear to you (or the user) that you have to click on the training slide to proceed?
• #18: JEFF: In these first few examples I want to talk about my biggest pet-peeve in training UX, consistency! On the screen you can see that the user is prompted with a choice of two response options, and the navigation button in the bottom left has disappeared. Is it clear to you (or the user) that you have to click on the training slide to proceed?
• #19: JEFF: Next, we see that in a later slide, the navigator window has disappeared entirely and the user must now click on a button in the content area, not a text box. These issues are admittedly small individually, but each little change can cause the user difficulty and these challenges can add up to create a negative user experience!
• #20: JEFF: Ok, game time! No looking ahead! On this slide we see the navigator, content, and a sticky note. These notes are meant to be the “more information” resource. Please note the location on the current slide!
• #21: JEFF: Now, where is the sticky note going to appear on this slide? I’ve blocked it out so you can guess, anyone? -slide- It actually appears in the bottom right, why did it move? Was this necessary or just an oversight?
• #22: JEFF: Ok, how about now? -slide- Bottom right! Ok, enough picking on Chris… This is just an example to show how inconsistency can give the appearance of disorganization and make the interface more noticeable to the user. You should provide the user an opportunity to learn how your system will perform and follow those rules as closely as possible so they are never left guessing.
• #23: JEFF: Next, I want to give you an example of good design and briefly talk about error prevention. Here you can see that the process is very clear, pick a response option and hit submit.
• #24: JEFF: An immediate change to the screen leaves the question and response options up, and now the user is already looking at that portion of the screen where the answer appeared. The background fades out and the user can now select “Continue” CHRIS: Too bad this is in the same training with the previous bad examples. JEFF: That’s true, but here you created a very clear path for the user and prevented them from making errors related to the interface.
• #25: JEFF: Another important heuristic is creating systems that match our understanding of how the real world works. Think of this as “don’t fix it if it isn’t broken”. Here the interface presents the alphabet in a continuous format from left to right, just as the user would expect. The actionable letters are highlighted and an alternate method of navigation is even presented on the left side. Just because this is an online interaction, we cannot separate the users knowledge of the real world from their understanding of how the system will work. Don’t just show the letters that want, help orient the user with something they already know.
• #26: JEFF: Many trainings have some form of navigator window with controls. -slide- Here we can see a list of menu options on the left, exit in the middle, and a navigation control on the right. The problem is that there is no visual distinction made between these different types of buttons. The ones on the left are drop down menus, exit will take the user out of the course entirely, and the previous/next buttons on the right navigate through the course. These should be distinguished. -slide- In the bottom right are icons… these icons need to be used consistently so a simple change to the “Hide CC” control would make the appearance consistent across the three buttons.
• #27: JEFF: This is an example of good controls. The buttons on the bottom have clear icons and there is a solid navigation strategy in the bottom right which includes a progress indicator. On the top right you can see a list of menu options, that all lead to pop-up windows.
• #28: JEFF: Earlier I mentioned visibility of system status, here we have a good example. You can see the progress bar shows the portion of the course the user has completed, and the bottom bar shows progress through the current video. This helps the user see quickly where they are and what the system is doing. This heuristic can also apply to things like loading messages, which help keep the user informed of actions the system may be taking in that are otherwise not readily apparent.
• #29: JEFF: Visual layout can present a unique challenge with eLearning courses, sometimes a course or window will not appear the way we intended, or an error in an uncontrolled element of the system will cause them to behave erratically. In this case, the system presenting the training automatically loads the player in a reduced size, and leaves the other windows open behind. This can be distracting to the user. In addition, the player is far to small for the user to read the slides. Errors like this can impact our user’s perceptions of the value of a training course. How can we expect the user to trust our course content when the technology used to deliver that content seems so obviously faulty. Does this look like a modern web interaction?
• #30: JEFF: For this final example, please raise your had if you think the screen has loaded correctly in the browser window. It did! This player simply does not use what we would consider a modern layout for the buttons used to navigate the exam. So, what can an instructional designer do when faced with this kind of challenge? The ISD can create additional instructions, use an overlay, or place reminders in the training course as an alternative solution when a technology modification just isn’t feasible. Remember, the course’s perception will be effected by the user’s expectations. If you show them a screenshot of how the exam is laid out, they won’t be surprised by the unusual button placement later. --------- CHRIS: Lets conclude today with a thought experiment… “One of the most common ways to keep users engaged in a eLearning course is by creating interactions where the user must click into the training window to progress dialogue, review additional content, or respond to a prompt. Do these interactions truly add value to the user’s experience with the training or does it simply keep them awake? What are some other methods that we could use to increase user interaction? I will give you a minute to discuss among yourselves…. Any thoughts?”
• #31: JEFF: So we leave you with this. The principles and tools of user experience can be fluidly integrated into your existing instructional design processes. Doing so will help you to better know and reach your users and ultimately contribute to the learning outcomes of your online training courses. CHRIS: I’m out! (drop the mic)