Explainable AI for non-expert users

Editor's Notes

• #6: We focus specifically on visual analytics for non-expert users. Non-expert users are defined as users that have little knowledge of data processing and analysis. We research two algorithmic foundations: predictions models like regression and clustering and recommender systems that suggest items to users. The key objective is to communicate the uncertainty of these models to support decision-making and increase trust. We do it through the use of visualization techniques that explain the models.
• #7: Amazon.com gebruikt een collaborative filtering techniek: zoekt gelijkenissen tussen gebruikers en gaat dan op basis van wat gelijkaardige gebruikers kopen aanbevelingen doen.
• #14: The procedure contains the following steps: \begin{enumerate} \item \textit{Tutorial of study} - Participants were invited to read the description of the user study and to choose a scenario for generating a play-list. Then, they were asked to watch a task tutorial. Only the features of the particular setting were shown in this video. The ``Start'' button of the study was only activated after finishing the tutorial. Users logged in with their Spotify accounts to our experimental system, so that our recommenders could leverage the Spotify API and user listening history to generate ``real'' recommendations. \item \textit{Pre-study questionnaire} - This questionnaire collects user demographics and measures user's personal characteristics such as musical sophistication and visual memory capacity. %and their trust in recommender systems. The visual memory capacity is measured by ``Corsi block-tapping test''. In the test, a number of tiles are highlighted one at a time, and participants are asked to select the tiles in the correct order afterward. The number of highlighted tiles increases until the user makes too many errors. In Experiments 1 and 3, we used a test with a more sophisticated implementation of the Corsi test~\footnote{\url{https://www.humanbenchmark.com/tests/memory}, accessed June 2018}, which allows us to better distinguish participants by the level of visual memory capacity. In Experiment 2, to control the workload of participants in the within-subjects design, we chose a simple version of the Corsi test~\footnote{\url{http://www.psytoolkit.org/experiment-library/corsi.html}, accessed June 2018} for measuring visual short-term memory. \end{itemize} \item \textit{Manipulating Recommender and rating songs} - To ensure that participants spent enough time to explore recommendations, the questionnaire link was only activated after 10 minutes. After tweaking the recommender, participants were asked to rate the top-20 recommended songs that resulted from their interactions. \item \textit{Post-study questionnaire} - Participants were asked to evaluate the perceived quality, perceived accuracy, perceived diversity, satisfaction, effectiveness, and choice difficulty of the recommender system. After answering all the questions, participants were given opportunities to provide free-text comments of their opinions and suggestions about our recommender. \end{enumerate}
• #15: Figure 2 shows that participants with low NFC are reporting a higher confident in their playlist with the explanations in- terface than in the baseline. Participants with a high NFC reported the opposite. Hence, the participants with low NFC have more confidence in the explanation interface than in the baseline, in contrast to user with low NFC. An explanation might be that low NFC participants benefited from the expla- nations because they did not spontaneously engage in much extra reasoning to justify the recommendations they received, and when they received the rational from the explanation this increased their confidence in their songs selection. Figure 2 also indicates that as the NFC increased, the con- fidence of participants in the playlist created in the baseline also increased. This result indicated that participants with a high NFC were more willing to understand their own musical preference in relation to the attributes of the recommended songs. This may have resulted in a higher confidence in their playlist. We did not see the same increase in trust as NFC increases in the explanation interface. As Figure 2 shows, the NFC scores in the third quartile were almost the same for both interfaces. At the highest NFC level, participants had a higher confidence in the baseline than in the explanation interface. The reduced confidence within the explanation interface could be an indication that users with a high NFC have less need for explanations.
• #21: We employed a between-subjects study to investigate the effects of interactions among different user control on acceptance, perceived diversity, and cognitive load. We consider each of three user control components as a variable. By following the 2x2x2 factorial design we created eight experimental settings (Table~\ref{tab:table1}), which allows us to analyze three main effects, three two-way interactions, and one three-way interaction. We also investigate which specific \textit{personal characteristics} (musical sophistication, visual memory capacity) influence acceptance and perceived diversity. Each experimental setting is evaluated by a group of participants (N=30). Of note, to minimize the effects of UI layout, all settings have the same UI and disable the unsupported UI controls, e.g., graying out sliders. As shown in section ~\ref{evaluation questions}, we employed Knijnenburg et al.'s framework~\citep{knijnenburg2012explaining} to measure the six subjective factors, perceived quality, perceived diversity, perceived accuracy, effectiveness, satisfaction, and choice difficulty~\citep{knijnenburg2012explaining}. In addition, we measured cognitive load by using a classic cognitive load testing questionnaire, the NASA-TLX~\footnote{https://humansystems.arc.nasa.gov/groups/tlx}. It assesses the cognitive load on six aspects: mental demand, physical demand, temporal demand, performance, effort, and frustration. The procedure follows the design outlined in the general methodology (c.f., Section \ref{sec:general-procedure}). The \textit{experimental task} is to compose a play-list for the chosen scenario by interacting with the recommender system. Participants were presented with play-list style recommendations (Figure~\ref{fig:vis1}c). Conditions were altered on a between-subjects basis. Each participant was presented with only one setting of user control. For each setting, initial recommendations are generated based on the selected top three artists, top two tracks, and top one genre. According to the controls provided in a particular setting, participants were able to manipulate the recommendation process.
• #22: Main effects: REC has lowest cgload and highest acceptance Two-way: All the settings that combine two control components do not lead to significantly higher cognitive load than using only one control component. combing multiple control components potentially increases acceptance without increasing cognitive load significantly. visual memory is not a significant factor that affects the cognitive load of controlling recommender systems. In other words, controlling the more advanced recommendation components in this study does not seem to demand a high visual memory. In addition, we did not find an effect of visual memory on acceptance (or perceived accuracy and quality). One possible explanation is that users with higher musical so- phistication are able to leverage different control components to explore songs, and this influences their perception of recommenda- tion quality, thereby accepting more songs. Our results show that the settings of user control significantly influence cognitive load and recommendation acceptance. We discuss the results by the main effects and interaction effects in a 2x2x2 factorial design. Moreover, we discuss how visual memory and musical sophistication affect cognitive load, perceived diversity, and recommendation acceptance. \subsubsection{Main effects} We discuss the main effects of three control components. Increased control level; from control of recommendations (REC) to user profile (PRO) to algorithm parameters (PAR); leads to higher cognitive load (see Figure \ref{fig:margin}c). The increased cognitive load, in turn, leads to lower interaction times. Compared to the control of algorithm parameters (PAR) or user profile (PRO), the control of recommendations (REC) introduces the least cognitive load and supports users in finding songs they like. We observe that most existing music recommender systems only allow users to manipulate the recommendation results, e.g., users provide feedback to a recommender through acceptance. However, the control of recommendations is a limited operation that does not allow users to understand or control the deep mechanism of recommendations. \subsubsection{Two-way interaction effects} Adding multiple controls allows us to improve on existing systems w.r.t. control, and do not necessarily result in higher cognitive load. Adding an additional control component to algorithm parameters increases the acceptance of recommended songs significantly. Interestingly, all the settings that combine two control components do \textit{not} lead to significantly higher cognitive load than using only one control component. We even find that users' cognitive load is significantly \textit{lower} for (PRO*PAR) than (PRO, PAR), which shows a benefit of combining user profile and algorithm parameters in user control. Moreover, combing multiple control components potentially increases acceptance without increasing cognitive load significantly. Arguably, it is beneficial to combine multiple control components in terms of acceptance and cognitive load. \subsubsection{Three-way interaction effects} The interaction of PRO*PAR*REC tends to increase acceptance (see Figure \ref{fig:margin}a), and it does not lead to higher cognitive load (see Figure \ref{fig:margin}c). Moreover, it also tends to increase interaction times and accuracy. Therefore, we may consider having three control components in a system. Consequently, we answer the research question. \textbf{RQ1}: \textit{The UI setting (user control, visualization, or both) has a significant effect on recommendation acceptance?} It seems that combining PAR with a second control component or combing three control components increases acceptance significantly. %KV: this paragraph refers to different RQs: either rephrase or omit? -SOLVED \subsubsection{Effects of personal characteristics} Having observed the trends across all users, we survey the difference in cognitive load and item acceptance due to personal characteristics. We study two kinds of characteristics: visual working memory and musical sophistication. \paragraph{Visual working memory} The SEM model suggests that visual memory is not a significant factor that affects the cognitive load of controlling recommender systems. The cognitive load for the type of controls used may not be strongly affected by individual differences in visual working memory. In other words, controlling the more advanced recommendation components in this study does not seem to demand a high visual memory. In addition, we did not find an effect of visual memory on acceptance (or perceived accuracy and quality). Finally, the question items for diversity did not converge in our model, so we are not able to make a conclusion about the influence of visual working memory on diversity. \paragraph{Musical sophistication} Our results imply that high musical sophistication allows users to perceive higher recommendation quality, and may thereby be more likely to accept recommended items. However, higher musical sophistication also increases choice difficulty, which may negatively influence acceptance. One possible explanation is that users with higher musical sophistication are able to leverage different control components to explore songs, and this influences their perception of recommendation quality, thereby accepting more songs. Finally, the question items for diversity did not converge in our model, so we are not able to make a conclusion about the influence of musical sophistication on diversity.
• #31: As I have already explained, this is the LADA Dashboard that predicts the chance of success and presents a set of components that are intended to help the student adviser to give feedback to the student.
• #32: We evaluated this dashboard with both laymen and experts. They used LADA based on real data of students to plan a semester for a student in two Conditions: Using the dashboard. Using the traditional system.
• #33: Results indicate that the prediction models enables users to explore more possible scenarios, but more transparency is required. The quality indicator is insuffient to increase user trust
• #35: The first application domain is agriculture Precision agriculture is an interesting domain to research the representation of data and uncertainty associated with both data and prediction models for non-expert users, such as farmers. This domain faces some typical challenges of Visual Analytics, missing data and uncertainty of predictions. In this work, we conducted a systematic review of visualisation techniques and the representation of uncertainty.
• #40: The transcribed data were coded and analysed following the thematic analysis approach (Braun and Clarke, 2006), which resulted in four main themes: potential use cases, trust, usability, and understandability. Marimekko charts difficult
• #41: Showing the dis/agreement of model outputs with expert’s knowledge can promote understandability and trust. ability to see dis/agreements between models' predictions and an expert's knowledge can help them inspect further and thus promote trust. u
• #43: Job recommender systems have become a well researched area. In this dissertation, we designed and evaluated two interactive dashboards that can help explain the reasoning behind job recommendations and predictions. A first dashboard has been elaborated that explain predictions of the chance to find a job in a particular job area. The second dashboard explains job recommendations by showing competences and competence gaps instead of the typical matching score used by recommender systems.
• #44: This is the first dashboard: We designed this dashboard on top of a prediction model to explain the inner workings to job mediators. We make a distinction between actionable and non-actionable parameters. Age is an example of a non-actionable parameter.
• #45: We used a user-centered design methodology consisting of the steps listed on this slide.
• #46: This is the first dashboard: We designed this dashboard on top of a prediction model to explain the inner workings to job mediators. We make a distinction between actionable and non-actionable parameters. Age is an example of a non-actionable parameter.
• #47: Job mediators highlighted the importance of customising the dashboard to be able to control the message. Five mediators used negative parameters to support their message Two mediators removed negative parameters to avoid demotivation. “age can be demotivating” “too much information might be difficult to process” “would like to see an overview of everything” “depends on the job seeker” Our explanatory tool helps mediators to control the message they wish to convey depending on the situation context.
• #49: Health is another interesting domain: here we try to provide relevant information to end-users that is trustworthy and has a positive impact on decision-making. We researched the representation of uncertainty of a prediction model that predicts the impact of a food product on weight as well as different layouts to present this data together with recommendations in an AR setting.
• #53: We found that the stack visualisation performs better with HMD devices with a limited field of view, like the HoloLens, at the cost of some usability affordances (RQ4). For handheld devices, the grid and the pie tended to score higher in terms of confidence in decision making, compared to the list and stack layouts (RQ1, RQ4).
• #54: The prediction model shows the impact of the food product on the weight of the participant. Opacity is used to represent the uncertainty of this prediction. (POINT to third card)
• #55: The prediction model shows the impact of the food product on the weight of the participant. Opacity is used to represent the uncertainty of this prediction. (POINT to third card)
• #56: We compared four different layouts to represent this information: a stack layout, a list layout, a grid layout and a pie layout. We compared their use in two implementations: one using the Microsoft HoloLens, and a second one using an Smartphone
• #57: We evaluated these layouts in a user study with 28 participants in a lab setting. and measured both subjective and objective data collected from the use of our application,
• #58: We found that the stack visualisation performs better with HMD devices with a limited field of view, like the HoloLens, at the cost of some usability affordances (RQ4). For handheld devices, the grid and the pie tended to score higher in terms of confidence in decision making, compared to the list and stack layouts (RQ1, RQ4).
• #64: Q11: not mych effort Q2, Q5 During the interviews, some participants also appreciated the explanations of the system, indicating in particular the detailed view of the call, where they can see the most frequent reasons for a call: \say{\textit{\textbf{P8}: I like that you see the most frequent reasons. Why they call most often. That is the most important to me. That you can see it again afterward. Maybe, when a person calls a lot, that you can reflect. they go a lot to the toilet, maybe they have a urinary infection. When many nurses visit a resident, maybe you don’t see this if you don’t look at the overview.}}Overview of resuldents