![Genres, conventions and usability
These expectations can guide user search for an element (Dillon, 2004) and when
violated, are not only immediately noticed by the user (Vaughan & Dillon, 2006) but
also seem to hinder user performance (Shneiderman & Plaisant, 2005).
Conventions
Thus, the concern with conventions expressed in usability literature is justifiable
since conforming to conventions can mean not only achieving consistency, but also
fulfilling users’ expectations. However the concept of conventions, specially genre
conventions, in usability, needs to be broken down further if one is to fully
understand the role these concepts play in usability.
As seen earlier in this chapter, genre conventions can be structural, linguistic, visual
or functional. These four types of conventions are different, and they may affect
usability differently. Some usability authors have given different recommendations
on how to create internally and externally consistent interfaces. Recommendations on
guaranteeing internal consistency do not directly mention conventions as they are
viewed more as rules which need to be followed. In that case, it is still possible to see
which type of conventions they will become by examining the recommendations.
Although structural, linguistic, visual and functional conventions seem to be
considered in usability literature, it seems that authors most often associate following
visual and linguistic conventions with usable design, than following structural
conventions.
Shneiderman and Plaisant (2005, p.74) recommend that ‘identical terminology should
be used in prompts, menus, and help screens; and consistent colour, layout,
capitalization, fonts and so on should be employed throughout [an interface]’. This
recommendation addresses the visual and linguistic aspects of internal consistency
but does not consider structural aspects of consistency.
Nielsen (1993, p. 132), recommends that ‘the same information should be presented
in the same location on all screens, and dialog boxes should be formatted in the same
way to facilitate recognition’ which also addresses the visual aspects of internal
consistency. A study which addressed all three aspects of internal consistency and
established twenty guidelines for designing consistent web interfaces was conducted
by Ozok and Salvendy (2004). When defining the concept of consistency, the authors
29](https://image.slidesharecdn.com/santa-maria-phdthesis2009visualgenreconventionsanduserperformanceontheweb-100716145158-phpapp01/85/Santa-maria-ph-d-thesis-2009_visual-genre-conventions-and-user-performance-on-the-web-51-320.jpg)
![Visual conventions and performance over time
However, the group using the non-conforming page started the test disagreeing
(Median=2, Mode=2) that ‘Learning to use the site was easy’ and ended the test
neither agreeing nor disagreeing (Median=3, Mode=4) with the statement. This result
shows an evident improvement in their perception of the ease of use of the website
which means that by the end of the test they did not think the website was as hard to
learn to use than when they started.
Furthermore, the difference between the median and the mode indicates that
although the middle value for the distribution was 3 (Median) the most frequent
choice was 4 (Mode). In fact, examining the results, by the end of the test 10
participants in the non-conforming group agreed with the statement while 5
participants neither agreed nor disagreed and 5 participants disagreed. It is plausible
to think that if participants in the non-conforming group spent a little more time
using the website they would agree that it was easy to learn to use it.
Thinking aloud
As explained earlier, in the 6th task participants were requested to think aloud as they
were searching for the answer to the question.
Analysis of the recording of the participants revealed that some participants in the
non-conforming forum indicated verbally having learned what the navigational
menu was or the meaning of the icons in the navigational menu. They made evident
in their comments that they have at some point during the experiment decoded the
symbols on the menu and learned their meaning. Some of their comments are listed
below:
! “Go to the symbol of sports on the menu, I know that is the menu now, click
there, there are four symbols the 1st is the football category.”
! “That carpet [icon], what is that? I can’t figure this one out… Oh! There is an
atom…so it must be science…still don’t get the carpet!
! “Football…I'm gonna look at the trophy thing because now I know that's
sports…”
! “Since I have been here before, I can tell that this [icon] is sports. I didn't
know that before…Anyway I can see it is sports because it has a logo I can
identify.”
Only one of the participants in the conforming group has indicated having to learn
the navigational menu by commenting “Well, now I know where the sports section
270](https://image.slidesharecdn.com/santa-maria-phdthesis2009visualgenreconventionsanduserperformanceontheweb-100716145158-phpapp01/85/Santa-maria-ph-d-thesis-2009_visual-genre-conventions-and-user-performance-on-the-web-292-320.jpg)
Santa maria-ph d-thesis 2009_visual genre conventions and user performance on the web
- 1. ! ! ! ! ! ! ! ! ! ! ! ! ! ! Visual genre conventions and user performance on the web Thesis Submitted for the Degree of Doctor of Philosophy Department of Typography & Graphic Communication Luis Santa-Maria December 2008
- 2. Declaration I confirm that this is my own work and the use of all material other sources has been properly and fully acknowledged. Luis Santa Maria December 2008
- 3. Abstract One problem which designers are faced with is whether they should follow existing conventions of a genre, or break with conventions, and innovate. Understanding the relationship between conventions and user performance is essential in order for designers to make informed decisions. Despite the fact that usability literature claims that not conforming to genre conventions can cause serious usability problems, there is not much empirical investigation on the topic. Through a review of the literature, a framework was defined which established genre conventions as having a visual, structural, linguistic and a functional component. This thesis examines the impact that violation of the visual conventions of a digital genre has on user performance. Web discussion forums were the category of websites chosen to be investigated because of the likelihood that they are a truly digital genre which is emerging. A survey was conducted to help establish the existing visual conventions in the genre and to help define the experimental material. A series of empirical studies were conducted to validate the experimental material and to measure the performance of users familiar and unfamiliar with the genre when using conforming and non-conforming web pages. Results show that not conforming to visual conventions of a web forum genre results in poorer performance and disorientation of participants familiar with the genre. But results also indicate that this poorer performance and disorientation is short-lived and participants who use a non-conforming forum over a brief period of time are able to improve their performance and feel less disoriented. Furthermore, participants who use the non-conforming forum page are able to maintain their performance improvement when using another non-conforming web forum. Results also show that performance varies depending on the conventionality of the website and whether the user is retrieving information from the interface or the content.
- 4. Abstract Finally, an empirical study was conducted on conforming and non-conforming online news genre and the findings obtained in web forums were reproduced in this other web genre. These results suggest that although not conforming to visual conventions can cause a decline in performance, the magnitude of the negative effect on user performance has not been thoroughly investigated in usability literature. The fact that users can quickly recover from problems caused by the violation of the visual conventions of a genre brings a new perspective to the negotiation between innovation and conventions. Furthermore, the results from the interaction between conventionality and the type of task users perform can help inform design decisions based on a task oriented approach. It also empowers designers to better consider when to conform and when to violate conventions, and thus better estimate what impact their decisions will have on the final users.
- 5. To Flavia for her love, friendship and care.
- 6. Acknowledgements The journey of a PhD is quite a long one and albeit solitary in many ways. In other ways you are surrounded by people who touch your life in so many different manners. Here I try to do justice to all these people who have in their own way helped me through this remarkable journey: ! First and foremost I would like to thank my wife Flavia for all her love, care, understanding, faith and motivation. Without her I would have never got here. ! My parents, who have always given me their support and love across the ocean and were always prompt to help with the technical issues in Brazil. ! To the Coordenação de Aperfeiçoamento de Pessoal do Ensino Superior (CAPES) and all the people who work there for the sponsorship and the friendly advice and help whenever I needed it. ! My supervisor Dr. Mary Dyson for walking this journey by my side. Her professionalism, prompt feedback and critical sense were invaluable to me and have taught me how to be an empirical researcher. ! Eric Kindel, Linda Reynolds and later Dr. Rob Waller for all their contributions in the Research Advisory Group sessions. ! Dr. Anamaria de Moraes, who introduced me to empirical research and whose advice made me consider doing a PhD abroad. ! Dr. Guilherme Cunha Lima and Edna Cunha Lima who encouraged me to come to Reading to do my research. ! My good friends Dr. Rathna Ramanathan, David Lewis, Jo De Baerdemaeker, Glenn, Dr. Heliana Pacheco and Dr. David Buss for their support, camaraderie and great laughs. ! My friends Carlos Alberto Munoz & Vanessa Toniolo, José Eduardo & Débora Guimarães, Matthew, Emma and Tristan Browne and Fernando and Rília Varandas for their friendship and all the great times we had together which kept me in touch with the ‘world outside my PhD’. ! My PhD colleagues who have gone on their own journeys and made me feel less lonely: José Marconi, Luciane Fadel and Silvio Campello. ! Helen Apted for her kindness, cheerfulness and for brilliantly managing Old Whiteknights House and always striving to make the life of all PhD students who work there easier.
- 7. ! Professor Stephen Taylor and Professor Pam DeNicolo for not only coming up with the idea of the Graduate Schools, but also for making Old Whiteknights House come true. ! My colleagues at Old Whiteknights House: Helen Ainslie, Colm MacCrossan, Maura O’Reagan, Seonghye Moon and Nathalie Montillot for the encouragement and silly debates which allowed me to remain sane. ! Brian Hoskins who has made adapting to life in the United Kingdom a lot easier and has always been available to help. ! Finally, to all the people who selflessly volunteered to do my experiments, specially the members of Webforumz.com who were very enthusiastic about the whole idea.
- 8. Table of contents List of figures vi! List of tables xiii! Chapter 1 Introduction 1! 1.1. Introduction 1! 1.2. Context 2! 1.3. Aims and scope of the thesis 3! 1.4. Methodology and structure of the thesis 4! Chapter 2 Genres, conventions and usability 8! 2.1. Introduction 8! 2.2. Genres 8! 2.2.1. Definition 8! 2.2.2. Categorization into genres 9! 2.2.3. A genre framework from a design point of view 11! 2.2.4. Genres and technology 19! 2.2.5. The social aspect of genres – discourse communities 20! 2.2.6. The importance of visual conventions of a genre 21! 2.2.7. Section summary 22! 2.3. The relationship between conventionality and usability 22! 2.3.1. Defining Usability 23! 2.3.2. Usability and genre conventions 26! 2.3.3. Visual appearance and Usability 34! 2.4. Conclusion 38! Chapter 3 Exploratory studies 40! 3.1. Introduction 40! 3.2. Exploratory study 1 - The effect of structured design on comprehension and structured recall of electronic documents 41! 3.2.1. Background 41! 3.2.2. Access structures & typographic cues 42! 3.2.3. Types of knowledge 49! 3.2.4. Objectives 51! 3.2.5. Method 51! 3.2.6. Study design 54! 3.2.7. Participants 54! 3.2.8. Experimental Procedure 55! 3.2.9. Results 56! i
- 9. 3.2.10. Discussion 60! 3.2.11. Conclusion 63! 3.3. Exploratory study 2 - Visual appearance and the identification of the category of a website. 63! 3.3.1. Introduction 63! 3.3.2. Objective 66! 3.3.3. Method 66! 3.3.4. Participants 68! 3.3.5. Procedure 68! 3.3.6. Results 69! 3.3.7. Discussion 75! 3.3.8. Conclusion 79! 3.4. Conclusion of exploratory studies 80! Chapter 4 Finding visual conventions in web forums 81! 4.1. Introduction 81! 4.1.1. Web forums 81! 4.2. Survey 1 83! 4.2.1. Introduction 83! 4.2.2. Method 89! 4.2.3. Sample 91! 4.2.4. Procedure 91! 4.2.5. Results 92! 4.2.6. Discussion 105! 4.2.7. Most frequent groups of elements 106! 4.2.8. Conclusion 108! Chapter 5 Assessing the results of the survey in terms of user perceptions 109! 5.1. Experiments to test templates created based on the results of the survey 110! 5.1.1. Experiment 1 – Perceptions of conventionality of the templates based on the results of the survey 110! 5.1.2. Experiment 2 - User perceptions of ease of use of the templates based on the survey results 130! 5.1.3. Conclusion 135! 5.2. Experiment 3 - Assessing the experimental material 137! 5.2.1. Introduction 137! 5.2.2. Objectives 139! 5.2.3. Method 139! 5.2.4. Procedure 145! 5.2.5. Results 146! 5.2.6. Discussion 148! ii
- 10. 5.2.7. Conclusion 149! 5.3. Final remarks 150! Chapter 6 Visual conventions and familiarity when searching in web forums 152! 6.1. Introduction 152! 6.1.1. Familiarity 152! 6.2. Experiment 4 – Visual conventions and searching for information in web forums 161! 6.2.1. Objectives 161! 6.2.2. Method 162! 6.2.3. Procedure 167! 6.2.4. Results 168! 6.2.5. Discussion 176! 6.2.6. Conclusion 179! Chapter 7 Visual conventions and performance over time 181! 7.1. Introduction 181! 7.1.1. Navigation and browsing strategies 181! 7.1.2. Expectations Conventions and usability 186! 7.1.3. Disorientation 192! 7.2. The pilot study 193! 7.2.1. Objectives 193! 7.2.2. Method 194! 7.2.3. Procedure 197! 7.2.4. Results 198! 7.2.5. Discussion 208! 7.2.6. Conclusion 211! 7.3. The main experiment 212! 7.3.1. Objectives 212! 7.3.2. Method 212! 7.3.3. Procedure 215! 7.3.4. Results for part 1 of experiment 5 217! 7.3.5. Results for part 2 of experiment 5 227! 7.3.6. Summary of results 233! 7.3.7. Discussion 235! 7.3.8. Conclusion 238! Chapter 8 The effects of conventionality on users searching for information in an online news pages 240! 8.1. Introduction 240! 8.2. Analysis of existing news websites 241! 8.3. Survey: Judgments of online news regarding the conventionality of the layouts 243! iii
- 11. 8.3.1. Objectives 243! 8.3.2. Method 243! 8.3.3. Results 246! 8.3.4. Discussion 248! 8.3.5. Conclusion 255! 8.4. Experiment 6 – Effects of conventionality on searching for information on online news websites 255! 8.4.1. Objectives 255! 8.4.2. Method 256! 8.4.3. Procedure 262! 8.4.4. Results 263! 8.4.5. Discussion 271! 8.4.6. Conclusion 274! Chapter 9 Conclusion 277! 9.1. Introduction 277! 9.2. Overview of the thesis 278! 9.3. Contributions 282! 9.3.1. Design 282! 9.3.2. Usability 285! 9.3.3. Final remarks on contributions 288! 9.4. Limitations and suggestions for future research 289! Appendix A Texts used in the first Exploratory study 1 which examined conventional access structures 292! Appendix B Summary task and multiple-choice questions and internet usage questionnaire used in the first exploratory study 295! Appendix C Elements which helped identifying the website in the second exploratory study 297! Appendix D Criteria used for determining the screen position of a visual element for the survey 300! Appendix E Explanation of elements found in all three levels of a web forum 302! Appendix F Tables with the symbolization mode frequencies of each element appearing on each of the three forum levels 305! Appendix G Tables with the configuration mode frequencies of each element appearing on each of the three forum levels 307! Appendix H Tables showing the survey 1 results for the most frequently occurring groups of elements 309! iv
- 12. Appendix I Templates used in experiments 1 and 2 to assess perceptions of conventionality and ease of use 312! Appendix J Results from experiment 3, the icon identification test 336! Appendix K Results for Experiment 3 which asked participants to identify the interactive elements of the non-conforming forum 340! Appendix L Text introducing PDAs presented for participants of experiment 4 342! Appendix M Tasks designed for the experiment 4 which tested the performance of users familiar and unfamiliar with web forums on conforming and non-conforming websites. 343! Appendix N Tasks designed for the pilot study which was conducted in preparation for experiment 5 344! Appendix O Tasks used on both parts of experiment 5 345! Appendix P Disorientation questionnaire applied on both parts of experiment 5 346! Appendix Q Questionnaire used in Experiment 6 347! References 348! v
- 13. List of figures Figure 1 – Model of the document design process as described by Waller (1987). 14! Figure 2 - Model for the design process involving genres and changes and functional constraints (based on the model developed by Waller (1987). 14! Figure 3 - Design process which relies solely on the document genre and its conventions. 15! Figure 4 - Design process which relies solely on the functional constraints of a document. 15! Figure 5 - Design process which relies on the genre-functional constraints negotiation to achieve the design solution. 16! Figure 6 - Model of contextual factors which might influence the usability of a product. 25! Figure 7 - Representation of the low space condition (Bernard et al., 2000). 44! Figure 8 - Representation of the medium space condition (Bernard et al., 2000). 44! Figure 9 - Representation of the high space condition (Bernard et al., 2000). 45! Figure 10 - Enhanced page layout (left) and poor page layout (right). 48! Figure 11- Sample of the unstructured text layout used int the experiment. 52! Figure 12 – Sample of the structured text layout used in the experiment. 52! Figure 13 - Mean reading time with standard error bars for the two layouts. 56! Figure 14 - Mean multiple-choice scores for both layouts. 58! Figure 15 – Mean multiple-choice structure question scores. 58! Figure 16- Chart of relative frequencies of elements for webmail sites. 71! Figure 17 – Chart of relative frequencies of elements for online news sites. 72! Figure 18 - Chart of relative frequencies of elements for search engine sites. 73! Figure 19– Chart of relative frequency of elements for e-commerce sites. 73! Figure 20 – Chart of relative frequencies x elements for online banking sites. 74! Figure 21 – Chart of relative frequencies of elements for online auction sites. 74! Figure 22 - Relative frequency of words being used as cues by participants in every category. 76! Figure 23 – Sample of an online auction website screen used in the experiment. 77! Figure 24 – Sample of an e-commerce website with advert used in the experiment. 78! Figure 25 - Sample of a search item extracted from a Swedish online news website. 87! vi
- 14. Figure 26 - Sample of the user's view of a web page that extends outside the screen dimensions. 89! Figure 27 - Grid used to determine the positioning of the elements on a web forum page. 90! Figure 28 – Some elements on the first level of a web forum. 93! Figure 29 – Some elements present in the forum index table of a web forum. 94! Figure 30 - Sample of the topic moderator element in the forum index table. 94! Figure 31 - Sample of the number of views in topic element on a forum index page. 95! Figure 32 - Elements in the second level of a web forum. 96! Figure 33 - Thread index table and its elements of a web forum. 96! Figure 34 – Elements in a post page of a web forum. 98! Figure 35 – Elements within the post area of a web forum. 99! Figure 36 – most occurring Positions of the elements in the forum index page. 102! Figure 37 – Least/non occurring positions of the elements in the forum index page. 102! Figure 38 – Most occurring positions of the elements in the threads page. Elements in bold italicized type appeared most frequently in two simultaneous positions. 103! Figure 39 – Least/non occurring positions of the elements in the threads page. 103! Figure 40 – Most occurring positions of the elements in the post page. Elements in bold italicized type appeared most frequently in two simultaneous positions. 104! Figure 41 – Least/non occurring positions of the elements in the post page. 104! Figure 42 - Example of the main conforming template (left) and its three variations (right) on symbolization mode, position and configuration mode (from top to bottom respectively) 112! Figure 43 – Example of the main non-conforming template (left) and its three variations (right) on symbolization mode, position and configuration mode (from top to bottom respectively) 113! Figure 44 - Comparison between the main conforming menu element (with all attributes on their most frequent values) and the menu variation where the configuration is at its least frequent value. 114! Figure 45 - Comparison between the main conforming menu element (all attributes on their most frequent values) and the menu variation where symbolization is at its least frequent value 114! Figure 46 - Comparison between the main non-conforming menu element (with all attributes on their least frequent values) and the menu variation where the configuration is at its most frequent value 115! Figure 47 - Comparison between the main non-conforming menu element (with all attributes on their least frequent values) and the menu variation where the symbolization is at its most frequent value 115! Figure 48 – Coding system used to name the forum templates. 116! vii
- 15. Figure 49 - Thumbnails of the 24 conforming and non-conforming 117! Figure 50 – Screenshot referring to the C11 template of the conforming group. This template has the symbolization, configuration and position attributes at their most frequently occurring values. 121! Figure 51 – Screenshot referring to the N11 template of the non-conforming group. This template has the symbolization, configuration and position attributes at their least frequently or non occurring values. 121! Figure 52- Sample of the non-conforming template from the threads page level which was chosen more frequently as similar to existing web forums. This template had the elements in their most common symbolization mode but their position and configuration in their least or non occurring values. 124! Figure 53 – Sample of the non-conforming template from the threads page level which was chosen more frequently as similar to existing web forums. This template had the elements in their most common symbolization mode but their position and configuration in their least or non occurring values. 124! Figure 54 - Elements present on the forum index page which also appear in the threads page. 127! Figure 55 - Elements present on the threads page which also appear in the forum index page. 128! Figure 56 - Elements in the post page which also appear in the forum index and threads page. 128! Figure 57 - Sample of section 1 of the survey. 141! Figure 58 - Sample of the non-conforming screen in section 2 of the survey. 143! Figure 59 - Division of areas in the non-conforming forum as displayed in the third part of the survey. 144! Figure 60 - Sample of a hypothetical learning curve to explain the small difference in performance betweeen experts and intermediate users found in Tabatabai and Shore (2005). 156! Figure 61 - Sample of the index page design (level 1) of the conforming web forum. 166! Figure 62 - Sample of the index page design (level 1) of the non-conforming web forum. 166! Figure 63 - Formula for the data transformation and calculation of the rate of right answers per second. 172! Figure 64 – Overall performance (measured by the RPS rate) of familiar participants compared to the unfamiliar participants. 172! Figure 65 –Interaction between conventionality and task type. 174! Figure 66 - Performance (measured by the RPS rate) on interface tasks compared to content tasks. 174! Figure 67 - Median value of the answer given to the question: ‘How would you compare the layout of this web forum with other forums?’ (1=very different, 5=very similar). 175! Figure 68 – Hypothetical illustration of a learning effect to explain the lack of difference in performance between the familiar users tested with the conforming and the familiar users tested with the non-conforming forum. 178! Figure 69 - Sample of the convention-conforming website 188! viii
- 16. Figure 70 - Sample of the non-conforming website 188! Figure 71 - Sample of the Convention-conforming news homepage 189! Figure 72 - Sample of the non-conforming online news homepage 190! Figure 73 - Sample of the experimental setting used for the pilot study and experiment 5. 196! Figure 74 - Task completion time for the non-conforming group. 198! Figure 75 - Task completion time for the conforming group. 199! Figure 76 - Mean time to complete each task in the non-conforming group. 200! Figure 77 - Mean time to complete each task in the conforming group. 200! Figure 78 - Overlapped graphs of the mean time to complete each task on the conforming and non-conforming groups. 201! Figure 79 – Overall mean time to complete each task. 201! Figure 80 - Mean time to complete interface and content tasks by condition. 203! Figure 81 - Mean number of right answers per task type from the conforming and non-conforming groups. 204! Figure 82 - Right answers per second (RPS) rate pert task type on the conforming and non- conforming groups. 205! Figure 83 - Number of total right answers over time and fit curve for the non-conforming group. 206! Figure 84 - Number of total right answers over time and fit curve for the conforming group. 207! Figure 85 - Mean revisits for the conforming and non-conforming groups. 208! Figure 86- Mean time to complete each task of the non-conforming group 217! Figure 87 - Mean time to complete each task of the conforming group 218! Figure 88 - Number of right answers in the non-conforming group through the test. 219! Figure 89 - Number of right answers in the conforming group through the test. 219! Figure 90 - Right answers per second ratio formula. 220! Figure 91 - Progression of right answers per second in the non-conforming group. 221! Figure 92 - Progression of the RPS ratio for both groups in the first part of the test. 222! Figure 93 – Average right answers per second rate (RPS) on content 224! Figure 94 - Mean number of quits for each group on each type of task. 224! Figure 95 - Mean number of right answers per group for each task type. 225! Figure 96 - Average quits on content and interface tasks per group. 226! ix
- 17. Figure 97 - Average time to complete each task in the conforming group. 227! Figure 98 - Average time to complete each task in the non-conforming group. 228! Figure 99 - Overlapped graphs of time to complete tasks on both groups. 228! Figure 100 - Progression of right answers and correspondent fit curve for the conforming group. 229! Figure 101 - Progression of right answers and correspondent fit curve for the non-conforming group. 230! Figure 102 - Average right answers per second rate (RPS) on content and interface tasks per group in the second part. 232! Figure 103 - Wireframes for the conforming online news pages designed for the experiment 242! Figure 104 - Wireframes for the non-conforming online news pages designed for the experiment 242! FIGURE 105 - PAGES OF THE CONFORMING ONLINE NEWS LAYOUT 244! FIGURE 106 PAGES OF THE NON-CONFORMING ONLINE NEWS LAYOUT 244! Figure 107 - Sample of a sheet used in the survey containing the two layouts 246! Figure 108 - Number of times each visual attribute/element was mentioned as influencing 247! Figure 109 - Number of times each visual attribute or element was mentioned as 248! Figure 110- Common configuration of the components of a piece of news 249! Figure 111 - Configuration of the components of a piece of news with the summary 250! Figure 112 - Type of configuration of the elements of an article in the story page of online news pages 250! Figure 113 - Configurations of news pieces used in the non-conforming online news pages 251! Figure 114 - Configuration used in the story pages of the non-conforming online news website where elements of the article were dissociated by partially separating them. 251! Figure 115 - Style and placement of the navigational menu considered as 252! Figure 116 - Style and placement of the navigational menu in 253! Figure 117 - Positioning and shapes of adverts commonly occurring on online news pages 254! Figure 118 - Icons designed for each category and subcategory of the conforming online news site 257! Figure 119 - Accordion menu designed for the non-conforming online news pages (a) with the news category selected and (b) with the money category selected 258! Figure 120 - Sample pages of the conforming news website 258! Figure 121 - Sample pages of the non-conforming online news website 259! x
- 18. Figure 122 - Sample of the experimental setting with the non-conforming website 260! Figure 123 - Mean time spent on tasks per group 264! Figure 124 - Mean time spent on tasks per group throughout the experiment 264! Figure 125 - Mean right answers per group 265! Figure 126 - Mean number of right answers per group throughout the experiment 266! Figure 127 - Mean RPS per group throughout the experiment 267! Figure 128 - Strategy 1 - Strategy used by the majority of the participants in the 269! Figure 129 - Variation on strategy 1 that was the second most used strategy 269! Figure 130 - Mean backtracking actions executed over time by participants 272! Figure 131 - Template C11 of the forum index page, most frequently chosen as more like other existing forums. 312! Figure 132 - Template C12 of the forum index page 313! Figure 133 - Template C13 of the forum index page 314! Figure 134 - Template C14 of the forum index page 315! Figure 135 - Template N11 of the forum index page 316! Figure 136 - Template N12 of the forum index page 317! Figure 137 - Template N13 of the forum index page 318! Figure 138 - Template N14 of the forum index page, chosen the least number of times as more like other web forums. 319! Figure 139 - Template C21 of the threads page, most frequently chosen as more like other web forums. 320! Figure 140 - Template C22 of the threads page 321! Figure 141 - Template C23 of the threads page 322! Figure 142 - Template C24 of the threads page 323! Figure 143 - Template N21 of the threads page 324! Figure 144 - Template N22 of the threads page 325! Figure 145 - Template N23 of the threads page, chosen the least number of times as more like other web forums. 326! Figure 146 - Template N24 of the threads page 327! Figure 147 - Template C31 of the post page, most frequently chosen as more like other web forums. 328! Figure 148 - Template C32 of the post page 329! xi
- 19. Figure 149 - Template C34 of the post page 331! Figure 150 - Template N31 of the post page 332! Figure 151 - Template N32 of the post page 333! Figure 152 - Template N33 of the post page 334! Figure 153 - Template N34 of the post page, chosen the least number of times as more like other web forums. 335! xii
- 20. List of tables Table 1 - Conditions under which each group of participants was tested by Satzinger and Olfman (1998). 30! Table 2 - Participants' gender across conditions. 54! Table 3 - Table with the correct answers for the multiple-choice tests by layout 57! Table 4 - Comparison of overall comprehension results across conditions. 60! Table 5 – Number of different screens captured in each category. 67! Table 6 –Identification results on each category screen by participant per pass. 69! Table 7 – Identification Ratio. 70! Table 8 - A comparison between categories and their respective elements. A tick mark signifies that the element indicated in the row has helped participants identify the respective website category. 75! Table 9 - No identification and misidentification per category. 76! Table 10 – Number of participants that cited each unique element in each category. 78! Table 11 - List of all elements found in the forum index page of web forums. The indented elements are found within the forum index table. 92! Table 12 - List of the elements found in the threads page of a web forum. The indented elements are found within the threads table. 95! Table 13 – List of the elements found in the post page of a web forum. The indented elements are found within the threads table. 97! Table 14 – Summary of the elements present in each of the three forum levels. 100! Table 15- Most frequent groups of elements in each forum level (shown in columns). Names in bold represent the elements which appear on all three forum levels and in italic the elements which appear on two forum levels. 107! Table 16 – Symbolization, position and configuration values for each conforming and non- conforming screen. 118! Table 17 – Summary of the chi-square tests on the number of times each group was picked as more like other web forums. 123! Table 18 – Least chosen and second least chosen conforming templates in each forum level. The table also indicates the values of each attribute of the elements and the number of times (frequency) each template was chosen. 125! Table 19 – Number of times each template in the conforming group was chosen as more like other web forum pages (results are out of a total of 48 comparisons per template per level. 126! Table 20 – Templates most frequently and least frequently chosen as similar to other web forums (results are out of a total of 48 comparisons per template per level). 130! xiii
- 21. Table 21 - Summary of the chi-square tests on the number of times each group was picked as easier to use. 133! Table 22 - Frequencies that each template in the conforming 134! Table 23 – Frequencies of the templates most frequently and least frequently chosen as similar to other web forums (results are out of a total of 80 comparisons per template per level). 134! Table 24 - Templates most frequently chosen in the conforming group. 136! Table 25 – Templates with the most frequent (MF) and least frequent (LF) values for each element attribute and which were used to design the conforming and non-conforming forums. 137! Table 26 - Icons presented in the survey and the text content they were replacing. 141! Table 27 – Correct answers for the areas highlighted in the third part of the survey. 144! Table 28 - Most frequent icon associations and frequency (answers conforming to the meaning of the icon are italicized). 146! Table 29 - Grouped results for the identification of the areas of the non-conforming template. 147! Table 30 - Distribution of groups within the familiar and unfamiliar categories and the forum layout each group was tested on. 162! Table 31 - Templates used to create each level of the conforming and non-conforming forums. 163! Table 32 - How values were applied to the variables symbolization, position and configuration for designing the forum templates. Values could be the most frequent (MF) or the least frequent (LF). 164! Table 33 - Questionnaire items with their respective extreme values meanings. 167! Table 34 - Mean values and Standard Deviation of the time to complete tasks in each group and condition with the significant differences highlighted and circled by a dotted line. The column means shows the means of the users who used the conforming and non-conforming forums respectively and the row means represent the means of the familiar and unfamilar users. 168! Table 35 - Median values of the number of correct scores in each group and condition with the significant differences highlighted and circled with a dotted line. The column medians shows the medians of the users who used the conforming and non-conforming forums respectively and the row medians represent the medians of the familiar and unfamilar users. 170! Table 36 – Mean values of the RPS rates in each group and condition with the significant differences highlighted and circled with a dotted line. The column medians shows the medians of the users who used the conforming and non-conforming forums respectively and the row medians represent the medians of the familiar and unfamilar users. 173! Table 37 - Results for the two within-subjects one between-subjects factor ANOVA. 173! Table 38 - Summary of the experimental tasks which were answered too quickly, too slowly or showed great variability. 202! Table 39 - One between-subjects, one within-subjects ANOVA results for time taken to complete the tasks. 203! Table 40 - One between-subjects, one within-subjects ANOVA on the number of right answers between the conforming and non-conforming groups. 204! xiv
- 22. Table 41 - One between-subjects, one within-subjects ANOVA reuslts for the right answers per second (RPS) rate in the conforming and non-conforming groups. 205! Table 42 - Questions in their order of appearance in the questionnaire. 215! Table 43 - Comparison between the linear and quadratic models measures of accuracy for the non-conforming group. 220! Table 44 - Average right answers per second for each type of tasks on each counterbalancing order. 223! Table 45 - Mean RPS rate per task type per task type in the two counterbalanced situations. 231! Table 46 - Summary of the results of each group in each part of the test. 233! Table 47- Online news websites analysed to prepare the experimental material 241! Table 48 - List of commonly occurring online news categories 256! Table 49 - Tasks designed for the experiment 261! Table 50 - Median values of the answers given in two instances 268! Table 51 - Frequency of symbolization mode of the elements on the forum index page. 305! Table 52 - Frequency of symbolization mode of the elements on the thread page. The indented elements are found within the thread table. 306! Table 53 - Frequency of symbolization mode of the elements on the post page. The indented elements are found withing the post area. 306! Table 54- Frequencies of configuration of elements in the forum index page. The indented elements are found within the index table. 307! Table 55- Frequencies of configuration of elements in the forum thread page. The indented elements are found within the thread table. 308! Table 56- Frequencies of configuration of elements in the forum post page. The indented elements are found withing the post area. 308! xv
- 23. Chapter 1 Introduction 1.1. Introduction The Internet has seen its world usage grow more than 300% in the past eight years, with approximately 1.46 billion people using it (Miniwatts Marketing Group, - Internet Usage Statistics). Currently the total number of registered and active website domains is reported to be over 100 million, and 74% of these are either commercial or other sites operating with the .com suffix (DomainTools.com, 2008). In 2007, about 70% businesses in the United Kingdom had their own websites which were used for selling products, giving access to catalogues or price lists, and providing access to job vacancies and application forms (Pollard, 2008). This increase in web usage and in businesses ‘going online’ is reflected in a higher demand for well-qualified web designers (Burleson, 2004). Besides the growing number of websites and the increase in businesses expanding on the Internet, there is one other very frequent activity in which designers are involved, that hardly gets mentioned in the statistics. Businesses are constantly updating their websites, and some studies which monitored business websites for changes have reported that most websites change over a one year period (Cosio & Dyson, 2002; Leichty & Esrock, 2001) and sometimes changes are made after only one or two months (Dontcheva, Drucker, Salesin, & Cohen, 2007; Tan, Foo, & Hui, 2001). The majority of changes to business websites are made to the content of the page (including text and update date), followed by changes to hyperlinks (change in the URL of the hyperlink) and finally by changes in layout and images (Tan, et al., 2001). Changes to the organizational structure of the websites seem to be less frequent, and when they happen, they tend to be quite minor, e.g. inserting of removing a small number of nodes (pages) to the website tree (Dontcheva, et al., 2007). Although hyperlink changes may indicate modifications in the organizational structure of the 1
- 24. Introduction website, this is not necessarily always true. For example, an e-commerce website may stop selling one brand of nappies and start selling a new brand which can lead to a change in the URL of the baby nappies hyperlink, but not to the organizational structure of the website (i.e. baby nappies will still be located in the baby products section). Another situation which has risen with the popularization and advances in mobile technology is the need for migrating websites to small screen devices (Albers & Kim, 2000; MacKay, Watters, & Duffy, 2004). This usually presents the designer with a lot of difficulties, since it is extremely hard to maintain the visual appearance of a website intact when moving to a reduced screen size. As a result, the original layout of the page is changed, and that is said to affect the user experience (Lehtonen, et al., 2006). Thus, whether creating new sites or modifying existing ones, it is important that designers be prepared to provide good and usable design solutions. To achieve this, the designer has to juggle between different aspects of the design which sometimes contradict each other (Borchers, 2001). Examples of the aspects involved in this process, usually called ‘forces’, are usability, innovation and genre conventions. But to be able to adequately judge the impact of these forces on the final product, the designer needs to have access to information on each of them. Most of the times this type of information is provided by research, but unfortunately, not all these aspects have been thoroughly investigated and therefore not many sources are available for designers to consult. 1.2. Context Genres have been studied and discussed in various fields of knowledge such as rhetoric, literature and folklore for many years. But it is only in the past twenty years that the importance that genres play in the design of documents has begun to be investigated (Bateman & Delin, 2001; Delin, Bateman, & Allen, 2002; Waller, 1987a). Since then, some authors have highlighted the importance of the structural (Dillon, 1991; Vaughan & Dillon, 2006), linguistic (Dillon, 2000) and visual (Toms, 2001) aspects of a document genre to its users. Genres are bound by rules and conventions which create and maintain the expectations of users (Vaughan & Dillon, 2006). Following these conventions is said 2
- 25. Introduction to provide simplicity, consistency and predictability, and facilitate the use of the document (Cappel & Huang, 2006). Some authors also warn that not conforming to conventions makes users feel lost, confused and disoriented (Cappel & Huang, 2006) and consequently usability will suffer (Nielsen, 1999a). The recommendations seem to somehow link usability and genres, although the connection is not entirely clear. The idea that not conforming to the conventions of a genre will cause usability problems is in part based on the concept that users develop mental models of documents which they interact with (Vaughan & Dillon, 1998). As users become more familiarized with documents which belong to a genre, they start to develop a mental model of the genre which is called ‘schematic knowledge’ (Vaughan & Dillon, 1998) or ‘superstructure’ (van Dijk & Kintsch, 1983). This knowledge leads the user to have certain expectations about a document which belongs to the genre they are familiar with. If, when interacting with the document the user finds certain types of cues, the schema is triggered (van Dijk & Kintsch, 1983) which can help users understand and remember the documents (Vaughan & Dillon, 1998). Some evidence exists to corroborate this theory, Dillon and Schaap (1996) and Vaughan and Dillon (2006) have shown that not conforming to genre conventions can hinder user performance. But other studies (Kalbach & Bosenick, 2006; McCarthy, Sasse, & Riegelsberger, 2003) obtained results which challenge this perception. These conflicting results illustrate that more research is needed on genres and usability, especially of electronic documents. Considering the enormous growth of the web and the update frequency of websites it is clear that of all existing electronic documents, websites are probably where this understanding is more urgently needed. 1.3. Aims and scope of the thesis This thesis aims at improving the understanding of how some of the forces, usability and genre conventions more specifically, which are part of the document design process, are related. This study has three main objectives. Firstly, guided by literature on genre theory it attempts to identify the relationship between genres and the design process. Then, based on usability and genre theory, it identifies a relationship between these two forces which influences the design process. Secondly, it attempts 3
- 26. Introduction to establish a systematic method for gathering information on conventional web layouts. Thirdly, it investigates whether not conforming to the visual conventions of a website genre affects user performance and disorientation. It is important to point out that this thesis is inserted in the field of knowledge of design which makes it an interdisciplinary work. As such it draws knowledge and theories from the fields of linguistics, rhetoric, human-computer interaction and psychology as well as graphic and interface design and typography. 1.4. Methodology and structure of the thesis To achieve the aim and objectives described in the previous section, this study follows the empirical approach which involves gathering information on phenomena from observation or experimentation. Conclusions will be based on the analysis of the resulting data which will include the adoption of statistical techniques. The thesis is structured to reflect the progress of this research. As such, the next chapter establishes the theoretical framework which will be used. This framework is based on an overview of existing relevant literature. The subsequent chapters describe empirical approaches taken to investigate issues relating to this framework. In each of these chapters there is an introduction which contains a literature review more focused on the specific issues relevant to the chapter. Below is a more detailed description of what each chapter addresses: Chapter 2 presents a review of the literature on genres and usability. The chapter discusses how genres are linked with the design process and design patterns. It then proposes the concept of genre conventions which can help describe a genre. Through an analysis of usability theories, the chapter determines how genres relate to usability. Chapter 3 describes two exploratory studies conducted in the initial stages of the research to probe different aspects of genre conventions as defined in the previous chapter, and then determines the investigative path to be followed in the thesis. The first exploratory study looks at how the presence of conventional text structuring artefacts affects the comprehension and recall of a text. Results of the first exploratory study indicate that when the content of a digital document is organized using 4
- 27. Introduction conventional structuring artefacts, users seem to better recall the order in which the content is organized. The second exploratory study tests if people can identify the type of service provided by a website based only on its visual appearance. The result of the second exploratory study seems to indicate that users may be capable of identifying the service provided by a website after a quick look at the visual appearance of the page. This seems to be in accordance with the ideas that web genres are emerging (Dillon & Gushrowski, 2000; Eriksen & Ihlström, 2000) and users can identify a document genre by its visual appearance (Toms, 2001). Based on the results from both exploratory studies, the research for this thesis will focus on the visual aspects of digital genres. From the outcomes of the two exploratory studies in chapter 3 and the literature review in chapter 2, the web forum category was chosen to be used as the main experimental material in this thesis. Chapter 4 explains the reasons for this choice and presents the results of a survey which gathered the most frequently occurring visual elements in a web discussion forum, along with their most frequent symbolization mode (i.e. graphic, alphanumeric or mixed), position and modes of configuration (see Twyman, 1979). The objective of this survey was to obtain information on the conventional configuration of visual elements present on a website to guide the preparation of the empirical material. Chapter 5 contains three experiments which were conducted to test the results of the survey in terms of user perception. The first two experiments described in the chapter also tested for a correlation between perceived conventionality and ease of use. For the first two experiments 24 web forum templates were created based on the results of the survey. The layout of the templates ranged from one which conformed to the conventions of the genre (based on the survey results), to one which did not conform to the conventions (based on the survey results). The first experiment tested the perceptions of the users on the conventionality of the templates. The second experiment tested the perceptions of the users on the ease of use of the templates. Based on the results of these two first experiments the templates for the conforming and non-conforming1 web forum were chosen. The third experiment described in the 1 The term ‘non-conforming’ is used to describe the experimental material which does not conform to the visual conventions of a web genre. Similarly the term ‘conforming’ is used to describe the experimental material which conforms to the visual conventions of a web genre. 5
- 28. Introduction chapter tested if participants were able to identify and recognize important aspects of the non-conforming web forum. Results of this experiment showed which elements and parts of the forum users had problems identifying and which they did not. This knowledge was subsequently used to inform the development of the tasks used in the experiments described in chapters 6 and 7. Chapter 6 describes an experiment, which tested the time taken to complete tasks and the number of correct answers of users familiar and unfamiliar with web forums2 when using conforming and non-conforming websites. The experiment also investigated if the effect of conforming or not conforming to conventions on performance is dependent on the type of task executed. Two different types of experimental tasks were created: content-related tasks and interface-related tasks. In the content-related tasks participants had to retrieve information from the content whereas in the interface-related tasks participants had to retrieve the information from the interface. Results showed that unfamiliar participants 3 had an equivalent performance on both conforming and non-conforming websites but it also showed no difference in performance of the familiar users in both the conforming and non- conforming web forums. This lack of difference motivated the next experiment (described in chapter 7), which further investigated this matter. This study also had some methodological implications for the next experiment: it indicated the need to improve the measuring apparatus used in the test, and also the need for counterbalancing tasks. Chapter 7 describes an experiment conducted to further investigate why the previous study found no difference in performance between familiar users interacting with a conforming and a non-conforming web forum. This experiment investigated what happens with the performance of participants familiar with web forums over time. 2 The term web forums is used to indicate the plural of the word web forum. Although the plural of forum is usually represented by the Latin word ‘fora’ the plural ‘forums’ is also grammatically accepted and was chosen over the latter for being used in scientific articles (Abbasi & Chen, 2005; Hanna & Nooy, 2003; Kazanis & Ginige, 2002) and the World Wide Web. 3 For reasons of simplification, throughout this thesis participants familiar and unfamiliar with using web forums will be referred to as familiar participants and unfamiliar participants respectively. 6
- 29. Introduction Participants using the non-conforming website showed a performance improvement throughout the experiment, while participants using the conforming site showed virtually no performance improvement. Chapter 8 describes the final experiment, which investigated whether the results obtained from previous studies could be replicated for another web genre. For this experiment the online news genre was used as experimental material. Results from this experiment confirmed that participants using a non-conforming online news genre improved their performance felt less disoriented as the experiment progressed. Finally, Chapter 9 concludes this thesis by presenting an overview of the study and discusses the practical and theoretical contributions of the work. It also points out the limitations of the results and suggests areas for further research. 7
- 30. Chapter 2 Genres, conventions and usability 2.1. Introduction This chapter describes the framework used in this thesis. It starts by looking at genre theory and how that relates to the document design process. This relationship leads to a suggestion of how genres can be linked with design patterns and points out the importance of understanding genres. In sequence it explains the importance of technology and discourse communities to the concept of genre. Finally, the chapter attempts to describe the relationship between genre conventions and usability. 2.2. Genres 2.2.1. Definition Several authors from different areas of knowledge have given various definitions of the term genre in the past decades and this has made the concept somewhat blurred. The etymology of the word shows that it originated from the Greek word ‘!"#$%’ which, according to the Liddell-Scott Greek-English lexicon, means ‘of a race, stock or family’. It was later borrowed by the Romans and transformed into ‘genus’ which means ‘of a kind’. Although both the Greek, and later Latin meanings, refer to groups that share common traits, it is the Latin counterpart that can give some clues to the contemporary use of the word. Genre theory started with the study of rhetoric and according to Waller (1987a) the idea of classifying types of discourse into groups seems to have been around since the time of Aristotle. The great majority of genre theories so far have been in the fields of literature, linguistics, rhetoric and some in folklore and film, each having the concept of genre adapted to its needs and idiosyncrasies. In an attempt to establish a broader 8
- 31. Genres, conventions and usability definition, Swales (1990) draws on ideas from four main fields that have developed studies on the concept of genre: folklore, literature, linguistics and rhetoric. His definition is indeed far reaching as it states that: A genre comprises of a class of communicative events, the members of which share some set of communicative purposes. These purposes are recognized by the expert members of the parent discourse community, and thereby constitute the rationale for the genre. This rationale shapes the schematic structure of the discourse and influences and constrains choice of content and style (Swales, 1990, p.58). There are many interesting concepts in Swales definition that are tightly connected with genres and are useful as a starting point to understanding genres. If genres are classes of communicative events, the involvement of at least two agents is necessary for a genre to be established, and it requires that the individual on the recipient end of the communicative process be familiarized with the structure, content and style of the discourse. When a group of people becomes familiarized with these aspects of a discourse (structure, content and style) that shape a genre, they develop a symbiotic relationship with the genre, and will simultaneously be able to decode and adapt it to their communication needs. These groups of people are called discourse communities by Swales (1990), and this concept will be addressed in more detail in sub section 2.2.5. Although Swales (1990) suggests that the rationale for a genre shapes the stucture of the discourse and influences content and style, he does not go further into defining a way of categorizing documents into genres. 2.2.2. Categorization into genres A concept that seems to be shared by some authors is one that associates the categorization of discourses into genres using a framework formed by the principles of rhetorical substance, form and rhetorical action (Miller, 1984). The concepts of substance and form are referred to by Yates and Orlikowski (1992, 2002) as also being characteristics of a genre. The authors explain that substance refers to the content as well as the social circumstances expressed in the communication, and form refers to the physical attributes and linguistic features of the communication. This expands the framework proposed by Miller (1984) allowing for the physical and social aspects of discourse to be considered. 9
- 32. Genres, conventions and usability With the advent of the Personal Computer and the Internet, a further expansion to the conceptual model of genre was proposed to include another component besides substance and form (Shepherd & Watters, 1999). Because users approach Internet documents with expectations with respect to functionality as well as form and content, Shepherd and Watters (1999) propose that a web genre (named cybergenre by the authors) be characterized by form, content(i.e. substance) and functionality. Considering that the focus of this thesis is on graphic design and typography, the form component of a genre is of particular interest. The role of the visual aspect of a document in genre categorization Because most of the genre theorists come from backgrounds such as rhetoric, linguistics and literature, the form component of a genre has previously been associated with the linguistic aspects of communication rather than its visual attributes. However, as seen earlier, some authors have mentioned the visual appearance of a document in their description or analysis of a genre (Yates & Orlikowski 1992, 2002) but despite that, not many authors have studied the visual aspect of a genre or its impact on genre identification and classification. The importance of the visual appearance of a genre has been illustrated in an experiment conducted by Toms (2001). To try to understand the role of document layout in genre recognition, the study compared three different versions of the same document: ! content version, where all structure, layout and formatting were reduced to a minimum (leaving only the words, phrases and sentences). ! form version, where structure, layout and formatting were preserved and all characters were replaced by ‘X’s or ‘x’ and digits were replaced by ‘9’s thus excluding the content. ! full version which had structure, layout, formatting and content intact. Participants were presented with these versions and were asked to identify what the page was. As expected, the full version had the higher identification rate, which suggests that content and form are both important to genre recognition. However, the results also showed that participants who were familiarized with one specific genre performed 10
- 33. Genres, conventions and usability better at identifying it through the layout (form version) rather than just through content (content version) (Toms, 2001). Also, when content from one literary genre was formatted by the rules or conventions of another genre (e.g. a letter content formatted as a dictionary), the visual aspect was the first clue people used to identify the genre (e.g. in the example identifying a personal letter as a dictionary). This experiment illustrates the important role that the visual aspect of a genre plays in the identification of a genre. Allowing users to identify a document genre accurately quickly enhances their interaction and makes the use of the document more efficient (Toms, 2001). In spite of that, not many genre theories have considered the visual appearance of genres in the depth which this empirical study suggests is needed. One exception is the work of Waller (1987a) which established a framework of genre bringing together language content and visual appearance. Waller also proposed in his framework how genres are inserted and contribute to the design process. This is extremely relevant to understanding that genres act as forces in the design process. 2.2.3. A genre framework from a design point of view The analysis of document genres from a typographical point of view and the model of the design process of a document proposed by Waller (1987a) are both highly relevant factors to this thesis. Although some other authors have developed Waller’s model further (Bateman & Delin, 2001; Delin, et al., 2002) this thesis will base its framework on the concepts of conventional genre structures (Waller, 1987a, p. 180) and the model of the design process developed by Waller (1987a, p. 298). Waller (1987a) proposes that any printed document can be accounted for by a combination of three structures (Waller, 1987a, pp.179-180): Topic structure – Typographic effects with the purpose of displaying information about the author’s argument. Artefact structure - The result of the medium where the document is being displayed and its production technology. Access structure – The features which make the document usable and accessible to readers. The separation of these three structures is theoretical as in practice these structures overlap and in most genres they are so strongly linked that it can become hard to think of an alternative way of designing the document (Waller, 1987a). These three 11
- 34. Genres, conventions and usability structural concepts form the basis for describing genres of documents (Waller, 1987a, p. 180). But because of the strong links between them, and the overlapping of these concepts, it can be quite hard to use these structures to describe the genres of documents. This thesis proposes another set of concepts which can be used to describe genres of documents. These concepts represent the types of conventions which can exist within a document genre (genre conventions). These genre conventions have their origin in the functional constraints proposed by Waller (artefact, topic and access structures) which state that when a document genre becomes more stable, these functional constraints tend to become highly conventionalized. The proposed concept of genre conventions consists of structural conventions, linguistic conventions, visual conventions and functional conventions. Although these four types of conventions also overlap and changes in one might affect another, it can be easier to separate the contribution of each type of convention to the genre, and to pinpoint where overlapping may occur. This makes them more appropriate for the work developed here. Structural conventions describe how information is organized within a genre of documents. Linguistic conventions describe the semantic, syntactical and lexical aspects of the genre (e.g. choice of words and abbreviations). Visual conventions describe how information is visually presented in a genre. Differently from the functional constraints defined by Waller (1987a), the functional conventions describe how the components of a particular genre respond when the user interacts with them. They are directly linked with the artefact used to portray the document. Waller (1987a) proposed a model which allowed him to represent the role of genre in the process of designing a document. What is not clear from Waller’s model (see Figure 1) is whether functional constraints are part of a genre (Waller, 1987a, p. 180) or if they are detached entities, since the author suggests that a negotiation process between genre and functional constraints can occur during the design process. Waller himself gives a clue as to how this apparently contradictory situation can occur: 12
- 35. Genres, conventions and usability When those technologies change, it may be necessary to separate out the three categories of functional imperatives in order to reassemble them to suit the new constraints of the new technology (Waller, 1987a, p. 181). In fact a change in technology or even a change of media provides new possibilities (and limitations) for the artefact structure of a document, and a designer has then to negotiate between the genre and the new functional constraints determined by the technological change. Thus, even though functional constraints over time do become part of the conventions of a genre, any changes in them may force a renegotiation to happen. Although technological change is probably the most common change in the functional constraints of a document, it is not by any means the only one, as topic structures or access structures can change from existing genre conventions and require the negotiation process between genres and functional constraints to happen. For the work proposed here, some adaptations to Waller’s model of the design process are needed. First, ‘genre’ in the model will be replaced by genre conventions, since in principle, the negotiation process occurs between the conventions which describe a genre and the functional constraints. Second, the term design task used by Waller (shown as ‘task’ in his diagrams) will be replaced by the term ‘design problem’ which can be considered as happening prior to when the design task is determined since a design task is usually established to solve a problem which arises. The belief adopted here is that the negotiating process between genre conventions and functional constraints is a continuous process which is initiated upon detection of a problem before a task is defined. 13
- 36. Genres, conventions and usability FIGURE 1 – MODEL OF THE DOCUMENT DESIGN PROCESS AS DESCRIBED BY WALLER (1987). Genre Design Task Solution Functional Constraints (Topic, artefact & access structures) FIGURE 2 - MODEL FOR THE DESIGN PROCESS INVOLVING GENRES AND CHANGES AND FUNCTIONAL CONSTRAINTS (BASED ON THE MODEL DEVELOPED BY WALLER (1987). Genre Conventions (structural, linguistic, visual and functional) Design Design Problem Solution Functional Constraints (topic, artefact & access structures) According to Waller (1987a), the design process can then occur in different ways: A) The designer can rely solely on the genre to provide a design solution (Figure 3). This solution is extremely conventional and is likely to meet the expectations of the users familiar with using that particular document genre. However innovation is not likely to happen here and in case of technological evolution occurring, the document might not be adapted to the media in which it is being portrayed. 14
- 37. Genres, conventions and usability FIGURE 3 - DESIGN PROCESS WHICH RELIES SOLELY ON THE DOCUMENT GENRE AND ITS CONVENTIONS. Genre Conventions (structural, linguistic, visual and functional) Design Design Problem Solution B) The designer considers in detail the functional constraints when generating the document, but does not look at the existing genre conventions for it (Figure 4). As a result, the document is likely to be innovative and well adapted to the media in which it is conveyed but users who are accustomed to the conventions of the genre may not know which strategy to use when interacting with the document. FIGURE 4 - DESIGN PROCESS WHICH RELIES SOLELY ON THE FUNCTIONAL CONSTRAINTS OF A DOCUMENT. Design Design Problem Solution Functional Constraints (topic, artefact & access structures) C) The designer either comes up with a conventional genre conforming solution, and then validates it against the functional constraints, or designs through an analysis of the functional constraints, and then checks the solution against the genre conventions (Figure 5). These two ways of designing are healthier than the previous ones (Waller, 1987a) and here a negotiation occurs between the genre and the functional constraints. When the design process occurs in this manner it can provide solutions which conform to the expectations of users and are better adapted to the possibilities of the media in which the document is portrayed. It can also allow for innovation to occur and it is possible that this is the path followed in order for genre evolution to happen. 15
- 38. Genres, conventions and usability FIGURE 5 - DESIGN PROCESS WHICH RELIES ON THE GENRE-FUNCTIONAL CONSTRAINTS NEGOTIATION TO ACHIEVE THE DESIGN SOLUTION. Genre Conventions (structural, linguistic, visual and functional) Design Design Problem Solution Functional Constraints (topic, artefact & access structures) For designers to be able to make good choices in the negotiation process between the genre and the functional constraints, it is necessary that they are aware of the possibilities and limitations imposed by both the functional constraints and the genre conventions. It is also important that they know what impact choosing one or the other will have on the users of the document. Only then can the outcome of the process provide a good design solution. Design solutions, genres and patterns. Good design solutions are also one of the main principles behind pattern languages since a pattern can be thought of as a good design solution to a recurring problem. This definition seems to indicate the existence of a relationship between genres and design patterns. The concept of pattern languages was developed by the architect Christopher Alexander as a way of describing ‘problems which occur over and over again in our environment’ (Alexander, Ishikawa, & Silverstein, 1977) and the solution to the problem. The concept of pattern languages has been adopted not only in architecture, but also in software engineering and Human-Computer Interaction (HCI). In the field of HCI the concept of a pattern language has been gaining momentum partly because HCI guidelines seem to be problematic when it comes to describing design solutions. HCI guidelines focus mostly on creating coherent user interfaces 16
- 39. Genres, conventions and usability rather than on the necessary knowledge for proper UI design such as knowledge of the user, the context and the task (Granlund, Lafrenière, & Carr, 2001). In fact, some authors claim that HCI patterns are closer to architectural patterns than software engineering patterns, since they are less rigid, and less detailed thus allowing more creative freedom (Granlund, et al., 2001) and they also share Alexander’s ideas of user-centered and participatory design (Borchers, 2001). The main motivation for creating a pattern language in HCI is to have a way of collecting, formalizing (Granlund, et al., 2001) and disseminating HCI design knowledge (Griffiths, Pemberton, Borchers, & Stork, 2000). It can also provide a common language of HCI design to anyone involved in the design, evaluation, development and use of interactive systems (Borchers, Fincher, Griffiths, Pemberton, & Siemon, 2001). But pattern languages are supposed to do much more than just allow writing good design ideas so that they can be discussed (Alexander, 1999). Pattern languages are ‘a more universal model of structuring design knowledge. They can be used in any discipline that requires structured creative work of some kind to be carried out’ (Borchers, 1999). This type of work is precisely that of graphic designers, typographers and information designers. A pattern language is a hierarchical structure comprised of several design patterns and when it comes to interface design a pattern ‘describes possible good solutions to a common design problem within a certain context’ (Tidwell, 1999). Although there are slight differences in the exact components of an interface design pattern, most authors agree that an HCI design pattern has the following components (Borchers, 1999, 2001; Granlund, et al., 2001): ! Name – This identifies the pattern and its central idea. ! Examples – Is used to clarify the problem. It shows existing situations in which the problem can be encountered and how it has been solved in these situations. ! Context – Describes the objective of the design, specifying the user and task requirements ! Problem – Describes the design problem, the major design issues that the pattern addresses. ! Forces – Describes all the factors that influence the design either directly or indirectly. As mentioned earlier in chapter 1, section 1.3 this thesis aims to contribute to the understanding of some of these forces. 17
- 40. Genres, conventions and usability ! Design solution – Generalizes from the examples a proven way to balance the forces for a given design context. Although from the description of a pattern it is possible to notice some similarities with genres, the relationship between the two is not entirely clear. This relationship, however, seems quite relevant not only in the HCI domain, but in the fields of graphic and information design. Some authors have attempted to link the concept of genres with design patterns and pattern languages. Pemberton (2000) explains that genres may be a useful tool for structuring design pattern sub-languages and facilitating the perception of links between them. But the question of whether genres are the same as pattern languages or if they overlap remains unanswered. In an attempt to link them Åkesson (2003) points out the following similarities between genres and pattern languages: ! Genres are formed from recurrent communication acts and design patterns from recurrent design problems. When design problems are communicational both concepts coincide. ! Both change over time. Pemberton (2000) proposes that the development of design patterns for each genre, in other words, representing the conventions of a genre through design patterns could be a way of ‘packaging the knowledge’ of the genre conventions in a way which would simplify the design task and allow for a better critical analysis of the implications of genre membership. Even though genre conventions may have their origins in design solutions based on certain functional constraints and at a certain point they may be a good design solution to a problem, this may not be perpetually true. It might be that any or all of the functional constraints have changed, and the solution which the genre provides is not a good one anymore. Thus it is not always viable to have a genre described through pattern languages. When the functional constraints change, achieving a good design solution requires negotiation between the genre and the functional constraints. The design solution which will then result from this process can be used as basis for the design pattern that will represent a better solution to the original problem, and consequently can be incorporated into a pattern language. This shows the necessity of having a good knowledge of the effects that the genre conventions may have on the users of a particular document genre, since that will 18
- 41. Genres, conventions and usability improve design solutions and consequently allow for the development of appropriate design patterns. This thesis will not aim at explaining the relationship between genres and design patterns. It will focus on genre conventions so that it can contribute to increasing the knowledge of genres and understanding the role genres play in the design of documents. 2.2.4. Genres and technology The influence of the medium used to convey a message on the visual appearance of the message itself can be easily observed. Changes and development in the media used to portray documents are usually connected with technological advances. Technological evolution causes the media or artefact used to portray a document to change thus allowing genres to change via modifications in the artefact structure functional constraint. Usually whenever a new technology of communication surfaces, the appearance of the messages is shaped to adjust to and take advantage of this new technology. Frye (1957) comments on the new possibilities that the printing press brought to the lyric poetic style; he says that it developed ‘an increasing tendency to address the ear through the eye’ (Frye, 1957, p.278). This statement clearly illustrates the new possibilities that the printing press offered for the laying out of a lyric style text. In a similar way, the computer and later the WWW, have again allowed for new possibilities in communication which have resulted in the emergence of new genres (Dillon & Gushrowski, 2000) or the adaptation of existing ones (Santini, 2005). This is corroborated by the results of a study carried out by Yates, Orlikowski and Rennecker (1997) which have shown that the introduction of a new communication medium in an organization can generate variations on an existing genre or even the emergence of new genres. Genre variations & emergence of new genres In the beginning of the WWW, many web pages seem to have used existing paper- based genres instead of taking advantage of the technological changes of this new communication medium to develop new genres or modify existing ones (Dillon & Gushrowski, 2000). This argument apparently has some truth in it since there is evidence that some paper-based genres seem to be adapting to the world wide web (Eriksen & Ihlström, 2000). But also, some web genres without an equivalent in print 19
- 42. Genres, conventions and usability such as the personal home page (Dillon & Gushrowski, 2000) have appeared. It is important to also have in mind that when the World Wide Web was initially conceived, there were great restrictions in the kind of information that could be displayed and how it could be presented. As seen in the previous section, this transfer of previously existing genres to a new technology is not a new phenomenon as it seems to have occurred previously when technology provided improved or new media which could be used to convey a document. A classical example occurred with the printing press where the first printed books imitated a genre that was known at the time - the manuscript illuminated book (Love, 1985). This imitation was not only visual (by copying the manuscript letter styles and using decorated capitals), but also linguistic by using the same written style and abbreviations that medieval scribes had used. It was not until later that the old manuscript document genre was adapted to the new technology and new genres were created. This shows that in order to better understand a genre and to be able to make appropriate design decisions between genres and functional constraints, it is also important to have a good understanding of the technology involved with the artefact in which the document is presented, and the technology relating to it. 2.2.5. The social aspect of genres – discourse communities As mentioned earlier, the idea of genres is directly associated with social interaction and this statement holds true in all fields that use the concept; in fact there is no sense in talking about genres devoid of a social context. Genres emerge from communicative needs of discourse communities under certain circumstances and certain constraints. Often the abilities to identify, decode and understand them are particular to those groups of people. The familiarity of these discourse communities with the particularities of a genre creates in their members expectations regarding the structural, linguistic, visual and functional features of the same genre of documents. These particularities are governed by the genre conventions and the expectations of members of the discourse community are in line with them. 20
- 43. Genres, conventions and usability Although discourse communities are mentioned in genre literature (Østerlund, 2006; Swales, 1990; S. Yates & Sumner, 1997), concerning the computer environment, some have long doubted if it is possible to have social groups within this context. If this is the case, it is highly unlikely that genres can emerge from the World Wide Web. However, evidence shows that a very strong sense of ‘belonging to a group’ regularly appears in members of web-based groups (Wallace, 1999). This is a strong argument in support the idea that there are digital genres emerging in the WWW (Dillon & Gushrowski, 2000; Ihlström & Åkesson, 2004; Shepherd & Watters, 1999). 2.2.6. The importance of visual conventions of a genre As defined earlier, visual conventions of a genre describe how the information is visually presented in a document from that genre. In design terms, visual conventions are tightly linked to the formatting and layout of the document. According to Waller (1999), the typographic genre originates from ‘design imperatives’ which initially are genuinely functional but over time may lose their original purpose and become signals to a genre. Analysing this concept under the light of the typographic genre defined in Waller (1987a) and the framework established for this thesis, the ‘design imperatives’ cited by Waller (1999) can be seen as one of the originators (if not the only one) of visual, structural and in the case of web genres, possibly some of the functional conventions of the genre. As seen earlier, a series of studies carried out by Toms (2001) in genre recognition shows that the formatting and layout of a document played an important role in helping users identify to which genre it belongs. Beghtol (2001) concurs with these findings by mentioning that a genre might be identified by its layout. The author adds that, when no distinct layout clues are available to help the user make a decision regarding the genre of the document, the need to read the content in order to associate it to the genre emerges. Apparently a user who is familiar with a particular genre (e.g. a member of a discourse community) has an organized, high level mental representation of the documents belonging to that genre (Vaughan & Dillon, 1998). Whenever a cue which relates to the genre is identified, the user activates the relevant representation from semantic memory (van Dijk & Kintsch, 1983) and this, according to Vaughan and Dillon (1998) can aid recall, encoding and comprehension of documents. 21
- 44. Genres, conventions and usability 2.2.7. Section summary In this thesis, a genre is described by a set of genre conventions, namely: structural, linguistic, visual and functional conventions. It is believed that the best possible design solution to a design task comes from the negotiation between the genre conventions and the functional constraints of a document. Technological advances can originate new media or promote changes in existing ones which can in turn alter the functional constraints of a document. This can not only trigger the evolution of a genre, but also account for the emergence of new ones. Conforming to the visual conventions of a document genre seems to facilitate immediate identification of the genre by the members of its discourse community. It may also trigger in these users a mental structure which may improve the recall, encoding and comprehension of the document. Finally, discourse communities (i.e. groups of people who share knowledge and understanding of a genre and its conventions) are an essential requirement for a genre to exist. These communities of users have certain expectations which are based on the genre conventions they share, and it is when these expectations are not met that usability problems with the document may appear. But how genre conventions relate to usability is not entirely clear and needs to be further examined. 2.3. The relationship between conventionality and usability This section will attempt to establish whether the conventions of a genre are in any way related to the usability of a document belonging to that genre. It is expected that this will help determine, for example, if not conforming to genre conventions can be considered a usability problem, or a different type of problem altogether. In order to establish this, existing definitions of usability and different views on how usability can be measured will be examined. Through an analysis of the concept of consistency, a link between genre conventions and usability will be formed. The analyses of existing research will help define if there is a relationship between genre conventions and usability and what this relationship might be. 22
- 45. Genres, conventions and usability 2.3.1. Defining Usability The term usability started to appear in Human-Computer Interaction literature in the beginning of the 1980s. Although many attribute its creation to Nielsen (1985), the term was being used long before by Gould and Lewis (1981), Shackel (1984) and Eason (1993). Usability seems to have derived from the concept of “user friendly” which was commonly associated with computers and machines in the 1970s. According to Nielsen (1993, p. 23) the term “user friendly” was inappropriate because it put users into one single dimension, when in reality the needs of different users are different (Bevan, Kirakowski, & Maissel, 1991). Van Welie, van der Veer and Eliëns (1999) present another possibility for the replacement of “user friendly” by usability by saying that in the early 1980s the term “user friendly” started to acquire some ‘vague and subjective connotations’. It seems, however, that the term usability did not improve matters much, and there is still no agreed upon definition of what usability means (Seffah & Metzker, 2004). The term usability has been used with different meanings which makes the concept a very confusing one (Bevan, et al., 1991). The many different aspects of usability were examined by the MUSiC project in an attempt to devise a consistent quantitative method of testing usability. The project which was set in the 1990s was coordinated by Jurek Kirakowski with Nigel Bevan as a collaborator. The objective of MUSiC was to develop a method of measuring usability that would be broad rather than restrictive. To do this, the researchers examined existing concepts and views on usability. According to Bevan et al. (1991) the many definitions of usability that exist derive from a number of different views on what usability is. Of these many definitions, Bevan et al. (1991) highlight four views which relate to how usability should be measured: ! The product-oriented view, that usability can be measured in terms of the ergonomic attributes of the product; ! The user-oriented view, that usability can be measured in terms of the mental effort and attitude of the user; ! The user-performance view, that usability can be measured by examining how the user interacts with the product, with particular emphasis on either ease-of-use (how easy the product is to use) or acceptability (whether the product will be used in the real world); 23
- 46. Genres, conventions and usability ! The contextually-oriented view which claims that the usability of a product is a function of the particular class of users being studied, the task they perform and the environment in which they work. According to Bevan et al. (1991), the contextually-oriented view can complement the other three views on usability. One definition which is mostly accepted by usability practitioners and researchers is the one given by the standard ISO 9241-11 (ISO/IEC 9241-11, 1998, p. 2). The ISO definition is performance-oriented, user-oriented and most importantly, contextually-oriented and it says that usability is the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO/IEC 9241-11, 1998, p. 2). Where: Effectiveness is the accuracy and completeness with which users achieve specified goals; Efficiency is the resources expended in relation to the accuracy and completeness with which users achieve goals; Satisfaction is the freedom from discomfort, and positive attitudes towards the use of the product (Bevan, et al., 1991; Bevan & MacLeod, 1994; ISO/IEC, 1998); The context of use of a product is thought of as composed of the characteristics of the users, the tasks they perform and the environment where the product is used (Bevan & MacLeod, 1994). The fact that usability is dependent on the context in which the product is used increases the complexity of attempting to measure usability since it becomes a mutable attribute of a product which can vary according to certain factors. This view is shared by some authors (Bevan & MacLeod, 1994; Holscher & Strube, 2000; Tabatabai & Shore, 2005) who claim that the characteristics of the context can be as important for usability as the characteristics of the product. The authors add that relevant changes to the context may change the usability of a system. Thus, it is possible to conclude that there are two main factors which can affect the usability of a system: the product features and context. These factors and their 24
- 47. Genres, conventions and usability relation to usability can be represented in a diagram (Figure 6). But it is important to notice that the diagram below is a simplification as it does not illustrate inter- relations which may occur between the contextual factors themselves. The environment might influence the user or the type of task may affect users with different characteristics in diverse ways. FIGURE 6 - MODEL OF CONTEXTUAL FACTORS WHICH MIGHT INFLUENCE THE USABILITY OF A PRODUCT. Contextual factors User Task Environment characteristics Usability Product features Although the environment and the task are important contextual factors which influence usability, understanding the user characteristics which affect it is the factor which falls within the scope of this research. User characteristics which may influence usability (such as familiarity with a genre) can be hard to determine, and even though much research has been done on the topic, more investigation is needed. However, results from existing research indicate some characteristics of the user which affect user performance, and consequently can change the usability of the product. These characteristics include user expertise (familiarity) with a system (Scott & Schwartz, 2007), metacognitive skills (Marchionini, 1995), age (Jenkins, Corritore, & Wiedenbeck, 2003; Wildemuth, 2004), domain knowledge (e.g. great knowledge of a topic being searched in an interface) (Lidwell, Holden, & Butler, 2003). Although these are only some user characteristics which can influence the usability of a product, the user characteristic which seems to 25
- 48. Genres, conventions and usability be most frequently addressed on usability guidelines, heuristics and recommendations, is the user familiarity with the system. 2.3.2. Usability and genre conventions One fundamental design principle which has been widely adopted by usability practitioners is that of consistency. This basically determines that related parts of a system should be expressed and behave in similar ways (UPA, 2005). The Usability Professional’s Association lists following appropriate standards/conventions for a system as one of its principles of usable design (UPA, 2005). It adds that the actions, terminology and commands inside an application should be used consistently (Nielsen, 1994). A similar recommendation is given by Nielsen (1994) who says that ‘users should not have to wonder whether different words, situations, or actions mean the same thing’. The author then adds that platform conventions (i.e. the conventions of the system) should be followed. Despite the fact that the recommendations made by the UPA and Nielsen show a justifiable concern with consistency within a system, they do not explicitly address another possible scenario very common in the web environment; that of different websites which belong to the same category (e.g. yahoo mail, hotmail which belong to the webmail category). This other scenario is addressed by Bevan (1997) who points out that for a website to be consistent it should ‘meet user expectations by following conventions established by other major sites’. He is supported by Jakob Nielsen with what he describes as his ‘Law of Web User Experience’. “Users spend most of their time on other sites. Thus, anything that is a convention and used on the majority of other sites will be burned into the users’ brains and you can only deviate from it on pain of major usability problems”(Nielsen, 1999a) . From these different recommendations it becomes apparent that, at least for the WWW, there are two ways in which a website can be consistent. One, which is addressed in the recommendations of Nielsen (2005) and UPA (1997; 2003) is the internal consistency which considers the consistency of the elements within the system. The other, called here external consistency, is concerned about the consistency between the system and other similar existing systems. In this sense, a genre of digital documents can be thought of as being externally consistent. The external consistency of a system is addressed in the recommendation of Bevan (1994) and Nielsen (1989). There is yet a third type of consistency defined by Grudin (1989) 26
- 49. Genres, conventions and usability which relates to the correspondence between the interface elements and elements which exist in the ‘world beyond computing’ (Nielsen, 1993). This type of consistency, although not addressed in this thesis, will be named consistency with the world for future reference. The concern of usability practitioners and researchers with consistency is justified since one of the attributes of usability as defined by Nielsen (1993) is learnability. According to Nielsen, systems should be easy to learn and keeping internal consistency is likely to facilitate learning. This happens because elements are kept consistent across an interface so that the user only has to learn what they mean and what they do once. Internal consistency is also likely to reduce the amount of information the user has to learn because each element will be represented and will function in the same way all across the interface. External consistency can be thought as being related to another important attribute of usability described in Nielsen (1993); efficiency. According to Nielsen, after users have learned the design of the system they should perform tasks rapidly and therefore perform more efficiently. Implicit in this statement is the idea that while learning the design of an efficient interface, users do not perform as fast as they could. From this follows that if an interface is consistent with another interface that the user already knows it will cut down system learning time and draw upon user memory thus improving the efficiency of a system and resulting in a more usable interface. Consistency, user expectations and conventions Most descriptions of consistency involve following conventions in some way or other. The internal consistency of an interface is usually achieved through a set of rules defined by the developers of a system to which the interface belongs and also take into account technological constraints. These rules can be thought of as the ‘system design conventions’ which developers will follow through the design process. When external consistency is considered, the link between conventions and consistency becomes even more evident. Usually external consistency involves following standards or conventions dictated by other similar existing interfaces (usually belonging to the genre of the interface being developed). The conventions, when followed, should guarantee an interface which is consistent with the genre. 27
- 50. Genres, conventions and usability Regardless of whether the consistency of an interface is internal or external (or both), after a certain time interacting with it, users are likely to develop expectations regarding what they will find in the interface and how it functions. User expectations As outlined above, user expectations are formed through prior experience with an interface (Vaughan & Dillon, 1998). The link between expectations and genres becomes evident when one thinks that prior experiences with a particular set of documents which share similar characteristics, generate in the user expectations of finding the same similarities on every document which belongs to that particular genre. This idea is echoed by Waller (1987b) who says that ‘to belong to a genre, a document is effectively conforming to a set of rules or expectations’. Waller (1999) also proposes a model to describe the origins of a genre which says that a genre emerges from early design necessities which were genuinely functional and that throughout time have lost their initial purpose. These early design necessities then start to work as a signal to the genre of a document and trigger appropriate expectations, interpretations and strategies amongst its users. Although the process through which expectations are formed is quite a complex one to describe and therefore understand, Vaughan and Dillon (1998) propose a model which is based on cognitive psychology research on mental representations of structure. According to the authors ‘Users begin by building up a mental model of an information space based on their interaction with a set of its digital documents. With repeated exposure and if the set of documents maintains some sort of regularity, the user will begin to develop a schematic knowledge of that regularity. This highly organized, abstract knowledge about digital documents would take the form of Kintsch’s superstructural knowledge. ’(Vaughan & Dillon, 1998) If the documents belonging to what the authors call an information space have regularities, then these regularities can become conventions of that particular information space, and the information space can be thought of as a document genre. The superstructural knowledge (Grier, Kortum, & Miller, 2007) which the user acquired through prior experience with a document genre, becomes the standard upon which their expectations are built. 28
- 51. Genres, conventions and usability These expectations can guide user search for an element (Dillon, 2004) and when violated, are not only immediately noticed by the user (Vaughan & Dillon, 2006) but also seem to hinder user performance (Shneiderman & Plaisant, 2005). Conventions Thus, the concern with conventions expressed in usability literature is justifiable since conforming to conventions can mean not only achieving consistency, but also fulfilling users’ expectations. However the concept of conventions, specially genre conventions, in usability, needs to be broken down further if one is to fully understand the role these concepts play in usability. As seen earlier in this chapter, genre conventions can be structural, linguistic, visual or functional. These four types of conventions are different, and they may affect usability differently. Some usability authors have given different recommendations on how to create internally and externally consistent interfaces. Recommendations on guaranteeing internal consistency do not directly mention conventions as they are viewed more as rules which need to be followed. In that case, it is still possible to see which type of conventions they will become by examining the recommendations. Although structural, linguistic, visual and functional conventions seem to be considered in usability literature, it seems that authors most often associate following visual and linguistic conventions with usable design, than following structural conventions. Shneiderman and Plaisant (2005, p.74) recommend that ‘identical terminology should be used in prompts, menus, and help screens; and consistent colour, layout, capitalization, fonts and so on should be employed throughout [an interface]’. This recommendation addresses the visual and linguistic aspects of internal consistency but does not consider structural aspects of consistency. Nielsen (1993, p. 132), recommends that ‘the same information should be presented in the same location on all screens, and dialog boxes should be formatted in the same way to facilitate recognition’ which also addresses the visual aspects of internal consistency. A study which addressed all three aspects of internal consistency and established twenty guidelines for designing consistent web interfaces was conducted by Ozok and Salvendy (2004). When defining the concept of consistency, the authors 29
- 52. Genres, conventions and usability address visual (e.g. ‘Use consistent sizes, shapes, and colours for screen elements such as menus, combo boxes, radio buttons, and check boxes’), linguistic (e.g. ‘Use your words consistently within and across interfaces’) and to some extent structural conventions (e.g. Use headings for different topics and sub-topics of the text’). External consistency is addressed in the Usability Body of Knowledge (UPA, 2005) which says that standards and conventions for a platform and suite of products should be followed. Despite using the word conventions in their guideline, the UPA does not specify which type of conventions these would be. However if one is to consider these conventions as Interface standards such as the Apple Aqua Interface Standard (available from http://developer.apple.com/documentation /UserExperience/Conceptual/AppleHIGuidelines/OSXHIGuidelines.pdf) it is possible to see that these cover structural, linguistic and visual aspects of consistency as well as functional aspects of the interface in order to achieve consistency. Despite the concern with conventions, not many investigations have been done on how not conforming or conforming to structural, linguistic, visual and functional conventions might affect interface usability. One of the first studies to examine the effects of not conforming to conventions by way of inconsistent interfaces is reported in Satzinger (1998) and Satzinger and Olfman (1998). Their study examined four groups of participants using two different modules belonging to the same system. The study investigated the effects of consistency and inconsistency on user performance and satisfaction. Inconsistency was achieved by manipulating the visual and linguistic aspects of the modules. Four groups were tested as seen in the distribution shown in Table 1. TABLE 1 - CONDITIONS UNDER WHICH EACH GROUP OF PARTICIPANTS WAS TESTED BY SATZINGER AND OLFMAN (1998). Group Tested on modules with 1 Consistent visual appearance and consistent language syntax 2 Consistent visual appearance and inconsistent language syntax 3 Inconsistent visual appearance and consistent language syntax 4 Consistent visual appearance and inconsistent language syntax Results showed that language syntax inconsistency (i.e. violation of linguistic conventions) resulted in poorer performance of the participants. However, visual inconsistency (i.e. violation of visual conventions) resulted in better performance. 30
- 53. Genres, conventions and usability The authors explain this result which contradicts usability literature by suggesting that because participants were using both modules simultaneously, a consistent visual appearance might have caused confusion in the participants. Thus, participants might have had difficulty in discriminating which of the modules they were using at one given time. Inconsistency on the other hand might have helped participants in determining which exact module they were using each time, thus allowing them to retrieve the appropriate schema to deal with it (Satzinger & Olfman, 1998). The authors do point out that more investigation ‘on the benefits of consistent visual appearance is certainly called for’ (Satizinger, 1991). This thesis investigates the effects of not conforming to visual genre conventions on user performance. Visual conventions are related with visual consistency since visual genre conventions are formed by maintaining visual consistency among the documents of a genre. Therefore, this research will certainly contribute to further expand the knowledge of the effects of the visual appearance and consistency of an interface on user performance. This thesis will also look at the issue from a more web-focused point of view where different websites are separate entities, as opposed to the software modules used by Satzinger (1991) and Satzinger and Olfman (1998). This approach seems more appropriate for examining genres than the approach taken by Satzinger (1991) and Satzinger and Olfman (1998) who looked at the internal consistency/inconsistency of a software system. Another study which investigated violation visual conventions was conducted by van Schaik and Ling (2001) varied the navigational position between the conventional left side, top, right and bottom of the screen along with the background (which could be either white or a specific shade of gray). Participants were presented with 72 screens and before each screen they were shown a target word. When a screen was displayed participants had to search to see if the target word was part of the navigational menu or not. Results showed the top menu position provided a higher ratio of correct identification, and a quicker response time when trying to locate the word. Screen contrast showed no significant differences on correct identifications or on response time. Although in this study the authors addressed only the visual search process and not an interactive search (where the search is carried out through interaction with the interface elements, i.e. clicking on links), it 31
- 54. Genres, conventions and usability shows that the conventional left side position is not necessarily the most visually efficient place for the navigational menu to be put. In terms of visual conventions on the Web, a more recent study which tested how altering the position of the navigation menu of a website could alter the performance of users searching for information on a website was conducted by McCarthy, Sasse and Riegelsberger (2006). In the study, the position of the menu varied from the left (conventional) to the top and right of the screen. The study showed that not conforming to the conventional menu position (left) did indeed cause poorer performance. However the authors mentioned that the performance deterioration was short-lived. In a similar study, Kalbach and Bosenick (as suggested by McCarthy, et al., 2003) investigated how altering the position of the navigation bar on a website from the conventional left side to the right side of the screen affected user performance when navigating. Although Kalbach and Bosenick (2006) did not find a significant difference in performance between the two conditions, participants in the right-sided menu group showed indications that they were learning because they scanned and read the page more often than their counterparts. Pearson and van Schaik (2003) conducted a study which looked at the effect of link colour and positioning of the navigational menu in both a visual search task and an interactive search task. Results for the visual search task agreed with van Schaik and Ling (2001) since the top menu position along with the bottom position elicited better response time than the left and right positions. There was no difference in response time between the top and bottom positions and no difference in response time between the left and right positions. No effect of position or link colour on correct identification rate was found. However, in the interactive tasks the authors found that performance was better on the left and right positioned menus than on the top and bottom positioned ones. All these studies have addressed the position of the navigational elements on the page, which is one aspect of visual conventions. Another study which went further and investigated visual and structural conventions was conducted by Vaughan and Dillon (2006). They investigated the impact of not conforming to digital genres on user comprehension, performance and navigation. In this study the authors used a web-news conforming page and a non-conforming page. The non-conforming web- news site altered structural conventions by manipulating the categories of the news stories and violated visual conventions by altering attributes such as the position of 32
- 55. Genres, conventions and usability the navigation bar, colour, type size, and number of columns. The performance of the participants who used the non-conforming site was worse than the performance of the other group. However, the authors reported that over time (in this case after 5 days using the website) participants using the non-conforming website learned to use the new website and improved their efficiency. Because Vaughan and Dillon (2006) have altered simultaneously the structural and visual conventions of a website it is impossible to separate the effects. Although there seems to be indications from McCarthy et al. (2006) that altering visual conventions hinders user performance, McCarthy’s study also shows users can quickly overcome performance problems caused by violations in the layout. Thus further investigation is needed to better understand the effect that not conforming to visual conventions has on user performance. Even though many usability authors seem to recommend that following conventions is important for designing consistent sites, and they say that consistency can improve usability, it is first necessary to establish whether one is talking of internal of external consistency. But it is even more important to understand how conforming or not conforming to each conventional component of a genre (structural, linguistic, visual and functional) can impact the usability of a website. As seen above, some experimental results seem to indicate that altering expected position of elements (visual convention) does not seem to cause undue problems to users as they seem to overcome any performance hindrance quite quickly. However, the position of an element in a page is only one aspect of a set of visual conventions which can be formed for an interface. The manner in which an interface element is presented as well as the way in which its components are organized can also become a convention. This issue will be addressed in more detail in Chapters 4 and 5; what is important to note here is that changing the position of one element on a page is not sufficient to fully understand how not conforming to visual conventions can impact user performance. The empirical evidence from Vaughan and Dillon (2006) only indicates that when visual and structural conventions are violated simultaneously, users face a more serious usability problem than when the position of only one element is altered. However, it is hard to establish how much of the performance hindrance is caused by the violation of the visual conventions and how much is due to the violation of the 33
- 56. Genres, conventions and usability structural conventions. Further complications can be added by other factors such as the type of task which users are performing or how familiar participants are with the conventions of the site used (in this case online news sites). It is even possible that not conforming to visual or structural conventions have not much of an impact on user performance but rather the combination of the two violations create a synergistic effect which can bring performance down to a very low level. The impact of not conforming to conventions on usability becomes extremely relevant if the possibility that the impact of each conventional component (structural, linguistic, visual or functional) on usability has a different magnitude. For example, altering the conventional structure of a website can cause greater performance hindrance than altering its visual convention. This thesis will consider both external and internal consistency as a relevant aspect of usability, but its main focus will be on investigating the impact of not conforming to genre conventions (i.e. external consistency), more specifically visual conventions, on user performance. As visual conventions are the focus of this thesis it is essential that the relationship between the visual appearance of a web page and its usability be clear. 2.3.3. Visual appearance and Usability Inherent and apparent usability Some usability researchers (Fu & Salvendy, 2002; Kurosu & Kashimura, 1995; Tractinsky, 1997) consider that the usability of an interface has two different facets: one, called inherent usability, determines how easy it is to use the system. The other aspect, called apparent usability, establishes how easy to use users perceive the interface to be. Quantifying inherent usability may vary depending on how the researcher views usability (see 2.3.1); while a product-oriented view will have the inherent usability measured by how much the product conforms or not with the guidelines, a performance-oriented view will quantify inherent usability by measuring dimensions such as the time users take to complete a task and the number of mistakes made while attempting to complete it. Because apparent usability is dependent on the 34
- 57. Genres, conventions and usability perceptions of the user, it is usually measured through techniques like questionnaires and interviews. Usability and aesthetics The fields of Human-Computer Interaction and Usability have for many years considered the visual appearance of web pages of secondary importance and frequently recommended that usability should be put before aesthetics. But some studies have contradicted this belief and showed that the aesthetics of an interface is related to how easy to use users perceive it to be (Kurosu & Kashimura, 1995). Tractinsky (1997) says that aesthetics can be more influential on how easy or difficult users believe an interface to be than the inherent usability of the page itself. Kurosu and Kashimura (1995) examined how apparent usability relates to the inherent usability of a system. The authors tested 26 Automated Teller Machine (ATM) interfaces which varied apparent usability (screen layout) and inherent usability. To establish the levels of inherent usability of the layouts, the researchers used a product-based approach. A session with interface designers was held where the designers were asked to establish which factors they considered determinant in a usable interface. The design factors determined in the session were: ! Glance sequence: top-left corner is looked at first so the main display should be placed there. ! Familiarity: people are more familiar with telephone style keypad than calculator style. ! Grouping: keys should be grouped according to their function. ! Operation sequence: special numeric keys should be arranged as: ten thousands key first and the thousands key next to it. Also, the Yen key should be placed in the lower corner of the screen. ! Hand dominance: numeric keys should be placed at the right side of the screen. The conformance of each interface with each factor suggested by the designers was then measured for each layout and an inherent usability coefficient was established for each interface. Then 248 participants were asked to judge the 26 ATM layouts in terms of their beauty (aesthetics) and ease of use (apparent usability). Results showed a strong correlation between the perceived beauty of the page and the apparent usability. Surprisingly to the authors, no correlation was found between apparent 35
- 58. Genres, conventions and usability usability and the inherent usability of the interfaces. It is important to note, however, that the authors adopted a product-oriented view on intrinsic usability since they determined the inherent usability coefficient based on the attributes of the system rather than on user performance. Another possible way to determine the inherent usability of each interface would be to have different users perform tasks on each interface and establish a usability coefficient based on that. In another study which tested the robustness of Kurosu and Kashimura’s study, Tractinsky (1997) adapted the experimental material used by the previous authors to the Hebrew culture and replicated the experiment to some extent (Tractinsky improved some methodological techniques). Tractinsky obtained the same correlation between aesthetics and apparent usability as Kurosu and Kashimura did. Another interesting find from Kurosu and Kashimura (1995) which was confirmed by Tractinsky (2000) was that the only design strategy which showed a correlation with the apparent usability of the interface was familiarity with the keypad. This seems to indicate that users perceive familiar layouts as easier to use than unfamiliar layouts. Another study which investigated the contribution of apparent and inherent usability to user satisfaction when using web pages was conducted by Fu and Salvendy (2002). To vary the apparent usability of the test material the authors varied the background of the web pages and the layout of the interfaces. To quantify inherent usability the researchers used a mixed product and performance-oriented view. First the authors manipulated the interfaces by making one of them violate some usability guidelines and recommendations. For this the authors changed the path necessary to complete the task, how the products were classified and the type of feedback that the interface gave. Then authors measured time and error rate for each participant performing the experimental tasks. To establish a value for the apparent usability of the material, the authors asked a group of participants to rate how easy to use they thought each design was, based on the look of the design. Then, to validate the apparent usability the experimenters showed each participant three pages considered as having high apparent usability and three considered as having low apparent usability and asked participants to rate them in terms of (i) liking and (ii) ease of use. 36
- 59. Genres, conventions and usability Similarly to the previous studies described above, their experiment found no correlation between inherent and apparent usability. There was no significant difference in time to complete the tasks and the error rate between the interfaces with high and low apparent usability. But the study did find that participants who used websites with high apparent usability were more active and put more items in the shopping cart than those who used sites with low apparent usability. Another interesting result comes from studies which investigated post-usage perceptions of ease-of-use. Tractinsky et al. (2005) conducted a study to investigate if perceptions of ease of use would be changed after participants had used the interfaces. The results confirmed once more the correlation between aesthetics and apparent usability before using the interface. Interestingly, post-usage perceptions of usability remained related to the aesthetics of the page regardless of the inherent usability of the page used. Chawda et al. (2005) confirmed this correlation between aesthetics and usability judgements both pre- and post-use of the interface. Testing aesthetic perceptions before and after website use van Schaik and Ling (2009) found no positive correlation between usability (i.e. task performance and mental workload) and aesthetic perceptions. However, the authors found a negative correlation between aesthetic perceptions and the number of tasks participants completed. In other words, the higher the aesthetic perceptions of a page, the fewer number of tasks participants completed using that page. Finally, a recent study (Michailidou, Harper, & Bechhofer, 2008) which investigated the relation of visual complexity, familiarity and website aesthetics showed a negative correlation between complexity and aesthetic perception (the more complex a page is considered, the less attractive it looks). However, the study also showed that familiarity with a web page is the most influential factor on the aesthetic perception of the same page i.e. participants considered pages they were familiar with as more aesthetically pleasant than pages they were not familiar with (Michailidou, et al., 2008). If users are familiar with a web page genre they regard pages belonging to that genre to be more aesthetically pleasant than other pages with which they are not familiar (Michailidou, et al., 2008). If web pages which are more aesthetically pleasant are considered by users to be easier to use (Kurosu & Kashimura, 1995; Tractinsky, 1997), 37
- 60. Genres, conventions and usability then it is possible that familiar interfaces (i.e. convention-conforming interfaces) might be perceived as easier to use than unfamiliar interfaces. These studies show that the visual appearance of an interface is extremely important to usability, since it not only gives users an idea of how easy or difficult using the site might be, but it can even make users feel that an interface is easy to use regardless of its inherent usability. Another interesting aspect which needs further investigation is the relation between familiarity and apparent usability. 2.4. Conclusion Although usability literature frequently mentions the importance of keeping consistency, following conventions, and meeting user expectations, no connection seems to have been made with the concept of genre. Even though research has pointed out the emergence of digital genres (Eriksen & Ihlström, 2000), and the transference and subsequent evolution of existing paper genres to the World Wide Web environment (ISO/IEC, 1998), little is known of the effects of genre violations on web usability. In this subsection an attempt is made to examine how genres are linked to usability. As seen before, according to the ISO definition of usability, for an interface to be considered as having good usability, it needs to be efficient, effective and make its users satisfied. In other words, in a usable system, users should accurately and completely achieve the goals specified without using many resources. They should also feel comfortable and show positive attitudes towards the system (Lidwell, et al., 2003; Shneiderman & Plaisant, 2005; UPA, 2005). As seen earlier in this chapter, the concept of consistency can be associated with usability as when the conventions of a document genre are followed, the resulting document is consistent with the other existing documents of the genre. This can be thought of as external consistency. However, when it comes to internal consistency, following genre conventions does not necessarily imply that the resulting document will be internally consistent as documents which belong to a genre are not necessarily internally consistent. According to the literature, both internal and external consistency are constantly associated with good usability (Satzinger & Olfman, 1998) and it is therefore plausible to assume that breaking apart from the conventions of a genre will hinder the usability of a document. 38
- 61. Genres, conventions and usability External consistency is said to improve learnability since users who are familiar with a certain interface can transfer their knowledge when using another interface which is externally consistent with the one they already know. This, in turn, can affect the effectiveness of the system, as users will be less likely to make mistakes if all elements of the system behave as expected. It can also affect the efficiency if the elements and the needed information are found in the expected place, and look as expected. However, results from studies which have investigated the relations above have either been not very clear or sometimes contradictory. Conforming to visual conventions has been detrimental to user performance (measured as accuracy and time to complete tasks) when compared to not conforming to visual conventions of the modules of the system (Satzinger & Olfman, 1998). However the same study showed that following linguistic conventions in the same system has in fact improved user performance when compared to a linguistic convention violating interface (Satzinger & Olfman, 1998). In two other studies, not conforming to visual conventions by altering the placement of the navigational menu of a website either caused: a) no difference in performance (Kalbach & Bosenick, 2006) or b) a poor performance when using the non- conforming website (McCarthy, et al., 2003). Finally, not conforming to both structural and visual conventions of a website seem to have been detrimental to user performance (Vaughan & Dillon, 2006). However, the extent to which each type of non-conformance contributed to the decline of performance could not be determined. Thus the type of effects and relations between conforming and not conforming to each type of genre conventions and user performance still needs to be determined if designers are to be able to better decide whether or not to follow genre conventions. The next chapter describes two exploratory studies which tested structural and visual aspects of websites. These studies were the first empirical studies conducted in this thesis and they were aimed at informing future directions of the research. 39
- 62. Chapter 3 Exploratory studies 3.1. Introduction This chapter describes two exploratory studies which were conducted in 2004 to investigate structural and visual aspects of websites. The idea for these studies was to gain better comprehension about the two main aspects of genre conventions which are directly associated with design: structural and visual conventions. The outcome of these two studies will also provide directions for the research. The first study examines whether reading from a website where the content is organized by the use of conventional structures rather than an unstructured website helps users comprehend the text, and recall procedural information in the correct sequence. The latter can be an indication of the formation of a schematic knowledge of the website structure by users. As seen in Chapter 2 the development of schematic knowledge is one step in the formation of user expectations, which are directly associated with genre conventions. The second study investigates if there are certain ways of visually organizing elements on a web page which can help users quickly identify the category of a website. This can indicate that the expectations that users have about a certain category of website are not restricted to the structural aspect of the site but also include its visual appearance. 40
- 63. Exploratory studies 3.2. Exploratory study 1 - The effect of structured design on comprehension and structured recall of electronic documents 3.2.1. Background When looking at a document genre, readers have expectations regarding not only its discourse and writing style, but also about the organization of the information within that document. These expectations are based on a mental schema built on previous experiences with that particular genre. This schema, called superstructure by van Dijk and Kintsch (1983), is responsible for giving the reader an idea of the organization of the global text content. As seen previously in section 2.3.2 of Chapter 2, a proposal about the formation of superstructures was made by Vaughan and Dillon (1998) where repeated exposure to documents with similar characteristics allows users to develop a structural knowledge of the document. When this structural knowledge is combined with spatial memory for layout it can facilitate searching and browsing the material (Dillon, 1991). This can be explained by the theory pointed by van Dijk and Kintsch (1983), that the relevant superstructure of a document genre is activated in the memory of the reader as soon as a cue is found in the text. However, the authors give no indication as to the nature of the cue (e.g. semantic, syntactic or graphic). Many cues used to organize texts and indicate ideas are (usually) semantic or syntactic, and therefore outside the scope of this research. There are, however, graphic cues which are also used to organize textual content. These cues usually consist of typographic elements and visual attributes which can be used to link and relate ideas within the text and to indicate thought units. Headings, subheadings and colour can be used to establish relationships between textual information, and spacing can indicate thought units (Dickson, Simmons, & Kameenui, 1995). These typographic elements, called access structures by Waller(1979), are instruments that help the reader gain access to the text using both spatial and typographic cues to group or order pieces of text (Hartley, 1987). Many studies have investigated the effects of access structures on reading and comprehension of texts both in print and on screen. Although this research will be primarily focusing on reading from screen, some results of print research will also be examined since they can provide relevant information. 41
- 64. Exploratory studies 3.2.2. Access structures & typographic cues Colour Colour can be used to express organization and structure within a text (Hartley, 1978). In electronic text, colour can be used to indicate categories of importance and can be used to convey organization and structure (Hartley, 1994, p. 158). But because there does not seem to be a range of colours which is intuitively associated with a hierarchy of importance, using colour to convey structure can be difficult (Hartley, 1994) despite that, some authors recommend using colour to support the spatial organization of text is recommended (Hartley, 1987, p. 68). In this exploratory study, colour will used to support the organization of the textual content. As recommended by Hartley (1987, p.70) colour will be used in headings to reinforce the organization of the textual content. Spacing Another very important typographical feature that helps organize information on screen is spacing, since it can reveal the text structure at a glance (Hartley, 1987, p.67). However, inappropriate spacing can lead to confusion and hinder comprehension (Hartley, 1987, p.67). Text spacing can be divided into four main types: inter-linear spacing, word spacing, character spacing and margins. The effects of some of these types of text spacing on reading performance were examined by Chaparro, Baker, Shaikh, Hull, and Brady (2004). The authors compared four layouts with different values for margins and inter-linear spacing. The values ranged from ‘Margin’ (with 10mm of space surrounding the text) to ‘No margin’ (with 2mm of space surrounding the text), and ‘Optimal inter-linear spacing’ (with 5mm between lines of text) to ‘Sub-optimal’ (with 4mm between lines of text). The typeface and font size remained constant throughout the experiment. The results showed that text margins affect reading speed and comprehension since participants read slower with the ‘Margin’ condition, but comprehended more than with the ‘No-margin’ condition. Considering that if type size is kept constant within an experiment, then the margin size bears an inverse relation to the number of characters per line and consequently line length. Thus, the differences in line length between the ‘Margin’ condition and the ‘No-margin’ condition in Chaparro et al. (2004) might have accounted for the observed differences in comprehension. This 42
- 65. Exploratory studies result seems to agree to a certain extent with the results from Dyson and Haselgrove (2001) that readers' comprehension declined as line lengths increased. Furthermore, Chaparro et al. (2004) found that inter-linear spacing did not seem to influence reading performance. This could have been caused by the small differences in inter-linear spacing between the ‘Optimal’ and ‘Sub-optimal’ conditions. Although not particularly studying space as a conveyor of text structure, the study conducted by Chaparro et al. (2004) indicates that space plays an important role in improving reading comprehension. The results from Chaparro et al. (2004) also illustrate a trade-off between reading speed and comprehension that can occur when testing for these variables (Dyson & Haselgrove, 2000, 2001; Muter & Kruk, 1984; Muter, Kruk, Buttigieg, & Kang, 1988). Considering that the possibility of trade-off exists, to get a broader picture of the performance, both comprehension and reading speed will be measured in the following experiment. One other study that tested how the amount of space affected the process of finding information on screen was conducted by Bernard and Chaparro (2000). In this experiment three mock websites with the same content but three different layouts were examined. Each layout varied the amount of space between chunks of text: The first layout (called low spacing - Figure 7) had only 3mm of space between columns of text and no space between paragraphs. The second layout (called medium spacing- Figure 8) had 9 mm between each column of text and between paragraphs, and the third layout (called high spacing-Figure 9) had 19 mm between each column and between paragraphs. 43
- 66. Exploratory studies FIGURE 7 - REPRESENTATION OF THE LOW SPACE CONDITION (BERNARD ET AL., 2000). FIGURE 8 - REPRESENTATION OF THE MEDIUM SPACE CONDITION (BERNARD ET AL., 2000). 44
- 67. Exploratory studies FIGURE 9 - REPRESENTATION OF THE HIGH SPACE CONDITION (BERNARD ET AL., 2000). Sixteen participants were asked to perform search tasks that were randomly assigned to them in each layout condition. Each participant answered 5 task questions in each condition so that they were in contact with all three layouts. After each task the participants were asked to rank the difficulty of finding information and after performing all the tasks they were asked to rank their preference to each layout. No significant differences in search time and search errors were found between the three spatial layouts. However, the fact that the content was the same in all three layouts and each participant interacted with all three layouts could have caused a carry-over effect to happen. In other words, the knowledge acquired by using one layout was transferred to another. In the preference assessment, participants responded that they did not like websites with low levels of space and preferred sites with medium levels of space. As well as this, despite the fact that this experiment did not separate the effects of paragraph spacing from column spacing, the results suggest that using some space after paragraphs seems to increase user satisfaction. This relates to the idea that the underlying structure of a text can be made more evident to the reader by using space consistently to separate elements of a text such as headings, paragraphs and sentences (Hartley, 2004). 45
- 68. Exploratory studies Considering that paragraphs are the smallest text elements in which one can express a complete thought (Peels, Janssen, & Nawijn, 1985), and that space can be used to indicate thought units (Dickson, Simmons, & Kameenui, 1995), it is not surprising that separating text paragraphs with space can improve reader satisfaction (Chaparro, et al., 2004). Headings The main function of headings is to group and sequence large segments of text therefore communicating its structure. Research has shown that headings facilitate recall of unfamiliar topics (Lorch & Lorch, 1996), and help users find and extract information from a text (Hartley, 1987, 2004). Many experiments have been carried out both in print and on screen to test variables relating to headings (Baker, Bernard, & Riley, 2002; Hartley & Trueman, 1985; Lorch & Lorch, 1996; Schultz & Spyridakis, 2004; Williams & Spyridakis, 1992) ranging from their position in the text, to their form (statements or questions) and to the task performed (Hartley & Trueman, 1985). Regarding the use of headings on computer displays Hartley and Jonassen (1985) mention that articles designed for screen display should be menu-driven and suggest that a content list of headings should appear at the beginning of the document. The influence of headings on printed text recall and summarization was evaluated by Lorch and Lorch (1996) throughout two experiments. The first experiment verified the effects of headings on a free recall task. After reading the texts, one condition with headings and the other without, the participants were asked to do a written recall test. The second experiment checked if the presence or absence of headings would influence the inclusion of topics in a summary. After reading the two texts, one layout condition with headings and the other without, the participants were asked to write a 15-sentence-long draft of the summary whilst referring to the text as much as they liked. This summary limitation was used to control variability in the data and to make participants selective in what they included in the summary. The results obtained showed that the presence of headings on printed text helped recall of unfamiliar topics but not of familiar ones. The authors also found that headings and familiarity with a topic had a great influence on whether or not that topic would be included in a reader’s summary of the text. 46
- 69. Exploratory studies On a study to verify which physical attributes of headings were better perceived by readers, Williams and Spyridakis (1992) varied type size, position, underlining and case (upper or upper-and-lower) on printed material. The results showed that relative differences in type size provided the most discernible visual cues to hierarchy for the readers. Another study, by Baker, Bernard and Riley (2002) evaluated the effects of headings and summaries on user satisfaction and task completion time when reading from screen. This was done by creating three different online news presentation formats with twelve articles taken from the New York Times website. The three layouts were described by the authors as: ! Full – containing full texts of all twelve articles. ! Summary – displaying link titles of all twelve articles (headlines) plus small abstracts. ! Links – displaying links to titles of all twelve articles (headlines) only. All participants were asked to perform ten different search tasks within the news articles using the three different layouts. After the tasks were completed, participants answered a 6 point Likert scale satisfaction questionnaire comprised of the following questions: "The layout made it easy to find information", "This site was visually pleasing", "The arrangement of this site promotes comprehension", "I am satisfied with this site", and, "The layout looks professional” The authors found no significant difference in the time taken to complete the tasks in the three layouts. However, the summary layout condition was considered the most easy to find information in and was also the one participants preferred. The lack of significant differences in task completion time could be explained if the headings (headlines) contained the key words necessary to finding the information. Because all three conditions had the headings this could have levelled all three layouts making performance equivalent in all three versions. The effects of headings, indentation and image placement on reading performance, comprehension and participants’ satisfaction were measured by Chaparro, Shaikh and Baker (2005). The authors compared the effects of an enhanced screen layout with a poor screen layout. The enhanced layout was designed to include optimal use of the headings, indentation and image positioning. The participants were asked to read three passages in each of the two layouts with the order in which each layout 47
- 70. Exploratory studies was presented chosen randomly for each participant. Each task was timed, and after reading each passage the participants were asked to answer eight comprehension questions on the passage. After reading the three passages from one layout each participant completed a questionnaire on their reading satisfaction and the same procedure was repeated for the other condition. Finally, after completing the tasks on both layouts the participant was asked to decide which layout they preferred. The difference between the number of words read per minute and comprehension score on the enhanced layout compared to the poor layout was not statistically significant and the authors concluded reading performance or comprehension was not influenced by the quality of the page layout. Unfortunately, the authors do not provide details on the criteria used to determine the characteristics of a good layout, and on examining the examples of the page layout provided in the article there is not much to differentiate both conditions except for the heading positioning and text wrap around pictures (Figure 10). One possibility for this lack of differences in reading performance and comprehension across layouts may have been due to there being little differentiation between the poor and enhanced layouts used by Chaparro et al. (2005). FIGURE 10 - ENHANCED PAGE LAYOUT (LEFT) AND POOR PAGE LAYOUT (RIGHT). Another possible explanation for no differences in the study may be due to how the authors structured the text. Because in both layouts most of the structuring of the text 48
- 71. Exploratory studies was in the same manner (i.e. contained the same access structures in the same parts of the text) and considering that some access structures (such as headings) help users find, extract (Hartley, 1987, 2004) and recall information (Lorch & Lorch, 1996) from a text it is possible that this have accounted for a lack of difference in what participants could remember. Eveland, Cortese, Park, and Dunwoody (2004) suggest that studies which evaluate the effects of information organization on the content of memory and not on the associations of different bits of information in memory are limited. According to the authors, it is necessary to extend measures of learning when looking for the effects of changes in layout beyond recall, to include knowledge structure or organization. As will be explained in the next subsection, knowledge of the way in which ideas, stored in the memory are interconnected is essential to the comprehension of these ideas and ‘problem solving depends on adequate structural knowledge of the ideas in the domain being explored’ (Jonassen, Beissner, & Yacci, 1993). In other words, the information stored in the memory must be adequately structured and connected for it to be useful in problem solving. The recall of stored information and some basic assessment of knowledge structure will be carried out in this experiment. More specifically, the ability of readers to recall information in the same order in which it is presented in the text will be investigated. This is especially important when procedural information is presented, such as assembly instructions, equipment operation instructions, software and hardware installation procedures and even cooking recipes. This type of recall will be referred to here as structured recall. Structured recall can also be an indicator that the structure of the information has been learned adequately and can be recalled correctly when needed. 3.2.3. Types of knowledge Knowledge is usually divided into two distinct but related categories: declarative and procedural knowledge (Jonassen, et al., 1993). Declarative knowledge, also known as factual knowledge (Eveland, et al., 2004), represents the consciousness of a concept, idea or object. However, the fact that people hold some declarative knowledge does not necessarily mean that they are able to use it since declarative knowledge does not by itself imply knowledge. 49
- 72. Exploratory studies Procedural knowledge describes the schemata through which the declarative knowledge is used to solve problems, form plans, and process arguments amongst other procedures (Jonassen, et al., 1993). During the learning process, information is stored in packets in the memory of the reader and relationships between the new information and existing information are established. So when faced with a problem, a person must retrieve from the declarative knowledge the relevant content pertaining to a particular schema in order to solve that specific problem. Since these two components of knowledge seem insufficient in explaining the whole process of storing and retrieving information from memory, Jonassen et al. (1993) propose an intermediate type of knowledge known as the structural knowledge, the function of which is to mediate the relations between procedural and declarative knowledge. Structural knowledge defines how the concepts of a particular domain are interrelated, and it manages and defines how declarative knowledge contents are connected in the memory. It is likely that the superstructural schema which users of a genre possess and use confronted with a document belonging to a genre (Dillon, 2000) is related to structural knowledge. And because structural knowledge defines how the concepts in memory are interrelated and connected, its existence can be evidence that not only factual but also procedural knowledge exist. Assessing structural knowledge Most comprehension assessments used in reading experiments simply ask participants to retrieve information from memory, and consider that the more correct information a person retrieves, the more that person is thought to have learned. In other words, these types of assessment only test declarative knowledge, and when structural changes to the text are part of the experiment, they fail to check the impact of these organizational changes in structural and procedural knowledge. Multiple-choice questions that merely present exact sentences extracted from the text and ask participants to compare it with their stored declarative knowledge are simply measuring one aspect of knowledge, and one that does not necessarily mean understanding. Therefore, experiments that evaluate the effects of text structure (e.g. use of access structures versus no access structures) on comprehension should present respondents with questions that encourage the use of both declarative and 50
- 73. Exploratory studies procedural knowledge on their tests. This way a better evaluation of acquired knowledge can be achieved. 3.2.4. Objectives This experiment is designed to investigate if organizing the contents of a web page through the use of conventional access structures can improve reading speed, comprehension and structured recall. This could give indications as to whether structuring web content in a conventional way can help with the generation of mental schemas as opposed to leaving the content unstructured. 3.2.5. Method Experimental texts The textual content chosen for this study was extracted from a book on wine, and it can be divided into two basic parts. The first one is procedural, and indicates the process of winemaking from the picking up of the grapes until the fermentation of the juice into wine. The second part, more hierarchical in nature, describes the categories of wine, from the lowest quality up to the highest quality wines. A sample of the text as it was structured in the book is presented in Appendix A. Two text layouts were produced with this content for this experiment. One is an unstructured text layout (from now on referred to as the unstructured layout – Figure 11); the other (from now on referred to as the structured layout -Figure 12) is laid out to present information in a structured manner allowing the participant to read either in a linear or non-linear form through the use of hyperlinks. The structure is indicated by using headings and subheadings, different typefaces with different type sizes. Colour is used consistently to indicate the organization of the text, as headings and subheadings which relate to the process of winemaking are represented in one colour (purple) and the ones which relate to the quality of the wine in another colour (green). Because of the left sided navigational menu, in the structured layout the left margin in this layout is larger than in the unstructured version. Consequently line length in the latter is longer than in the former. Line spacing, character spacing and paragraph spacing are the same in both versions. But in the structured layout only one section 51
- 74. Exploratory studies (or subsection) is visible on the screen at one time, whereas in the unstructured layout the whole text is visible. FIGURE 11- SAMPLE OF THE UNSTRUCTURED TEXT LAYOUT USED INT THE EXPERIMENT. FIGURE 12 – SAMPLE OF THE STRUCTURED TEXT LAYOUT USED IN THE EXPERIMENT. 52
- 75. Exploratory studies Through the use of the menu in the structured layout, the reader was allowed to move freely through the text topics in any order. Comprehension and Recall Assessments Two methods for assessing comprehension were chosen for this experiment: a summarization task and a multiple-choice test about the text. Summary Task The summarization task was open-ended and asked each participant to summarize what was read during the experiment. A time limit of 5 minutes to complete the summary was established to encourage the participants to write a short piece, be selective (Lorch & Lorch, 1996) and to keep participants from getting too tired. To evaluate the written summaries of the participants, three main criteria were defined. If all three criteria were met, the participant was considered as having achieved maximum comprehension. ! General comprehension If the reader shows a general idea of what the text is about, it is assumed that they acquired a general comprehension of the text. To be considered as having grasped a broad idea of the text, the participant has to write at least that the text is about winemaking and wine classification. ! Structured recall If the reader can recall the sequence of events as they appear in the text, this counts as evidence of structured recall. To be considered as having achieved that, the participant has to write about all the following topics in this particular order: grape crushing, fermentation, ordinary wine, vin-de-pays, good wine, fine wine and great wine. ! Further comprehension If the reader can recall any detail from the text, this is considered as having obtained further comprehension of the text. Any specific detail that does not give a general idea of the text and does not express a sequence of events as they appeared in the text indicates that the reader has acquired extra information from the text. Any of the three criteria can appear independently in the summary, although it seems odd that a participant might achieve ‘Further comprehension’ of the text without achieving ‘General comprehension’. 53
- 76. Exploratory studies Multiple-choice The multiple-choice questions are designed to assess the text comprehension and structure recall. As can be seen in Appendix B, the assessment consists of 6 questions, and although some measured more than one particular item, they can broadly be categorized as follows: two measured recall of specific information from the text (questions 1 and 5), two measured the structural recall by asking the participants to recall the order of appearance of sentences, winemaking processes or topics from the text (questions 2 and 3) and two measured if the reader could arrive at conclusions based on what they read in the text (questions 4 and 6). The order of questions was randomized for each participant. 3.2.6. Study design This exploratory study was designed to have two independent groups of participants. Each group was tested on one version of text layout (structured or unstructured). 3.2.7. Participants Ten participants volunteered for this study. They were randomly allocated to one of the two experimental layouts so that each participant read the text from one condition only. Five participants were tested on the structured layout and five on the unstructured layout. Four participants were women and six were men. All participants were fluent in English: two of the participants were native English speakers and eight had English as a second language. All the participants were graduates and 90% used the Internet on a daily basis. It is relevant to note that after randomization each native English speaker was assigned to one of the conditions. Table 2 illustrates the gender distribution among conditions. TABLE 2 - PARTICIPANTS' GENDER ACROSS CONDITIONS. Gender Male Female Unstructured Condition 3 2 Structured Condition 3 2 54
- 77. Exploratory studies Participants’ Internet Usage Profiles An Internet usage questionnaire was applied to users before the test to assess their level of familiarity with navigating the Net and using hypertexts. Nine of the participants said they used the Internet on a daily basis and one used it every other day. Only two participants used the Internet for less than one hour every day and the majority (six) used it from 1 up to 5 hours. Two participants said that they used the Internet on a daily basis for more than 5 hours. All the participants used search engines and sent and received e-mails. More than half of them used bank facilities online. The two least common activities were shopping/comparing price information and playing games online. 3.2.8. Experimental Procedure Participants were asked to read a printed text which described the experiment and told to adjust the equipment (i.e. mouse, keyboard and computer screen) and the height and position of the chair to their liking. Participants were told to read the text at their own pace and to do it as carefully and as accurately as possible. The importance of reading the whole text was reinforced and participants were informed that they would be answering questions on it later. It was also emphasized that the participants should signal to the experimenter when they started and ended the reading task for timekeeping purposes. Each participant was observed and notes were taken on their interactions with the interface, such as the use of the mouse, scrolling patterns, text positioning and of any comments they made. The participants in the structured layout group were also observed for the reading sequence they chose. When the reading task was completed, each participant was asked to write a short summary about what they had just read and were given a maximum of 5 minutes to do this. They then had to answer 6 multiple-choice questions about the text, and answer a questionnaire about their Internet usage habits (see Appendix B). 55
- 78. Exploratory studies 3.2.9. Results Reading time The mean reading time for the unstructured layout was 613s and for the structured layout was 669s. A Mann-Whitney non-parametric test adjusted for ties found that the time to read the texts on each layout were not significantly different (W=35, n=5, p=0.676). FIGURE 13 - MEAN READING TIME WITH STANDARD ERROR BARS FOR THE TWO LAYOUTS. Multiple-choice Questions The table below displays the outcome of the multiple-choice questions for each condition. 56
- 79. Exploratory studies TABLE 3 - TABLE WITH THE CORRECT ANSWERS FOR THE MULTIPLE-CHOICE TESTS BY LAYOUT (U.L. = UNSTRUCTURED LAYOUT AND S.L. = STRUCTURED LAYOUT). correct answers Question U.L. S.L. Tick the option containing countries where you still find the traditional method of 1 5 5 treading grapes by bare feet being used. 2 Tick the option which shows the right order of processes necessary to make wine. 2 5 Number the sentences according to the order in which they happen during the winemaking 3 0 2 process. 4 Why is it so important not to crush the grapes’ pips during the process of pressing them? 2 1 5 Tick the option in which all the names refer to ordinary wine. 2 4 6 Assign (T) for true or (F) for false to each of the sentences below 6a Good wine is usually an “ordinary” wine with a birth certificate 2 4 6b Usually the yeast cells can work up to a 15% concentration of alcohol 1 3 6c The world’s largest wine presses are located on small farms in Chianti. 4 3 6d Not many regions of the world have been able to produce great wine regularly 3 5 6e Fine wine is the rare and exquisite result of perfect conditions in a perfect vineyard. 3 1 Total per group 13 16 Two of the multiple-choice questions (numbers 3 and 6) allowed for more than one answer: one asked the respondent to order the sentences and the other to establish if a sentence was true or false. In the true or false question (number 6), each of the 5 sentences were marked individually, and in the sentence ordering question (number 3) the criteria used to mark these questions was that participants had to get the complete sentence order correct in order to be considered as having answered the question correctly. Participants who read the text through the unstructured layout achieved a mean of 4.8 out of 10 possible correct answers, whilst participants who read the text through the structured layout had a mean of 6.2 out of 10 possible correct answers. A Mann- Whitney test adjusted for ties revealed that this result was not significant (W=23.0, n=5, p=0.459) and therefore there was no evidence that layout affected comprehension. Although the difference in score is relatively high, the lack of significance of the result might be due to the small number of participants in this experiment, the small number of questions and high variability of the results. 57
- 80. Exploratory studies FIGURE 14 - MEAN MULTIPLE-CHOICE SCORES FOR BOTH LAYOUTS. Structured Recall Two of the multiple-choice questions (2 and 3) required not only recall of the text content but also required structured recall from the reader. Participants who read the structured layout text got more of those questions right (Mean=1.4) than participants who read the text from the unstructured layout (Mean=0.4). A Mann-Whitney test adjusted for ties showed that this difference was statistically significant (W=18, n=5, p=0.041) and therefore there is indication that the differences in structured recall scores were due to the differences in the layouts. However, due to the small sample size and reduced number of structural questions further investigation is required. FIGURE 15 – MEAN MULTIPLE-CHOICE STRUCTURE QUESTION SCORES. 58
- 81. Exploratory studies Summary Task In the summary task, participants who used the structured layout wrote an average of 71 words against an average of 61 for participants of the unstructured layout group. General comprehension All participants in both groups showed a general comprehension of the text since all participants wrote in the summary that the text was about winemaking and wine classification. As a result the general comprehension score was 5 for both groups. Structured recall In the summary, four participants who used the structured layout described the sequence of topics as they appeared in the text correctly. Only one participant from the unstructured layout wrote correctly the sequence of topics as they appeared in the text and one participant, although not mentioning specific topic names, demonstrated having conscience of the relative position of each topic in the text. Further comprehension Only two participants in the unstructured layout group included details from the text (e.g. names of vinification processes, countries, wine classification names, measurements or quantities) in the summary. Three participants who used the structured layout included details from the text. Overall comprehension (summary and multiple-choice) All ten participants demonstrated having acquired a general idea of the text subject. This can be concluded by the lack of significant differences in scores both in the multiple-choice questions and the summary task. 59
- 82. Exploratory studies TABLE 4 - COMPARISON OF OVERALL COMPREHENSION RESULTS ACROSS CONDITIONS. Multiple-choice Summary Task (mean scores) (out of 5) U.L. 4.8 5 S.L. 6.2 5 Observations Unstructured layout One participant placed the mouse pointer at the beginning of each line that was being read. Generally, most of the participants scrolled the text one paragraph at a time and usually placed the paragraph they were reading at the top of the screen. Structured layout One participant read the text not following the order established by the navigation menu. Although indicating the structure in the summary, the participant only got one multiple-choice structural question correct. Two participants did not read the first link (How Wine is Made) and one of them also missed the fourth link (What is Great Wine). Three participants, while reading the text, left the mouse pointer on the menu indicating the last clicked link, and two used the mouse pointer to help read the lines (using it as people use their fingers while reading). 3.2.10. Discussion Comprehension The results suggest that the difference in layout does not seem to influence reading time. Having a structured layout does not seem to influence comprehension and consequently declarative knowledge. Looking at the scores from the multiple-choice questions and the summary task, it can be observed that participants who read from the structured layout achieved higher scores than participants from the unstructured layout group (Table 4). However, this result was not statistically significant for the multiple-choice test and there was no score difference in the summaries written by the participants. Although it cannot be assumed that the use of headings and colour-coded menus to structure a 60
- 83. Exploratory studies text had any effect on the text comprehension, the lack of significance in the results might be due to the reduced number of participants in this exploratory study. It is possible that a study conducted with a larger sample might reveal a significant difference in this aspect. Another factor which may have contributed to the lack of significant differences is the fact that paragraph spacing was used on the unstructured layout may have given readers some idea of the text structuring, thus influencing how they understood it (Hartley, 2004, p.922). Structured recall The difference in the multiple-choice structured recall scores between the two groups was significant. The participants who read from the structured layout answered more structural questions correctly than the participants from the unstructured layout group. This result was echoed in the summarization task where four participants who read from the structured layout mentioned correctly the sequence of topics as they appeared in the text, against only one participant who read from the unstructured layout. This is strong indication that the use of colour-coded headings to structure and organize a text helped participants to recall information in the sequence it appeared in the text. Remarks on the methods used Observations Observing participants throughout the experimental tasks has revealed some interesting patterns such as using the mouse pointer to indicate the word being read on a line. This suggests that the readers may have had some trouble following lines with their eyes, and probably returning to the beginning of the following line. Another valuable observation was that two participants missed the first link to text (represented by the heading ‘How Wine is Made’ on the menu) and one of them missed another link to the text ‘What is Great Wine’. This might have been due to the fact that this heading was on the top left of the page where the logo of a website is usually located. Therefore, participants may have interpreted it as the ‘title’ or ‘name’ 61
- 84. Exploratory studies of the page. Yet another explanation is that participants perceived those links as a section header devoid of any content used only to group the links. Summary Task The summarization task was an important tool not only for encouraging participants to read the text attentively, but also for indicating differences between conditions. However limiting the time each participant had available for this task prevented some participants from writing more in their summaries. Perhaps combining a time limit with the number of sentences as suggested by Lorch and Lorch(1996) might encourage participants to write more on the summary task. Multiple-choice questions on comprehension Although results of the multiple-choice questions on comprehension showed no significant result, they did encourage participants to read the text carefully. The lack of significance was probably due to the small number of questions (only 6). In future research, a greater number of questions will be used. Structured recall assessment Despite the fact that the method used to assess structure recall provided valuable results, different and more precise techniques of tackling this particular aspect of knowledge might be developed for future research. Other issues It would also have been interesting to have some reader preference test to evaluate both conditions and to get a view of participants’ impression of each layout. Knowledge assessment The comprehension assessments from this experiment dealt solely with two types of knowledge: declarative and structural. Regarding structural knowledge, only a very simple and superficial method of structural recall test was applied. Future research might improve on these methods of assessment of structural knowledge and declarative knowledge. 62
- 85. Exploratory studies 3.2.11. Conclusion Results from this exploratory study seem to indicate that using conventional access structures such as headings, and subheadings along with colours appears to help readers of digital documents organize and better retrieve sequential information. This indicates that these conventional structures may help readers to store the read information in an organized manner or, at least it may help them retrieve this information in an orderly fashion. Furthermore, when it comes to web genres there is another aspect of content structuring which is worth investigating. Users who belong to the discourse community of a particular web genre expect the content of a website from that genre to be organized in a certain way. This involves not only using the access structures conventional to that genre, but is also related to how the content is distributed throughout the website. 3.3. Exploratory study 2 - Visual appearance and the identification of the category of a website. 3.3.1. Introduction From the previous experiment it was gathered that presenting text in a structured layout text can improve the structure recall of the textual information by the readers. The structuring of the text is usually achieved by the use of visual elements such as headings, colours, paragraph spacing, typeface, type size and icons. To understand a text, readers use three different types of cues they find in texts: graphic, semantic and syntactic (Dillon, 2000; Loines, 1991; Stamboltzis & Pumfrey, 2000). Graphic cues (e.g. levels of headings, colours, typeface and type position) provide the reader access to the information (Loines, 1991), pointing out what is relevant or not. Syntactic cues define how the information in the text is related, and semantic cues help readers recover the meaning of the text (Loines, 1991). As mentioned in Chapter 2, whenever approaching a document, readers have some expectations of how it will be structured and what the organization of its key elements will be (Dillon, 2004). These user expectations are generally based on previous experiences with certain ways of organizing and laying out a particular type of document which over time has become standardized. These standards have 63
- 86. Exploratory studies given rise to highly conventional document forms that are instantly recognized as being part of a genre (Dillon & Gushrowski, 2000). The formation of these genres according to Dillon (2002) is usually thought of as a very slow process, taking sometimes decades for a genre to emerge. However, some authors argue that, that at least for web genres, this process seems to occur faster (Crowston & Williams, 2000; Dillon, 2000; Dillon & Gushrowski, 2000; Shepherd & Watters, 1999). This idea is backed up by some research that has reported the emergence of genres exclusive to the web such as the personal homepage (Dillon & Gushrowski, 2000) and weblogs (Herring, Scheidt, Bonus, & Wright, 2004). Dillon and Gushrowski (2000) point out that when it comes to digital documents, many existing paper conventions are adopted (e.g. online newspaper and magazines). But as digital documents are an entirely different media from paper ones, the demands involved in accessing information on screen cannot be solved by simply transporting a printed genre to the electronic medium (Hartley & Jonassen, 1985). It seems that readers use their ‘knowledge of genre conventions and grammatical rules to make predictions about the text in order to comprehend it’ (Loines, 1991). If the layout of a text conflicts with the expectations of the reader then it will increase their uncertainty by inhibiting the understanding of the meaning of the text (Loines, 1991). In the digital domain, genre violation seems to leave the reader without an adequate representation of the text, confusing them so that they rely on navigation buttons and links to locate themselves (Dillon & Vaughan, 1997). As many of the genres found in electronic documents seem to follow genres from print they are regarded by some authors (Dillon & Gushrowski, 2000) as inappropriate for the digital media. These authors believe that only when genres are bred within the digital space they will be adequate for electronic documents. This statement, however, is quite restrictive and it does not observe that even though transferring a paper based genre to the digital space may result in the short term in inadequate documents, since each media offer different functional constraints. But, if this transference is considered only a step in the evolution of a genre towards a new media, then it is also reasonable to think that the resulting evolved genre will be as 64
- 87. Exploratory studies adequate to the new media as a genre bred in the digital space. In fact, this point of view is shared by other authors who see this transference of a paper genre to the digital domain as a step in the natural evolution of a document genre (Eriksen & Ihlström, 2000; Santini, 2007). In search of an emerging truly digital genre, Dillon and Gushrowski (2000) considered the personal homepage as a strong candidate since it has no paper equivalent with little or no possibility of acquiring conventions from print. The authors looked at over 100 homepages and identified great similarities between them. Sampling the pages, a list of elements was identified and their frequency of occurrence was noted. Then eight experimental homepages were created using these elements (the five most and least occurring plus the remainder). Participants were then asked to rank these pages in preference as a home page design, and then to look at the list of elements made by the authors and select the ones they thought should be included in a good personal homepage design. Participants were free to add other elements they felt were relevant to the list. The results showed that users broadly agree on the contents of a personal web page and they generally reflect the existing home page contents on the web. The preferences and expectations across the groups of participants suggest that the personal home page might be the first unique digital information genre. This being true, genre emergence could be happening faster than initial studies from paper suggested (Dillon & Gushrowski, 2000). Some questions arise from these results obtained by Dillon and Gushrowski (2000) given that genre emergence might be happening faster than was expected; are there other genres on the Web? What are the visual elements that compose them?To answer the first question, it is necessary to categorize the types of existing websites currently present on the internet. According to an Internet Activity Index published by the Online Publishers Association (OPA, 2005), Internet usage can be divided into the four following activities: ! Content - websites and Internet applications that are designed primarily to provide news, information and entertainment. Examples of those included in this class are CNN.com, ESPN.com, Windows Media Player and MapQuest. 65
- 88. Exploratory studies ! Communications - websites and Internet applications that are designed to facilitate the exchange of thoughts, messages, or information directly between individuals or groups of individuals. Examples of those included in this category are Yahoo! Mail, AOL Instant Messenger and MSN Groups. ! Commerce - websites and Internet applications that are designed for shopping online. Examples of those included in this category are Amazon, eBay, Shopping.com and Dell.com. ! Search - websites and Internet applications that scan the Web to provide prioritized results based on specific criteria from user-generated requests. Examples of those included in this category are Google Search, MSN Search and Yahoo! Search. Apparently, the OPA classification seems to miss one online activity which McGhan (2005) reports to have increased in usage in the U.S. since 2002: online banking. Whether or not these categories have indeed already bred a digital genre is something that remains to be evaluated. 3.3.2. Objective The objective of this exploratory study is to investigate whether web users will be able to determine the category of a website (based on the type of service offered) by only examining the visual appearance of the page. Knowing that a document which complies to a set of conventional forms can be instantly recognized as belonging to a certain genre (Dillon & Gushrowski, 2000; Waller, 1987b), the study will aim to find out if any of these identifying elements are part of a conventional form, thus verifying if there are any genres in the process of formation. 3.3.3. Method Based on the OPA (2005) and McGann (2005) reports, six categories of online activities were defined for this study; webmail (an Internet interface that allows users to read and write e-mail using a Web browser), online news, search engine, e- commerce, online banking, and an online auction. Website pages which illustrated the service offered by the website (e.g. a balance screen for online banking or an inbox for webmail) were captured from the Internet. 66
- 89. Exploratory studies The total number of pages captured was 20 and they were arranged in the categories as illustrated in Table 5. The decision to use captured screens instead of designing them was made because the study aimed at investigating if Web users were able to identify real-life websites and designing them could create constructs which were not representative of existing web categories ad consequently not be properly identified. TABLE 5 – NUMBER OF DIFFERENT SCREENS CAPTURED IN EACH CATEGORY. Category Nº of screens Web mail 3 screens Online news 3 screens Search engine 3 screens E-commerce DVD 3 screens Supermarket 3 screens Online banking 2 screens Online auction 3 screens The supermarket and DVD selling sites were grouped together under the e-commerce category because both have selling products as their main objective and do this in a similar manner. Although an online auction website also sells products, the selling process occurs in a different manner to the e-commerce websites, thus it was placed in a different category. The smaller number of online banking screens was due to the difficulty of finding available bank accounts the screens of which could be captured. To keep the participants from using the logo of the website as a cue, each screen had its logo edited so that identification would not be possible, and references to the name of the website written on the screen were blurred so that they could not be read. But even with the logos removed, participants could still look for words within the page to help them with guessing the category of the website. Considering that the study was investigating visual appearance and not semantic or syntactic cues the possibility of participants gathering cues from text had to be minimized. The option of using placeholder text would mean altering all text in the 20 website screens which would be extremely time-consuming. The alternative decision of displaying each screen for an extremely short period of time (1 second) was made. This way, participants would not have much time to look for words and use them as cues and if within this short time frame they showed that they had used 67
- 90. Exploratory studies words as cues, it could be an indicator that the visual appearance was not distinctive enough for them to define the category of the website. To make sure that the screens were not be displayed without the participant being ready each participant was allowed to determine when each screen was displayed by pressing the space key on the keyboard. The sequence in which the screens appeared was randomized but each participant was presented with the same screen captured images. Finally, participants were not given the categories of websites to choose from. This allowed for more free answers to be given. 3.3.4. Participants Five people, three female and two male, volunteered for this study. Of the five participants two were PhD students, three had a MSc. degree, one was an undergraduate, and the other was a secondary school student. All the participants confirmed using the Internet at least four times a week for a minimum of one hour. This ensured that participants had a minimum exposure to the Web so that they were more likely to have seen the website categories used in the study beforehand. Participants in this experiment have not taken part in the previous exploratory study described in 3.2. 3.3.5. Procedure The study was divided into two parts, called first and second pass. The first pass aimed at obtaining an immediate identification which would only rely on the overall appearance of the page without allowing participants to concentrate on detail. The objective of the second pass was to investigate if any particular visual feature would have more influence in the process of identification of the page. In the first pass each participant was told that the test consisted of identifying the type of service provided by a website by looking at an image of its page. Participants were told that a total of twenty website screens would be displayed, one at a time, as they pressed the space key on the computer keyboard. They were informed that each screen would be shown for a very short period of time and that all the website logos had been removed. After every screen, each participant was asked to write down 68
- 91. Exploratory studies and/or comment on the type of service the website provided, and to say which visual elements seen had that led to that conclusion. In the second pass participants went through all the 20 screens again, but this time they determined the length of time that each screen was displayed by pressing the space key on the computer keyboard to change to the next one. This time participants were allowed to examine each screen for as long as they liked. After each screen was examined, participants were reminded what their answer in the first pass had been. They were asked to either confirm or change the answer they had given in the first pass. In the case where they changed their mind, they were asked to say whether there were any elements on the screen that made them change their opinion and, if the answer was affirmative, they were asked what these elements were. Each session was recorded on a Digital Voice Recorder so that every comment made by the participants could be analysed later. 3.3.6. Results Identification The following table shows the results for each screen for the first and second passes: TABLE 6 –IDENTIFICATION RESULTS ON EACH CATEGORY SCREEN BY PARTICIPANT PER PASS. Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Total Pos. Id 1st 2nd 1st. 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd pass pass Web mail ! ! X ! X ! Web mail ! ! ! ! X ! 11 2 Web mail UI UI ! ! ! ! Online news ! ! ! ! ! Online news ! ! ! ! ! 14 1 Online news ! UI ! ! ! ! Search engine ! ! ! ! ! Search engine X X X ! ! ! ! 12 1 Search engine X X ! ! ! ! E-commerce (DVD) UI UI ! ! X ! X ! E-commerce (supmkt) X X UI ! ! X ! X ! E-commerce (DVD) ! ! X ! ! X ! 16 8 E-commerce (supmkt) ! X ! ! ! ! E-commerce (supmkt) X X ! X ! ! X X E-commerce (DVD) X X ! ! ! ! Online banking ! X ! ! ! ! 8 2 Online banking ! ! ! ! X ! Online auction ! ! X ! X X X ! Online auction X X ! ! X X X ! 6 2 Online auction ! X X X X X X ! Total Correct Ident. 12 0 14 5 15 3 15 2 12 7 67 16 != Correct Identification X = Incorrect Identification UI = Unidentified 69
- 92. Exploratory studies The total number of correct identifications in the first pass for each participant and for each category can be seen in Table 6. These figures do not allow a fair comparison between categories since the number of screens in each category varied (e.g. e- commerce had a total of six screens whilst online banking had only two). So, to have a better idea of the ease of identification of each category, the positive identifications for each category on the first pass were added, and divided by the total number of screens displayed in each category, producing an identification ratio per category as seen in the following page: TABLE 7 – IDENTIFICATION RATIO. Online Online Online search web mail e-commerce newspaper auction banking engine identification ratio 0.73 0.53 0.93 0.4 0.8 0.8 The most easily identified category was the online newspaper followed by online banking, search engines, and web mail. It seems that the hardest category to identify was online auctions. Most of the screens that were not identified in the first pass were identified in the second. The category that presented the least correct identifications on the second pass was the online auction: out of the 8 incorrect identifications on the first pass only 3 were corrected on the second pass. The only other category that presented one incorrect identification on the second pass was the e-commerce category, where one participant was not able to identify a screen either the first or second time. Visual cues Regarding visual cues helping participants to identify the categories of the websites, observations made when the participants misidentified the website were ignored. Although misleading visual cues may give clues to relevant aspects of other categories, for the purposes of this study only the elements that led to correct identifications will be focused on. When analysing the comments on which visual cues were perceived as helpful towards identification, the comments were grouped together according to element similarity. 70
- 93. Exploratory studies Finally, whenever a participant mentioned a word or a sentence from the screen that helped guess the category of the website, it was assumed that the content was more powerful as an indicator of the type of site than visual elements for that particular participant. As this study focuses on visual elements, specifying which words were read was not necessary, therefore any comments like this were counted under the ‘Words’ keyword. In Appendix C, the way in which the comments made by each participant were grouped can be observed. Once all the elements were sorted, the number of times each one was mentioned showed the frequency of each element (Ef). The frequency provides a simple way of evaluating the influence a determined element had on readers’ decisions about the function of the website. To analyse the frequency results for every element on each category, the frequency values will be ordered from the most frequent element to the least frequent providing at a glance, a general idea of the elements that were cited per category. Webmail The graph in Figure 16 shows that there were not great discrepancies amongst the frequencies for each element, with ‘Checkboxes’, ‘List’ and ‘To-From columns’ (columns in a table that indicate respectively the recipient and sender of an e-mail) being the most cited elements. Considering that the element ‘To-From columns’ indirectly implies a table, it can be said that the configuration of elements in a table can be considered quite a relevant visual element on a webmail site. FIGURE 16- CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR WEBMAIL SITES. 71
- 94. Exploratory studies Online News In this category, the visual configuration of text near a picture seems to have been the most poignant visual cue for the participants (Figure 17). It was followed by the use of ‘Headlines’ which are a particular element of newspapers. The least frequent elements in the participants’ commentary were the use of colours and the current date being displayed. These results suggest that perhaps the way of organizing pictures and text together with headlines helps readers identify the site as online news at a glance. FIGURE 17 – CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR ONLINE NEWS SITES. Search Engine The search engine category had as the most frequent element the repetition of a name (e.g. searched results about London showing several times the word ‘London’ on the screen). This might be due to the fact that the predominant feature in a search engine is words. The second most cited element was ‘Links’ (Figure 18) but this, in this case, is directly related to ‘Words’ as they also contain words which can be used as cues. This might indicate that search engines are still more textual than graphic interfaces. The third most frequent element comment made by the participants (‘Layout’) does not refer to a precise element, but to a form of arranging graphical and textual elements on the screen. This possibly indicates that there is a layout that signals to readers that the page they are looking at is a search engine, although it is not clear from the data obtained how this layout is described. 72
- 95. Exploratory studies FIGURE 18 - CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR SEARCH ENGINE SITES. E-commerce For this category, ’Words’ was the most frequent element cited by participants as a cue to the function of the site (Figure 19). The second most frequent element in participants’ comments was ’Pictures’. All the other six elements mentioned showed frequencies well below the first two. FIGURE 19– CHART OF RELATIVE FREQUENCY OF ELEMENTS FOR E-COMMERCE SITES. 73
- 96. Exploratory studies Online Banking For the online banking screens, ’Words’ was again the most frequent element and, in this category, the frequencies of the other elements were very small when compared to it. FIGURE 20 – CHART OF RELATIVE FREQUENCIES X ELEMENTS FOR ONLINE BANKING SITES. Online Auction The graph in Figure 21 shows that for online auctions, once again the pattern of looking for words to be used as cues to the function of the site was repeated as the ’Words’ elements were the elements most frequently mentioned by participants. Pictures of products next to textual descriptions (‘Pictures & description’) followed as the second most frequent element. ‘Colours’ and ‘Pictures’ were the least cited elements. FIGURE 21 – CHART OF RELATIVE FREQUENCIES OF ELEMENTS FOR ONLINE AUCTION SITES. 74
- 97. Exploratory studies 3.3.7. Discussion Looking at the elements that were mentioned in each category and analysing the Unique Elements (elements that were mentioned in only one category) and Common Elements (elements that were mentioned in more than one category), it can be observed that the two categories with the lowest identification ratio; online banking and the online auction, both had few or no unique elements (see Table 8). This might have contributed to the misidentification and to the inability to identify the category of the website. TABLE 8 - A COMPARISON BETWEEN CATEGORIES AND THEIR RESPECTIVE ELEMENTS. A TICK MARK SIGNIFIES THAT THE ELEMENT INDICATED IN THE ROW HAS HELPED PARTICIPANTS IDENTIFY THE RESPECTIVE WEBSITE CATEGORY. Categories Web Online e-commerce Search Online Online Total mail News Engine Auction Banking Elements Advertisement " " 2 Checkboxes " 1 Colours " " " 3 Date " 1 Envelope Icon " 1 Headlines " 1 Layout " " 2 Links " " 2 Links and Summaries " 1 List " " 2 Ordered items " 1 Picture " " " " 4 Picture and Text " 1 Picture and " " " 3 description Pictures and Links " 1 Price " 1 Repeated Names " 1 Shopping Cart " 1 Table " " 2 Tabs " 1 To-From Columns " 1 Underlined Text " 1 Words " " " " " 5 Unique Elements 3 4 3 3 0 1 Common Elements 3 5 5 4 4 5 Of all the categories, online banking, e-commerce and online auction showed the greatest occurrence of words being used as cues by the participants (Figure 22). The categories e-commerce and online auctions also presented the greatest number of screens not identified or misidentified (Table 9) . 75
- 98. Exploratory studies One explanation for the greater use of words for those categories when compared to the others might be that some particular categories of websites have not yet achieved a visual standard. This might be due to the fact that they are sites where words are more relevant than graphic elements, or that these sites belong to new categories or, even, these categories of websites are not as frequently used as some others. FIGURE 22 - RELATIVE FREQUENCY OF WORDS BEING USED AS CUES BY PARTICIPANTS IN EVERY CATEGORY. TABLE 9 - NO IDENTIFICATION AND MISIDENTIFICATION PER CATEGORY. No Identification Misidentification 1st pass 2nd pass 1st pass 2nd pass Web mail 1 0 3 0 Online news 1 0 0 0 Search engine 0 0 3 0 E-commerce 3 0 10 1 Online Banking 0 0 2 0 Online Auction 0 0 8 5 The auction websites were particularly interesting as they were repeatedly categorized as e-commerce: 7 out of the 15 instances on the first pass and 5 out of 15 instances on the second pass. Amongst them, one screen in particular (Figure 23) had three participants believe it was an e-commerce site on the first pass and even on the second pass these participants still thought it was e-commerce. This might be 76
- 99. Exploratory studies explained by the similarities of purposes (both selling products) influencing the elements used to organize and convey information in both categories. An alternative explanation is perhaps that there are enough visual similarities between e-commerce and auction websites to make it difficult for participants to distinguish between the two. Either way it is possible that the e-commerce and online auction categories belong to a same web genre and therefore not only have the similarity of purpose but also share similar structural, linguistic, visual and functional genre conventions. The other two participants who managed to identify this particular screen (Figure 23) used a semantic cue, the word ‘Auctions’ to help them. These results might be explained by this particular site possibly having some features of an e-commerce website. It can be verified by looking at the elements mentioned in each category (Table 8) that three of the four visual elements (75%) cited for online auctions are common to e-commerce websites. FIGURE 23 – SAMPLE OF AN ONLINE AUCTION WEBSITE SCREEN USED IN THE EXPERIMENT. Another interesting phenomenon occurred on a particular e-commerce website that displayed an advert of a ‘summer sale’; it took up about a quarter of the screen area (Figure 24) with an illustration of a woman on a sunny beach. Three out of the five participants mistook the website for a travel agency website. On a webmail site, another participant saw an advert for Sky Cable TV and thought the site was of a TV channel. The same thing happened on an e-commerce website where a participant saw a picture of two girls and believed it was a dating site. This might indicate that sometimes adverts can confuse website users, making them at first glance, misinterpret the function of a site, and that pictures are an important visual cue. 77
- 100. Exploratory studies FIGURE 24 – SAMPLE OF AN E-COMMERCE WEBSITE WITH ADVERT USED IN THE EXPERIMENT. On the other hand, it seems that there were some visual elements and a particular way of organizing the information in some categories, such as online news and search engines, that might have helped people immediately identify them as such. Looking into the Unique Elements in each category and how many times each was cited by the participants, it is possible to see which Unique Elements were more commonly associated with each category (Table 10). TABLE 10 – NUMBER OF PARTICIPANTS THAT CITED EACH UNIQUE ELEMENT IN EACH CATEGORY. Web mail Nº Online News Nº e-commerce Nº Search Engine Nº Online Nº citing citing citing citing banking citing Picture and Repeated Checkbox 4 6 Shopping Cart 2 5 Tabs 1 Text Names To-From Ordered 3 Headlines 4 Price 2 1 Columns items Underlined Pictures and Links and Envelope Icon 1 1 1 1 Text Links Summaries Date 1 On webmail sites, checkboxes seemed to be the most associated with this category whilst on online news websites, it was the way of positioning together pictures with text along with headlines. On e-commerce sites, it was the shopping cart, in search engines, the presentation of a name repeated several times seemed to be the most perceived cue, and on online banking web sites, it was tabs. 78
- 101. Exploratory studies The fact that some elements were cited in more than one category (e.g. tables, cited on web mail and online banking) does not discard them as being identifiers of a particular category, but it seems that the existence of additional visual cues (unique or not) might be necessary in the process of identifying the site. The e-commerce and online banking categories had some unique visual elements, but these were not cited by many participants. This was observed not only on e- commerce but also on online banking screens where the most-used cues were words and sentences. This might indicate that, when lacking visual conventions, users tend to look for words and sentences to use as cues on guessing a website’s function, or that for those websites, words are more relevant than graphic elements. 3.3.8. Conclusion Despite the small scale of this study, it seems to indicate that some visual elements play an important role in helping participants identify website categories. The study suggests that these elements may be part of a visual convention, thus reinforcing findings from Dillon and Gushrowski (2000) about the emergence of genres in some website categories. The impact of these genres on the performance of users when searching for information is yet to be investigated. The role played by the relative position of the elements on screen in the identification of the website categories was not studied here. This issue will be further investigated in Chapter 5 where the extent to which the position of the elements on a web page can influence how conventional users perceive that page will be examined. Considering the comment made by Waller (1987b), that a genre conforms to a set of rules and expectations of the reader, the fact that users were able to identify certain categories of websites by looking at their layout and sometimes singling out some visual elements in the process, suggests that users associate the existence of certain visual elements and layout rules with particular websites. This generates expectations in the users about what a certain category of website should look like, which can be an indication that some digital genres might be emerging or even already be established. 79
- 102. Exploratory studies Finally, it is important to note that as seen in Chapter 1 (page 1) websites are being constantly updated and although most changes made are to content, it is likely that some, if not all the pages used in this experiment have changed. 3.4. Conclusion of exploratory studies The results of both exploratory studies described here suggest interesting research paths which are worth pursuing in this thesis. However, due to time constraints only one of them can be investigated in this research. Text organization through the use of access structures and its impact on readers of both printed and electronic texts have been well researched and discussed (Dyson & Gregory, 2002; Hartley & Jonassen, 1985; Lorch & Lorch, 1996; Schultz & Spyridakis, 2004; Waller, 1979; Williams & Spyridakis, 1992). But access structures are one aspect of the functional constraints of a document which can belong to all three types of genre conventions: visual, structural and linguistic. Waller (1979) illustrates this with an example of the visual difference between an educational textbook and a novel. The author mentions that although both books have a structure, the structure of the educational text is made explicit by typographic (i.e. visual) signalling whereas in the novel the structure is linguistically signalled. This typographic signalling can be achieved via the use of access structures such as headings, summaries and lists. In the case of headings, their position in relation to the text can become a visual convention, the word used in the heading can become a linguistic convention, and the place of that particular heading in the hierarchy of the document and the type of content associated with it can become a structural convention. This example shows how difficult it can be to link the functional constraints of a document with a particular type of genre convention. Taking all this into account, the research in this thesis will concentrate on visual conventions of a genre. This will hopefully add to an understanding of genres, conventions and functional constraints involved in the process of creating electronic documents. Furthermore it will investigate how not conforming to the visual conventions of a document genre will affect user performance and consequently usability of the document. 80
- 103. Chapter 4 Finding visual conventions in web forums 4.1. Introduction Results of the last exploratory study (see chapter 3, section 3.3) seem to indicate that some web page layouts are becoming easily recognized with web users being able to identify the category of a website with a quick glance at its layout. This could mean that visual genre conventions may be emerging in the Web. As mentioned previously, this thesis proposes to investigate the impact on user performance of not conforming to the visual conventions of a genre. However, to generate the appropriate experimental material to be tested, it is necessary to establish what is and what is not conventional in web forums. With this in mind, a survey was conducted to look at the visual features of web forums and gather knowledge of what a conventional web forum looks like. The survey is described in this Chapter and its results will provide a basis for the development of both a conforming and a non-conforming forum to be used in future experiments. 4.1.1. Web forums As seen in Chapter 2, genres are communicative events; in other words can only exist when communication occurs. A genre also relies heavily on a having a discourse community with certain communicative needs or purposes associated with it. These two characteristics of genres indicate that in the World Wide Web, web forums are strong candidates for an emerging digital genre and consequently for having established visual conventions. Therefore in this thesis they will be the category of electronic documents chosen for the experiments. 81
- 104. Finding visual conventions in web forums Web forums are ‘ongoing conferences in which participants start topics (or threads), post replies to each other, and read what others have said’ (Wallace, 1999, p.5). Within forums, discussions on a particular subject are held, and after some time, web forum users are said to develop a sense of belonging to a virtual community (Wallace, 1999, p.56). Forums also have their own particular discourse style conventions which even include some rules of social interaction (Fayard & DeSanctis, 2006; Fayard, DeSanctis, & Roach, 2004) and differentiates them from other styles of online communication. Web forums can be created using several programming languages such as php, Perl, Java and ASP. However, php has been the original language and until today is one of the mostly used forum programming languages due to the popularity of a web forum creation package called phpBB (which means php Bulletin Board). The phpBB web forum creation package is extremely efficient in terms of creating a functional web forum, but in terms of flexibility it is quite restrictive for the average developer. The restrictiveness of this development environment can be thought of as being a ‘design imperative’ (Waller, 1999), which is likely to have had great influence in the visual conventions of the web forum genre. As a result, even though more modern languages and resources are available, the conventions are still based on the design constraints originally imposed by phpBB. Forum structure The vast majority of web forums have at least three distinct nested levels: (a) the first level (index page), where a list of topics contained in the forum is presented; (b) the second level (threads page) where a list of sub-topics (threads) from a chosen topic are listed and (c) the third level (post page) where messages from members (posts) are written and presented. For reasons of simplicity, in this thesis the three levels will be named forum index page, threads page and post page respectively. The forum index page, the first and topmost level, presents users with all the topics that the forum contains, and allows them to choose which subject(s) most interest them. This level, although accessible to the user, usually does not allow them to modify any content. Generally forum moderators are responsible for indexing and creating topics in this area. The threads page, which is the second level, contains all the threads that are posted on a particular topic. Threads are groups of linked messages that share a common 82
- 105. Finding visual conventions in web forums theme. The threads page allows greater freedom to the users since they can create new threads and sometimes can even remove them. Finally, the post page, the third and innermost level, is where users have greater interaction with the forum. Here, users communicate by reading and writing messages (posts) to each other and replying to other users’ previous posts. This is the level that offers users most freedom of expression. Here they can generally, depending on the forum interface, write, edit, quote, reply to posts and also post images and emoticons (small symbols or icons that represent moods and feelings). 4.2. Survey 1 4.2.1. Introduction As mentioned earlier, the aim of this survey is to look at the visual elements which appear on web forums and see how frequently they occur. The main objective here is to obtain a picture of what a conventional web forum looks like. This will help to design forums that either comply with, or do not comply with the visual conventions of the web forum genre. These forums will be used in future experiments to test the influence of visual conventions on user performance. Similar surveys to verify the occurrence of elements in specific web pages have been conducted before (Cosio & Dyson, 2002; Dillon & Gushrowski, 2000; Ihlström & Åkesson, 2004; M. R. Patel, 2003; Scheidt & Wright, 2004). Of these surveys, the ones which are more relevant to this study will be addressed here. Dillon and Gushrowski (2000), although mentioned in the previous chapter, have important methodological contributions and will be analysed in more depth first. Then, Ihlström and Åkesson (2004) and Cosio and Dyson (2002) will be analysed since they have contributions in terms of the visual organization, positioning and visual appearance of the elements on the pages. To verify if the personal homepage was a candidate for an emerging digital genre, Dillon and Gushrowski(2000) examined 100 websites for a list of occurring elements. Elements were defined by the authors as features and components of the pages such as email addresses, graphics and external links. The authors categorized all the elements and counted their frequencies of occurrence. After everything was counted the final ranking of elements was divided into three groups: the 5 most common elements, the 5 least common elements and the remainder. From that division the 83
- 106. Finding visual conventions in web forums authors created 8 experimental pages which combined the three categories in a systematic way to create four ‘common-element’ pages and four ‘uncommon- element’ pages. Then printouts of the pages were shown to 57 participants who were asked to rate them from 1-8 in terms of preference of personal homepage design. However, the experimenters were not clear on whether they had taken any measures to ensure that the participants judging the layouts were familiar with the personal homepage. Considering that genres are closely related to a community of people who use them and are not easily interpretable by someone outside this community, asking participants who are not familiar with the personal homepage to judge it can lead to results which might not be representative of the personal homepage genre. After ranking the pages, participants were then shown the list of elements occurring in personal homepages and were asked to choose any elements from the list which they thought should be included in the personal homepage. With this, the authors managed to verify if there was agreement between what was commonly used in personal homepages and what users thought should be in a personal homepage. The authors concluded that there is agreement between the elements users think should be in personal homepage and what actual elements of the existing personal homepage. However, while listing the elements present in a particular category of websites and checking the agreement of the findings with users are valid procedures, some important aspects of the page design should also be considered. As the authors themselves have acknowledged, the number of graphic elements present on a page can affect its ‘look and feel’ (Dillon & Gushrowski, 2000, p. 203). However, it is not just the number of graphic elements on a page that can influence its ‘look and feel’, but also the visual form in which any element appears on a page can influence its ‘look and feel’. Whether elements occur in a graphic, textual or combined form should be considered when surveying for web genres, since these factors can influence the appearance of a page. Another aspect not considered by the authors but which can significantly influence the layout of the page is the positioning of the elements. The location of key features on a page can not only be learned as users consistently return to it (Ehret, 2002), but users come to expect them to be placed in those common positions (Bernard, 2001b). A study which goes a step further into analyzing the visual aspect of the elements present on web pages was conducted by Cosio and Dyson (2002). The authors looked 84
- 107. Finding visual conventions in web forums at a sample of 50 websites to establish the basic graphic components these websites used. The authors were interested in finding which elements were more frequent and which information structures were used on the websites examined. Their website sample covered diverse activities: education, commerce, financial, public services, government, cultural, sports related and personal pages. The authors restricted their analysis to the homepage and two subsequent levels of each website. To categorize the elements found in the websites analysed, the authors developed modes of symbolization and configuration of graphic language proposed in Twyman (1979, 1982). The modes of symbolization are of three different types: verbal, pictorial and schematic (Twyman, 1982). However, when it comes to defining these types Twyman (1982) simply mentions that the ‘verbal and pictorial categories speak for themselves’ and goes on to define the schematic category as that whose graphic images are neither verbal nor pictorial’(Twyman, 1982). Cosio and Dyson (2002) define the pictorial category as referring to any image, photograph or illustration and the verbal mode of symbolization the authors define as referring to ‘any letter of the alphabet, number or any written words (Cosio & Dyson, 2002). For the survey the authors adopted three categories of configurations of graphic language: linear interrupted, linear branching and matrix (see Twyman, 1979 for samples of each mode of configuration). After examining the websites, Cosio and Dyson (2002) present a list with the elements found, the frequency counts and the various ways the elements appear (e.g. the background could appear as a picture or as a solid colour). By looking at the way the elements could appear the authors take an approach not taken by Dillon and Gushrowski (2000) but extremely relevant considering the layout of the interface. But as in Dillon and Gushrowski (2000) for most elements analysed the authors do not consider their positioning on the page. The only exception is the navigation menu of the pages analysed. For the menu, the authors consider its location as relevant and they established its location by using a grid on the screen. But probably due to the fact that the menu is normally visible on a page, the authors do not consider areas of the screen which are only accessible by scrolling down the page. Although it was not the purpose of the survey carried out by Cosio and Dyson (2002) to obtain a ‘picture’ of one particular website genre, the method used to categorize the symbolization of web visual elements based on Twyman (1979) is very useful. However, as some of the elements present on a web page can have more than a single component (e.g. a menu usually has various words representing menu items and 85
- 108. Finding visual conventions in web forums sometimes graphic icons as well), describing only the method of symbolization of the element is not enough, it is also necessary to explain how the components of the element are arranged within the element. For this the categories of configuration of visual language (Twyman, 1979, p.8) used by Cosio and Dyson (2002) are extremely useful. As seen in Chapter 2, genre theory usually categorizes genres according to substance and form (Yates & Orlikowski, 1992); this description is further expanded by Schmid- Isler (2000) to encompass another component: function. But this categorization still does not cover all visual possibilities of a genre so Ihlström and Åkesson (2004) propose four genre concepts: content, form, functionality and positioning. With the addition of positioning to the description of a genre (Ihlström & Åkesson, 2004), it is possible to draw a parallel with the layout structure of a genre as defined in Bateman and Delin (2001) and Delin et al. (2002). A genre layout structure involves the nature, appearance and position of the communicative elements on a page (see Chapter 3). So, bringing together the concept of genre components (Ihlström & Åkesson, 2004) and the layout structure of a genre (Bateman & Delin, 2001; Delin, et al., 2002) the nature and appearance of an element would relate to the form concept and position would be linked with the positioning concept. So, the final survey to be analysed was conducted by Ihlström and Åkesson (2004) goes a step further than Cosio and Dyson (2002) and considers not only the visual aspect of the elements, but also the positioning of these elements on the screen. To establish a framework for identifying the characteristics of a genre, the authors examined 85 front pages of Swedish online newspapers. The authors were looking for which elements were present (content), how they were presented (form), what functionality they required, and where the elements were located on the screen (positioning). To establish the location of the elements on screen the authors divided the pages into a grid. Two features stand out when analysing the positioning system used by the authors: first the grid used was not divided regularly (i.e. the sections were not equally sized). The second interesting feature is that unlike what was done by Cosio and Dyson (2002), Ihlström and Åkesson (2004) have devised a positioning system which took into account the areas of the page which are only visible when scrolling. The process used was the following: the whole page (not just the visible area) was positioned under the grid, then each element was detected, positioned and then 86
- 109. Finding visual conventions in web forums related to form and functionality (Ihlström & Åkesson, 2004). The authors however do not enter into much detail as to how elements were categorized or what on the page was considered to be an element. This is extremely relevant since an element could have different components which could appear in different relative positions to each other and in different formats (e.g. pictorial, alphanumeric, etc.). For example, when looking at a search element on a page (such as the one in Figure 25) have the authors considered the element to be the text input box, the button next to it? Or did they consider the search element to be a combination of the input box and the button? FIGURE 25 - SAMPLE OF A SEARCH ITEM EXTRACTED FROM A SWEDISH ONLINE NEWS WEBSITE. Similarly the authors do not explain how the criteria used for establishing the form of the item. The example given by the authors of a ‘video item’ which could be presented as an icon, an image or a link, is also not very clear since an icon can simultaneously be an image and a link on a web page. Summary The analysis of these three surveys provide useful inputs on how to conduct a survey to identify what is conventional in web forums. It is important to point out that the main aim of the survey conducted here differs from all of the three surveys examined earlier. This survey is focused on identifying what is visually conventional in web forums and to use this knowledge to develop two web forums to be used in future experiments. From an analysis of the previous surveys some interesting points were noted; since the focus of this thesis is on visual conventions the functionality aspect of genres studied by Ihlström and Åkesson (2004) will not be considered. Similarly to what was done by Cosio and Dyson (2002), elements will be classified using the modes of symbolization found in Twyman (1979). However, here Twyman (1979) will be used as a basis for the classification system as for reasons of simplicity some of his categories will be adjusted. Considering the way elements are 87
- 110. Finding visual conventions in web forums presented in web forums, and the fact that they can have more than a single component, the categories of symbolization will be the following: ! Graphic - the element is an image or icon either concrete or abstract (joining pictorial and schematic representations from Twyman (1979) into one category). ! Alphanumeric - the element is made of a combination of characters containing the letters of the alphabet, numbers, punctuation signs and/or other non-Latin character. ! Hybrid - the element is composed of at least one graphic and one alphanumeric part. When defining modes of configuration of graphic language, Twyman (1979), takes into account how the reader uses the text. Since this thesis is concerned with the visual conventions of the genre, rather than structural or functional, the configuration modes adopted here will be based on some configuration modes from Twyman (1979) but will consider only the visual characteristic of the elements without concerning with how the reader approaches the element. ! Horizontal - when the components of an element are arranged as a horizontal line without breaks. ! Linear interrupted - when the components are arranged in a linear fashion but lines are artificially broken and stacked. ! Matrix - when the components are organized in a matrix fashion with rows and columns. The positioning of the elements on a page will be done using an asymmetric grid considering the whole extent of the page (including the areas off-screen) as was done by Ihlström and Åkesson (2004) and to certain extent by Cosio and Dyson (2002). Finally, as in Dillon and Gushrowski (2000) the results from the survey will be tested on users in order to obtain feedback on their validity. It is also important that the survey identifies: ! The elements which are present in the forums. ! Where these elements are located on a page. ! How these elements are presented. ! How the subcomponents of the elements are arranged. 88
- 111. Finding visual conventions in web forums 4.2.2. Method The survey examines existing web forums listing the existing elements, their symbolization modes, position and the modes of configuration. Henceforth the symbolization modes will be referred as symbolization and modes of configuration as just configuration for reasons of simplicity. As mentioned before, an element could be categorized in terms of its symbolization as graphic, alphanumeric or hybrid. The configuration of the elements will either be horizontal, linear interrupted or matrix. For establishing the positioning of the elements the whole length and width of the page was taken into account, not just the part which was visible on the screen. If this was not done, the positioning of the elements would depend on the amount of horizontal/vertical scrolling. Considering the page shown in Figure 26 it is clear by looking at the scrollbar that there is more content below the one displayed in the browser window. By scrolling the screen downwards all elements which were positioned at the bottom of the screen will be moved to the top of the window, thus altering their position. FIGURE 26 - SAMPLE OF THE USER'S VIEW OF A WEB PAGE THAT EXTENDS OUTSIDE THE SCREEN DIMENSIONS. If only the screen area was considered in positioning the elements, their positions would be relative to the amount of scrolling. The approach was then to divide the 89
- 112. Finding visual conventions in web forums forum page (not the screen) into a grid with 9 quadrants (Figure 27). The sections were given different dimensions, with the central row having the largest area since it is usually where the largest elements in the forums are located, such as the forum index table, threads table or the post area. Horizontally the screen was divided in three equal parts and vertically it was divided so that the top and bottom parts contained 20% of the height and the central part consisted of 60% of the height. The areas were labelled: top left, top centre, top right, centre left, centre, centre right, bottom left, bottom centre, bottom right. FIGURE 27 - GRID USED TO DETERMINE THE POSITIONING OF THE ELEMENTS ON A WEB FORUM PAGE. 90
- 113. Finding visual conventions in web forums The criteria for determining the positioning of each element in the quadrant are described in Appendix D. Configuration This criterion described how the components of some elements are arranged in relation to each other. In this survey, each multi-component element can be classified as horizontal, linear interrupted or matrix. When the element has only one component it is not possible to establish a configuration for it (e.g. the forum status indicator usually consists of a single graphic icon, so it is neither horizontal, nor linear interrupted, nor matrix). These elements are indicated by the abbreviation ‘N.A.’ (not applicable) on the tables. An alphanumeric element was considered as being multi-component when it had two or more words. The matrix class comprised of elements which were neither horizontal nor vertical alone, but a mixture of both. For example an element such as the forum index table which contains rows and columns of pictorial and alphanumeric elements, is classified as having a matrix configuration. 4.2.3. Sample As mentioned before, many forums are generated using templates and forum creating packages such as phpBB. This causes many forums to have a very similar appearance and elements, which suggests that a small sample of pages should represent the great majority of existing web forums. Thus a sample of 15 web forums was chosen randomly from search engine results. For a forum to be selected it had to contain at least an index page, a threads page and a post page. The first 15 different web forums which appeared as the results on a search in www.google.com and had at least all three levels were selected to be used in this survey. 4.2.4. Procedure Initially, all occurring elements on all forums were listed. On each web forum, the forum index page was examined first, then the threads page and finally the post page. The elements were listed and put in a table and then the forums were re- examined level by level, and the symbolization, position and configurations of each element on each level were noted. 91
- 114. Finding visual conventions in web forums 4.2.5. Results Identification of the elements Forum Index (first level) The elements found in this level are listed in Table 11 and illustrated in Figure 28. For a full explanation of the elements see Appendix E. TABLE 11 - LIST OF ALL ELEMENTS FOUND IN THE FORUM INDEX PAGE OF WEB FORUMS. THE INDENTED ELEMENTS ARE FOUND WITHIN THE FORUM INDEX TABLE. Forum index elements Forum index table Topic title Topic summary Nº of threads Nº of posts Last post Topic moderator Nº of views Status indicator Topic type Menu Logo Login field Captions Current time & date User information area Breadcrumbs Forum noticeboard Adverts Forum search field From all the elements detected, it seems that the forum index table is probably the most important in this level given that it is the starting point for any user who searches the interface. It contains several sub-elements that portray information for the users and are relevant for this research. The following indented elements seen in Table 11 were found in the forum index table. Some of the elements are illustrated in Figure 29, since it was not possible to find a single web forum which contained all the elements found. Figure 30 and Figure 31 illustrate the elements missing from Figure 28. 92
- 115. Finding visual conventions in web forums FIGURE 28 – SOME ELEMENTS ON THE FIRST LEVEL OF A WEB FORUM. Logo Menu Breadcrumbs Login Field Forum Noticeboard Forum Index Table User info area Captions Current time & date 93
- 116. Finding visual conventions in web forums FIGURE 29 – SOME ELEMENTS PRESENT IN THE FORUM INDEX TABLE OF A WEB FORUM. Number of threads in topic Topic title Number of posts in topic Last post in topic Topic summary Topic status indicator FIGURE 30 - SAMPLE OF THE TOPIC MODERATOR ELEMENT IN THE FORUM INDEX TABLE. Topic moderator 94
- 117. Finding visual conventions in web forums FIGURE 31 - SAMPLE OF THE NUMBER OF VIEWS IN TOPIC ELEMENT ON A FORUM INDEX PAGE. Number of views in topic Thread page (second level) On the thread page the menu, breadcrumbs, captions, logo, current time and date as well as the login field elements found in the forum index are still present. This, allied to the presence of the thread table, makes the overall appearance of this page similar to that of the first level. In the following page, Table 12 shows a list of the elements found; the new ones are explained in Appendix E. TABLE 12 - LIST OF THE ELEMENTS FOUND IN THE THREADS PAGE OF A WEB FORUM. THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREADS TABLE. Thread page elements Thread table Thread name Thread author Thread status Last Post Number of replies Number of views Emoticons Menu New thread button Number of thread pages Breadcrumbs Forum jump Captions Logo Posting rules Current time & date Forum search Login field Popular topics 95
- 118. Finding visual conventions in web forums FIGURE 32 - ELEMENTS IN THE SECOND LEVEL OF A WEB FORUM. Menu Logo Breadcrumbs Login Field New thread button Forum thread table Captions Forum jump Current time & date Posting Rules As with the Forum index table, within the thread table there are other elements that convey information to the users (see the indented elements on Table 12). Some of these ( Figure 33) are similar to the ones that appear on the Forum index table, others are different. FIGURE 33 - THREAD INDEX TABLE AND ITS ELEMENTS OF A WEB FORUM. Number of Emoticons Last post views Thread name Thread author Thread status Number of replies indicator 96
- 119. Finding visual conventions in web forums Post page (third level) Despite containing some elements that are common to the second and first levels (menu, breadcrumbs, logo, jump to forum, posting rules, time and date, login field, adverts, forum search field), the third level is visually quite different from the previous levels. In the following page is a list of the elements found on this level (Table 13) and the new elements are explained in Appendix E. The elements are also illustrated in Figures 34 and 35. TABLE 13 – LIST OF THE ELEMENTS FOUND IN THE POST PAGE OF A WEB FORUM. THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREADS TABLE. Post page elements Post area Time & Date of Post Post content Author Information Quote button Post title Signature User profile button Private message button Post number Edit Post Author’s website button E-mail author button Menu Next/previous thread button Breadcrumbs Reply to post button Jump to forum Logo Posting rules Thread title(out post area) Current time & date Adverts Login field New post button Search topic field Table of contents 97
- 120. Finding visual conventions in web forums FIGURE 34 – ELEMENTS IN A POST PAGE OF A WEB FORUM. Menu Logo Login Breadcrumbs field Post reply button Post area Next/Previous thread buttons Forum jump Current time Posting Rules & date 98
- 121. Finding visual conventions in web forums FIGURE 35 – ELEMENTS WITHIN THE POST AREA OF A WEB FORUM. Time and date of post Author information Post number Post content Quote button Signature A summary of which elements appear on each web forum level is shown on Table 14. From this table it is possible to see that many elements appear across all forum levels and this is an indicator that usually there is a fair degree of consistency across the levels. As mentioned in Nielsen (1993, p.91) to some extent this might be due to the standards imposed by the web forum creation engines, but still there is the case of some elements where repetition could be solely due to a design choice. 99
- 122. Finding visual conventions in web forums TABLE 14 – SUMMARY OF THE ELEMENTS PRESENT IN EACH OF THE THREE FORUM LEVELS. Forum index Threads page Post page Menu " " " Logo " " " Login field " " " Captions " " Current time and date " " " Users information area " Breadcrumbs " " " Forum notice board " Adverts " " Forum search field " " " Topic title " Topic summary " Status indicator " " Number of threads in topic " Number of posts in topic " Last post in topic " Topic moderator " Number of views in topic " New thread button " Number of thread pages " Forum jump " " Posting rules " " Popular topics " Thread name " Thread author " Last post " Number of replies " Number of views " Emoticons " Post area " Next/previous thread button " New post button " Table of contents " Time & date of post " Post content " Author info " Quote button " Post title " Signature " Profile button " Private message button " Post number " Edit post " Author’s website button " Email button " 100
- 123. Finding visual conventions in web forums The next step was to find out how these elements were displayed across different web forums. As seen before, there are three possible formats for presenting the information. These are graphic, alphanumeric and a hybrid form which mixes graphic and alphanumeric information into one element. Symbolization Most elements in all three levels of a web forum are presented in an alphanumeric format or hybrid format (graphic and alphanumeric). In the Index page of the forum only one element was presented in a pure graphic style and this was repeated in the threads page of the forum. Apparently it was only in the post page that a greater variety of elements were presented in a pure graphic style. But even this does not happen frequently and the graphic symbolization mode of elements is quite rare. Tables 51, 52 and 53 in Appendix F show the frequency of symbolization modes of the elements at the first, second and third levels of the forums respectively. As in previous tables, the indented elements are found within the topmost non-indented element. Note that not all the elements listed appear together on the same web forum page; hence most totals do not add up to 15. Position As mentioned before, the results of this survey will be used to create the conforming and non-conforming material to be tested in future experiments. For this reason, as in Dillon and Gushrowski (2000), the more frequently occurring and the least frequently occurring values for the position of the elements are the most relevant and will be examined further. However, determining the position of the least frequently occurring elements can present a problem. Problems can emerge when the element (which appears only a few times) appears simultaneously in two screen positions (see criterion (d) in Appendix D). This double position which would be the least frequently occurring position for the element coincides with the most frequently occurring position. Whenever this happened, a position in which the element never occurs (i.e. a non- occurring position) was chosen as the least occurring position for that element. To simplify the visualization of the position results, an image of the grid was created and the name of the element was placed in the quadrant which represents either its most occurring or least occurring position. The most occurring positions of each element in the forum index, threads and post pages are displayed in Figures 36, 38 101
- 124. Finding visual conventions in web forums and 40 respectively. Figures 37, 39 and 41 show the least/non occurring positions for every element in the forum index, threads and post pages respectively. It is important to note that whenever an element is shown in bold italicized type, this indicates that on the same level page the element appeared most frequently in two simultaneous positions (e.g. in Figure 38, the element ‘new topic button’ appears on the top left quadrant and the bottom left quadrant on the same page at the same time). When an element appears the same number of times in different positions, the name of the element was placed on all tied positions but without bold or italic. For example, in Figure 39 the element login field appears the same number of times in the top left and centre-top positions so it is placed on both. FIGURE 36 – MOST OCCURRING POSITIONS OF FIGURE 37 – LEAST/NON OCCURRING POSITIONS OF THE THE ELEMENTS IN THE FORUM INDEX PAGE. ELEMENTS IN THE FORUM INDEX PAGE. Breadcrumbs Menu Login Field Login field Logo Menu Logo Adverts Forum search Noticeboard Captions Noticeboard Adverts Time & date Index table Index table User info area Adverts Captions Login field Adverts Time & date Time & date User info area Forum search 102
- 125. Finding visual conventions in web forums FIGURE 38 – MOST OCCURRING POSITIONS OF THE FIGURE 39 – LEAST/NON OCCURRING POSITIONS OF ELEMENTS IN THE THREADS PAGE. ELEMENTS IN BOLD THE ELEMENTS IN THE THREADS PAGE. ITALICIZED TYPE APPEARED MOST FREQUENTLY IN TWO SIMULTANEOUS POSITIONS. New topic Menu Nº of thread New topic New topic btn Menu button pages button Breadcrumbs New topic btn Breadcrumbs Forum search Login field Logo Nº of thread Logo Login field pages Login field Thread table Forum search Forum jump New topic Time & date Nº of thread Thread table Captions Button pages Forum search New topic btn Forum search Captions Forum jump Captions Time & date Posting rules Popular topics Posting rules 103
- 126. Finding visual conventions in web forums FIGURE 40 – MOST OCCURRING POSITIONS OF THE FIGURE 41 – LEAST/NON OCCURRING POSITIONS OF THE ELEMENTS IN THE POST PAGE. ELEMENTS IN BOLD ELEMENTS IN THE POST PAGE. ITALICIZED TYPE APPEARED MOST FREQUENTLY IN TWO SIMULTANEOUS POSITIONS. Breadcrumbs Menu Adverts Login field Next / previous Logo button Reply button Adverts Login field New post btn Menu Topic title Logo New post btn Reply button Breadcrumbs Topic title Forum search New post Logo button Post area Post area Post area Table of Forum jump contents Search field Reply button Next/previous Forum jump Breadcrumbs Forum jump Time & date button Posting Rules New post btn Reply button Posting rules Time & date Time & date Adverts New post btn Search field 104
- 127. Finding visual conventions in web forums Configuration Most elements on web forums seem to appear in a horizontal configuration mode and the second most frequent configuration mode seems to be the matrix mode closely followed by the linear interrupted one. Some elements, however, only have one component which does not allow for any configuration mode to be established (e.g. the forum status indicator is usually a graphic icon, so it is impossible to establish a configuration mode). The frequency counts of the configuration modes of each element present on the web forums are listed in the tables in Appendix G. 4.2.6. Discussion An overall examination of the results obtained seem to indicate that most web forums have their elements presented in a very similar manner, positioned in similar places and configured in similar ways. This seems to suggest that web forums have highly conventional layouts and could be strong candidates for a digital genre. Some particularities of each of the three attributes surveyed are commented on below. Position At first glance, the top and bottom of the web forum levels seem to have a high concentration of elements compared to the centre. This however is not necessarily true since the centre areas of the forums usually contain a table which is made up of a considerable amount of sub elements which also densely populate this area of the screen. So in the end, the central area of the forum appears as densely populated as the top and bottom of the screen and in some cases, even more. Looking at the elements which are present in all three forum levels (Figure 36, Figure 38 and Figure 40) it is possible to see that these tend to be in the same position in all levels where they appear. This to some extent contributes to the internal consistency of the pages as mentioned before. Symbolization mode Considering that the sole purpose of a web forum is to be a media for discussion, it is not surprising that the great majority of elements are presented in an alphanumeric or a hybrid form. After all, written text is currently the best way to communicate on the web. 105
- 128. Finding visual conventions in web forums Configuration The configuration of most elements in a web forum appear to be either horizontal or linear interrupted. This is not surprising considering that the symbolization mode of most elements is alphanumeric. There is also a sense that some alphanumeric elements are configured in horizontal format just because of their length, and were they longer, they would be configured as linear interrupted. For this survey a distinction was made between these two configuration modes regardless of this issue. The reason for this decision is based on the concept that if some long alphanumeric elements which would normally be configured as linear interrupted were configured as horizontal, it would add to the non-conformance of the design. The inverse situation also applies. 4.2.7. Most frequent groups of elements The next chapter will describe two experiments which asked web forum users to compare several templates and choose the ones they thought were more conventional (experiment 1) and the ones they thought would be easier to use (experiment 2). These templates ranged from highly conventional to highly unconventional, and were created based on the results obtained from the survey. These results indicated (a) which elements most commonly occur on web forums and (b) the most frequently occurring and least frequently occurring values for the symbolization, position and configuration of these elements. Although the results of the survey indicate which elements most frequently occur on web forums, simply using these elements to create the templates could generate unrealistic combinations. This can make the experimental material not representative of existing forums. Another approach (the one chosen) is to choose all the elements which occur in a web forum and see which combination of elements happens more frequently and use this. This approach is more likely to result in a design which more resembles existing forums than merely picking the most frequently occurring elements. In Appendix H a table shows all the groups of elements appearing on each web forum examined, and the resulting patterns. The results for the most frequent patterns for each forum level are shown here in Table 15. 106
- 129. Finding visual conventions in web forums TABLE 15- MOST FREQUENT GROUPS OF ELEMENTS IN EACH FORUM LEVEL (SHOWN IN COLUMNS). NAMES IN BOLD REPRESENT THE ELEMENTS WHICH APPEAR ON ALL THREE FORUM LEVELS AND IN ITALIC THE ELEMENTS WHICH APPEAR ON TWO FORUM LEVELS. Forum index page Threads page Post page Menu Menu Menu Logo Logo Logo Login Field Login Field Login Field Current Time & Date Current Time & Date Current time & date Breadcrumbs Breadcrumbs Breadcrumbs Captions Captions next/previous thread button User Info Area Forum Jump Forum jump Forum Noticeboard Nº of Thread Pages reply to topic button Forum index table Posting Rules Posting Rules Forum Title New Topic Button Post area Status Indicator Thread table Author Info Forum Summary Thread Name Post Content Nº of Threads in Forum Status Indicator Time & Date of Post Nº of Posts in Forum Thread Author Post Title Last Post Nº of Replies Quote Button Nº of Views Signature Last Post Profile Button Private Message Button email Button Poster website button From this table, it is possible once more to see that there is a group of elements that appear repeatedly across the forum levels (shown in bold) and some smaller sets of elements appearing in two levels (shown in italic). This indicates that these elements are likely to be more relevant in terms of maintaining the visual consistency within the website. It is possible that changes in the symbolization, configuration and position of these elements will bring more problems to users than the same changes in the other elements. The most frequent groups of elements in each forum level (shown in Table 15) will be used on both the conforming and non-conforming forums. If different elements were used it would be difficult to establish whether variations in the perceptions of conventionality and ease of use were due to the presence (or absence) of an element or the variation of the attributes of the elements. 107
- 130. Finding visual conventions in web forums 4.2.8. Conclusion The survey gathered data on the elements present on web forums. The data indicates the frequency of the type of symbolization, configuration and the positioning for each forum element. As previously explained in the introduction to the Chapter, these results will serve as basis for constructing the material to be used in the next experiments. But before any material is developed, it is important to ensure that the system used in this survey to establish the set of ‘conventional’ elements represents what is considered by users to be a convention-conforming web forum layout. It is also interesting to see if in terms of user perceptions ease of use and conventionality are correlated. Considering that users may be more willing to use web pages which they perceive as easy to use, this could give indications as to whether conventionality may be a page attribute which can make users more willing to use a web page. With that in mind, three experiments described in the following Chapter will be conducted next; the first one will ask users to choose from a series of templates (created based on the results of the survey), those which they think look more like web forums they have seen before; the second experiment will use the same templates which were used in the first experiment to ask users which templates they perceive as easier to use. The results of these first two experiments will then be used to create the experimental material (one conforming forum and one non-conforming forum). Finally, after the materials are created, a survey will assess some aspects of the non-conforming interface regarding identification and user comprehension. Then the experimental material will be used in subsequent experiments. 108
- 131. Chapter 5 Assessing the results of the survey in terms of user perceptions This Chapter describes three experiments which were conducted to validate the results obtained in the survey. This validation process consisted of asking web forum users about different aspects relating to the results obtained from the survey. The purpose of the two initial experiments is twofold: first they will attempt to confirm that the results obtained from the survey agree with what web forum users perceive a conventional and easy to use web forum layout to be. Second, they will attempt to verify if there is a relationship between what users perceive as a conventional web page and a web page which is easy to use. This relationship is important to establish as it can give indications as to whether conventionality and ease of use are perceived as one thing or two separate concepts. Based on the results of these two experiments, the conforming and non-conforming experimental material will be created. Finally, a third experiment, which tested if web forum users would be able to correctly identify relevant aspects of the design of the non-conforming experimental material which was created, is described. 109
- 132. Assessing the results of the survey in terms of user perceptions 5.1. Experiments to test templates created based on the results of the survey 5.1.1. Experiment 1 – Perceptions of conventionality of the templates based on the results of the survey Introduction This experiment consists of presenting to participants a number of forum page templates which were created based on the results of the survey. Participants will be asked which templates they think are most similar to pages they have seen before in other web forums. The layout of the templates will range from conforming (most frequent symbolization, configuration and position values for the elements) to non- conforming (least frequent symbolization, configuration and position values for the elements). Objectives The survey previously described gathered elements which were present on web forums and their most common symbolization modes, positions and configuration modes. From these results a set of elements was established as most common to web forums and the most frequent values for the three attributes (symbolization, position and configuration) were noted as likely candidates for the creation of a conforming page layout. However, a web forum generated by these results in an ‘artificial’ manner might not properly represent the web forum genre. So, the need to obtain the feedback of web forum users is quite important. Also, in the second exploratory study (see Chapter 3, section 3.3.7) it was detected that the way some elements are positioned in relation to other elements cued participants as to the category the web page belonged to. This seems to indicate that perhaps position or configuration of elements may have more influence on user perceptions of conventionality. Finally, as seen in Chapter 2, genres emerge from communication needs of discourse communities and members of these communities are able to identify, decode and understand them. Thus, testing the forums designed on the basis of the results of the survey with web forum users, seemed the best way of ensuring that they are appropriate designs and meet the expectations of users for the genre. So, this first experiment was put in place to: 110
- 133. Assessing the results of the survey in terms of user perceptions ! Verify if the conclusions drawn from the survey regarding a conforming forum layout are in agreement with what is perceived as convention- conforming by web forum users. ! Find which web forum layouts users consider least conventional so that it can be used as a template for designing the non-conforming material for future experiments. ! See if any of the three attributes of an element (symbolization, position or configuration) could have stronger influence on user perception of conventionality when compared to the other attributes. If that is the case, it can be extremely valuable for designers to know what impact changing each of theses attributes will have in terms of user perceptions of conventionality. The results of this experiment will be used (along with the results of the following study) to define the templates to be used to create the experimental material for future experiments. Method Material Based on the results of the survey, 24 web forum screens were designed in a way that each level comprised of two groups of forum templates: one group, named conforming and the other group non-conforming. Each group had four templates per forum level; one main template and three variations. The main template in the conforming group was meant to represent a highly conventional layout and it was designed with the attributes of the elements in the most frequently occurring values. The main template in the non-conforming group was meant to represent a non-conforming forum layout so it was designed with the attributes of the elements in the least frequently or non-occurring values. Because of the possibility that these main templates do not represent what users perceive as more conventional or least conventional in a web forum, template variations were created. The variations were intended to also help verify if any of the three attributes would have more influence than the other two on the perception of conventionality of web forum users. Knowing if one of the attributes has more 111
- 134. Assessing the results of the survey in terms of user perceptions influence on user perception than the other will allow for better understanding of how users may perceive the conventionality of web pages. Because in this experiment three attributes of the elements are studied (symbolization, position and configuration) there were three variations per level per group, and on each variation template one attribute was altered. Samples of main conforming and non-conforming templates with their variations are shown in figures Figure 42 and Figure 43. FIGURE 42 - EXAMPLE OF THE MAIN CONFORMING TEMPLATE (LEFT) AND ITS THREE VARIATIONS (RIGHT) ON SYMBOLIZATION MODE, POSITION AND CONFIGURATION MODE (FROM TOP TO BOTTOM RESPECTIVELY) 112
- 135. Assessing the results of the survey in terms of user perceptions FIGURE 43 – EXAMPLE OF THE MAIN NON-CONFORMING TEMPLATE (LEFT) AND ITS THREE VARIATIONS (RIGHT) ON SYMBOLIZATION MODE, POSITION AND CONFIGURATION MODE (FROM TOP TO BOTTOM RESPECTIVELY) For example, a variation of configuration on a conventional template (see Figure 44) was designed with the symbolization and position attributes fixed at the most frequently occurring values, and the configuration changed to the least or non- occurring value. In the same way, a variation of symbolization was designed with the position and configuration attributes at their most frequently occurring values and the symbolization at its least or non-occurring value (Figure 45) 113
- 136. Assessing the results of the survey in terms of user perceptions FIGURE 44 - COMPARISON BETWEEN THE MAIN CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR MOST FREQUENT VALUES) AND THE MENU VARIATION WHERE THE CONFIGURATION IS AT ITS LEAST FREQUENT VALUE. FIGURE 45 - COMPARISON BETWEEN THE MAIN CONFORMING MENU ELEMENT (ALL ATTRIBUTES ON THEIR MOST FREQUENT VALUES) AND THE MENU VARIATION WHERE SYMBOLIZATION IS AT ITS LEAST FREQUENT VALUE A variation of configuration on a non-conforming template was designed with the symbolization and position of the elements fixed in their least frequent values and the configuration in their most frequently occurring value (Figure 46). Similarly a variation of the symbolization attribute on a non-conforming template was done with the configuration and position attributes fixed on their least frequent values and the symbolization in its most frequently occurring value (Figure 47). The variation of an attribute value was always made to the opposite extreme value since subtle changes (e.g. from the most frequently occurring value to the second most occurring value) might not generate a difference big enough to be detected by the participants. 114
- 137. Assessing the results of the survey in terms of user perceptions FIGURE 46 - COMPARISON BETWEEN THE MAIN NON-CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR LEAST FREQUENT VALUES) AND THE MENU VARIATION WHERE THE CONFIGURATION IS AT ITS MOST FREQUENT VALUE FIGURE 47 - COMPARISON BETWEEN THE MAIN NON-CONFORMING MENU ELEMENT (WITH ALL ATTRIBUTES ON THEIR LEAST FREQUENT VALUES) AND THE MENU VARIATION WHERE THE SYMBOLIZATION IS AT ITS MOST FREQUENT VALUE 115
- 138. Assessing the results of the survey in terms of user perceptions To simplify the classification of the templates a coding system was put in place. It consists of a letter and two numbers. The letter indicates whether the template is from the conforming (C) or non-conforming group (N). The next number represents the forum level with which that template is associated (1-3) and the last number (the rightmost one) represent the number of the template (i.e. 1 to 4 as each forum level had four templates). The coding system is explained in Figure 48. FIGURE 48 – CODING SYSTEM USED TO NAME THE FORUM TEMPLATES. The templates which were generated in order to test the results of the survey against web forum user perceptions are illustrated on Figure 49. The templates C11, C21, C31 are the main conforming templates with all the attributes set at their most frequently occurring values. The templates N11, N21 and N31 are the main non- conforming template with all three attributes set at their least frequently occurring values. 116
- 139. Assessing the results of the survey in terms of user perceptions FIGURE 49 - THUMBNAILS OF THE 24 CONFORMING AND NON-CONFORMING WEB FORUM PAGES DESIGNED FOR ASSESSING THE RESULTS OF THE SURVEY 117
- 140. Assessing the results of the survey in terms of user perceptions Table 16 shows how the three attributes (symbolization, position and configuration) were used to design each template in each forum level in both the conforming and non-conforming groups. TABLE 16 – SYMBOLIZATION, POSITION AND CONFIGURATION VALUES FOR EACH CONFORMING AND NON-CONFORMING SCREEN. Conforming Symbolization Position Configuration Template Forum index MF MF MF C11 MF MF LF C12 MF LF MF C13 LF MF MF C14 Threads page MF MF MF C21 MF MF LF C22 MF LF MF C23 LF MF MF C24 Post page MF MF MF C31 MF MF LF C32 MF LF MF C33 LF MF MF C34 Non-conforming Symbolization Position Configuration Template Forum index LF LF LF N11 MF LF LF N12 LF LF MF N13 LF MF LF N14 Threads page LF LF LF N21 MF LF LF N22 LF MF LF N23 LF LF MF N24 Post page LF LF LF N31 MF LF LF N32 LF MF LF N33 LF LF MF N34 MF – Most frequently occurring value LF – Least frequently or non-occurring value Research has shown that repeated exposure to a stimulus increases the sense of familiarity in users (Zhang & Ghorbani, 2004). Thus, because all participants will be exposed to all the templates, if the same content is used over and over it is possible that the sense of familiarity of users with the templates will increase. However, this familiarity can be a result of repeated content exposure, rather than layout. This improved sense of familiarity could be mistaken for conventionality by the participants. In an attempt to minimize the possibility of the content influencing the judgment of participants, two different content sources are used in the templates. Both contents used in the creation of the templates were retrieved from existing web forums. 118
- 141. Assessing the results of the survey in terms of user perceptions Because it is the layout of the interface that is under scrutiny in this test, some aspects of the layout which are not being investigated were held constant. The same sans- serif typeface (Arial) was used on all templates because it is extremely common in web forums. The same colour scheme was also adopted for all templates. These two layout aspects were fixed at a constant value to keep them from influencing the perception of conventionality of users. The graphic elements used in the design of the conforming templates were obtained from existing web forums. However, because the non-conforming templates have as their basis the least, or non-occurring symbolization modes for each element, some symbolization modes are hardly ever (if at all) found in existing forums. This happens especially when the most frequently occurring symbolization mode is alphanumeric and the non-occurring symbolization mode is graphic. Whenever this happened (e.g. a textual element needed to be replaced by a graphic element in a non-conforming template) the following procedures were adopted: (1) If the textual element was not exclusive to web forums (e.g. FAQ menu item) and in another interface, a graphic was used to represent the element (e.g. a question mark icon used to represent FAQ), then this graphic element was used in the forum. (2) If (1) was not possible then an attempt was made to find an icon which could represent the element. The initial criterion was to use a representational image but if that was not possible then an abstract one was used. The aesthetic consistency (consistency of style and appearance) with the other elements was maintained as much as possible. As mentioned before, to achieve greater likeness to a real web experience, each screen was designed in HTML and was displayed using a Firefox browser. This allowed participants to use the mouse as they normally would when browsing the Internet. However, all links and clickable buttons on the templates were disabled. Experimental design For the test, the templates (see Table 16) were grouped into pairs. Each pair was composed of one template from the conforming group and one from the non- 119
- 142. Assessing the results of the survey in terms of user perceptions conforming group. In a pair both templates being used were from the same forum level so that two different levels were never put together. To create a setting that would emulate as close as possible a real browsing experience, with scroll bars and real web page dimensions, the templates were presented to participants in a browser engine (e.g. Firefox). While participants could only see one screen at a time, they were given free mobility to switch to and fro between the screens as many times as needed in order to compare them. The pairs were put together so that all templates on one level of a group were compared with all templates on the same level of the other group. For example C11 was compared with N11, N12, N13 and N14 and the same happened for C12, C13 and C14. This resulted in a total of 48 pairwise comparisons per participant. For each participant the order in which each screen was displayed within each pair was randomized. For example, for one participant a pair could appear as C11, then N13; for another participant the same pair could appear as N13 then C11. The sequence in which the pairs appeared was also randomized so that each participant saw the pairs in a different order. Following each pair, a white background screen was displayed to separate the comparisons and ensure that pairs remained together. On the top right corner of each screen, the number of the pair and screen that were being displayed was shown. This information was displayed using very small characters so as to interfere visually as little as possible with the layouts. It had the purpose of both guiding the participant when choosing the screens, and the experimenter when taking note of the choices made (some samples can be seen in Figures 50 and 51). 120
- 143. Assessing the results of the survey in terms of user perceptions FIGURE 50 – SCREENSHOT REFERRING TO THE C11 TEMPLATE OF THE CONFORMING GROUP. THIS TEMPLATE HAS THE SYMBOLIZATION, CONFIGURATION AND POSITION ATTRIBUTES AT THEIR MOST FREQUENTLY OCCURRING VALUES. FIGURE 51 – SCREENSHOT REFERRING TO THE N11 TEMPLATE OF THE NON-CONFORMING GROUP. THIS TEMPLATE HAS THE SYMBOLIZATION, CONFIGURATION AND POSITION ATTRIBUTES AT THEIR LEAST FREQUENTLY OR NON OCCURRING VALUES. Participants Twelve people, all familiar with web forums, took part in the experiment. Participants were either graduate or undergraduate students from different academic backgrounds. All participants were fluent in English either as their first or second 121
- 144. Assessing the results of the survey in terms of user perceptions language. Participants in this experiment have not taken part in any previous experiments conducted for this thesis. Procedure Each participant was told that they were going to see 48 pairs of screens and on seeing each pair was going to be asked to choose the screen layout which they thought looked more like what they would see on a web forum. Participants were also told that only the visual aspect was being considered, thus they should not pay attention to the content. It was also reinforced that the objective was not to choose the best design, but the design that was more similar to a design they would expect to see on a web forum. Before the test started, each participant listened to a brief description on how to navigate through the interface, and was allowed to use a simulation to get used to using the keyboard to move forward and back on the screens. For each pair of templates, participants could use the ‘<’ and ‘>’ keys on the keyboard to switch between the templates. On each template the participant was allowed to scroll up and down (and left to right when necessary) as they normally would when browsing the Web. After looking at both the templates of a pair the participant was taken to a screen with a white background which asked the participant to tell the experimenter which screen from the pair looked more like what the participant would expect to see in a web forum layout. After the choice was made, the experimenter noted on a table which screen was chosen, and by pressing the ‘>’ key the participant was taken to the next pair. During the test, participants were allowed to make comments on anything that they found relevant and the experimenter wrote down the comments. Results Templates chosen as more similar to other web forums Of the 576 (48 pairs seen by 12 participants) times that each pair of templates was displayed during the experiment, the templates from the conforming group were chosen 482 times (83.7%) over the non-conforming ones as being more similar to other web forum layouts. In 94 (16.32%) cases the templates from the non-conforming 122
- 145. Assessing the results of the survey in terms of user perceptions group were thought to look more like what users would expect to see on a web forum. Thus, overall the templates from the conforming group were perceived as more similar to other forums than the templates belonging to the non-conforming group. A Chi-square goodness-of-fit test was applied to the data and the results are summarized in Table 17. TABLE 17 – SUMMARY OF THE CHI-SQUARE TESTS ON THE NUMBER OF TIMES EACH GROUP WAS PICKED AS MORE LIKE OTHER WEB FORUMS. Group Conforming Non-conforming P Forum index (level 1) 160 32 < 0.01 Threads page (level 2) 146 46 < 0.01 Post page (level 3) 176 16 < 0.01 Total 482 94 < 0.01 Non-conforming templates frequently chosen as similar to other web forums Another interesting result which came out of this experiment is that some non- conforming templates were chosen as similar to other web forums a considerable number of times. Of all the non-conforming templates, the one which was chosen the most was N22 (chosen 20 times). This template represented a threads page layout (second level) which had the elements presented in their most frequent value, but their position and configuration in their least frequent or non occurring value (Symbolization=MF, position=LF, configuration=LF). The template is shown in Figure 52. 123
- 146. Assessing the results of the survey in terms of user perceptions FIGURE 52- SAMPLE OF THE NON-CONFORMING TEMPLATE FROM THE THREADS PAGE LEVEL WHICH WAS CHOSEN MORE FREQUENTLY AS SIMILAR TO EXISTING WEB FORUMS. THIS TEMPLATE HAD THE ELEMENTS IN THEIR MOST COMMON SYMBOLIZATION MODE BUT THEIR POSITION AND CONFIGURATION IN THEIR LEAST OR NON OCCURRING VALUES. Similarly, in the post page (third level) the most frequently chosen non-conforming template was N32, chosen 7 times (see Figure 53) which was also designed with the same attribute pattern (symbolization=MF; position=LF; configuration=LF). However, in the forum index page (first level) the non-conforming template which presented the highest frequency of times chosen (10 times) was N11 which had all three attributes in their least occurring values (symbolization=LF, position=LF, configuration=LF). The template is shown in Figure 51. FIGURE 53 – SAMPLE OF THE NON-CONFORMING TEMPLATE FROM THE THREADS PAGE LEVEL WHICH WAS CHOSEN MORE FREQUENTLY AS SIMILAR TO EXISTING WEB FORUMS. THIS TEMPLATE HAD THE ELEMENTS IN THEIR MOST COMMON SYMBOLIZATION MODE BUT THEIR POSITION AND CONFIGURATION IN THEIR LEAST OR NON OCCURRING VALUES. 124
- 147. Assessing the results of the survey in terms of user perceptions Conforming templates chosen the least times as similar to other forums Looking at the symmetrical situation, at the templates of the conforming group which were chosen the least number of times, it is possible to see that on the forum index page, template C12 was chosen 38 times. This template had the symbolization and position of the elements in their most frequently occurring values, and configuration in its least occurring value. On the threads page (second level) and post page (third level), the conforming templates which were least picked were C23 (24 times) and C33 (40 times). These two templates presented the same arrangement for the three attributes (symbolization=MF, position=LF, configuration=MF). However, looking at the second least chosen conforming templates, a pattern emerges: all three templates had the symbolization of the elements in its least occurring value and the position and configuration in their most occurring values (symbolization=LF, position=MF, configuration=MF). It is also possible to observe from Table 18 that for the templates in the first and third levels the difference in frequency between the least chosen and second least chosen templates is not very big which could indicate that there was no significant difference in perception from the perspective of the participants. TABLE 18 – LEAST CHOSEN AND SECOND LEAST CHOSEN CONFORMING TEMPLATES IN EACH FORUM LEVEL. THE TABLE ALSO INDICATES THE VALUES OF EACH ATTRIBUTE OF THE ELEMENTS AND THE NUMBER OF TIMES (FREQUENCY) EACH TEMPLATE WAS CHOSEN. Symbolization Position Configuration Frequency C12 MF MF LF 38 Level 1 C14 LF MF MF 40 C23 MF LF MF 24 Level 2 C24 LF MF MF 38 C33 MF LF MF 40 Level 3 C34 LF MF MF 43 Least chosen conventional template Second least chosen conventional template Non-conforming templates least chosen as being similar to other forums Similarly, the non-conforming templates which were least chosen as being similar to other web forum layouts were N14 at the index page level, and N23 at the threads page level. Both these templates had the symbolization and configuration at their least occurring values and the position at its most occurring value. However, the template on the post page which was never picked (N34) had a different arrangement of these attributes (see Table 16). 125
- 148. Assessing the results of the survey in terms of user perceptions Looking at the number of times that each template from the conforming group was chosen as looking more like other web forum pages (Table 19), it is possible to observe that the frequency difference between the most often chosen template and the second most chosen one is relatively small. TABLE 19 – NUMBER OF TIMES EACH TEMPLATE IN THE CONFORMING GROUP WAS CHOSEN AS MORE LIKE OTHER WEB FORUM PAGES (RESULTS ARE OUT OF A TOTAL OF 48 COMPARISONS PER TEMPLATE PER LEVEL. Forum index page Threads page Post page C11 41 C21 43 C31 48 C12 38 C22 41 C32 45 C13 41 C23 24 C33 40 C14 40 C24 38 C34 43 Discussion The templates, which according to the survey results were designed to be conforming (i.e. conforming group), were indeed chosen by participants as more like other web forums when compared with the non-conforming group. This seems to indicate that the method used to combine the results of the survey to create a conforming forum interface are in accordance with what users expect a convention-conforming web forum page to look like. The templates chosen the least number of times as more similar to other forum pages can be considered as samples of pages which violate the expectations of users regarding the visual appearance of a web forum page. This can be an indication that these pages to a certain extent, do not meet the expectations of web forum users and thus, possibly, do not conform with the visual conventions of the web forum genre. Considering that the template variations on the conforming layout had only one of the three element attributes modified to its least frequent value, the fact that these templates were picked more often as similar to other web forums than the non- conforming template and its variations suggests that altering just one of the three element attributes (symbolization, position or configuration) does not seem sufficient to alter perceptions of conventionality. However, it was noted that between all conforming variation templates, the ones where the symbolization mode was in its least occurring values were the ones which were picked the least number of times as similar to existing web forum pages. 126
- 149. Assessing the results of the survey in terms of user perceptions Despite that, this change on the symbolization attribute was not enough to make participants perceive these conforming templates as less similar to other forums than the non-conforming screens which had the symbolization mode in its most occurring value. The results also indicate that the perception of conventionality of the forum index page and the threads page is affected similarly by similar changes in the attribute of the elements. The templates chosen the least number of times for the index page (N14), and the threads page (N23), shared the same values for the three attributes (symbolization and configuration=least/non occurring values and position=most occurring). But the least chosen template for the post page (N34) did not share the same attribute configuration (see Table 16). Considering that these two levels (index page and threads page) share a more similar layout and have a greater number of common elements (as can be seen from Figure 54 and Figure 55), when compared with the post page (Figure 56), it is not surprising that changes in the same attributes might influence them in a similar manner. FIGURE 54 - ELEMENTS PRESENT ON THE FORUM INDEX PAGE WHICH ALSO APPEAR IN THE THREADS PAGE. Logo Login field Breadcrumbs Menu Forum index table Status Last post indicator Captions Time & date 127
- 150. Assessing the results of the survey in terms of user perceptions FIGURE 55 - ELEMENTS PRESENT ON THE THREADS PAGE WHICH ALSO APPEAR IN THE FORUM INDEX PAGE. Logo Login field Breadcrumbs Menu Forum index table Status Last post indicator Captions Time & date FIGURE 56 - ELEMENTS IN THE POST PAGE WHICH ALSO APPEAR IN THE FORUM INDEX AND THREADS PAGE. Logo Breadcrumbs Login field Menu Time & date 128
- 151. Assessing the results of the survey in terms of user perceptions Furthermore, this result could be an indicator that the symbolization and configuration of an element might have a greater influence on the perception of conventionality than its position. But when the template which was perceived as the least conventional in the post page (N34) is examined along with the least picked templates of the other levels (N14 and N23), the common attribute value to all three is the symbolization of the elements set in its least occurring value. This indicates that even more than the configuration of the elements, the symbolization is more likely to influence user perception of conventionality. The indication that the symbolization mode of the elements could have more influence on the perception of conventionality than the position or configuration of the elements is extremely relevant, since 2/3 of the templates in the non-conforming group have the symbolization mode in its least occurring value (see Table 16). However, changing the symbolization mode of web forum elements to their least/non occurring values means that some elements will be displayed on a web forum in a purely graphic manner. This graphic symbolization mode may be quite difficult to decode and comprehend resulting in problems such as poorer performance, comprehension and higher cognitive load. This may cause users to consider these forums as less easy to use than the other ones. The next experiment will address the impact of the symbolization mode of the elements on perceptions of ease of use thus, results may show the need for yet another experiment to be conducted in order to determine if participants are able to use the elements in the least conventional template without much difficulty. In summary, the templates which participants chose most and least frequently as similar to other existing forums are described in Table 20, and indicated in Appendix I. These are strong candidates to be used in the creation of the test material for future experiments. But before it is decided which templates will be used to create the experimental material, another experiment (described in 5.1.2) will be conducted to test how web forum users perceive the templates in terms of ease of use. The results of this and the following experiment will be the determinants as to which templates will be used in the main experiments described in Chapters 6 and 7. 129
- 152. Assessing the results of the survey in terms of user perceptions TABLE 20 – TEMPLATES MOST FREQUENTLY AND LEAST FREQUENTLY CHOSEN AS SIMILAR TO OTHER WEB FORUMS (RESULTS ARE OUT OF A TOTAL OF 48 COMPARISONS PER TEMPLATE PER LEVEL). FORUM INDEX THREADS PAGE POST PAGE TIMES CHOSEN TIMES CHOSEN TIMES CHOSEN C11 41 CONFORMING C21 43 C31 48 C13 41 NON-CONFORMING N14 6 N23 3 N34 0 5.1.2. Experiment 2 - User perceptions of ease of use of the templates based on the survey results Introduction As seen in Chapter 2 (see 2.3) genre conventions and usability are linked via external consistency. Some usability authors claim that not conforming to conventions established by the majority of websites (i.e. external consistency) can cause major usability problems (Nielsen, 1999a). Other authors take the contrary position that violating some expectations of users regarding conventions does not seem to badly affect usability (Kalbach & Bosenick, 2006; McCarthy, et al., 2003). Furthermore, user acceptance (willingness to use) of software or interface seems to be influenced by the perceived ease of use (Morris & Dillon, 1997). If that is the case, understanding whether users perceive ease of use and conventionality as the same thing may provide additional knowledge as to how not conforming to genre conventions can affect users’ acceptance of a website. This second experiment is a follow-up to experiment 1 and attempts to further investigate the relationship between conventions and usability. This investigation will focus on user perceptions of ease of use, which will be considered as indicators of good apparent usability (Fu & Salvendy, 2002; Tractinsky, 1997). The same templates from the previous experiment (see 5.1.1) will be used, and participants will be asked to compare them in terms of ease of use. 130
- 153. Assessing the results of the survey in terms of user perceptions Objectives ! Examine which of the templates created from the results of the survey and used in the previous experiment web forum users perceive as easier to use. ! Investigate if there is a correlation between the perceptions of conventionality and ease of use amongst web forum users. Methods Material The material used for this experiment was basically the same as the one used in experiment 1. However, unlike experiment 1, this one was conducted entirely online. This allowed it to be completed in a shorter timeframe, and provided a more efficient way of recruiting by focusing the recruitment advertising to web forums thus ensuring that participants were web forum users. For the online experiment to function it is necessary to adapt the interface so that it can work on different computer configurations. It is also necessary to devise a way of sending the answers to the experimenter. It is vital that the test instructions are presented to participants as a start page for the test. The solutions were as follows: the interface was adapted to work on the two most popular Internet browsers: Internet explorer 6 and Mozilla Firefox. To send the answers, the interface was encapsulated inside an online php form which would transmit the answers to the experimenter via e-mail. Finally an instruction page was created and placed before the test. Experimental design Here once more the experimental design of experiment 1 was reproduced. However, all pair orders and the order of each template in each pair were again randomized. Participants Participants were recruited by posting an invitation with a link to participate in the test on several web forums. All forums where the post was placed were written in English. Posting in English web forums ensured that all participants who responded to the announcement were web forum users, and had a minimum level of language understanding to perform the test. Although less was known about the participants in this experiment than in the previous one, the sample was probably more varied since participants could come from any part of the world. Because the characteristic which was more likely to influence the outcome of the test were the familiarity of 131
- 154. Assessing the results of the survey in terms of user perceptions users with forums (and this was guaranteed by the way in which participants was recruited) the lack of further information on participants does not constitute a major problem. Furthermore, the more varied sample improves the generalization of the results. In total 20 participants volunteered to do the test and submitted their results via email to the experimenter. The emails contained data from the test and the number of the participant. When the number reached 20 (which was the established number of participants for this test) the test was taken offline. Participants in this experiment have not taken part in any previous experiments conducted for this thesis. Procedure The experiment instructions and how to navigate the interface were displayed to the participants. Participants were informed that they were going to see 48 pairs of screens and after each pair they would be asked to choose one page layout they thought would be easier to use. At the end of the instructions participants were told to press a key to start the test. By doing so, participants were taken to the first pair of templates. As in experiment 1, participants were allowed to scroll up and down (and left to right when necessary) on each template as they normally would when browsing the Web. After looking at both templates in a pair, a white background screen was displayed saying that one pair had ended and asking participants to choose which of the two templates they thought looked easier to use. After the 48 pairs of templates were compared an email was sent to the experimenter with the participant number and the answers. Then a screen thanking the participants for taking part in the test was displayed. Results Of the 960 times (48 pairs presented to 20 participants) that each pair of templates was displayed during the experiment, the conforming templates were chosen 822 times (85.63%) over the non-conforming ones as easier to use. The conforming samples included not only the conforming templates where the symbolization, configuration and position attributes assumed their most occurring values, but also the variations around the conforming pages. 132
- 155. Assessing the results of the survey in terms of user perceptions In 138 (14.37%) cases the non-conforming sites were thought to be easier to use than the conforming templates. Overall the conforming templates were seen by participants as easier to use than their non-conforming counterparts. The total number of times the templates of each level were compared was 320 times per level. Again looking at each level, the conforming templates were chosen more frequently as easier to use than the non-conforming templates. In the forum index (level 1) the conforming templates were chosen 286 times as easier to use than the non- conforming templates. The non-conforming templates were chosen 34 times as easier to use. In the threads page (level 2) the conforming templates were chosen as easier to use than their non-conforming counterparts 275 times, while the non-conforming templates were picked only 45 times. Finally, in the post pages, the conforming templates were again picked more frequently as easier to use than the non-conforming template. The conforming templates were chosen 261 times against 59 times that the non-conforming were picked. A Chi-square test was applied to the data and the results are summarized in Table 21. TABLE 21 - SUMMARY OF THE CHI-SQUARE TESTS ON THE NUMBER OF TIMES EACH GROUP WAS PICKED AS EASIER TO USE. Group Conforming Non-conforming P Forum index (level 1) 286 34 <0.01 Threads page (level 2) 275 45 <0.01 Post page (level 3) 261 59 <0.01 Total 822 138 <0.01 The non-conforming templates which were picked more frequently as easier to use when compared to their conforming counterparts were N12 in the forum index level (13 times), N22 in the threads page level (18 times), and N32 in the post page level (27 times). The non-conforming templates which were least often chosen as easier to use when compared to a conforming template were N14 in the forum index level (4 times), N23 in the threads page level (6 times), and N34 in the post page level (5 times). 133
- 156. Assessing the results of the survey in terms of user perceptions When it comes to the templates in the conforming group the results are generally much closer than the results from the non-conforming group with not much difference between the first and second most chosen templates in each forum level. The frequency that each template in the conforming group was chosen as the easier to use when compared to a non-conforming template is shown in Table 22. TABLE 22 - FREQUENCIES THAT EACH TEMPLATE IN THE CONFORMING GROUP WAS CHOSEN AS EASIER TO USE (RESULTS ARE OUT OF A TOTAL OF 80 COMPARISONS PER TEMPLATE PER LEVEL). Forum index page Threads page Post page C11 72 C21 74 C31 70 C12 66 C22 71 C32 65 C13 71 C23 61 C33 58 C14 76 C24 69 C34 69 The number of times the most frequently and least frequently chosen templates were picked as easier to use is shown in Table 23. TABLE 23 – FREQUENCIES OF THE TEMPLATES MOST FREQUENTLY AND LEAST FREQUENTLY CHOSEN AS SIMILAR TO OTHER WEB FORUMS (RESULTS ARE OUT OF A TOTAL OF 80 COMPARISONS PER TEMPLATE PER LEVEL). Forum index Threads page Post page Times chosen Times chosen Times chosen Conforming C14 76 C21 74 C31 70 Non-conforming N14 4 N23 6 N34 5 Discussion Overall, the conforming group templates were more frequently chosen as easier to use than the non-conforming group ones. Below a correlation test is applied to the results of both this and the previous study. Here, it suffices to say that participants considered the templates of the conforming group easier to use. In this experiment, some non-conforming templates were chosen as easier to use when compared to some conforming templates. Although the number of times these templates were chosen is much smaller than that of the conforming templates, some interesting observations can still be made looking at which templates were chosen by participants. 134
- 157. Assessing the results of the survey in terms of user perceptions All of the non-conforming templates more frequently chosen as easier to use had the position and configuration of their elements in their least occurring values, while their symbolization was set in its most occurring value. This finding reinforces the idea from the previous experiment that the symbolization of the elements on a page might be quite influential not only on the user perception of the conventionality of the page, but also on their perceptions regarding the ease of use of that page. The non-conforming templates which were least chosen in this experiment (experiment 2) were the same ones that were least often chosen as similar to other existing forums in experiment 1 (N14, N23 and N34). This seems to suggest that participants might perceive non-conforming layouts as harder to use or vice versa. The results from these two experiments show that there seems to be a relationship between what web forum users perceive as conventional, and what they perceive as easy to use in a forum interface. A Spearman-& test showed a strong correlation (r=0.83) between the perceived conventionality and the perceived ease of use of the templates on both experiments. 5.1.3. Conclusion The correlation found between the two studies seems to indicate that users who are familiar with web forums seem to find web forum interfaces which follow the visual conventions of the genre also easier to use. The analysis of the experimental results also seem to indicate that the conventionality parameters gathered in the survey are a fair representation of what users expect a web forum to look like, and what they consider as easier to use. Future studies will consist of getting users to search for information in a conforming forum and a non-conforming forum. To define the layout of these pages the results from these two experiments discussed above will be taken into account. To create a forum which violates the established visual conventions, the templates which were chosen the least number of times as more like other web forums and least number of times as easier to use in each study will be used. These templates (N14, N23 and N34) were perceived by users as those least visually similar to other web forums they previously used and also as being less easy to use than other layouts. 135
- 158. Assessing the results of the survey in terms of user perceptions However, choosing templates to be used to create a conforming forum is not so easy. The reason for that resides in the fact that there was no complete agreement between the two experiments regarding the most frequently chosen templates. For the threads page and the post page, the same templates were chosen in the first and second experiment. But in the forum index page, there was no complete agreement: C11 and C13 were equally considered as the templates which looked more like other web forums, and C14 was the template users thought was easier to use (see Table 24). However the conforming template which was the second choice for users in terms of ease of use was C11, which was chosen 72 times. The frequency difference between the first choice (C14) and the second choice (C11) is not large enough to suggest users would feel that the latter would be more difficult to use than the former. TABLE 24 - TEMPLATES MOST FREQUENTLY CHOSEN IN THE CONFORMING GROUP. TEMPLATES FORUM INDEX THREADS PAGE POST PAGE TIMES CHOSEN TIMES CHOSEN TIMES CHOSEN MORE LIKE OTHER C11 41 (85.4%) C21 43 (89.6%) C31 48 (100%) WEB FORUMS C13 41 (85.4%) C14 76 (95%) EASIER TO USE C21 74 (92.5%) C31 70 (87.5%) C11 72 (90%) Thus for the purpose of creating a conforming web forum to be used in future experiments it seems appropriate that C11 be used instead of the other most chosen templates C13 and C14. One reason is that the frequency difference between C11 and C13 in the first experiment was non-existent, and the difference between C11 and C14 in the second experiment is not big enough to justify the use of C14 instead of C11. Furthermore, by using C11, the whole web forum would benefit from consistency since the templates chosen for the other two levels had the same values for the three element attributes as C11. All three templates have the symbolization, position and configuration of their elements in their most occurring values. The templates which were chosen the least number of times as more like other web forums and considered the least easy to use (N14, N23 and N34) all have symbolization at its least occurring value. Both the conforming and non-conforming templates seem to confirm that symbolization has a considerable influence on user perceptions of conventionality and ease of use. The templates which will be used to design the web forums for the upcoming experiments are shown in Table 25 and are indicated in Appendix I. 136
- 159. Assessing the results of the survey in terms of user perceptions TABLE 25 – TEMPLATES WITH THE MOST FREQUENT (MF) AND LEAST FREQUENT (LF) VALUES FOR EACH ELEMENT ATTRIBUTE AND WHICH WERE USED TO DESIGN THE CONFORMING AND NON-CONFORMING FORUMS. Conforming Symbolization Position Configuration Template Forum index MF MF MF C11 Threads page MF MF MF C21 Post page MF MF MF C31 Non-conforming Symbolization Position Configuration Template Forum index LF MF LF N14 Threads page LF MF LF N23 Post page LF LF MF N34 5.2. Experiment 3 - Assessing the experimental material 5.2.1. Introduction From the results of the previous experiment it is possible to see that the symbolization mode of the elements plays an important role in user perceptions of ease of use. The templates which will be used to create the non-conforming material were the ones perceived least often as easy to use by participants. One of the reasons for these templates being perceived as not as easy to use as the others could be due to the considerable number of elements which are presented as graphic images. There is a possibility that user perceptions may be converted into usage difficulties when using the interface. Thus, an indication of how hard it is for users to identify the elements which are presented in this manner is necessary. Furthermore, it is important to test not only the ability to identify the elements but also the ability to recognize the main interactive elements of a forum. By doing so it will be possible to have an idea about how hard or easy it is for forum participants to use the non- conforming interface. Extreme difficulties with interpreting the graphic elements or the interactive elements of the non-conforming forum might indicate that participants find it difficult to use the interface. On the other hand, if participants have no difficulties in identifying the graphic elements or the interactive ones, this 137
- 160. Assessing the results of the survey in terms of user perceptions might suggest that there will be no complications when using the non-conforming forum. Many elements which are textual in the conforming forum are represented graphically in the non-conforming forum by the use of icons. Literature divides icons into three categories: abstract (composed of geometric marks and shapes), representational (simple pictures of familiar objects and operations) and semi- abstract (formed by combining both abstract and representational images) (Blattner, Sumikawa, & Greenberg, 1989). Representational icons are believed to be easier to interpret than the abstract and the semi-abstract ones (Blattner, et al., 1989; McDougall & Curry, 2004; Moyes & Jordan, 1993, p. 50). However, differences in accuracy when identifying representational versus abstract icons tend to disappear as the user acquires more experience with them (McDougall, Curry, & De Brujin, 1999). This seems to indicate that, as expected, icon identification improves according to familiarity with the icon. This familiarity can be understood as either familiarity with the object depicted in the icon or as experience with the icon in an interface (McDougall & Curry, 2004). Grouping icons together (in clusters) is also believed to allow users to learn icon- function relationships more quickly (McDougall & Curry, 2004). Clustering icons allows inferences about categories of icon-function relationships to be made and this, in turn, helps users interpret the meaning of the icons in the groups. There is another important variable to icon identification which was not considered above that is, position. It seems that when identifying an icon, users first look at its shape and if that is not sufficient for identification, they use the position of the icon on the interface to help identify it (Moyes, 1994). This assumes that the user is familiar with the placement of the icon and expects to find it in a determined position on screen. Research carried out by Moyes (1995) shows that contextual information, especially the position on the icon on the interface, is extremely relevant in identifying abstract icons, whereas the form of the icon is more important when users are faced with a representational icon. As the non-conforming interface contains both representational and abstracts icons it is possible that the place where the icon is located might be relevant for identifying the abstract ones. However, unlike the conforming forum, the non-conforming (designed from the templates which participants chose least often as conventional 138
- 161. Assessing the results of the survey in terms of user perceptions and easy to use) are not internally consistent. Thus, in the non-conforming forums the elements are positioned in their most common place except for the post page (the third level) of the forum, where they are placed in their least common position. So, the lack of consistency on the positioning of the icons in the non-conforming forum may deprive users of using this contextual cue to help identify the icons. But it is also plausible to assume that to get to the post page (third level) of the forum participants are likely to go through the other two levels and that by then they may have inferred the meaning of the icons from their position on these pages. 5.2.2. Objectives This study aims at further examining the experimental material generated based on the templates selected from the two previous experiments. The results obtained will serve to inform future experimental designs and the design of tasks. More specifically, this experiment has three main objectives: ! Verify if web forum users can identify the icons which will be used to replace the textual elements in the non-conforming forum. It will give information as to which icons are easier/hard to identify in the non-conforming interface so that tasks can be designed avoiding the use of elements which participants found hard to identify. ! Obtain perceptions of the participants regarding the conventionality of a finished page of the conforming and non-conforming forums. This will indicate that the experimental material is adequate and will further validate previous results. ! Investigate how well web forum users can recognize the interactive forum elements (e.g. menu, login area, index table) in the non-conforming template. This will give some indication as to how difficult it is for participants to locate the information within the forum and which interactive elements are harder to identify. Then tasks can be designed to avoid elements which are harder to identify. 5.2.3. Method The survey was divided into three distinct sections, each devised to assess one of the objectives. The first section investigates the first objective; the second section investigates the third objective and the third section, the second objective. 139
- 162. Assessing the results of the survey in terms of user perceptions All the text in the forum templates was replaced with placeholder text (‘Lorem ipsum’) so that participants could not use the content to help them throughout the test. Due to the nature of the survey and the fact that it needed participants that were familiar with web forums, the simplest solution was to devise an online questionnaire that people would be able to access from wherever they were. Then, by placing links to the test on different web forums the chances that participants were familiar with web forums would highly increase. Section 1 On the templates chosen to generate experimental material there are thirteen elements which are textual in the conforming forum, and graphic in the non- conforming version. This happened because the symbolization mode of the elements was changed from its most frequent value (conforming) to the non occurring value (non-conforming). This means that these particular elements are not presented graphically on web forums. In the design of the experimental material these thirteen elements are presented as graphic icons. The thirteen icons are never used on web forums so they were obtained from icon databases available on the Internet. For this survey, the icons were positioned vertically on a web page and participants were given empty fields next to each icon to fill in what they thought the icon represented (Figure 57). Icons were displayed out of context in order to keep participants from being exposed to the forum interface in the first section and this could influence the last part of the test where participants had to identify the interactive elements of the forum. 140
- 163. Assessing the results of the survey in terms of user perceptions FIGURE 57 - SAMPLE OF SECTION 1 OF THE SURVEY. The icons were presented to all participants in the same order. Table 26 shows the icons that were presented to the participants and the textual content that they were replacing. TABLE 26 - ICONS PRESENTED IN THE SURVEY AND THE TEXT CONTENT THEY WERE REPLACING. Icon 1 Icon 2 Icon 3 Icon 4 Icon 5 Icon 6 Icon 7 FAQ Password Register Today’s posts Members List Calendar User ID Icon 8 Icon 9 Icon 10 Icon 11 Icon 12 Icon 13 Time and date Thread starter Last post Threads Replies Views 141
- 164. Assessing the results of the survey in terms of user perceptions Section 2 This section investigated the second objective which is to check the perceptions of web forum users regarding the conforming and non-conforming forum screens created from the templates chosen in 5.1.3. Since section was designed as a quick check on the perceptions of conventionality of the users regarding two finished experimental screens (one conforming and one non- conforming) only one screen of one level of each forum was tested. The level chosen to be displayed to users was the forum index page for two reasons (a) it is the first screen users will see when they start the test and (b) in a conforming forum the index page and the threads page share a similar design (see page 127). So the experimental material based on the templates C11 for the conforming forum, and N14 for the non- conforming forum were used. This section consisted of two separate screens; one displayed the conforming page and the other displayed the non-conforming page (Figure 58). At the bottom of each screen was a question which asked participants to rate how the page they were looking at compared with web forum pages they had seen before. The rating was done through a Likert type scale ranging from 1 to 4 in increments of 1 where the value 1 stood for ‘Very different’ and 4 to ‘Very similar’. 142
- 165. Assessing the results of the survey in terms of user perceptions FIGURE 58 - SAMPLE OF THE NON-CONFORMING SCREEN IN SECTION 2 OF THE SURVEY. To keep the order in which the two templates were displayed from affecting the results, half of the participants were shown the non-conforming screen first and the other half were shown the conforming screen first. Section 3 This section investigated if participants are able to identify the elements which they would have to interact with while using the non-conforming forum page. If they can do this without difficulty then it is likely that they will be able to know where to retrieve the information they require when performing future tests. The ability to identify interactive elements of the non-conforming forum can indicate the difficulty participants might have when using the non-conforming template in comparison with the conforming template. 143
- 166. Assessing the results of the survey in terms of user perceptions In this section of the survey, an image of the index page of a non-conforming forum page was displayed. The relevant elements were circled and numbered 1 to 6 (Figure 59). Below the forum image, input fields corresponding to the key areas were shown, and a text asked participants to write what they thought these parts of the forum were. The names of each element which was to be identified are shown in Table 27. TABLE 27 – CORRECT ANSWERS FOR THE AREAS HIGHLIGHTED IN THE THIRD PART OF THE SURVEY. Area 1 Login Area 2 Menu Area 3 Thread/Topic / Post area Area 4 Thread/topic / post Area 5 Key or legend Area 6 Time and date FIGURE 59 - DIVISION OF AREAS IN THE NON-CONFORMING FORUM AS DISPLAYED IN THE THIRD PART OF THE SURVEY. 144
- 167. Assessing the results of the survey in terms of user perceptions Participants The questionnaire was made available online by postings on 8 web discussion forums. The posts contained requests asking people to participate with a hyperlink to the test page. All forums chosen for posting had English as their main language and the discussion topics ranged from old books to postgraduate research. This ensured that participants were not only web forum users but also had some fluency in English and had different backgrounds. Most forums were not based in the UK and the ones which were based in the United Kingdom, were not related to the University of Reading These measures were put in place to minimize the possibility that participants in this experiment had already participated in other experiments in this thesis. The answers were sent through e-mail to the experimenter. A total of 34 web forum users participated in the survey. 5.2.4. Procedure By clicking on the link provided on the post on the web forums, participants were redirected to a page containing Section 1 of the survey. Here participants were asked to consider the text elements which appear in a web forum page and associate them with one of the icons displayed by writing the name of the element in the field next to the icon image. Upon completion of this part of the survey participants clicked on a ‘continue’ button and were sent to a page thanking them for answering the first part of the survey and informing them that whenever they felt ready they should click on a link placed below the text to move on to the second part of the test. Clicking on the link directed participants to the second part of the test. As mentioned before, the second part consisted of a comparison between both the conforming and non-conforming templates with existing web forum screen participants had seen before. Each template was shown (the order depending on the counterbalancing) together with its Likert type scale. After participants had rated the first template, the second template was displayed with another Likert type scale. The third and final part of the test consisted of the non-conforming template screen with 6 key areas highlighted. Participants were then asked to fill in what they thought each area was for in the forum. When completing this final part a screen thanking participants for taking part in the test was shown. 145
- 168. Assessing the results of the survey in terms of user perceptions 5.2.5. Results The average time that participants took to complete the test was 5 minutes and 39 seconds. The fastest participant took approximately 2 minutes to complete the test and the slowest took 25 minutes. Section 1 The results for the first part of the survey are very diverse with some icons being easily identified and others not being identified at all. Because participants were free to write what they wanted for every icon, the words they used to express an idea were not always the same (e.g. identification, identity and user id were used to describe the ‘User ID’ icon), so for the purpose of analysis words were grouped into similar concepts and the most frequent results are shown on Table 28. A complete list of all the entries and their frequency count can be found in Appendix J. TABLE 28 - MOST FREQUENT ICON ASSOCIATIONS AND FREQUENCY (ANSWERS CONFORMING TO THE MEANING OF THE ICON ARE ITALICIZED). Icon1 Icon2 Icon3 Icon4 Icon5 1 Help 23 Password 14 Write new post 19 Calendar 1 1 Messenger FAQ 9 Security 8 Reply 4 Dates 4 2 Member’s list Get info 1 Lock 3 Register 0 Today’s post 5 0 Icon6 Icon7 Icon8 Icon9 Icon10 Support Calendar 32 User profile 17 Time & date 12 9 Last post 5 VOIP Gallery 1 User ID 5 Appointment 3 6 Next page 3 Thread Archives 1 Video 2 Events 2 0 Go to 3 starter Icon11 Icon12 Icon13 Thread 7 Post/Reply 7 Search 15 Cotton reel 1 Quote 3 View 2 Tailor 1 Comments 2 Zoom 2 Section 2 In this part both the conforming and non-conforming pages were assessed for their similarity with other forum pages that participants had used before. Twenty-seven 146
- 169. Assessing the results of the survey in terms of user perceptions participants considered the conforming page more similar to other forums than the non-conforming one. Only one participant thought that the non-conforming page was more like other web forums than the conforming. Six participants found no differences between the layout of the conforming and non-conforming pages in relation to existing web forums (i.e. attributed both pages the same value on the scale). Overall, the conforming page was thought of as more like other web forum pages with a median of 3.5 points on the scale and the non-conforming templates was thought of as different from other web forum pages with a median of 2.0 points in the scale. A Wilcoxon signed rank test showed that the difference between the two scores was significant (W=396.0, n=28, p<0.001). Section 3 As happened in Section 1, in this Section participants were allowed to write as they pleased therefore some answers expressed the same concept with different wording. So a similar procedure as the one used to group the answers in Section 1 was used here. The grouped results can be seen in Table 29 and a complete table with the ungrouped results can be seen in Appendix K. TABLE 29 - GROUPED RESULTS FOR THE IDENTIFICATION OF THE AREAS OF THE NON-CONFORMING TEMPLATE. Area 1 (login field) Area 2 (Menu) Area 3 (Threads table) Login 29 Menu 12 List of threads 22 Don’t know 2 Don’t know 3 Main content 3 Entry 1 Access/action 3 Don’t know 3 Search 1 User details 3 Entry box 2 User function 1 toolbar 3 Background 1 navigation 2 Conversation 1 information 2 Page number 1 links 2 Search result 1 other 4 Area 4 (Thread) Area 5 (Key/legend) Area 6 (Time and date) Thread/topic 24 Don’t know 8 Time & date 23 Don’t know 3 Key / legend 6 Don’t know 5 link 2 Online users 5 Save 2 Thread listing 2 Change colour 3 Copyright 1 search result 1 Copyright 1 Close 1 criterion 1 Footer text 1 Login 1 Forum status 1 Stats 1 Guide to new posts 1 links 1 Meta data 1 Site details 1 Permissions 1 147
- 170. Assessing the results of the survey in terms of user perceptions 5.2.6. Discussion Section 1 – Icons The results from the icon identification section were not clear–cut, with some icons being associated with the expected web forum textual elements, while others not at all. Although participants were informed that they had to associate the icons with web forum elements, results show that some icons were associated with certain software (e.g. icon 5 with MSN or icon 9 with Skype) which are not part of web forums. Although all icons came from icon libraries available online, it is possible that their shape resembles that of the icon from the software cited. Other icons were associated with features which are not present in web forums (e.g. icon 9 with Voice over IP, icon 6 with gallery). It is likely that because these icons are not commonly present in web forums that the associations which participants might have for them are related to previous experiences with similar icons in other software or interface. This could explain why even when asked to relate the icons to a forum, participants kept associating them with meanings they were familiar with. The icons that were most correctly identified were: icon 2 (Password), icon 6 (calendar), icon 8 (time & date) and icon 12 (reply). Icon 4 (Today’s Post), icon 3 (register) and 9 (thread starter) were not correctly identified by any participant. A large number of participants incorrectly identified other icons: icon 1 (FAQ), icon 5 (member’s list), icon 7 (user ID), icon 10 (last post), and icon 11 (thread). However, some participants did get the right meaning for these icons with the frequency of correct textual elements being cited staying between the most and the least cited words. This mixed result could be explained by the fact that the icons were presented out of context so participants had trouble identifying them (Moyes, 1995). So it is likely that when placed in context and in position within the interface, participants will be able to better identify the icons. However, in the following experiments tasks shall be designed to require that participants only interact with the icons that had a high frequency of correct identification. Section 2 – Conventionality of the templates The conforming forum page was perceived as the one most similar to other web forum interfaces participants have used. The fact that participants were recruited from 8 different web forums strengthens the conviction that the conforming templates chosen in the previous experiments faithfully represent what is perceived 148
- 171. Assessing the results of the survey in terms of user perceptions by web forum users as a convention-conforming web forum design. The non- conforming template was rated by most participants as quite different from the web forum interfaces they were accustomed to seeing. This also reinforces the idea that the templates violate some existing visual conventions for web forums and consequently will suit the purposes of the future experiments. Part 3 – Identification of interactive elements of the forum in the non- conforming template As mentioned previously, this section was intended to verify if users who were familiar with web forums would be able to identify the interactive elements of a forum on the non-conforming template. The results show that all interactive elements had high rates of identification and the key/legend element, although having a high number of people who could not identify it at all (8 participants), had 6 participants identifying it for what it was. This poor identification rate might be explained by the fact that in a conforming web forum, the key/legend element usually has graphic and textual components whereas in the non-conforming it is completely textual. Overall, identification of the interactive elements on the non-conforming page was fairly straightforward. The fact that participants were able to identify the interactive elements of the interface (e.g. menu) may provide some form of contextual cues which may help them to infer meaning to the icons present in the interface (e.g. FAQ icon, Calendar icon) more easily. 5.2.7. Conclusion The results of this experiment (experiment 3) show a clear distinction in the perception of the participants regarding the conventionality of the templates. However it indicates that participants have no great difficulties in working out what the interactive elements of a non-conforming web forum are. The icons have presented some difficulties when it comes to relating them to textual elements present in web forums. However, because the icons were presented out of context and not clustered, no extra cues (such as positioning and grouping) were given to participants to help them on the identification process. The interactive elements on a web forum page were easily identified by the participants. It seems that users familiar with web forums might be able to overcome some usage difficulties in the non-conforming forum by inferring and interpreting 149
- 172. Assessing the results of the survey in terms of user perceptions the interface based on their existing knowledge of forums. As mentioned before, it is possible that that the icon identification problems might be minimized when these are placed in context within the interface. Finally, this experiment indicates that the templates selected to build the material for the next experiments are good, valid representations of what a forum does and does not look like. The conforming template is in conformity with what users expect a forum page to look like. Users did not perceive the non-conforming template as similar to other existing web forums which does indicate that, at least to some extent, they violate what users expect to see in a web forum page layout. It also shows that any difficulties participants may have regarding the interpretation of the icons and recognition of the interactive elements in a web forum are not difficult to overcome. 5.3. Final remarks Users perceive non-conforming interfaces as harder to use than ones that follow conventions. However, results from the last experiment which asked participants to identify the interactive elements in a non-conforming forum page indicate that this difficulty might not be as significant as users may believe. As seen before, there seems to be a connection between the identification of icons in an interface and the level of familiarity with it (McDougall & Curry, 2004). Even though literature highlights the importance of positioning to icon identification, and the results of the first two experiments shown that symbolization of elements has an important effect on user perceptions of how easy an interface is to use, no research has yet fully investigated the impact which changing the position, symbolization and configuration of interface elements has on the performance of users who are familiar with a specific interface. Research points out (McDougall, et al., 1999), that by using a certain interface for some time, users will be familiarized with the icons and identification performance will no longer suffer. It is possible that after using an interface for some time users will become familiar with not just the icons it contains, but the whole interface. However, it is possible that the time necessary for this familiarization to happen may be too long. 150
- 173. Assessing the results of the survey in terms of user perceptions The next Chapter describes an experiment which investigates this matter further, by looking at what happens with the performance of both users who are familiar and users who are unfamiliar with web forums when using a conforming and a non- conforming web forum. 151
- 174. Chapter 6 Visual conventions and familiarity when searching in web forums 6.1. Introduction The exploratory study conducted earlier (see Chapter 3, section 3.3) showed that some web page layouts are quickly recognized by users. Users are then able to predict what type of content will be found within the page. As this thesis aims at investigating the effect of conforming to and not conforming to visual conventions on the time and accuracy to complete tasks by web users, a survey was conducted (Chapter 4) in an attempt to establish what is conventional for a particular category of website. The survey listed the most frequently occurring elements within a specific category of websites: the web forums. The results of the survey were then used to create several templates which varied their degree of conventionality. Web forum users were then shown the templates and asked to compare these in terms of their similarity with other existing web forums and their perceived ease of use (see Chapter 5). The outcome of this experiment was a rating of templates ranging from those considered more similar to existing web forums and those considered least similar to existing web forums and those considered easy to use and those considered not so easy to use. This Chapter continues the study of visual conventions. It describes an experiment designed to investigate how not conforming to visual conventions of a web forum affects the performance of users who are familiar and users who are unfamiliar with the conventions. The experimental material used here was based on the results obtained in the survey and experiments as reported in Chapter 5. 6.1.1. Familiarity The Concise Oxford English Dictionary defines familiar as something ‘well known from long or close association’. From this definition it is possible to establish that for 152
- 175. Visual conventions and familiarity when searching in web forums familiarity with a stimulus to be achieved it requires prior contact with the stimulus for a certain period of time. From this definition it is also possible to directly associate familiarity with genres. As seen in Chapter 2, section 2.2.5 genres emerge from the communicative needs of a discourse community. This discourse community becomes familiar with particularities of the genre and comes to expect these to be followed by each document belonging to that genre. But it seems that prior experience with a stimulus is not the only requirement to breed familiarity. Although some authors do indeed point out that prior experience with a stimulus can produce a feeling of familiarity (Whittlesea, 1993; Zhang & Ghorbani, 2004), prior experience is not the only factor which influences the feeling of familiarity. According to Zhang and Ghorbani(2004), the following factors also contribute: ! Repeated exposure – the feeling of familiarity increases with repeated exposure to a stimulus. ! Level of processing - associated with the amount of familiarity which is obtained. The deeper the processing of the stimulus, the greater the feeling of familiarity achieved. Deep processing (processing meaning) results in an increase in familiarity when compared with shallow processing (processing perceptual aspects). ! Study duration – longer times spent studying something lead to an increase in familiarity. For web navigation purposes the authors define study duration as the period of time a user stays navigating and using a website. ! Forgetting rate - defined by the interval of time between two consecutive exposures to a stimulus. In terms of web navigation the exposures can be thought of as visits to one website. The longer the time interval between exposures , the greater the decrease in the feeling of familiarity (Zhang & Ghorbani, 2004). An experiment conducted by Gefen (2000), where 84 web users answered questions on familiarity and website trust, shows that in the World Wide Web, familiarity is an important component in building user trust towards a website. Apparently familiarity is not only important for creating trust in users but, in fact some authors share the opinion that web users have extreme desire for familiarity (Usborne, 2003; Zhang & Ghorbani, 2004). 153
- 176. Visual conventions and familiarity when searching in web forums Familiarity, however, is a fairly broad concept and a person can be familiar with different aspects of different things. The fact that research has shown that one kind of familiarity can be affected by other types of familiarity (Lim, Benbasat, & Todd, 1996) makes it important to look at other types of familiarity in order to understand how familiarity with a genre can be achieved. Thus a brief review of the types of familiarity relevant to this study is outlined below. Nielsen (1993, p.43) has subdivided user experience into three categories: experience with the system, experience with computers and experience with the task domain. Since familiarity comes from experience (Zhang & Ghorbani, 2004), after acquiring experience with each of the aspects outlined above, it is possible to say that a user achieves: familiarity with the system, familiarity with computers and familiarity with the task domain. Although this division seems quite useful, at least in terms of navigating the Web users can be familiar with a certain task type but be unfamiliar with the topic being addressed in the task (Capra, 2006, p. 3). A good example would be a user searching for financial information online: the user can be extremely familiar with searching for information online but have no familiarity with the financial topic. Yet another type of familiarity which should be considered is familiarity with a text genre. A reader can be familiar with a text genre such as academic papers, but be ignorant of a particular topic addressed in one specific paper they are reading. Therefore, a more adequate subdivision of familiarity for WWW users which covers a broader range of research done in the area is proposed below: ! Familiarity with the topic. ! Familiarity with the text genre. ! Familiarity with the task. ! Familiarity with the interface (or system). ! Familiarity with computers. All these types of familiarity are somehow linked with genres and in order to become familiar with a genre a user has to achieve some degree of each type of familiarity. However, mapping these types of familiarity to the conventions of a genre is extremely complex since there is much overlapping and some types of familiarity are linked with different genre conventions. 154
- 177. Visual conventions and familiarity when searching in web forums For the purposes of this thesis only the first four types of familiarity will be addressed since the type of users being tested will all be familiar with computers. Familiarity with a specific interface is sometimes referred to as user expertise and users are usually considered novices, experts or somewhere in-between (Nielsen, 1993). Users who are new to an interface (and in a more specific case to a website) are novices and users with intensive use of the interface are considered experts. The problem with this classification starts when one tries to place within the novice- expert interval users who are not novices but have not yet reached expertise status. Those are usually referred to as intermediate users, but their degree of familiarity could be situated closer to the novice or closer to the expert status and this could make a difference in the outcomes of an experiment. It seems that the distinction between the expert users and those who are not novices but also not experienced enough to be expert can be quite hard to establish. In fact in a study which attempted to identify success on web search tasks of experts, intermediate users and novices using different strategies, Tabatabai and Shore (2005) found little difference between experts and intermediate users. The authors attribute the small difference in performance to the fact that intermediate users had Internet search training just before the test started while experts did not. An alternative explanation is that the learning curve for the interface in question showed a steep incline in the initial part and carried on more horizontally from then on (Nielsen, 1993). An example of this situation is illustrated in Figure 60 where the difference in performance (vertical axis) is quite small between intermediate and expert users when compared to the novices. 155
- 178. Visual conventions and familiarity when searching in web forums FIGURE 60 - SAMPLE OF A HYPOTHETICAL LEARNING CURVE TO EXPLAIN THE SMALL DIFFERENCE IN PERFORMANCE BETWEEEN EXPERTS AND INTERMEDIATE USERS FOUND IN TABATABAI AND SHORE (2005). Either way, this study illustrates the difficulty in defining where the boundary between experts and intermediate users should be. Differences in performance between intermediate and expert users can be quite hard to obtain. For the next experiment in the thesis, intermediate and expert users will be grouped together as users who are familiar with web forums. By allocating users who have used web forums before (intermediate and experts) in one group, and users who have never used a forum before (novices), the differences between the two groups are maximized. Familiarity with a topic Research has shown that familiarity with a certain topic can increase online search efficacy (Hirsch, 1997; Kelly & Cool, 2002) and it can also improve performance when searching online for information on a familiar topic (Lazonder, Biemans, & Wopereis, 2000; McDonald & Stevenson, 1998; Patel, Drury, & Shalin, 1998). When users who are familiar with a topic use a familiar interface, a synergistic effect seem to occur and these users change their search strategy (Jenkins, et al., 2003). They tend to explore the content of the site further, going deeper within the structure of the website (Holscher & Strube, 2000; Jenkins, et al., 2003). This illustrates the importance of taking familiarity with the search topic into account when designing 156
- 179. Visual conventions and familiarity when searching in web forums the material for the next experiment. Having a topic with a high probability of users being familiar with can make some participants perform in an extraordinary fashion which could compromise the validity of the results. So, topic familiarity will be minimized by choosing a search topic participants are not familiar with. This will keep user performance from being improved by a participant already knowing about a topic. An extreme example of this situation would be a participant knowing the answer to the task without having to search the forum. Familiarity with text genre Familiarity with a text genre helps familiar users locate themselves within the text structure as shown by Dillon and Schaap (1996). In this study, the experimenters asked users familiar and unfamiliar with reading scientific journals to place paragraphs extracted from journals in the appropriate section (introduction, method, results or discussion). All paragraphs were stripped of explicit structural cues (such as headings) and half of them had other embedded cues (such as references to the section it belonged to, references to literature, formulae and acronyms of tests or research tools) removed. The authors concluded that users familiar with the text genre rely on the semantic details to locate themselves in the information space. Commenting on the results of this experiment, Dillon(2000) added that unfamiliar users, however, have to depend on more explicit spatial indicators such as headings and titles on the screen. Similarly to scientific journals, web discussion forums do have a particular writing style (see Chapter 5) and linguistic conventions which forum users become familiar with. So if the results from Dillon and Schaap (1996) hold true to any hypertext and are extendable to user interfaces, it is possible that in the next experiment, the group of unfamiliar users (i.e. unfamiliar with the interface) might perform better in the conforming forum than in the non-conforming one. Since users in all the groups will be unfamiliar with the topic and unfamiliar users will not have familiarity with the interface either, they will have to rely on explicit cues in order to locate themselves. Because many cues which are textual in the conforming forum will be graphic- in the non-conforming, unfamiliar users tested on the latter are likely to have their sense of location impaired. 157
- 180. Visual conventions and familiarity when searching in web forums Familiarity with the task Familiarity with the task has also been associated with improved performance (Marchionini, 1995), and successful retrieval (Marchionini & Shneiderman, 1988) when searching for information in an electronic environment. Familiar tasks are more likely to be executed through automatic processing whereas unfamiliar tasks are executed via controlled processing (Lim, et al., 1996). While automatic processing is fast, effortless and unconscious (Logan, 1988), controlled processing is time-consuming, effortful and performed in a step-by-step manner (Lim, et al., 1996). Task familiarity is achieved mainly through the frequency with which a user performs a task (Capra, 2006, p.129). A direct implication of this is that users who are accessing a certain website with some regularity are more likely to be familiar with the type of tasks which are commonly executed when using this website. This seems to suggest a connection between task familiarity and interface familiarity. In fact, some believe that when using computer interfaces such as a website, the presence of familiar elements may trigger automatic processing of tasks (Lim, et al., 1996). This could mean that members of a discourse community who are familiar with a certain web genre and more specifically its interface and tasks, when faced with a website interface they are not familiar with (i.e. does not conform with the genre) they will process familiar tasks in a controlled manner. In other words, because the unfamiliar interface does not have elements which are familiar to the user, it can not trigger the automatic processing of the tasks. As a result users would perform tasks in a controlled manner resulting in slower performance. This concept relates to the idea that some cues may trigger the schema that a user has for a particular genre of documents (Vaughan & Dillon, 1998). It follows that automatic processing may be used by the members of a discourse community when faced with a document genre they are familiar with. Familiarity with the interface Research has shown that users who are familiar with an interface seem to perform better than unfamiliar users when searching for information on that interface (Tabatabai & Shore, 2005). One explanation for these results could be that users who are familiar with an interface are also familiar with the tasks it involves while 158
- 181. Visual conventions and familiarity when searching in web forums unfamiliar users are not. So, when faced with a familiar interface (with elements they were familiar with), they use automatic processing of the tasks while unfamiliar users have to rely on controlled processing in order to complete the tasks. Also, users are less likely to become confused or lost in a familiar interface than in an unfamiliar interface (Galletta, Henry, McCoy, & Polak, 2006). This might be explained by the idea presented in Dillon(1991), that repeated exposure to a particular type of hypertext allows users to acquire/develop a structural model which in turn, helps them predict with a certain degree of accuracy the location of information and possibly their location within the hypertext. This fits with the idea expressed earlier in Chapter 2 that not conforming to the expectations of users regarding a genre can cause disorientation, confusion and hinder performance. On the other hand, users who are unfamiliar with an interface, seem to not only perform worse than users who are familiar with it, but also rely more on using the back button when searching (Holscher & Strube, 2000). Furthermore, when unfamiliarity with the interface is allied to unfamiliarity with the search topic, users seem to have severe problems searching for the information they need (Holscher & Strube, 2000). Considering that: ! Users familiar with a certain type of website are familiar with the tasks associated with it, while unfamiliar users are not used to these types of tasks. ! Users familiar with a certain type of website are likely to have a structural model of the website which helps them predict the location of information. ! Even though all users (familiar and unfamiliar) will not have knowledge of the search topic, because unfamiliar users will also not know the interface, they are likely to have severe problems when searching for information. It seems plausible to expect that in the experiment described in this chapter users who are familiar with web forums will perform better than unfamiliar users on both the conforming and non-conforming interfaces. It is also expected that familiar users will perform better on the conforming interface than on the non-conforming one. Finally based on the literature reviewed here there are two possibilities for the performance of unfamiliar users: (a) they will show no difference in performance between the two layouts or (b) they will perform worse on the non-conforming 159
- 182. Visual conventions and familiarity when searching in web forums layout since many of its elements are pictorial thus requiring them to be correctly identified (see page 157). As mentionaed earlier, it is believed that web users make mental models of the sites they visit (Danielson, 2002a, p. 77; Vaughan & Dillon, 1998) and that this mental model is based on the structural organization of the information on the page (MacKay, et al., 2004). For the creation of this mental model the layout of the page and visual and semantic cues all contribute. However, despite the layout of the page and its visual cues contributing to the formation of the mental model of the site, the contribution of the layout of the page to user performance once the structural mental model of the site is created is unknown. So here it is proposed that familiarity with the interface be split into familiarity with the interface layout, and familiarity with the interface structure. These two types of familiarity with an interface are directly connected with the concept of visual and structural genre conventions respectively. Although the distinction between the two types of interface familiarity might not seem very clear at first it is not difficult to imagine examples of websites which hold similar structures but different layouts. For instance, websites belonging to the online news genre tend to have their news organized in the same or very similar categories and to share similar structures. However, their interface elements might have different symbolization modes, positions and configurations (see Chapter 5, section 5.2.1 for more explanation on these) resulting in different layouts. Here, these two interface characteristics will be considered separately. Although research has pointed out that when layout and structure are changed, familiar users reported feeling disoriented (MacKay, et al., 2004) and performance suffered (Vaughan & Dillon, 2006) the impact of radically changing the conventional layout, but maintaining the structure of a web site on users who are familiar and unfamiliar with the original interface has not yet been investigated. This type of information is extremely valuable for web designers who need to visually alter the layout of a website and need to know how that might affect its users. Summary Defining familiarity seems a daunting task since there are many types of familiarity most of which are interlinked, influencing each other and consequently user performance. Familiarity with a task, as well as familiarity with an interface has been reported to improve user performance. Furthermore, there seems to be a relationship 160
- 183. Visual conventions and familiarity when searching in web forums between the two: users who are familiar with an interface are highly likely to become familiar with the tasks performed in that particular interface. Also, familiarity with the search topic can improve user performance, and when combined with a familiar interface seems to boost user performance even more. Since the focus of this next experiment will be on visual genre conventions and familiarity with an interface layout (i.e. the visual conventions of the genre), it seems inevitable that familiar users will have some sort of familiarity with the tasks to be performed. However, some tasks designed to assess user ability to retrieve information from the interface will be included. These tasks have the purpose of testing if there is a variation in performance when the layout of the site changes. They also have the added benefit of not being tasks which are commonly performed on web forums. Topic familiarity will be minimized by having a search topic which participants will not be familiar with. In summary, the main point this study will address is the impact on user performance of changing the visual conventions of a page, and keeping its other genre conventions intact. This study aims to investigate this issue by testing the performance of users familiar and unfamiliar with web forums on both the conforming and non-conforming web forums websites generated from the survey and the experiments described in Chapters 4 and 5 respectively. The hypotheses here are that (a) overall, familiar users will perform better than unfamiliar users; but (b) within the group of users who are familiar with web forums, those using the non- conforming forum version will perform worse than the participants using the conforming one. 6.2. Experiment 4 – Visual conventions and searching for information in web forums 6.2.1. Objectives The objectives of experiment 4 are to: ! test the effect of familiarity with a web forum interface on user performance (time to complete a task and accuracy of task answer). ! test the effect of website conventionality on user performance (time to complete a task and accuracy of task answer). ! test if user performance varies between tasks which involve retrieving information from content and tasks which involve retrieving information from the interface. 161
- 184. Visual conventions and familiarity when searching in web forums 6.2.2. Method Participants 40 participants, 20 familiar with web forums and 20 unfamiliar with web forums participated in this experiment. Participants who had never searched for information and/or posted on a web forum before were considered unfamiliar with web forums. Participants who had either searched for information and/or posted messages on web forums were considered familiar. However, it was ensured that participants had navigated the interface of a web forum and not simply reached the post they wanted via and external link (e.g. a search engine). Also to minimize familiarity with the search topic participants were not expert PDA (content of the experimental forums) users. In fact only one participant has reported using PDAs a few times. All of the participants were fluent in English either as a first or second language. The academic background of participants ranged from completed GCSEs to PhD students. Participants in this experiment have not taken part in any previous experiments conducted for this thesis. Experiment design The 20 familiar and unfamiliar participants were divided into two groups of 10 each. In total the test used four groups, two in the ‘familiar category’ and two in the ‘unfamiliar category’. In each category one group was tested on the conforming forum and the other on the non-conforming forum (see Table 30). TABLE 30 - DISTRIBUTION OF GROUPS WITHIN THE FAMILIAR AND UNFAMILIAR CATEGORIES AND THE FORUM LAYOUT EACH GROUP WAS TESTED ON. Participants Tested on Group FC Familiar with forums Conforming forum Group FNC Familiar with forums Non-conforming forum Group UC Unfamiliar with forums Conforming forum Group UNC Unfamiliar with forums Non-conforming forum The variables that were being changed (independent variables) were the conventionality of the web forum, the familiarity of the participants with web forums and the type of tasks participants were performing. The variables that were being measured (dependent variables) were time to complete the tasks, and number of correct answers. The perception of the participants regarding some aspects of the 162
- 185. Visual conventions and familiarity when searching in web forums forum was also measured. Because familiar participants have had prior experience with other forums, they were also asked to judge the conventionality of the forums in the experiment. Materials Web forum designs Two different designs were created for this experiment. The designs were based on results of the previous experiments (experiments 1 and 2 - see Chapter 5), which tested whether the assumptions of conventionality drawn from a survey on web forums were in accordance with what forum users perceived as conventional and what they perceived as easy to use. One design consisted of a web forum which followed the templates from the survey which was perceived as conventional by users. The other site was meant not to conform to conventions of the genre, so its design was based on the templates from the survey which forum users thought were the least conventional. As seen earlier in Chapter 5, a web forum usually consists of three distinct levels: the forum index, a threads page and a post page and for the purposes of this experiment each experimental forum had these three levels. The template from the experiments 1 and 2 in Chapter 5 that was most frequently chosen as conventional and easy to on each forum level, was used to create each level of the conforming forum. The template which was chosen as conventional and easy to use least frequently on each forum level was used to create each level of the non- conforming forum (see Appendix I). The templates used to make each forum level are shown in Table 31. TABLE 31 - TEMPLATES USED TO CREATE EACH LEVEL OF THE CONFORMING AND NON-CONFORMING FORUMS. Forum level Conforming Non-conforming Index page C11 N14 Threads page C21 N23 Post page C31 N34 163
- 186. Visual conventions and familiarity when searching in web forums The templates chosen to be used in the design of the conforming forum: C11 (level 1), C21 (level 2) and C31 (level 3) had the symbolization, position and configuration of the elements fixed at the most frequent values gathered from the survey. As explained in the previous chapter (Chapter 5), the templates used for creating the non-conforming forum screens: N14 (level 1), N23 (level 2) and N34 (level 3), were chosen from the templates which were least frequently picked as similar to existing forums. Because of this, the resulting templates did not have the symbolization, position and configuration of the elements fixed in a similar manner. While level 1 and 2 had the symbolization and the configuration of the elements fixed at the least frequent values, and the position fixed at the most frequent value, level 3 had symbolization and position at the least frequent value, and configuration at the most frequent value (Table 32). TABLE 32 - HOW VALUES WERE APPLIED TO THE VARIABLES SYMBOLIZATION, POSITION AND CONFIGURATION FOR DESIGNING THE FORUM TEMPLATES. VALUES COULD BE THE MOST FREQUENT (MF) OR THE LEAST FREQUENT (LF). Symbolization Position Configuration C11 MF MF MF C21 MF MF MF C31 MF MF MF N14 LF MF LF N23 LF MF LF N34 LF LF MF Graphic elements The graphic elements used (e.g. icons, buttons) are the same as those used in experiment 3 which tested the results of the survey (see Chapter 5, section 5.2). Due to the way the non-conforming forum was designed, elements which were presented in a certain style in the conforming version (e.g. text), were presented differently in the non-conforming (e.g. graphic). However, whenever the symbolization mode of the element was the same on the conforming and non-conforming forums, the same graphic element was used. As explained in Chapter 5, the elements which varied their symbolization mode on the conforming and non-conforming forums were designed keeping aesthetic consistency (consistency of style and appearance) with the other existing elements. 164
- 187. Visual conventions and familiarity when searching in web forums The same colour scheme was used to design both the conforming and non- conforming forums Content The experiment involves participants searching for answers within a web forum. As mentioned in the introduction, familiarity with the search topic can help improve the performance on search tasks. So it is important that the knowledge participants have on the subject of the forum is minimal in order to keep them from knowing the solution to the tasks beforehand. The content of choice was related to Personal Digital Assistants (PDAs) since it not only provides a reasonable amount of questions to be answered, but is also simple enough to be comprehensible without being common knowledge. Because the forum content is divided according to different PDA brands and/or operating systems it is possible that most participants will not have this knowledge. Without this, it will be quite hard for participants to grasp the organizational structure of the website, so a brief written introduction to PDAs (see Appendix L) was created. This introduction outlines the brands, operating systems and pictures of the PDAs found within the forum. This introduction will not only help participants to understand the technical terms used in the organization of the forum but will also ensure all participants have the same basic level of knowledge about PDAs. Both the conforming and non-conforming forum convey the same content so that any influence that content can have on the performance of the participants is balanced out. The content for the experimental forums was gathered from an existing web forum. However, the textual content used in the experiment is a reduced version of the existing one, so the breadth of the experimental forum is smaller than the real version. This should make searching for answers not too tiring for participants. In the following pages there are samples of the index page (first level of the forum) of the finished conforming (Figure 61) and the non-conforming (Figure 62) designs. The final designs consisted of the template chosen for each level (of both the conforming and non-conforming forums) with the content included and with the colour scheme added. 165
- 188. Visual conventions and familiarity when searching in web forums FIGURE 61 - SAMPLE OF THE INDEX PAGE DESIGN (LEVEL 1) OF THE CONFORMING WEB FORUM. FIGURE 62 - SAMPLE OF THE INDEX PAGE DESIGN (LEVEL 1) OF THE NON-CONFORMING WEB FORUM. 166
- 189. Visual conventions and familiarity when searching in web forums Tasks A set of five tasks (see Appendix M) was created to test the participants on the forums. There were two main ideas behind the creation of the experimental tasks: one was to emulate the range of possible activities which a web forum user would normally perform while searching for information within a web forum; the other was to assess user ability to retrieve information from the interface. So, two types of tasks were designed: one where participants had to retrieve information from the written content (normal web forum activity), and the other where participants had to retrieve information from the interface of the forum (interface retrieval). Tasks were given to participants in the same order. Questionnaire A questionnaire was developed to assess what participants thought of the design of the forums and the ease/difficulty of finding information. The questionnaire contained 3 questions: the first two were directed to all participants and assessed what participants thought of the website they had used in the experiment. The last question was only to be answered by participants who were familiar with web forums, and asked them to compare the appearance of the forum they used in the experiment with other existing forums. The possible answers to the questions were given on a Likert scale using five steps ranging from 1 to 5. Table 33 illustrates the questions and the meaning of the extreme values of the scale in each case. Participants answered these questions by marking the number on the scale they thought best represented their perception. TABLE 33 - QUESTIONNAIRE ITEMS WITH THEIR RESPECTIVE EXTREME VALUES MEANINGS. Question 1 5 1. In your opinion, how easy or difficult was it to find information in Very easy Very difficult the forum? Very well 2. Overall, how would you rate the layout of the web forum pages? Badly designed designed 3. How would you compare the layout of this web forum with other Very different Very similar web forums? 6.2.3. Procedure Each test started with the experimenter explaining to the participants what the tasks would be. Participants were told that the experiment consisted of them looking for 167
- 190. Visual conventions and familiarity when searching in web forums answers to different questions (tasks) within a web forum. It was explained that the tasks would be handed to them on a printed sheet of paper one at a time and that the answers should be written on the same sheet. Participants were asked to look for the answers as quickly as possible and to answer the tasks as accurately as they could. They were told not to feel pressured as what was being tested was the design and not them. Then the participants were given the sheet of paper with the introduction to PDAs. After that, the participants were shown the forum condition to which they were assigned (either conforming or non- conforming) and given a sheet of paper containing the first task. As soon as the participants finished reading the task and looked at the screen the time measurement started. Time measurement stopped as soon the participants had finished writing the answer to the task. The participants were then asked to return to the forum index page and were given the next task. After the participants had answered all five tasks they were presented with the questionnaire. 6.2.4. Results Time A Mann-Whitney test was used on all time results to determine if differences were significant. The results found on the analysis of the time to complete the tasks are shown on Table 34 with the significant differences highlighted. TABLE 34 - MEAN VALUES AND STANDARD DEVIATION OF THE TIME TO COMPLETE TASKS IN EACH GROUP AND CONDITION WITH THE SIGNIFICANT DIFFERENCES HIGHLIGHTED AND CIRCLED BY A DOTTED LINE. THE COLUMN MEANS SHOWS THE MEANS OF THE USERS WHO USED THE CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEANS REPRESENT THE MEANS OF THE FAMILIAR AND UNFAMILAR USERS. Time to Conforming Non-conforming Row Means complete the tasks Mean SD Mean SD Mean SD 449.2s Familiar 115.7 509.7s 165.3 479.4s 142.3 627.3s Unfamiliar 627.6s 132.0 627.0s 214.5 173.3 Column 538.4s 151.6 568.4s 195.9 Means 168
- 191. Visual conventions and familiarity when searching in web forums Familiar and Unfamiliar users Overall participants who were familiar with web forums took less time to answer the tasks than participants who were not familiar with web forums (see ‘row means column’ in Table 34). A Mann-Whitney test showed this result was significant (W=308.0, n=20, p=0.006). Conforming and Non-conforming forums There was no significant difference in time to complete the tasks (W=404.0, n=10, p=0.887) between participants who used the conforming forum and participants who used the non-conforming forum (see the column means for the conforming and non- conforming forums in Table 34). Familiar-conforming and Unfamiliar-conforming Comparing the time the familiar group took to complete the tasks on the conforming forum with the time the unfamiliar group took to complete the tasks on the same forum, it is possible to observe that the familiar group performed significantly faster than the unfamiliar group (W=67.0, n=10, p<0.005). Familiar-non-conforming and Unfamiliar-non-conforming A Mann-Whitney non-parametric test showed no significant difference in the time taken to complete the tasks between participants in the familiar group who used the non-conforming forum and the participants in the unfamiliar group who used the non-conforming forum (W=92.0, n=10, p=0.345). Familiar-conforming and Familiar-non-conforming A Mann-Whitney non-parametric test showed no significant difference in the time taken to complete the tasks between participants in the familiar group who used the conforming forum and the participants in this group who used the non-conforming forum (W=96.0, n=10, p=0.521). Unfamiliar-conforming and Unfamiliar-non-conforming A Mann-Whitney test showed no significant difference in the time to complete a task between participants in the unfamiliar group who used the conforming forum and the participants of this group who used the non-conforming forum (W=109.0, n=10, p=0.791). 169
- 192. Visual conventions and familiarity when searching in web forums Correct scores Scores were defined according to the number of correctly answered tasks by each group. A Mann-Whitney test was used on all the correct score results to determine if differences were significant. Looking at the significant results shown on Table 35 it is possible to see that the significant differences in score follow the same pattern as the significant differences in time taken to complete the tasks. TABLE 35 - MEDIAN VALUES OF THE NUMBER OF CORRECT SCORES IN EACH GROUP AND CONDITION WITH THE SIGNIFICANT DIFFERENCES HIGHLIGHTED AND CIRCLED WITH A DOTTED LINE. THE COLUMN MEDIANS SHOWS THE MEDIANS OF THE USERS WHO USED THE CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEDIANS REPRESENT THE MEDIANS OF THE FAMILIAR AND UNFAMILAR USERS. Correct scores Conforming Non-conforming Row Medians Familiar 3.0 3.0 3.0 Unfamiliar 2.0 3.0 2.4 Column Medians 3.0 3.0 Familiar and Unfamiliar users Overall, the familiar group answered more tasks correctly than the unfamiliar group (see row medians in Table 35). This difference was significant (W=483.0, n=10, p=0.041). Familiar-conforming and Unfamiliar-conforming Comparing the scores of participants using the same forum type across the familiar and unfamiliar groups, it is clear that when using the conforming forum familiar participants achieved better scores than the unfamiliar participants. A Mann- Whitney test showed this result to be significant (W=135.0, n=10, p=0.019). Familiar-non-conforming and Unfamiliar-non-conforming Comparing the scores of the familiar and unfamiliar participants who used the non- conforming forum no significant difference is evident (W=111.5, n=10, p=0.635). Also, comparing the scores for participants within the same group who used each version of the forums showed no significant results. Familiar-conforming and Familiar-non-conforming A Mann-Whitney test found no significant difference in scores between participants in the familiar group who used the conforming forum, and participants in this group who used the non-conforming forum (W=117.0, n=10, p=0.3847). 170
- 193. Visual conventions and familiarity when searching in web forums Unfamiliar-conforming and Unfamiliar-non-conforming In the unfamiliar group a Mann-Whitney test showed no significant difference in scores between participants who used the conforming forum and participants who used the non-conforming forum (W=95.0, n=10, p=0.458). Right answers per second (RPS) rate One way of looking at the results to better assess any improvement or hindrance in performance is to combine time taken and correct answers into one rate (Whiteside, Jones, Levy, & Wixon, 1985). Thus, the number of correct answers and the time to complete the tasks were combined into one measurement. However, because of the great number of zero correct scores (participants not answering a task correctly), in most cases combining scores and time to complete tasks caused the contribution of the time factor to be lost. A data transformation was then applied to the data in order to adjust the scores that were zero to a value that would not cancel out the contribution of time. The problem caused by this can be appreciated when considering the hypothetical example of two participants: one that takes a long time to before quitting the task and another participant who quickly quits the task. Even though these two participants did not complete the task correctly, the time they took to complete the task is relevant and can not be ruled out. Kirk (1995) suggests four types of transformations that can be applied to the data: square-root transformation, logarithmic transformation, reciprocal transformation and sine transformation. Kirk also suggests a method for selecting which transformation is more appropriate for a particular data set. After applying the method described in Kirk (1995), the square-root transformation came out as the most appropriate to the data since it resulted in the distribution that was closest to the normal. To make the resulting number easier to work and understand it was multiplied by constant value of 100. This resulting number indicates the rate of right answers per second so that a higher value indicated better performance. The formula used to calculate it is shown in Figure 63. 171
- 194. Visual conventions and familiarity when searching in web forums FIGURE 63 - FORMULA FOR THE DATA TRANSFORMATION AND CALCULATION OF THE RATE OF RIGHT ANSWERS PER SECOND. Familiar and Unfamiliar users Familiar users showed an overall better performance than unfamiliar users (see Figure 64) with a mean RPS rate of 1.44 (S.D.=0.55) against a mean of 0.85 (S.D.=0.62) in the unfamiliar group. A Mann-Whitney test showed that this difference was significant (W=522.0, n=20, p<0.03). FIGURE 64 – OVERALL PERFORMANCE (MEASURED BY THE RPS RATE) OF FAMILIAR PARTICIPANTS COMPARED TO THE UNFAMILIAR PARTICIPANTS. Familiar-conforming and Unfamiliar-conforming When using the conforming layout, participants in the familiar group performed better (Mean=1.61, S.D.=0.62) than the participants in the unfamiliar group (Mean=0.64, S.D.=0.25). Again a Mann-Whitney test on the data revealed a significant difference in performance between familiar and unfamiliar participants when using the conforming forum (W=518.0, n=20, p<0.01). Similarly to the analysis of the time to complete the tasks and number of correct answers, no other significant results were found between: 172
- 195. Visual conventions and familiarity when searching in web forums (a) Familiar participants and unfamiliar participants on the non-conforming layout. (b) Familiar participants on the conforming layout and non-conforming layout (c) Unfamiliar participants on the conforming layout and non-conforming layout The results are summarized in Table 36. TABLE 36 – MEAN VALUES OF THE RPS RATES IN EACH GROUP AND CONDITION WITH THE SIGNIFICANT DIFFERENCES HIGHLIGHTED AND CIRCLED WITH A DOTTED LINE. THE COLUMN MEDIANS SHOWS THE MEDIANS OF THE USERS WHO USED THE CONFORMING AND NON-CONFORMING FORUMS RESPECTIVELY AND THE ROW MEDIANS REPRESENT THE MEDIANS OF THE FAMILIAR AND UNFAMILAR USERS. Correct scores Conforming Non-conforming Row Medians Familiar 1.61 1.27 1.44 Unfamiliar 0.70 0.99 0.85 Column Medians 1.15 1.13 Task type As mentioned before experimental tasks were divided in two categories: content- related and interface-related. On content-related tasks, participants had to read through the content of the forum posts to find the answer. On interface-related tasks participants had to search the visual elements of the interface (e.g. looking for an icon that indicates that a post is locked) to find the answer to the task. Of the five tasks, three were content-related and two were interface-related. To analyze whether familiarity and conventionality had a different effect on performance depending on the type of task (content or interface) being performed, a two within-subjects, one between subjects factor ANOVA was applied to the RPS rate. From the results (Table 37) an interaction between task type and conventionality is evident. The main effects of familiarity and task type were also detected. TABLE 37 - RESULTS FOR THE TWO WITHIN-SUBJECTS ONE BETWEEN-SUBJECTS FACTOR ANOVA. Source (between-subjects) SS df Mean Sq. F-ratio p Familiarity 216.773 1 216.773 4.811 0.035 Conventionality 41.321 1 41.321 0.917 0.345 Familiarity x Conventionality 49.639 1 49.639 1.102 0.301 Error 1622.126 36 45.059 Source (within-subjects) SS df Mean Sq. F-ratio P Task type 407.523 1 407.523 11.600 0.002 Task type x Familiarity 56.710 1 56.710 1.614 0.212 Task type x Conventionality 254.676 1 254.676 7.249 0.011 Task type x Familiarity x Conventionality 16.135 1 16.135 0.459 0.502 Error 1264.686 36 35.130 173
- 196. Visual conventions and familiarity when searching in web forums The graph indicates (see Figure 65) that participants using the non-conforming site performed better than the conforming group on interface tasks, however, on content tasks both groups showed an equivalent performance. Surprisingly, these results suggest that the non-conforming interface resulted in better performance. FIGURE 65 –INTERACTION BETWEEN CONVENTIONALITY AND TASK TYPE. A main effect of task type showed performance to be overall better on interface tasks than on content tasks (Figure 66). FIGURE 66 - PERFORMANCE (MEASURED BY THE RPS RATE) ON INTERFACE TASKS COMPARED TO CONTENT TASKS. 174
- 197. Visual conventions and familiarity when searching in web forums Questionnaire For all the analysis of the questionnaire data the Mann-Whiney test was used. Overall there was no difference (W=351.5, n=20, p=0.095) in perception of the difficulty of finding information in the non-conforming forum (Median=4.00) and in the conforming forum (Median=3.00). However there is a trend towards significance of this result and it is possible that a larger sample size would reveal this difference to be significant. For the second question, the test also showed no difference in the overall perception of participants regarding the quality of the design of the conforming (Median=3.00) and non-conforming (Median=3.00) websites (W=405.5, n=20, p=0.908). Familiar users Looking at the questionnaire answers in the group of users who were familiar with web forums no difference in perception of ease/difficulty in finding answers between the conforming and non-conforming forum was found (W=88.5, n=10, p=0.201). In terms of the quality of the design of the conforming and non-conforming websites the test revealed no difference in user perception (W=105.0, n-10, p=1.000). However, when comparing the conforming and non-conforming design to previously visited web forums (Figure 67) participants found the layout of the conforming forum (Median=4.00) more like other forums than the non-conforming one (Median=3.00). This result was statistically significant (W=137.0, n=10, p<0.01). FIGURE 67 - MEDIAN VALUE OF THE ANSWER GIVEN TO THE QUESTION: ‘HOW WOULD YOU COMPARE THE LAYOUT OF THIS WEB FORUM WITH OTHER FORUMS?’ (1=VERY DIFFERENT, 5=VERY SIMILAR). Participants who used Participants who used the the conforming forum non-conforming forum 175
- 198. Visual conventions and familiarity when searching in web forums Unfamiliar users In the unfamiliar group of users, similar results to the familiar group were found. There was no difference in the perception of the ease/difficulty of finding information between the conforming and non-conforming websites (W=92.0, n=10, p=0.312). Similarly, no difference was found between the perceived quality of the design of conforming pages and non-conforming ones (W=103.0, n=10, p=0.904). Participants in this group were not asked to compare the forums with existing ones since by being unfamiliar with them it was assumed that they had not come across web forums and so could not have an opinion about their design. 6.2.5. Discussion Performance Search performance seems to improve according to familiarity with the interface where the search is taking place, and familiarity with the type of task being executed. Considering that the familiar group encompasses users ranging from intermediate to experts and the unfamiliar group encompasses novice users in terms of web forum usage, this result agrees with what was found by Holscher and Strube (2000). The authors found that users with high Web expertise had higher success rates on the Web search tasks when compared to novice users. In this experiment (experiment 4), users who were familiar with web forums completed tasks faster than users who were unfamiliar with this type of web pages. They also obtained a better score than the unfamiliar group and a higher RPS rate than the unfamiliar participants. This confirms hypothesis (a), which stated that overall, familiar participants will perform better than unfamiliar ones. Participants in the familiar group were web forum users and were familiar not only with the interface but also with the tasks which are executed in web forums. Thus it is no surprise that they performed better than participants who never used a web forum before. After all, despite the interface in the non-conforming forum being changed, the processes (e.g. decisions, paths to take) necessary to complete the tasks remained the same as in the conforming forum. Because familiar participants knew web forums they were familiar with the type of tasks performed in them. On the other hand, unfamiliar participants were not used to the style of tasks which can be performed in a web forum and might have had difficulty in knowing how to process 176
- 199. Visual conventions and familiarity when searching in web forums the tasks. An explanation which stems from this one is that because of familiarity with forums and the tasks it involves (see Lim, et al., 1996), users in the familiar group processed tasks automatically while unfamiliar participants used controlled processing of tasks. This could have resulted in faster and more accurate task completion for the former group. However, a much more puzzling result was the fact that no significant difference in performance was found between participants of the familiar group using the conforming forum and those using the non-conforming forum. This contradicted hypothesis (b) which predicted that familiar participants would perform better in the conforming layout than in the non-conforming one. This also goes against the literature (Galletta, et al., 2006; Holscher & Strube, 2000; Tabatabai & Shore, 2005) according to which familiar participants should have performed significantly better in the conforming version than in the non-conforming one. There could be various explanations for the lack of significant difference in performance between familiar users using conforming and non-conforming pages. One explanation could be that since the experimental forums did not have a lot of breadth compared to existent web forums, the scope of the search was reduced. Thus, participants had a reduced number of links to search for information. As a result the familiar participants (who were familiar with the forum structure and with the tasks performed in web forums) found answers very quickly independently of the interface they were using. However, there was no difference in performance between familiar and unfamiliar participants who used the non-conforming forum. Considering that unlike the familiar participants, the unfamiliar participants were not accustomed to the forum structure and tasks, it would be expected that their performance would have been worse than the performance of their counterparts. But, on the contrary, when using the non-conforming forum there were no significant differences between the groups. Another explanation is that a learning process could have occurred. In the case of participants who were familiar with forums and the associated tasks, they could have learned the non-conforming forum fairly quickly and from then on performance was equivalent to the familiar participants using the conforming forum (see Figure 68). In the unfamiliar group, learning could have occurred as well, but here the group was unfamiliar with both the conforming and non-conforming interfaces, and both had to be learned. However, the time to learn either interface would be very similar, resulting in similar learning curves and consequently no differences in performance. The fact that learning could have occurred does not necessarily rule out that the 177
- 200. Visual conventions and familiarity when searching in web forums narrowness of the site and the familiarity of some users might have influenced the lack of differences. In fact it is possible that all these factors have influenced user performance. FIGURE 68 – HYPOTHETICAL ILLUSTRATION OF A LEARNING EFFECT TO EXPLAIN THE LACK OF DIFFERENCE IN PERFORMANCE BETWEEN THE FAMILIAR USERS TESTED WITH THE CONFORMING AND THE FAMILIAR USERS TESTED WITH THE NON- CONFORMING FORUM. Familiar users using a conforming forum Familiar users using a non-conforming forum Task type Another interesting result was the interaction between conventionality and task type. Results showed that when conventionality is violated, participants tend to perform better on interface related tasks than on content related tasks. One explanation for this could be because interface related tasks were written with an explicit cue to where the information was located (in this case the exact link where participants had to click), and required participants to depend less on the layout than content tasks. So, when visual conventions were not followed, the cues embedded in the interface tasks could have made finding interface elements easier than finding content. Furthermore, because task order was not randomised or counterbalanced it is very likely that learning occurred through the experiment. Therefore further investigation of this interaction is needed. 178
- 201. Visual conventions and familiarity when searching in web forums Questionnaire The lack of difference in user perceptions of the ease/difficulty of finding information on both versions of the forum is consistent with the absence of differences in performance between the two conditions. This result could be due to the fact that participants felt equally about the ease/difficulty of finding information on both versions of the forum. An alternative explanation would be that by the end of the test (when participants answered the questionnaire) participants who were using the non-conforming version were more familiar with the new interface and judged it according to their final impression and not their first. When comparing the quality of the design of the conforming and non-conforming forums, both the participants in the familiar and unfamiliar groups showed no difference in perception. This could indicate that because forums were created using the same design features (colour scheme, graphic elements or when elements differed, the design kept the aesthetic consistency), they were quite similar visually. This could indicate that the variations on symbolization, position and configuration of the graphic elements on the non-conforming page were not apparent to the participants. An alternative explanation comes from the perception of the researcher when conducting the test. Many participants, when asked to rate the design, turned to the researcher and asked if he had designed the screens. This could indicate that the participants were conscious about ‘hurting the feelings’ of the experimenter and therefore might not have expressed any negative opinion they could have had. Finally, users who were familiar with web forums found the conforming design more similar to other existing web forums than the non-conforming design. This result again confirms what the studies in Chapter 5 have shown. 6.2.6. Conclusion Although the study did not find any indication that not conforming to conventionality can compromise user performance, the results obtained show that participants who are familiar with web-forums performed better than unfamiliar participants on both the conforming and non-conforming design. This suggests that familiarity may play an important role in searching for information online, but might also enable users to quickly learn the interface, picking up on the existing visual elements when the expected visual conventions are not present. The interaction between task type and conventionality is also very interesting, indicating that a 179
- 202. Visual conventions and familiarity when searching in web forums violation of visual conventions might have a different effect on performance, depending on the type of task the user is performing. However, the lack of difference in performance between familiar users who were tested with the conforming design, and the familiar users tested with the non- conforming design, was unexpected. This needs further investigation and another experiment, which can look at the performance of familiar users over time, needs to be carried out. There are also some limitations in this experiment, which need to be resolved in the next one. In this experiment the analysis of the effects of task type and interactions were made more difficult because there were only 5 tasks and only 2 of them were interface related. Therefore, any task which participants found too easy or too difficult could bias the results. Tasks were not designed to maximize the requirements of each type of activity (i.e. retrieving information from content vs. retrieving information from the interface). This is particularly true for the interface tasks. The fact that the order of the tasks was not randomized also presented a problem since a carry-over effect could have influenced the results. The wording on the interface tasks may have made them very simple to answer as they indicated to the participant where to click. The topic chosen, PDAs, also caused some problems as some participants were not familiar with the technological terms used in the forums. Even though a basic explanation of PDAs was given before the test started, it seems this it was not sufficient, as participants had problems completing some of the tasks. Finally, the number of pages of the experimental websites could have oversimplified the tasks making it too easy for familiar users to find the answers to the tasks. The next Chapter describes an experiment which investigates further the unexpected lack of difference in performance of familiar users when using the conforming and non-conforming layouts. The experiment will look at the performance of users on both layouts over time to identify if participants using the non-conforming version may be improving their performance as they use the website. 180
- 203. Chapter 7 Visual conventions and performance over time 7.1. Introduction Results from experiment 4 showed no significant differences in performance of users familiar with web forums when presented with a website which violated their expectations. This finding was contrary to what is suggested in the literature that not conforming to conventions results in poor user performance (Gery, 1995; Vaughan & Dillon, 2006; Walton & Vukovic, 2003). This chapter describes an experiment which was devised to investigate why in the previous experiment participants who were familiar with a certain set of visual conventions showed no overall difference in performance, whether using a website which violated these conventions or a website which conformed to the conventions. Much has been researched on the expectations of users when faced with a website, how these users browse the Web, their strategies and navigation patterns. However, the issue of not conforming to visual conventions and the impact it has on user performance over time has not been explored thoroughly, and the existing research merely gives indications as to what could have caused the equivalent results. Some relevant findings are discussed here. 7.1.1. Navigation and browsing strategies Browsing is an information-seeking strategy which is exploratory in nature. Some authors describe browsing as being dependent on serendipity (Marchionini & Shneiderman, 1988). 181
- 204. Visual conventions and performance over time There are three generally agreed strategies which can be used while browsing: (a) search browsing, which happens when the goal is known and tends to be highly systematic; (b) general purpose browsing – here, even though the target is known, it is broadly defined, obliging users to probe various sources with a high probability of having items of interest relevant to the search and (c) serendipitous browsing which is fortuitous and has no real goal defined (Catledge & Pitkow, 1995; Cove & Walsh, 1988; Danielson, 2002b; Marchionini, 1995). Although there seems to be general agreement regarding these three browsing strategies, there seems to be a contradiction in the naming of the category ‘search browsing’. If all browsing activities involve serendipity as stated by Marchionini and Shneiderman (1988), then a systematic search with a well-defined goal in mind (search browsing) would not qualify as browsing. A more appropriate naming for the ‘search browsing’ strategy then is provided by Salomon (1989): goal-directed search. General purpose browsing on the other hand, although having a goal defined, involves probing different sources which would allow for serendipity to happen, thus qualifying it as browsing. But are goal-directed search, general purpose browsing and serendipitous browsing used equally when navigating through the World Wide Web? To answer this question it is crucial to look at the activities performed while navigating through the Web. It is not surprising to find that next to e-mailing, the primary use of the World Wide Web is information retrieval (Broder, 2002; Gordon & Pathak, 1999; Rumpradit & Donnell, 1999). Taking into consideration the amount of information available on the World Wide Web, and the fact that searching for information is part of a great number of human activities (Tabatabai & Shore, 2005), it is likely that information retrieval will remain one of the most frequent Internet activities. This makes the three browsing styles mentioned above very important since in one way or another they all involve the retrieval of information. But further analysis into the serendipitous browsing style shows that although it is extremely rewarding, it is not very efficient for information retrieval (Stanoevska-Slabeva, Hombrecher, Handschuh, & Schmid, 1998). Because users ‘want to find the information they seek quickly and easily’ (Cappel & Huang, 2006) they will use the most efficient strategies they have available. Therefore it is very likely that goal-directed search and general purpose browsing are the two more common browsing styles on the web. 182
- 205. Visual conventions and performance over time Navigation models and efficiency When searching for information on the World Wide Web, users tend to navigate in a hub and spoke model, where search starts at a central page (hub) and then navigates to a new page (spoke), then returning back to the central page (Catledge & Pitkow, 1995; Tauscher & Greenberg, 1997). This browsing style is marked by intensive use of ‘backtracking’ a technique where the user returns to the starting point through the same route they have navigated. This is achieved by clicking on the back button that is widely available on WWW browsers. Another navigational technique which is not as often used as backtracking, but also allows return to the central page (hub), is called loopback (Catledge & Pitkow, 1995). Here, instead of using the back button to return, the user clicks directly on a ‘return to home/ entry point’ link and is traversed immediately to the desired location. However, considering the speed with which technology evolves and the growth of the World Wide Web in the past ten years, it is possible that the hub-and-spoke navigation model is no longer used and backtracking and loopback are not the preferred strategies when browsing for information on the web. However, a recent study by Milic-Frayling et al (2004) has confirmed that the results of previous studies are still valid. This study analysed the Web usage log of 9 participants for a period of 3 weeks. Results showed that 28% of all page visits were visits to hub pages and 30% of all the navigation was done using the back button. Although these browsing styles explain broadly how users navigate on the web, they alone are not enough to explain if the navigation is being carried out in an efficient manner. Navigation is considered efficient when a user is able to find the desired information using the minimum necessary number of movements from link to link (Schoon & Cafolla, 2002). Since the pilot study to be carried out will be looking at user performance, it is important to have means of measuring the efficiency of the navigation of the participants. One attempt to measure the efficiency with which users navigate through a website was proposed by Smith(1996). This measurement, called Efficiency Indicator by the author, was found by dividing the number of nodes (information items in the hypertext base) required to complete a task (R), by the number of different nodes visited (D). So the more the indicator moved away from zero and towards 1, the more efficient the search was. 183
- 206. Visual conventions and performance over time The author also claims that the indicator informs when the user did not complete the task. Whenever the indicator is greater than 1, it means that the task was not completed (Smith, 1996). This affirmation is not always valid since it does not take into account the situation where the user has visited more nodes than the minimum required to complete the task (R), and then quits the task. This situation would have an indicator smaller than one (R<D) but the task would not have been completed. Unreliability regarding the incomplete tasks might prove problematic for the next experiment to be carried out. Therefore, the efficiency indicator will be used to quantify the efficiency of each group but not to detect whether a participant has quit a task or not. Another critique of the efficiency indicator (Smith, 1996) is made by Herder and Dijk (2003). The authors claim that a study by McEneaney (2001) found that efficient hypertext readers seem to work by going back and forward from landmark nodes at the root of the pages and that this strategy would make efficient users look inefficient according to Smith’s indicator standards. What Herder and Dijk (2003) fail to notice is that the proposed efficiency indicator (Smith, 1996) takes into account the number of different nodes visited rather than the number of nodes visited. So if readers were to apply the browsing strategy mentioned in McEneaney (2001), the revisits would not be counted towards the indicator. However, Herder and Dijk(2003) question both McEneaney (2001) and Smith (1996) methods since neither mention what pages users returned to, nor how much time they spent on each page. Although it is important to know which pages users are revisiting when navigating because it would help separate lost users from those using a navigation strategy, this would not affect the efficiency indicator proposed by Smith for the same reasons described above. However, not knowing how long users spend on each page does affect the idea of efficiency since time is an important indicator of efficiency. Despite the critiques, Smith’s methods have been successfully used and endorsed by some researchers (Larson & Czerwinski, 1998; Otter & Johnson, 2000; Selvidge, Chaparro, & Bender, 2002) and the efficiency indicator is an important instrument in analysing navigational efficiency when it is considered along with time. 184
- 207. Visual conventions and performance over time Since the efficiency indicator will be adopted in this experiment, a system must be used to allow definition of the number of steps necessary to complete a task, and later analyse the steps participants took while doing the experiment. One which serves the purposes of sorting the data to apply the efficiency indicator well is a taxonomy developed by Byrne et al.(1999). By examining participants’ browsing of the web as they would in a normal work day, the authors observed six general classes of web tasks: ! Use information – when the user attempts to use a piece of information from the WWW (e.g. by reading, listening, copying and pasting). ! Locate on page – when the user tries to find a piece of information on the site through visual search. ! Go to page – when the user performs an action that makes the browser display a particular page (e.g. clicking a hyperlink, using the back button, typing a web address). ! Provide information – when the user sends information through the web (e.g. providing authentication information, shipping addresses). ! Configure browser – when users changed the user configurable options of the browser to suit their needs (e.g. changing the size of a window, hiding the toolbar). ! React to the environment – These tasks are typically when the browser demands something from the user (e.g. a dialog box opens). Each of these main processes is broken down into smaller sub-processes (e.g. a ‘go to page’ action can involve clicking on a hyperlink, using the Back/Forward button, clicking on a bookmark, etc). This taxonomy system will be adopted in this study since it facilitates the setting of the minimum number of pages (nodes) necessary to complete each task. It also allows a detailed transcription of the actions of each participant including which page they revisited and which pages were different. However, due to the kind of tasks being created in this experiment, only 3 of the 6 classes defined by Byrne et al.(1999) will be used. The classes used will be: (1) Use information; (2) Locate on page and (3) Go to page. 185
- 208. Visual conventions and performance over time 7.1.2. Expectations Conventions and usability As seen previously in chapter 2, whenever looking at a document, users have certain expectations on how to handle it in order to extract the necessary information (Dillon, 2004, p. 117). According to the authors, when users have some knowledge of the document, this provides them with information on the probable structure and organization of the elements of the document (Dillon, 2004: p.117). These expectations are built up from long periods in contact with a particular set of conventions that are characteristic of that type of document. Many usability and human-factors experts warn about the problems of breaking these conventions. It is said that not following an established set of design conventions, thus violating the expectations of users for a particular document, is likely to generate major usability problems (Nielsen, 1999a). The author however gives no details as to which major usability problems are caused by these violations of conventions. Although the experiments conducted in Chapter 5 have shown that users perceive conforming layouts as easier to use than non-conforming ones, the fact that something is conventional does not necessarily mean that it is also easier to use than something that is does not conform to conventions. Sometimes, it is even necessary to break with standards and conventions in order to improve usability, for example interfaces designed according to the principles of usage-centered design (Constantine & Lockwood, 2002). Norman(2005) goes even further by stating that ‘great design comes from breaking the rules, by ignoring the generally accepted practices’. A good example of this can be found in an experiment conducted by Blackler et al.(2004) which compared user performance on remote controls with different layouts. The authors found that user performance was improved by using a specific layout which consisted of intuitive symbols (different from the conforming ones) located in positions different from the conforming ones. Nielsen (1999b) also exemplifies this point well by affirming that if the navigation menu of a web page was placed on the right side it would improve its usability because: (a) It would be closer to the scroll bar and that would save users time (b) Users always look at content first (cf. DeVigal, 2003) 186
- 209. Visual conventions and performance over time Since in cultures that read left-to-right people tend to look at the left side of the screen first, this would leave the left side of the window for content to be placed (Nielsen, 1999b). But Nielsen continues to reinforce the following of conventions even ‘when a different design would be better’ (Nielsen, 1999a). Despite this however, some other authors list other problems related to breaking with conventions. According to Dillon and Vaughan(1997) the consequence of breaking conventions is that it makes users feel disoriented having to rely more on interface elements such as the browser back and forward buttons. As seen before, using back and forward buttons on the browser can indicate that the user is lost (which is a serious problem) but it can also be used as part of a navigation strategy (such as hub-and-spoke model). It is also said that not conforming to conventions can affect comprehension by making it difficult for readers to gather the textual information into a meaningful representation of the text (Loines, 1991). Loines (1991), however, gives no empirical evidence to support this comment. Despite that, the idea of conforming to conventions is one widely practised and reinforced. This poses a constant dilemma to designers – should they innovate and break new ground or should they stick to the established set of rules and conventions that users are accustomed to, and ensure that their expectations are met? As literature suggests, users do have expectations on the positioning of web page elements (Bernard, 2001a; Markum & Hall, 2003) and a web design that meets these expectations helps users find interface elements faster (Oulasvirta, Kärkkäinen, & Laarni, 2005) and reduce task completion time (Ehret, 2002). As seen in Chapter 2, a study to check these statements was carried out by Kalbach and Bosenick (2006). The study manipulated the visual convention of a website by altering the position of the navigation bar. According to the study, if the statements were true, user performance on websites which placed the navigation bar on the expected left-side position (Figure 69) would be better than navigation bars placed in an unexpected the right-side position (Figure 70). 187
- 210. Visual conventions and performance over time FIGURE 69 - SAMPLE OF THE CONVENTION-CONFORMING WEBSITE USED BY KALBACH AND BOSENICK (2006) FIGURE 70 - SAMPLE OF THE NON-CONFORMING WEBSITE USED BY KALBACH AND BOSENICK (2006) The study found no differences in overall user performance (measured by the task completion time) regarding the positioning of the menu on the screen. But 188
- 211. Visual conventions and performance over time researchers found that participants who used the right-side menu spent more time scanning the homepage on the two first tasks and that this effect diminished on the subsequent tasks. This lack of significant results in Kalbach and Bosenick (2006) could be explained by another study which compared user performances on genre-conforming and non- conforming online news pages (Vaughan & Dillon, 2006). In this study, Vaughan and Dillon went further than Kalbach and Bosenick (2006) by not only not conforming to the visual conventions of a website but also its structural conventions (e.g. the conventional way of structuring the information within the website). To develop the experimental material, the authors asked expert web news readers and developers to come up with what a genre-conforming web page would be like. Participants drew a sketch of the homepage, category page and story page, and then all sketches were discussed and the agreed features were noted. Because participants of this group had trouble coming up with non-conforming categories and design ideas, the experimenters asked a panel of experts from computer science, cognitive psychology and human-computer interaction what a non-conforming website would be like. Samples of the convention-conforming and non-conforming websites used by the authors are shown in figures 71 and 72 respectively. FIGURE 71 - SAMPLE OF THE CONVENTION-CONFORMING NEWS HOMEPAGE USED BY VAUGHAN AND DILLON (2006) 189
- 212. Visual conventions and performance over time For the non-conforming sites, not only the layout of the page was changed from what was deemed conventional, but also the categorization of the news was totally reinvented, demanding a complete semantic rethink of the information organization within the site by the user. FIGURE 72 - SAMPLE OF THE NON-CONFORMING ONLINE NEWS HOMEPAGE USED BY VAUGHAN AND DILLON (2006) The study found that the participants who used the genre-conforming sites had a more efficient performance on the information-seeking tasks compared to those who used the non-conforming site. But by the fifth day they used the website, users of the non-conforming sites had learned the new interface, increasing their efficiency while doing the tasks. So it is possible that the learning effect which took place in the study by Vaughan and Dillon (2006) also happened in the study by Kalbach and Bosenick (2006). Perhaps the fact that less page scanning was done towards the end of Kalbach and Bosenick’s experiment was an indication that some sort of learning had happened. This still does not explain why there was a discrepancy between the results between of the two studies. Comparing the two studies it is clear that Vaughan and Dillon violated the visual and structural conventions of the genre, but Kalbach and Bosenick only altered the visual conventions of the genre. This could have made the learning easier in the latter case consequentially speeding up the process, and making the initial differences which might have existed, disappear. 190
- 213. Visual conventions and performance over time People can learn to do a task by one of three ways: (a) by hearing, (b) by seeing or (c) by doing. Out of these three ways, the most important and effective is learning by doing (Constantine & Lockwood, 2002). Based on learning by doing, the authors developed the Anticipatory Learning paradigm which consists of four elements: novelty, anticipation, action and confirmation. Usually the user finds an unfamiliar element, and guesses what the element is or how it works. Then the user tries it out and is rewarded by finding out that the guess was correct (Constantine & Lockwood, 2002). Constantine and Lockwood (2002) mention that when all the elements necessary for anticipatory learning to take place are present, no prolonged period of trial-and-error is needed. With one use the user learns the interface and uses it correctly thereafter (Constantine & Lockwood, 2002). So in the previous study by Kalbach and Bosenick (2006), the fact that the structure of the site was not changed might have given users a better chance to correctly anticipate the function of a particular interface element thus reducing the learning process. This could indicate that the violation of the visual conventions of a website might not be as bad as currently suggested by usability experts simply because users might pick up on the ‘new layout’ quite quickly. In fact, some research has shown that users do learn fairly easily where elements of the interface are positioned (Ehret, 2002; McCarthy, et al., 2003; Oulasvirta, et al., 2005). This indicates that the difficulties caused by breaking visual conventions by repositioning elements on a page can easily be overcome by users. The study on interface menu positioning carried out by McCarthy, Sasse and Riegelsberger (2003) has shown that sites that conformed to users’ expectations regarding the menu positioning did lead to faster search times when compared to non-conforming websites. But according to the authors this performance difference was short-lived and when sites were revisited for the second time, there was no search time difference. Although there seem to be indications that not conforming to conventions might have a negative impact on user performance some experiments seem to suggest that this negative impact might not last for long. Furthermore, when only visual conventions are violated it seems that this negative impact can be even more ephemeral. But no study so far has looked at whether users can improve their 191
- 214. Visual conventions and performance over time performance as they use a site which violates visual conventions for a certain period of time or how long this process would take. Also, in this experiment the effect of not conforming to conventions on disorientation as suggested by Dillon and Vaughan (1997) will be investigated along with user performance. 7.1.3. Disorientation As seen before, not conforming to the expectations users have regarding a particular genre has been reported to make users feel disoriented (Dillon & Vaughan, 1997). Disorientation in hypertext has been defined as the ‘tendency to lose one’s sense of location and direction in a nonlinear document’ (Conklin, 1987, p. 40). It has been considered one of the major problems in navigating through hypertext (Conklin, 1987; Nielsen, 1990; Xu, Cockburn, & McKenzie, 2001). Measuring disorientation, however seems to be quite difficult (Herder, 2003) and several methods have been proposed. Basically the methods revolve around two main ideas: one that claims a link between the actions of users when navigating and disorientation (Rosenberg, 1996) and another which claims that user disorientation can only be measured by asking the user about their perceived disorientation (Ahuja & Webster, 2001). Regarding the actions of users while navigating which could indicate disorientation, there are page revisits and backtracking. According to a psychological model of disorientation proposed by Yatim (2002) disoriented users face two options: (a) quit the task altogether or (b) try to re-orient themselves. This re-orientation process can involve using the back button (backtracking), exploring alternative routes or re- reading the information available (Yatim, 2002). The concept that lost users try to re- orient themselves through the use of backtracking does reinforce the use of backtracking as a measure of disorientation. However, pressing the back button does not always mean that the users are lost and trying to re-orient themselves. The user could be revisiting a page to read it again (Rosenberg, 1996) which would not necessarily mean that they are lost. This builds up a strongest argument against using backtracking and revisits alone to measure disorientation, since backtracking can also be used as a web navigation strategy (Catledge & Pitkow, 1995). 192
- 215. Visual conventions and performance over time Recently researchers have been questioning the concept of dividing these two measurements (user action and perceived disorientation) and have actually found correlations between them (Gwizdka & Spence, 2007; Herder, 2003; Yatim, 2002). This correlation could indicate that although backtracking and revisits might not be a foolproof way of measuring disorientation, they could give good indication as to whether or not a user is lost, especially when combined with another measure. 7.2. The pilot study Before the main experiment was executed, a pilot study was conducted in order to investigate what happens to performance over time (i.e. whether performance changes as participants progress on the non-conforming site) and test the experiment design and methodology. This pilot study compares what happens when web forum users use a non- conforming forum with when they use a conforming forum. The hypotheses here are that (a) participants who use the non-conforming forum will familiarize themselves with the new interface, as happened with the studies from McCarthy et al.(2003) and Vaughan & Dillon (2006). This familiarization will be reflected in their performance so that the group which uses the non-conforming website will initially perform worse than the group which uses the conforming forum, but they will show a performance improvement throughout the test. (b) participants who use the conforming forum will show no performance improvement throughout the experiment. 7.2.1. Objectives The objectives of this pilot study are: ! Check if the performance of participants using a non-conforming forum will improve over time. ! Verify if the efficiency indicator (Smith, 1996) is suitable for use in this experiment and test the adequacy of the taxonomy proposed by Byrne et al.(1999) to gather data to be used in the indicator. ! Evaluate if measuring navigation corrections (e.g. using the back button or traversing links) is a reliable way of assessing user disorientation. 193
- 216. Visual conventions and performance over time 7.2.2. Method Experiment design This pilot experiment used two independent groups of participants with one group being tested on a conforming web forum design, and the other group on a non- conforming web forum design. The variable that was being manipulated (independent variable) was the conventionality of the web forum and the type of task participants were answering. The variables that were being measured to look at the effect of conventionality on performance (dependent variables) were the task completion times, the number of right answers, the number of navigational adjustments (revisits, backtracking and loopback) done by each participant and the number of different pages each participant visited in order to complete the tasks. Participants 10 participants, all graduates or postgraduates, volunteered for this pilot experiment. They were randomly allocated to one of the experimental conditions in such a way that 5 participants were tested with the conforming web forum and 5 participants with the non-conforming web forum. All participants were frequent web forums users and although their nationalities varied, they were all fluent in English. The participants who volunteered for this experiment had not taken part in any previous experiments in this thesis. Materials The web forum designs for the conforming and non-conforming conditions were based on the templates used in the previous experiment (see Chapter 6) which originated from the survey that looked into the web forum elements and their most common position, configuration and symbolization values (see Chapter 5). As seen earlier, the content used in the material for the experiment 4, described in Chapter 6 (extracted from a PDA forum) produced some interpretation problems since some participants were not familiar with the technical PDA terminology. Therefore, the content used in these experimental forums was obtained from an online book discussion forum. This choice of content was made based on the assumption that books are a topic that the participants would be more familiar with, consequently minimizing any comprehension difficulty due to the vocabulary used. 194
- 217. Visual conventions and performance over time In the experiment described in Chapter 6 which tested user performance when searching for information in a forum, some participants expressed the view that the forum designs were not very good and needed improvement. Although the forums were systematically designed based on the results of the survey, some minor alterations could be done to attempt to improve the visual aspect of the designs. So for these new designs the colour scheme was changed from shades of green to shades of blue, maintaining the figure/background contrast and saturation values. Some of the icons used in the previous experiment for the conforming and non-conforming designs were also replaced by other icons commonly occurring on the Web, but with better resolution and more aesthetically pleasing. A screen-capture software was set to capture an image of the screen on each click the participants made. Thus whenever a participant clicked with the mouse, a screen would be saved on the hard drive along with the time when the click happened. To solve the time measure at the end of the tasks, a pop-up window with a button labelled ‘finished’ was created so that when the participants finished the task they would click on the button signalling that they were satisfied with the answer found and were ready to move on to the next task. This button also gave the opportunity for the participants to quit any tasks to which they could not find the answer. One of the difficulties in the previous experiment (see Chapter 6) for the experimenter was stopping the measurement of time precisely when the participant ended each task. The difficulty was due to several reasons e.g. the participants did not state when they have finished the task, or they would say they had finished but then change their mind; there would be a lapse between the moment the participant verbalized finishing the task and the moment the experimenter stopped the measure. So, in order to improve the reliability of the time measurement on each task, a very small and discrete software stopwatch was strategically placed on the screen so that every time a mouse click occurred and a screen was captured, the time of the click was registered. To minimize the effect that the stopwatch may have on participants by adding time pressure, the following measures were put in place: the stopwatch was placed on the bottom right corner of the browser and was set to have the same colour as the browser frame so it blended smoothly with the interface (see Figure 73). 195
- 218. Visual conventions and performance over time FIGURE 73 - SAMPLE OF THE EXPERIMENTAL SETTING USED FOR THE PILOT STUDY AND EXPERIMENT 5. Tasks design To have a clearer picture of the performance changes during the experiment, this pilot had a greater number of tasks than the previous experiment. It consisted of 10 tasks each requiring a different minimum number of steps to be taken in order for each one to be solved. The reason for the difference in the number of steps is mainly due to the way in which forums are structured, with a forum index page, thread pages and post pages. Since the answers could be found in any of these three places, the minimum number of steps to get to them would vary. The steps were based on the taxonomy mentioned earlier, devised by Byrne et al.(1999) which establishes the processes that are involved with a World Wide Web task. Since this experiment only involved searching for information, the only steps used were: (a) locate information on page; (b) use information and (c) go to a page. Each of these main processes was considered a step. 196
- 219. Visual conventions and performance over time The number of steps necessary to complete a task ranged from 5 to 9 depending on the task. Even though longer tasks may lead to more participants quitting the task than shorter ones, it is interesting to look at how each task type and length can affect user performance. This will allow a better selection of task style and length (number of steps) for the main experiment, so that task styles which are too easy and the ones which are too difficult can be detected and not used. As in the previous experiment (see Chapter 6) there were two types of task: one (called content tasks) required the participant to look for the answers on the written content of the forum (e.g. ‘What is the difference between a 1st edition and a 1st trade edition?’), the other type (called interface tasks) required that at some point in the search for the answer participants would need to obtain some information from the interface (e.g. ‘Which thread in the ‘Books wanted’ topic has the last post dated Nov 22?’). Extra care was taken to design interface tasks that would only require the participants of the non-conforming condition to use the icons that were easily recognized in the previously mentioned survey (see Chapter 5). A total of 10 tasks, 6 content-related and 4 interface-related, were created. Due to the icon identification limitation described above the number of interface tasks was smaller than the content tasks. Task orders were randomized for each participant. The task questions can be seen in Appendix N. 7.2.3. Procedure Each participant was individually tested in a quiet room. Initially the experiment was explained to the participant. Participants were told that the test consisted of finding answers to questions on a web forum and it was explained how to proceed when they had found an answer. They were told to behave as they would normally behave when browsing the Web. They were told that, as in a real situation, they could quit a task whenever they wanted. When participants were ready to begin, the experimenter handed them the first task printed on a sheet of paper. When the participants finished reading the task and looked at the computer screen, the experimenter started the time measurement and screen capture. Participants then searched within the forum for the answer to the task and when they thought they had found the answer they would select it (by clicking and dragging the mouse over it), and would click on the ‘finished’ button in the pop- up window. By clicking on the button participants were taken back to the homepage 197
- 220. Visual conventions and performance over time of the forum and were ready to start a new task. The experimenter wrote down the time displayed on the stopwatch and whether the participant got the question right (this was a safeguard for any system failure). These procedures were repeated for every task for each participant in both groups. 7.2.4. Results Time The mean task completion time for the conforming group was 76.8s (SD=33) and 87.6s (SD=23.7) for the non-conforming group. A one-tailed Mann-Whitney test found no significant difference in the times taken by each group (conforming and non-conforming) (W=23.5, n=5 and p=0.23). To look at how task completion time of the participants in both groups progressed over time, the results were averaged for each group in the order each task was answered by each participant. This was done because tasks were randomised and participants did not receive tasks in the same order. The results for the non- conforming group (Figure 74) and conforming group (Figure 75) are illustrated in the graphs below. FIGURE 74 - TASK COMPLETION TIME FOR THE NON-CONFORMING GROUP. Non- conforming 198
- 221. Visual conventions and performance over time FIGURE 75 - TASK COMPLETION TIME FOR THE CONFORMING GROUP. Conforming On these graphs it is possible to notice that that the group which used the non- conforming design showed a performance improvement throughout the experiment, while the group which used the conforming design apparently had no change in performance. A Mann-Kendall trend test indicated a significant downward time trend on the non-conforming group (Z=-27, n=5, p<0.01). The Mann-Kendall test for the conforming group did not reveal any significant trend (Z=-13, n=5, p=0.13). Efficiency indicator The efficiency indicator was calculated as proposed by Smith(1996) dividing the number of nodes required to complete a task by the number of different nodes visited. Both groups showed similar efficiency with the conforming group showing a mean efficiency indicator of 0.52 (S.D.=0.187) and the non-conforming group showing a mean value of 0.55 (S.D.=0.188). A Mann-Whitney test showed the difference not to be significant (W=2216.0, n=5, p=0.623). Tasks To have an idea of the difficulty of the tasks in this experiment a graphs of the average times to complete each task for both the non-conforming (Figure 76) and conforming groups were plotted. 199
- 222. Visual conventions and performance over time FIGURE 76 - MEAN TIME TO COMPLETE EACH TASK IN THE NON-CONFORMING GROUP. Non- conforming FIGURE 77 - MEAN TIME TO COMPLETE EACH TASK IN THE CONFORMING GROUP. Conforming Looking at the graphs it is possible to notice large variabilities in both groups on most of the tasks. Finding tasks which show different performance results between the two groups could indicate a specific kind of tasks for which one of the layouts would be more adequate. To verify if any of the tasks showed this effect, the two graphs were overlapped (Figure 78). From this overlapping it is possible to see that apparently the only task where a significant difference between the two groups 200
- 223. Visual conventions and performance over time might show is task B. However a Mann-Whitney test on the time to complete task B on both groups showed that even though the difference was not significant there was a trend towards significance which indicates that with a higher sample size this difference may become significant (W=19.0, n=5, p=0.097). Therefore it is worth having task B in the next experiment. FIGURE 78 - OVERLAPPED GRAPHS OF THE MEAN TIME TO COMPLETE EACH TASK ON THE CONFORMING AND NON-CONFORMING GROUPS. Conforming Non- conforming So, to better analyse the global effect of each task the mean time to complete each task overall was plotted (Figure 79). FIGURE 79 – OVERALL MEAN TIME TO COMPLETE EACH TASK. 201
- 224. Visual conventions and performance over time From the graph it is possible to see that some tasks were answered very quickly, some took a long time to be answered and some showed high variability. These tasks are shown in Table 38. From the graph in Figure 79 it is clear that task H was quite difficult, taking participants longer to complete, and task I was quite easy taking little time to be answered and presenting very little variability. Task D showed a lot of variability among participants despite taking on average a similar time to complete as task B. This variability indicates that some participants found this task quite easy and others found it quite difficult. This variability could be due to the fact that not all the participants were able to associate the term ‘Frequently Asked Questions’ with the menu item FAQ (in the conforming design), or the question mark icon (in the non-conforming design), thus making the task quite difficult. Task I was not only easy for both groups but also showed virtually no variability. This was probably due to the way in which the wording of this task was structured; it contained all the path the participant had to navigate through to reach the page where the answer was and also told the participant where to look for an answer. Furthermore, the answer to task I was only one forum level away from the point where participants started the search, so that with just one click on the right link, the participants were already on the page where the answer was. TABLE 38 - SUMMARY OF THE EXPERIMENTAL TASKS WHICH WERE ANSWERED TOO QUICKLY, TOO SLOWLY OR SHOWED GREAT VARIABILITY. Task Question H How much is a T.S. Eliot signed 1st edition worth? I Which thread under the “What I’m reading now” has the greatest number of views? Find the answer for this frequently asked forum question: “How do I prevent my user name from D appearing in the online user’s listing”? Which forum member wrote the last post on the thread “Trimming pages” under the “Adding value C to your books” topic? B What is the difference between a 1st edition and a 1st trade edition? Time per task type To analyse the time taken to complete each task type a one within-subjects, one between-subjects ANOVA was applied to the data. Results showed a main effect of task type but no interaction of conventionality and task type (see Table 39). 202
- 225. Visual conventions and performance over time TABLE 39 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS ANOVA RESULTS FOR TIME TAKEN TO COMPLETE THE TASKS. Between subjects Source SS df Mean Squares F-Ratio p-value Conventionality 31,203.420 1 31,203.420 0.630 0.450 Error 396,353262 8 49,544.158 Within Subjects Source SS df Mean Squares F-Ratio p-value Task type 1,037,091.533 1 1,037,091.533 19.676 0.002 Task type x Conventionality 26,229.076 1 26,229.076 0.498 0.501 Error 421,659.823 8 52,707.478 Examining the graph (Figure 80) it is possible to see that interface tasks were answered more quickly than content tasks in both the conforming and non- conforming conditions. It is also possible to perceive that interface tasks took virtually the same time to complete in both the conforming and the non-conforming groups. FIGURE 80 - MEAN TIME TO COMPLETE INTERFACE AND CONTENT TASKS BY CONDITION. Conforming Non-conforming Right answers per task type A one between-subjects, one within-subjects ANOVA of the number of right answers achieved by each group on each task type showed again a main effect of task type. No interaction of task type and conventionality or main effect of conventionality was detected. 203
- 226. Visual conventions and performance over time TABLE 40 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS ANOVA ON THE NUMBER OF RIGHT ANSWERS BETWEEN THE CONFORMING AND NON-CONFORMING GROUPS. Between subjects Source SS df Mean Squares F-ratio p-value Conventionality 0.450 1 0.450 0.643 0.446 Error 5.600 8 0.700 Within subjects Source SS df Mean Squares F-ratio p-value Task Type 14.450 1 14.450 28.950 0.001 Task Type x Conventionality 0.050 1 0.050 0.100 0.760 Error 4.000 8 0.500 Examining the graph for the number of correct answers per task type in both groups it is clear that participants in both groups achieved a better overall score on content tasks than on interface tasks. FIGURE 81 - MEAN NUMBER OF RIGHT ANSWERS PER TASK TYPE FROM THE CONFORMING AND NON-CONFORMING GROUPS. Conforming Non-conforming Right answers per second rate (RPS) per task type As in the experiment described in Chapter 6, the rate of right answers per second was also examined. The rate was obtained by dividing the number of right answers by the time to complete each task. One between-subjects, one within-subjects ANOVA (Table 41) applied to the data revealed once more a main effect of task type on performance. 204
- 227. Visual conventions and performance over time TABLE 41 - ONE BETWEEN-SUBJECTS, ONE WITHIN-SUBJECTS ANOVA REUSLTS FOR THE RIGHT ANSWERS PER SECOND (RPS) RATE IN THE CONFORMING AND NON-CONFORMING GROUPS. Between subjects Source SS df Mean Squares F-ratio p-value Conventionality 0.001 1 0.001 0.001 0.979 Error 6.611 8 0.826 Within subjects Source SS df Mean Squares F-ratio p-value Task Type 5.842 1 5.842 7.827 0.023 Task Type x Conventionality 0.042 1 0.042 0.061 0.812 Error 5.603 8 0.700 Looking at the graph in Figure 82 it is possible to see that the overall performance was better on interface task than on content tasks. FIGURE 82 - RIGHT ANSWERS PER SECOND (RPS) RATE PERT TASK TYPE ON THE CONFORMING AND NON-CONFORMING GROUPS. Conforming Non-conforming Quits Both groups had the same number of task quits throughout the experiment (5 in each group). While the participants who used the conforming forum took an average of 227 seconds before giving up on a task, participants who used the non-conforming version gave up on the tasks on average after 195.8 seconds. A Mann-Whitney test showed that this difference was not statistically significant (W=32, n=5, p=0.40). 205
- 228. Visual conventions and performance over time Right answers Each of the 5 participants in each condition was presented with 10 tasks totalling 50 tasks executed per condition. The conforming group totalled 40 (out of 50) right answers, while the non-conforming group totalled 43 right answers. A two-tailed Mann-Whitney test showed no significant difference in the scores (W=23.5, n=5 and p=0.456). However, when the number of right answers was analyzed as participants progressed through the test, a Mann-Kendall analysis showed no significant trend for the conforming group (Z=0.39, n=5, p=0.34) but found a significant uptrend for the non-conforming group (Z=2.48, n=5, p=0.006). Looking at the graph for the non- conforming group (Figure 83) it is noticeable that after the fifth task the number of right answers constantly remains at the value of 10 which is the maximum value possible. Interestingly enough the same does not occur in the conforming group (Figure 84), where the number fluctuates between 8 and 10. FIGURE 83 - NUMBER OF TOTAL RIGHT ANSWERS OVER TIME AND FIT CURVE FOR THE NON-CONFORMING GROUP. Non-conforming Less conv entional 10 Fit curv e 8 Nº of right answers 6 4 2 0 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th Task (first to last) 206
- 229. Visual conventions and performance over time FIGURE 84 - NUMBER OF TOTAL RIGHT ANSWERS OVER TIME AND FIT CURVE FOR THE CONFORMING GROUP. Conforming Navigational adjustments The screen-capture file for one participant of the non-conforming group was corrupted and could not be retrieved or analyzed so that the information on navigation was based on four participants in the non-conforming group and five participants in the conforming group. Backtracking In this experiment, participants tested on the conforming forum used backtracking as their main navigational correction strategy (57 times out of 81 total corrections) while participants tested on the non-conforming forum used backtracking only 27 times out of a total of 71 corrections. However, the difference in the number of times backtracking was used in each group was not significant (W=29, n=5, p=0.39) Revisits Each time during navigation that a participant returned to a page that they had already been to, was considered a page revisit. Participants who used the non- conforming forum had an average of 19.5 page revisits per participant while participants who used the conforming forum had an average of 15.2 page revisits per participant. A Mann-Whitney test showed, however, that this result was not significant (W=23, n=5, p=0.71). This lack of significance might be due to the high variability within the groups (Figure 85). 207
- 230. Visual conventions and performance over time FIGURE 85 - MEAN REVISITS FOR THE CONFORMING AND NON-CONFORMING GROUPS. Conforming Non-conforming 7.2.5. Discussion Task completion time No significant differences have been found in the total time taken for each group to complete the tasks. This could have happened due to the participants of the non- conforming group having developed a performance improvement throughout the test, as was hypothesized. Looking at the progression of the time required to complete the tasks in the non-conforming group, a downward trend is clear. This means that as the test progressed, participants from this group found answers for the tasks more quickly. This seems to indicate that these participants, although familiar with a certain set of visual conventions, when using a website which violated their expectations they were quick to learn the new layout. No trend was noticeable for the group which used the conforming forum. This was expected since these participants did not have to learn any new layout and therefore should show no improvement or worsening of performance. Right answers There were no significant differences in the total number of right answers in each group. But the experiment indicated that while there was no significant trend in the conforming group, the non-conforming group showed a slight but significant upward trend in the number of right answers. This result allied with the downtrend 208
- 231. Visual conventions and performance over time in time seem to suggest that the non-conforming group had an improvement in performance throughout the test. Task type The results showed that in both groups participants took longer to answer content tasks than they did to answer interface tasks. But results also show that participants in both groups answered more content tasks correctly than they did interface tasks. When results were combined into the RPS rate it shows that there were no difference in the performance between the groups in each type of task. In other words, the performance on interface tasks between the conforming and non-conforming groups was equivalent. The same happened with the content tasks. Overall, the RPS rate was higher on interface tasks which is expected, since finding information within content requires that participants read or skim through content and comprehend what they are reading, which takes longer, whereas retrieving information from the interface is just a matter of locating it. And although the non- conforming forum violated the visual conventions of the genre participants were familiar with, it still conformed with the linguistic, structural and functional conventions of the web forum genre. This may have simplified the task of retrieving information from the interface. Efficiency indicator Although the efficiency indicator (Smith, 1996) is a valid measure of efficiency (Larson & Czerwinski, 1998; Otter & Johnson, 2000; Selvidge, et al., 2002) the absence of a significant difference in the results in this pilot study raises the question of whether it should be applied in the main experiment. Considering that in the main experiment backtracking, revisits and loopbacks will be measured as well as right answers and time taken it seems there is no need to go through this analysis since similar conclusions can be drawn from the other variables. Tasks From observations during the experiment, it was clear that the main reason for the difficulty of answering task H was due to the wrong and misleading wording of the task (‘How much is worth a T.S. Eliot signed 1st edition?’). The forum had two 209
- 232. Visual conventions and performance over time different topics on the index page where participants looked for the answer, one entitled ‘First editions and signed books’ – where the answer really was, and the other entitled ‘How much is my book worth?’. Because of the wording of the questions, participants tended to first go in the ‘How much is my book worth?’ topic and that delayed the answer. This indicates that this task should not be used in the main experiment. Also, greater care should be taken with the wording of the tasks so as not to mislead the participants since doing so would add a confounding variable to the experiment. Task I (‘Which thread under the “What I’m reading now” has the greatest number of views?’) was answered quite quickly and with little variability among participants and could be explained in several ways. First, it required a small number of steps to complete since the answer to the task was in the threads page (the second level of the forum). It also showed the participant exactly where to go to find the answer (‘What I’m reading now’ topic). When participants got to the second level (where the answer was) they were faced with only 12 threads on a single web page so finding the answer was straightforward. Finally, the high variability of task D (‘Find the answer for this frequently asked forum question: “How do I prevent my user name from appearing in the online user’s listing”?’) could have different explanations; one could be that some participants did not know that the menu item ‘FAQ’ was an acronym for ‘Frequently Asked Questions’ (in the conforming version) or that the question mark icon stood for the FAQ section (non-conforming version). Another explanation would be that participants simply did not know where to find the answer to a ‘Frequently Asked Question’. In summary, the problems found in the design of the tasks can be categorized into the following types: ! Difficult and misleading wording of the task which could lead participants down the wrong path to find the answer. This adds to the difficulty of the task. ! Tasks which easily indicate the path to the answer and where the participant should find it on a page. This makes the task easy. ! Answers which are located quite close to the start page, requiring little navigation. In other words, the answers are not deep enough within the web site structure. 210
- 233. Visual conventions and performance over time ! Answers which are located in pages with too few textual elements making it very simple for the participant to find the answer. ! Participants not being familiar enough with the terminology used in web forums. Backtracking and revisits The small number of participants on each condition could have led to the lack of significant results for the usage of backtracking or the number of page revisits on this test. Since these two variables are considered good indicators of disorientation (Xu, et al., 2001; Yatim, 2002) they will be considered in the next experiment. However a perceived disorientation measurement through a questionnaire will be added to the main experiment. 7.2.6. Conclusion The results from this pilot indicate that participants familiar with a certain set of visual conventions might learn a layout which violates these conventions quite quickly. This could explain the difficulty in finding significant performance differences when comparing user performance on conforming sites with performance on non-conforming sites. However, further investigation with a larger sample is needed in order to confirm these results and verify if the violation of visual conventions has any further impact on user disorientation and performance. The lack of significant results on the disorientation measures (backtracking and revisits) may indicate, as suggested by Ahuja and Webster (2001), that perceived disorientation may be a more reliable predictor of disorientation than actions themselves. Therefore it might be a good strategy in the main experiment to measure not only the physical aspects of disorientation, but also the perceived disorientation, as suggested in Gwizdka and Spence (2007) and Herder (2003). 211
- 234. Visual conventions and performance over time 7.3. The main experiment 7.3.1. Objectives Similarly to the pilot study, this experiment will attempt to verify if participants using a non-conforming forum will have their performance improved as they progress through the test compared to participants using a conforming one. But based on what the pilot showed, this experiment will go further by testing whether the participants who used a non-conforming forum will continue to show a performance improvement when using the same non-conforming interface on another web forum. The effect of the type of task on the performance of participants using both conforming and non-conforming forums will also be analysed. This experiment will also try to confirm if not conforming to conventions will result in user disorientation. 7.3.2. Method Design This experiment used two independent groups of participants, with one tested using the conforming forum design and the other using the non-conforming forum design. As in the pilot study and the experiment described in Chapter 5, each group had to perform a number of tasks which were divided into two types: content and interface. The variables that were being measured in this experiment (dependent variables) were the task completion time, the number of wrong answers, the number of navigation adjustments done by each participant and the number of page revisits done of participant. This test was divided in two parts with a 5 minute interval between them. In the first part, participants were tested on the same web forum used in the pilot study. In the second part participants were tested on the same web forums used in the experiment described in Chapter 6 (one conforming and one non-conforming). The group of participants which was tested on a conforming forum in the first part of the experiment was tested on a conforming forum in the second part as well. The group of participants which was tested on a non-conforming forum in the first part was tested on a non-conforming forum in the second part. 212
- 235. Visual conventions and performance over time Participants Twenty-eight participants, all graduates or post-graduates from different disciplines volunteered for the experiment. Participants were randomly allocated to one of the experimental groups in a way that each group contained 14 participants. Volunteers for this study had not previously participated in any of the experiments conducted so far neither in the pilot study. All participants were web forum users having reported continuously using at least one web forum site. Participants were fluent in English (either as first or second language). Materials First part As mentioned above, for this part the forums used were the same as those used in the pilot study and no alterations were made to their designs. Second part Since participants for this experiment were entirely new to this research and did not have any contact with previous experimental material, it was decided that the forum used for this part would be the one used in the experiment of Chapter 6. To keep the interface in this part consistent with the interface of the previous material the same modifications that were considered for the first part material (see 7.2.2, materials section) were made on this forum. To keep consistency with the procedures that participants used in the first part, the same pop-up windows system which allowed participants to indicate the end of a task was added to the material for the second part. Tasks design From the pilot results it was possible to see that the design of the tasks would need rethinking for the main experiment. Because the main experiment consists of two parts it was decided that in the first part the number of tasks would be the same as in the pilot study. The second part of the study was shorter than the first, so only six tasks were designed for this part. 213
- 236. Visual conventions and performance over time From the results of the pilot study, the following measures were taken when designing the tasks for this experiment: ! the wording used was as simple as possible and whenever possible some key words were added in the sentence to act as cues to the places where the answers were. For example, the answer to the task ‘Identify which company now owns the publisher called Doubleday’. (Obs.:italics added for highlighting purposes only and were not present in the experimental task given to participants) would be found in : ‘Help on identification > Identification of publishers’. ! whenever possible tasks would not indicate the entire path to the answer and tasks were designed to have answers on pages with not too little textual elements. ! tasks would be designed whenever possible to have answers as far away as possible from the forum start page. Again in this study tasks were of two main types: content tasks and interface tasks. However, in order to better analyze any possible interactions between question type and conventionality, the number of interface and content tasks was made the same in both parts, with 5 content and 5 interface tasks in the first part and 3 content and 3 interface tasks in the second part. Because giving participants all tasks of one type, followed by all tasks of another type, could introduce a confounding variable (task order), tasks were counterbalanced in an CI-IC fashion, in other words, one participant would be given all their tasks in this order: a content task first, followed by an interface task (CI) and so on, while another participant would do an interface task followed by a content task (IC) and so on. This would counterbalance any effects that starting with a particular task type might have on the performance of participants. Also, within the counterbalancing scheme, tasks were also randomized. The counterbalancing scheme was maintained in the second part of the experiment so that a participant who started with an interface task in the first part would start with an interface task in the second part and a participant who started with a content task in the first part would start with a content task in the second part. 214
- 237. Visual conventions and performance over time The tasks presented to participants in both parts of the test can be seen in Appendix O. Disorientation questionnaire Based on the work by Ahuja and Webster(2001) a questionnaire to assess the perception of users regarding their disorientation when using the web forums was created. Although the same ideas given by the authors were followed, this questionnaire contained fewer questions than the original questionnaire designed by Ahuja and Webster. This was because the original questionnaire had some of the questions repeated but reverse-coded, so for reasons of simplicity and in order not to tire the participants of the experiment excessively, only 5 of the authors’ original questions were included in the questionnaire. The answers to the questionnaire were arranged in a 5 point Likert scale but no numbering was placed on the scale, only squares with the respective answer below it. Table 42 shows the questions used in the questionnaire. For a view of the complete questionnaire used in the present study refer to Appendix P. TABLE 42 - QUESTIONS IN THEIR ORDER OF APPEARANCE IN THE QUESTIONNAIRE. Question 1 I felt lost / disoriented Question 2 I felt like I was going around in circles Question 3 It was difficult to find a page I had previously viewed Question 4 Learning to use the site was easy Question 5 The site was easy to navigate Questions 1, 2 and 3 asked participants about their perceived sense of disorientation and questions 4 and 5 were concerned with their perceived ease of learning the interface. 7.3.3. Procedure Participants were tested individually and when a participant entered the room, they were asked to sit in front of the computer and the experiment was explained. They were told that the test would be divided into two parts with a 5-minute break between each part. They were informed that each part consisted basically of looking for answers to questions within a web forum and that there would be a different 215
- 238. Visual conventions and performance over time forum for each part. It was then explained that whenever the participant found a satisfactory answer they should select the text (with a click and drag mouse movement) and click on the finished button that was on the pop-up window to the left of the browser. Participants were told to look for the answers as fast and as accurately as possible. They were told that there were no questions with double meanings and all the answers were on the forum. Participants were told to behave as they would in a normal web environment, and were allowed to give up on the search for an answer at any time they wanted. The researcher reinforced to the participant that the experiment was concerned with testing the design of the site and not the participant so they should not feel pressured. When the participant was ready the web forum was presented and the paper with the printed task was given to them. The participant would read the paper and as soon as they finished reading and looked at the screen the stopwatch would be started. Screen-capture software was set to capture a copy of the current screen whenever the participant clicked the mouse. Whenever a participant found an answer they would select it and click on the ‘finished’ button that was located on a window to the top left of the web browser. That action would take them back to the first level of the forum and another paper with a task would be given to them and the procedure would be repeated until all tasks for the first part were completed. Then the participant would be requested to answer the questionnaire on their perceptions of disorientation and ease of use of the forum in the first part. After that the participant would take a 5 minute break while the researcher prepared the material for the second part. The procedures for the second part were the same as the procedures in the first part. When the participant completed all the tasks from part two they were requested to fill in the questionnaire on their perceptions of disorientation and ease of use of the forum in the second part. 216
- 239. Visual conventions and performance over time 7.3.4. Results for part 1 of experiment 5 Time The group which was tested on the conforming forum took on average 594.1 seconds (SD=192.1) to complete the first part of the test while the group which was tested on the non-conforming forum took on average 750.5 seconds (SD=304.4) to complete the first part of the test. This difference was not significant (F(1,26)=2.64, p=0.116). Looking at the time taken to complete the tasks over the course of the first part, it is evident that both the non-conforming (Figure 86) and the conforming (Figure 87) groups seemed to have reduced the time to complete the tasks over time. A Mann- Kendall trend test showed a downward trend in both the non-conforming group (z=- 2,14, p=0.015) and the conforming group (z=-2.86, p=0.002). FIGURE 86- MEAN TIME TO COMPLETE EACH TASK OF THE NON-CONFORMING GROUP OVER THE COURSE OF THE FIRST PART OF THE EXPERIMENT AND THE CORRESPONDENT FIT CURVE. Non-conforming 217
- 240. Visual conventions and performance over time FIGURE 87 - MEAN TIME TO COMPLETE EACH TASK OF THE CONFORMING GROUP OVER THE COURSE OF THE FIRST PART OF THE EXPERIMENT AND THE CORRESPONDENT FIT CURVE. A trend curve fit test showed that the best fit for both cases was a quadratic trend model, but despite both groups having a quadratic downtrend it is possible to tell from the graphs that the decrease in the time taken to complete the tasks happens faster in the non-conforming group than in the conforming. Initially participants in the non-conforming group take much longer to complete the tasks than participants in the other group. However, towards the end of the experiment the difference in time taken to complete the tasks between the groups is much less. Right answers and quits There was no significant difference in the number of right answers in the conforming and non-conforming group (F(1,26)=0.76, p=0.390). There was not a significant difference in the number of times participants in each conditions quit a task (F(1,26)=0.43, p=0.517). Right answers over time The number of correct answers obtained by the participants of the non-conforming group (Figure 88) increased as the test progressed while there was no particular trend in the conforming group (Figure 89). 218
- 241. Visual conventions and performance over time FIGURE 88 - NUMBER OF RIGHT ANSWERS IN THE NON-CONFORMING GROUP THROUGH THE TEST. Non- conforming FIGURE 89 - NUMBER OF RIGHT ANSWERS IN THE CONFORMING GROUP THROUGH THE TEST. Conforming A Mann-Kendall trend test showed a significant upward trend in the number of right answers for the participants of the non-conforming group (z=1.720, n=10, p=0.042). The test showed no upward (z=1.106, n=10, p=0.134) or downward (z=1.106, n=10, p=0.865) trend in the number of right answers of the conforming group. The results of a trend curve fit test on the progression of right answers on the non-conforming 219
- 242. Visual conventions and performance over time group showed a very small difference between the fit accuracy measures on the linear and quadratic model although the Quadratic model was the best fit. TABLE 43 - COMPARISON BETWEEN THE LINEAR AND QUADRATIC MODELS MEASURES OF ACCURACY FOR THE NON-CONFORMING GROUP. Measures Quadratic Linear MAPE (Mean Absolute Percentage Error) 24.0057 24.0871 MAD (Mean Absolute Deviation) 1.8797 1.8873 MSD (Mean Squared Deviation) 4.8829 4.9018 This means that although the equation which generates the curve which best fits the data is quadratic, its exponential component has less influence than the linear one meaning that the performance increased practically in a linear way. This causes the quadratic fit curve to behave almost as a straight line as can be seen in Figure 88. Right answers per second (RPS) rate As it was previously done in the past two experiments the right answers per second rate (RPS) was obtained by dividing the number of right answers by time. However, to simplify calculations without compromising the analysis, this time the ratio was multiplied by a constant. The formula to achieve the RPS is illustrated in Figure 90. FIGURE 90 - RIGHT ANSWERS PER SECOND RATIO FORMULA. A one-way ANOVA on the RPS ratio for the first part of the test showed a significant difference (F(1,26)=5.70, df=1, p=0.025) between the conforming (M=1.30, SD=0.41) and non-conforming group (M=0.98, SD=0.30). The participants in the non-conforming group showed a lower mean RPS than the conforming group which indicates that in this part of the test their performance was not as good as the other group of participants. 220
- 243. Visual conventions and performance over time RPS rate over time A Mann-Kendall trend test on the right answers per second data showed no trend in the conforming group (z= -1.25, pup= 0.895, pdown= 0.105) and an upward trend (Figure 91) in the non-conforming group (z=2.50, p=0.006). FIGURE 91 - PROGRESSION OF RIGHT ANSWERS PER SECOND IN THE NON-CONFORMING GROUP. Non- conforming A trend curve fit test showed the linear trend to be the best fit for the data. Overlapping the RPS progression graphs of the conforming and non-conforming groups it is possible to see that by the end of the test, the performance of the non- conforming group was close to the performance of the conforming group (Figure 92). 221
- 244. Visual conventions and performance over time FIGURE 92 - PROGRESSION OF THE RPS RATIO FOR BOTH GROUPS IN THE FIRST PART OF THE TEST. Conforming Non-conforming Looking at the graphs of the two groups it is noticeable that towards the end of the first part, the RPS graphs for the two groups are very close. This happens after the point where the non-conforming trend line crosses the graph of the conforming group. After that there are three RPS scores for each group. A post-hoc analysis on these points showed no significant statistical difference between the last 3 RPS scores of the two groups (F(1,26)=1.85, p=0.186). This result confirms that by the end of the first part the performance of the two groups was equivalent. Navigational adjustments Revisits Participants who used the non-conforming forum performed more page revisits (M=20.36) than the participants who used the conforming forum (M=7.93). A Mann- Whitney test adjusted for ties on the number of page revisits carried out by each group showed that the difference was significant (W=140.5, n=14, p=0.0043). Backtracking Again, participants who used the non-conforming forum used the back button more times (M=17.36) than participants who used the conforming forum (M=5.07). A Mann-Whitney test adjusted for ties on the number of backtracking actions carried 222
- 245. Visual conventions and performance over time out by each group showed that the difference was significant (W=137.5, n=14, p=0.003). Loopback A Mann-Whitney test adjusted for ties showed there was no significant differences in the number of times participants used the loopback strategy to go back to a page where they have been before (W=224.0, n=14, p=0.323). RPS rate per task type Analyzing the ratio of right answers per second (RPS) per task type on each of the counterbalancing options (CI and IC) showed no carryover effect on the time to complete tasks (Table 44). TABLE 44 - AVERAGE RIGHT ANSWERS PER SECOND FOR EACH TYPE OF TASKS ON EACH COUNTERBALANCING ORDER. Content task time Interface task time Content tasks first (CI) 0.92 2.25 Interface tasks first (IC) 0.96 2.53 An analysis of the RPS to complete each task type and the conventionality of the forums using a one between-subject, one within-subject ANOVA showed no main effects of task type (F(1,26)=2.73, p=0.111) or conventionality (F(1,26)=2.31, p=0.140) but found a significant interaction between task type and conventionality (F(1,26)=46.47, p<0.01). This type of interaction is called a disordinal interaction and in it the group differences tend to reverse their sign at some level of the other variable (Figure 93). As expected, in this part of experiment 5, participants who used the conforming website performed better on interface tasks than on content tasks. With the participants who used the non-conforming website the situation was reversed, they performed better on content tasks than on interface tasks. 223
- 246. Visual conventions and performance over time FIGURE 93 – AVERAGE RIGHT ANSWERS PER SECOND RATE (RPS) ON CONTENT AND INTERFACE TASKS PER GROUP IN THE FIRST PART OF THE EXPERIMENT IN THE FIRST PART. Number of quits per task type Analyzing the number of quits per task type on each group with a 1-between subjects 1-within subjects ANOVA showed no main effects of task type (F(1,26)=0.00, p=1.00) or conventionality (F(1,26)=0.432, p=0.517) but found a significant disordinal interaction (F(1,26)=24.27, p<0.01) between the task type and the conventionality of the forum (Figure 94). FIGURE 94 - MEAN NUMBER OF QUITS FOR EACH GROUP ON EACH TYPE OF TASK. Conforming Non-conforming 224
- 247. Visual conventions and performance over time In this test participants of the conforming group quit fewer times on interface tasks than on content tasks, and with the participants of the non-conforming group the situation was reversed: they quit fewer times on content tasks than on interface tasks. This could explain the RPS rates shown before; however, further examination of the data reveals that participants in the conforming forum quit more often on content tasks than on interface tasks (Figure 94). This was followed by them having completed fewer content tasks correctly when compared with interface tasks (Figure 95). This consequently led to a lower RPS rate. FIGURE 95 - MEAN NUMBER OF RIGHT ANSWERS PER GROUP FOR EACH TASK TYPE. Conforming Non-conforming But the inverse is not true for the non-conforming group. Although participants quit more on interface tasks than content ones, they did not complete more content tasks correctly compared with interface tasks. In fact, they still completed more interface tasks correctly than content tasks. Therefore their higher RPS rate on content than on interface tasks is explained by the fact that the participants who used the non- conforming forum completed content tasks much faster than they completed interface tasks, as can be seen in Figure 96. 225
- 248. Visual conventions and performance over time FIGURE 96 - AVERAGE QUITS ON CONTENT AND INTERFACE TASKS PER GROUP. Navigational adjustments per task type Revisits Overall participants made more page revisits in the content type tasks (M=10.75, SD=12.21) than on interface tasks (M=3.42, SD=5.14); this result was significant (F(1,54)=8.55, p=0.005). There was no interaction of revisits and task type, and nor was there a main effect of conventionality. Backtracking Participants also used the back button more when answering content tasks (M=8.54, SD=11.28) than when answering interface tasks (M=2.67, SD=4.21). A ANOVA showed that this result was significant (F(1,54)=6.63, p=0.013). There was neither interaction of backtracking and task type, and nor was there a main effect of conventionality. Loopback Loopback was also used more when participants were answering content tasks (M=1.29, SD=1.90) than when they were answering interface tasks (M=0.32, SD=0.72). This result was significant (F(1,54)=6.29, p=0.015). Questionnaire In the first part participants in the conforming and non-conforming group only differed in their perceptions of the ease of learning the website (see Question 4 on 226
- 249. Visual conventions and performance over time Table 42). Participants who used the conforming version thought it was easy to learn to use their site, while the other group neither agreed nor disagreed. A Mann- Whitney test comparing the answers of the conforming (Median=4) and non- conforming (Median=3.5) groups for this question showed that this result was significant (W=242.5, n=14, p=0.046). 7.3.5. Results for part 2 of experiment 5 Time A one-way ANOVA showed no difference in total time to complete the tasks for the second part of the test (F(1,26)=0.77, p=0.388). The conforming group showed an average of 253.8s (SD=123.2) and the non-conforming group an average of 298.4s (SD=144.9). The progress of time to complete the tasks showed no trend in the conforming group (z= -0.37, p=0.354) nor in the non-conforming group (z= 0.37, p=0.646) (Figure 97 and Figure 98). FIGURE 97 - AVERAGE TIME TO COMPLETE EACH TASK IN THE CONFORMING GROUP. 227
- 250. Visual conventions and performance over time FIGURE 98 - AVERAGE TIME TO COMPLETE EACH TASK IN THE NON-CONFORMING GROUP. Overlapping the two graphs (Figure 99) it is obvious that the shapes of the graphs of the groups are similar, which seems to suggest an equivalent performance in both groups in the second part of the test. Another interesting feature visible in the graph is the longer time it took both groups to complete the fifth task. Remembering that the fifth task was not the same for all the participants, rules out the possibility of something intrinsically more difficult about the task. One possible explanation could be that participants were getting tired since the variability on the fifth and sixth tasks is larger than the variability of the other tasks. Still, this does not explicate why the variability in the sixth task is smaller than the variability of the fifth task. FIGURE 99 - OVERLAPPED GRAPHS OF TIME TO COMPLETE TASKS ON BOTH GROUPS. 228
- 251. Visual conventions and performance over time Right answers and quits The difference in the number of right answers in the conforming group (M=4.57, SD=1.95) and in the non-conforming group (M=5.43, SD=0.65) was not significant (F(1,26)=2.44, p=0.131). The difference between the number of times each participant quit the tasks in the conforming (M=0.36, SD=0.63) and the non-conforming group (M=0.14, SD=0.36) was not significant (F(1,26)= 1.21, p=0.282). Right answers over time A Mann-Kendall analysis of the progression of the number of right answers for both groups showed no particular trend for the conforming group (z= -0.60, p=0.727). The non-conforming group presented an upward trend in the number of right answers as the test progressed (z=1.81, p=0.035). These results are illustrated in Figure 100 (conforming group) and Figure 101 (non-conforming group). FIGURE 100 - PROGRESSION OF RIGHT ANSWERS AND CORRESPONDENT FIT CURVE FOR THE CONFORMING GROUP. Conforming 229
- 252. Visual conventions and performance over time FIGURE 101 - PROGRESSION OF RIGHT ANSWERS AND CORRESPONDENT FIT CURVE FOR THE NON-CONFORMING GROUP. Non-conforming Mean RPS rate Again, to have a better idea of the performance of the participants in the second part of the test, the right answer and time results were combined following the same principles applied before. A one-way ANOVA found no significant difference between the mean RPS rate (F(1,26)=0.02, p=0.882) of the conforming (M=2.16, SD=1.39) and non-conforming group (M=2.23, SD=1.10). RPS rate over time A Mann-Kendall test found no trend in the conforming group (z= -0.375, pup=0.646, pdown=0.354) and no trend in the non-conforming group (z= 0.00, pup=pdown=0.5). Navigational adjustments Revisits The difference in the number of page revisits by the participants in the conforming group (M=3.00, SD=3.21) and the participants in the non-conforming group (M=8.36, SD=12.93) was not significant (F(1,26)=2.26, p=0.144). Both groups showed a lower mean number of page revisits than in the first part of the test, with the non- conforming group reducing the number of page revisits quite considerably. Backtracking Here also the difference in the usage of backtracking by the participants in the conforming group (M=1.79, SD=2.08) and the participants in the non-conforming group (M=6.14, SD=8.94) was not significant (F(1,26)=3.15, p=0.088). 230
- 253. Visual conventions and performance over time Loopback The number of times the participants in the conforming group (M=0.64, SD=0.93) used the loopback strategy was not significantly different (F(1,26)=0.45, p=0.509) from the number of times the participants in the non-conforming group used it (M=0.43, SD=0.76). RPS rate per task type As in the first part of the test, in the second part there was no carryover effect of task on the right answers per second ratio (Table 45). TABLE 45 - MEAN RPS RATE PER TASK TYPE PER TASK TYPE IN THE TWO COUNTERBALANCED SITUATIONS. Content task RPS Interface task RPS Content tasks first (CI) 2.87 2.62 Interface tasks first (IC) 2.57 1.40 A one-way ANOVA showed that the difference between the two interface task RPS ratios (CI vs. IC) was not significant (F(1,11)=2.20, p=0.169). A one between-subject, one within-subject ANOVA showed an interaction between task type and the conventionality of the website (F(1,26)=9.71, p=0.004). This time the interaction was ordinal (Figure 102) and the ANOVA also showed a significant main effect of task type (F(1,26)=21.47, p<0.01). Looking at the graph, the main effect of task type is quite clear; participants in both groups got more right answers per second on the content tasks than on the interface tasks. The interaction shows no difference in task type performance on a conforming page but there is a great gap in performance between content and interface tasks when the page does not conform to conventions. 231
- 254. Visual conventions and performance over time FIGURE 102 - AVERAGE RIGHT ANSWERS PER SECOND RATE (RPS) ON CONTENT AND INTERFACE TASKS PER GROUP IN THE SECOND PART. There was no difference in RPS rate of the conforming group for interface and content tasks in the second part (F(1,26)=0.472, p=0.498). Close examination also shows that in the second part of the test (see Figure 102), the RPS rate of the non-conforming group on interface tasks was equivalent to the rate of the conforming group (F(1,26)=0.65, p=0.426). Interestingly, on content tasks the non- conforming group outperformed the conforming group (F(1,26)=4.77, p=0.038). Number of quits per task type There were no main effects or interaction on the number of quits per task type in each group. Navigational adjustments per task type Revisits There were no main effects or interactions of conventionality and task type on the number of page revisits made by the two groups in the second part of the test. Backtracking There were no main effects or interactions of conventionality and task type on the number of backtracks made by the two groups in the second part of the test. Loopback A 1-between subjects, 1-within subjects ANOVA showed a main effect of task type (F(1,26)=6.382, p=0.018) in the number of times the loopback strategy was used by the participants of the groups. Participants used more loopback when responding to 232
- 255. Visual conventions and performance over time interface tasks than when responding to content tasks. There was, however no significant interaction between conventionality and task type (F(1,26)=0.709, p=0.407) on the use of loopback. Questionnaire There were no differences between groups in the feelings of disorientation and ease of learning and navigating the web forums in the second part of the test. However when comparing the answers given by a group in the first part with the answers given by the same group in the second part, some interesting results came up. A Wilcoxon signed-rank test showed that the non-conforming group felt more lost in the first part than in the second part (Z=-2.72, n=14, p=0.006) and they felt as if they were going round in circles more in the first part than in the second part of the test (Z= -2.73, n=14, p=0.006). There were no differences in the perception of the non- conforming group regarding: (a) the difficulty of finding a previously viewed page; (b) the ease of learning the site; (c) the ease of navigating the site. There were no differences between the answers given by the conforming group in the first and second parts of the test. 7.3.6. Summary of results Below is a summary of the significant results obtained from this experiment. TABLE 46 - SUMMARY OF THE RESULTS OF EACH GROUP IN EACH PART OF THE TEST. Exp. Part Conforming Non-conforming Part I Downward trend Downward trend Time to complete task Part II No trend No trend Part I No trend Upward trend Number of right answers Part II No trend Upward trend Part I Higher Lower RPS Rate Part II No difference No difference Part I No trend Upward trend RPS Rate trend Part II No trend No trend Part I Fewer More Page revisits Part II No difference No difference Part I Fewer More Backtracking Part II No difference No difference Part I Constant Higher Feeling of disorientation Part II Constant Lower 233
- 256. Visual conventions and performance over time Interactions In the first part of the experiment, the conforming group performed better on interface tasks than on content tasks, and the situation was reversed for the non- conforming group where performance was better on content tasks than on interface tasks. In the second part of the experiment the conforming group showed no difference in performance between content and interface tasks, and the non- conforming group maintained a better performance on content tasks than on interface tasks. The interaction observed in the second part of the experiment was different from the interaction in the first part of the experiment. The difference in performance of the participants from the conforming group detected in the first part of the experiment could be due to the forum used in the first part being larger, and consequently having more content than the one in the second part. The same difference in performance of the conforming group was shown in the pilot study described in this chapter which used the same forum as in the main experiment. However, looking at the interaction between conventionality and task type of the experiment described in Chapter 6, the conforming group showed no difference in performance between the task types. The web forum used in the experiment in Chapter 6 was the same as the one used in the second part of the main study in this chapter. Again this web forum had less content than the one used in the first part. This suggests that the performance on task type may also be affected by the ammount of content the website has. In the first part of the test participants in the conforming group quit more on content tasks than on interface tasks, but in the non-conforming group the situation was reversed with participants quitting more on interface than on content tasks. In the second part of the test there were no effects or interaction between task type and conventionality. In the first part of the test, participants made more page revisits, more use of backtracking and of loopback when answering content tasks than when answering interface tasks. In the second part there were no differences in page revisits or backtracking between the two task types. However participants used the loopback strategy more often when answering interface tasks than content tasks. 234
- 257. Visual conventions and performance over time 7.3.7. Discussion This study has demonstrated that as the literature suggests, not conforming to visual conventions can leave users disoriented (Dillon & Vaughan, 1997) and hinder their performance (Nielsen, 1999a). But it has also showed that their disorientation and hindered performance can be short-lived. Time and right answers There was no overall difference in time to complete the tasks nor in the number of right answers between the groups in either part of the test. Considering what the literature says, this result is quite unexpected. However, as will be seen below, a decrease in time to complete tasks and increase in number of right answers of the participants was observed in the non-conforming group throughout the first part of the test. This could have compensated the poorer performance they showed in the beginning of the test, making their overall time and right answers results close enough to that of the group which used the conforming template so a difference was not detected. RPS rate In the first part, performance (measured by the RPS rate) for the group using the non- conforming website was not as good as the performance of the group using the conforming website. But, as expected, their performance improved throughout the first part of the test, so that by the end of the first part, participants in the non- conforming forum had an equivalent performance to the participants of the conforming group. This suggests that although not conforming to visual conventions might cause usability problems thus hindering performance (Nielsen, 1999a), this performance hindrance is short lived and users familiarize themselves with the new interface layout quite quickly. In the second part of the test there was no difference in the RPS rate of both groups. This indicates that the performance of both groups were equivalent in the second part of the test. One plausible explanation is that participants who used the non- conforming website had familiarized themselves with the layout of the interface in the first part of the test so that their performance was equivalent to the group which used the conforming version in the second part. An alternative explanation would be that because the site used in the second part had fewer topics and fewer pages than the one in the first part, it was easier to find the 235
- 258. Visual conventions and performance over time answers to the tasks. However, an examination of the answers given in the questionnaire by the group who used the non-conforming website regarding how easy it was to learn to use and navigate the forum showed that participants attributed the same difficulty to both the first part and the second part forums. This indicates that they did not think the site in the second part was easier to learn than the one in the first part. Disorientation In the first part of the test, participants using the non-conforming site did more page revisits and backtracking movements than the conforming group. This seems to indicate that they needed to re-orient themselves more than the other participants (Yatim, 2002) and could consequently be more disoriented than their counterparts. But, according to the questionnaire answered in the first part, they have felt as lost and disoriented as the participants in the conforming group. One explanation for this difference between the perceived and measured disorientation of the non- conforming group is that by the end of the test, participants were not as lost as they were in the beginning of the test. Since the questionnaires were answered at the end of each part, it is possible that their final disorientation state had more influence on the answer than the initial one. However, looking at the perceived disorientation of the non-conforming group across the two parts of the experiment, there was an improvement in their condition; they had felt more disoriented in the first part than in the second. These results are in accordance with the measured disorientation of the group, since in the first part they made more page revisits and backtracking movements than in the second part. The perceived disorientation, ease of learning and navigation of the site by the participants of the conforming group were not significantly different between the first and second part of the test. Task type Both in the first and second parts of the test, participants who used the non- conforming forum performed better on content tasks than on interface tasks. But for the group of participants who used the conforming forum, the situation was different; in the first part participants performed better on interface tasks than on content tasks but in the second part there was no difference in performance between content and interface tasks. So, when visual conventions are violated, users who are familiar with the conventions perform better on search tasks which depend solely on 236
- 259. Visual conventions and performance over time written content, than on search tasks which depend on interface information. It makes sense to think that when visual conventions of a website are violated the interface elements do not meet users’ expectations thus causing their performance to suffer. If structural conventions of a website are not changed, the organization of content in the website should remain quite the same, regardless of any violation visual conventions. Considering that all participants were familiar with the visual conventions of web forums, it is not wrong to assume that finding information in the conforming interface would be easier than finding content, which seems to support the results of the group who used the conforming forum in the first part of the test. But their results in the second part of the test showed no difference in performance on interface and content tasks. One explanation could be that the content tasks in the second part were easier than the interface tasks; however in both parts tasks were designed based on the analysis of the results from the pilot study. The only factor which could have caused content tasks to become easier in the second part would be that the forum used in that part was purposely built with less content than the one used in the first part. As a consequence, answers were not as deep in the forum in the second part as in the first part and the pages where answers were located did not have as many elements as the pages in the first part. The results for the group of participants who used the non-conforming forum showed their behaviour to be consistent across the two parts of the test. They performed better on finding content than retrieving information from the interface. One simple explanation would be that, once learned, the non-conforming forum was easier to use than the conforming one. But this would probably be reflected in the interface tasks as well as the content ones. An alternative explanation would be that in order to adapt to the new interface, participants who used the non-conforming forum had to make more cognitive effort in the first part than their counterparts. This cognitive effort is reflected in the higher number of page revisits (Tauscher & Greenberg, 1996) and disorientation that the non-conforming group showed in the first part compared to the conforming group. Literature has shown that the higher the cognitive effort needed to acquire some information, the more likely the person will be to recollect the information at a later stage (Dewhurst & Hitch, 1999). So it could be that the cognitive effort made by them in the first part helped participants to recall the interface better than the conforming 237
- 260. Visual conventions and performance over time group in the second part, thus improving their performance. Another explanation which shares similar grounds with the one above is found in familiarity literature (Yonelinas, 2002; Zhang & Ghorbani, 2004). As seen in the previous chapter, deeper levels of processing provide users with greater familiarity (Yonelinas, 2002; Zhang & Ghorbani, 2004) and increase recollection (Dewhurst & Hitch, 1999; Yonelinas, 2002). In this experiment it is possible that participants who used the non-conforming forum had to use deeper processing in the first part of the test. This could have improved their recollection and made them achieve a high level of familiarity with the interface which could be equivalent with the familiarity they might have had with the conforming forum. 7.3.8. Conclusion This study has shown that when users who are accustomed to a certain set of visual conventions have to use a website which violates these conventions they adapt to them quite quickly. This can explain the lack of significant results not only in the study described in Chapter 6, but also in the study described earlier conducted by Kalbach and Bosenick (2006). It is likely that the changes in the menu positioning made in their study were such a small violation of visual conventions that users were able to quickly adapt and no differences could be detected. Reinforcing this idea are the results of a similar study conducted by McCarthy et al.(2003). The study has also violated the conventional positioning of the navigation menu and although in this case a difference in performance was detected, the authors have indicated performance improved quite quickly for the users who used the non-conforming layout. From the results of this experiment and indications from Kalbach and Bosenick(2006) and McCarthy et al.(2003) it can be concluded that the performance and disorientation problems that originate from the violation of the visual conventions of a website genre are quickly overcome. Therefore, looking back at the outcomes of the experiment conducted by Vaughan and Dillon (2006) which violated visual and structural conventions of a web news genre, it is possible to ponder whether violation of any of the four types of genre convention will have the same impact on user performance. As explained earlier, Vaughan and Dillon (2006) reported that in their experiment the violation of genre conventions resulted in a significant performance hindrance. However, the authors also pointed out that after 5 trials with the violation website the participants improved their performance and the differences between them and the participants using the convention-conforming website disappeared. 238
- 261. Visual conventions and performance over time Although the performance improvement of participants using the non-conforming website shown in Vaughan and Dillon (2006) took longer to happen than the performance improvement shown in this thesis, it reinforces the idea that users can adapt to violations of genre conventions. The difference in time participants using a non-conforming website took to improve performance in this thesis compared to participants in the Vaughan and Dillon (2006) study suggests that participants may take longer to improve their performance when structural conventions are violated in comparison with visual conventions. Another possibility is that not conforming to visual or structural conventions will have the same impact on user performance but when both visual and structural conventions are violated simultaneously ( as in Vaughan & Dillon, 2006) they produce a synergistic effect. This illustrates not only the importance of the work developed in this thesis but also the need for more understanding of how each type of genre convention affects the users. Another relevant outcome of the experiment described in this chapter is the interaction between conventionality and task type. Although this interaction was detected earlier in Chapter 6 the results from the pilot and the main experiment described in this chapter confirm this interaction. This interaction strengthens the importance of considering not only the user in the design process but also the tasks which users will be carrying out in the system. Limitations Although results from the experiment described here indicate that participants may quickly develop a schema for a non-conforming website and suggest that this schema may be carried over to other similar websites, the time interval between the two parts of the experiment was too small. This time interval may not have been sufficient to establish whether participants will be able to maintain the performance improvement after a longer time without interacting with the non-conforming website. It might be that they have acquired short term knowledge of the site which might have helped to improve their performance. But it is possible that were they asked to use the site again after a longer period of time, performance would again be hindered. 239
- 262. Chapter 8 The effects of conventionality on users searching for information in an online news pages 8.1. Introduction A final experiment was put in place to evaluate if the results obtained for web forums are likely to be replicated in other web genres. Mainly, the experiment attempted to confirm the results obtained in the first part of experiment 5 (see Chapter 7) which indicated that participants in the non-conforming group showed a performance improvement throughout the experiment. This is important as it can indicate whether or not the results can be extended to other web genres, adding to the generality of the outcomes of this research. Websites belonging to the online news category were chosen for this experiment, because they have been consistently cited in the literature as a web genre (Åkesson, 2003; Eriksen & Ihlström, 2000; Ihlström & Åkesson, 2004; Vaughan & Dillon, 1998, 2006). To design the material for this experiment an analysis of existing newspapers was made. Seven mainstream newspaper websites (commonly known in Britain by the term ‘quality newspapers’) were analysed. All seven newspapers feature in the list of most visited United Kingdom newspaper web news sites created by the ABCe (Audit Bureau of Circulation – new media division) and five were among the top 10 most visited UK newspaper websites list created by Nielsen Online (available from www.nielsen-online.com/pr/pr_071011_UK.pdf - accessed on 17/04/2009). Even though the BBC News website does not appear on Nielsen’s nor on ABCe lists of newspaper websites, for not having a print version, it is one of the most accessed web news sites in the United Kingdom. For this reason it was decided that the BBC News website should be included in the analysis. A list of the websites analysed for this research is shown in Table 47. 240
- 263. Visual conventions and performance over time TABLE 47- ONLINE NEWS WEBSITES ANALYSED TO PREPARE THE EXPERIMENTAL MATERIAL Newspaper name Website URL Financial Times www.ft.com The Guardian www.guardian.co.uk The Independent www.independent.co.uk Daily Telegraph www.telegraph.co.uk The Times www.timesonline.co.uk BBC News www.bbc.co.uk/news The Herald www.theherald.co.uk The Scotsman www.scotsman.com 8.2. Analysis of existing news websites Following the methodology applied in Survey 1 (described in Chapter 4), these online news pages were examined for the most frequently occurring elements and the most frequent values for the three attributes (symbolization, position and configuration). For this analysis, wireframes of the main page, category page and story page of each of the 8 news sites were created. Then, based on the observation of the attributes of the most frequently recurring visual elements of these news websites, design decisions were made regarding the layout, and wireframes for the main, category and story page were designed for the conforming (Figure 103) and non-conforming (Figure 104) sites. 241
- 264. Visual conventions and performance over time FIGURE 103 - WIREFRAMES FOR THE CONFORMING ONLINE NEWS PAGES DESIGNED FOR THE EXPERIMENT FIGURE 104 - WIREFRAMES FOR THE NON-CONFORMING ONLINE NEWS PAGES DESIGNED FOR THE EXPERIMENT 242
- 265. Visual conventions and performance over time Once the wireframes were completed a preliminary survey was put in place to verify with online news users that the conforming layout was indeed perceived as more conventional. 8.3. Survey: Judgments of online news regarding the conventionality of the layouts 8.3.1. Objectives This survey was put in place to test: ! If the design of the conforming website was more in conformity with other online news sites which users are accustomed to visit when compared to the non- conforming one. ! Whether particular visual elements or visual configurations may lead participants to perceive one layout as ‘more conforming’ than another. ! Verify if the previously found correlation between perceived conventionality and ease of use is replicated for online news pages. 8.3.2. Method Material The wireframes produced in the previous stage were edited to include colour, pictures and adverts so that they would resemble a real website as much as possible. Every picture and advert image used in the conforming layout was also used in the non- conforming layout. The same colour scheme was applied to both layouts. As seen earlier (see figures 103 and 104), the text used in the templates was ‘greeked’ but the hierarchy of information of an online news page was indicated through different type sizes and italic types. The final conforming and non-conforming templates used in the survey are presented in Figures 105 and 106 respectively. 243
- 266. Visual conventions and performance over time FIGURE 105 - PAGES OF THE CONFORMING ONLINE NEWS LAYOUT FIGURE 106 PAGES OF THE NON-CONFORMING ONLINE NEWS LAYOUT 244
- 267. Visual conventions and performance over time Participants Twenty participants volunteered for this survey. All participants were Internet users and read news online at least three times a week. They were all fluent in English and used at least one of the seven mainstream online news websites or BBC News. Participants of this survey had not participated in any previous experiments in this thesis. Procedure Participants were approached and asked some questions which aimed at assessing their online news reading profile. To avoid giving away the true purpose of the questionnaire and risk biasing the answers, participants were told that the questionnaire aimed at investigating their Internet usage profile. Besides asking about their online news reading habits, the questionnaire also addressed their emailing, online banking and online shopping profile. If the participant read online news three times a week or more, a second set of questions were asked, if they did not they were thanked and the survey ended. The second set of questions in the survey aimed at obtaining judgment of online news users regarding the conventionality of the two layouts. To facilitate comparison, each page of the conforming layout was presented next to the equivalent page of the non-conforming layout and both were printed one on top of the other on a sheet of paper (Figure 107). In other words, the main pages of the conforming and non-conforming layouts were printed together on one sheet of paper, the category pages of both layouts were printed in another sheet and the same was done for the story pages of both layouts. The position in which each page appeared on the paper (top or bottom) was randomized. Participants were shown one sheet at a time and were asked to respond which of the two layouts they thought looked more like an online news page they have seen before and what did they see that led them to that conclusion. Then they were asked which of the two layouts they thought would be easier to use and why they thought that. Participants were allowed to give more than one reason for choosing a particular layout as more similar to other existing online news and for choosing a particular layout as easier to use. 245
- 268. Visual conventions and performance over time FIGURE 107 - SAMPLE OF A SHEET USED IN THE SURVEY CONTAINING THE TWO LAYOUTS Pages were presented to the participants in the same order in which participants would see them if they were navigating on a real website. Thus, they were shown the main pages first, then the category pages and finally the story pages. When given the sheet of paper with the printed pages, participants were informed at which level they would be in an online news website. For example, when shown the sheet with the main page, participants were informed that they should consider the layouts as the main page of an online news website. The same was done for the category page sheet and the story page sheet. 8.3.3. Results Conventionality All participants chose the conforming layout as the one more similar to other existing online news websites for all three pages (main, category and story pages). 246
- 269. Visual conventions and performance over time Overall, the visual elements and their attributes which seem to mostly influence participants’ perceptions of conventionality are first the configuration of the picture, the headline and the text on a single news piece and second the use of tabs to represent the menu (symbolization mode) and the location of the menu on the top of the page (see Figure 108). Interestingly enough, adverts (both their position and shape) seem to be more influential in participants’ perceptions of conventionality than the position of the logo on the screen, location of the ‘Other stories’ box and the use of more than one column of text on online news pages. FIGURE 108 - NUMBER OF TIMES EACH VISUAL ATTRIBUTE/ELEMENT WAS MENTIONED AS INFLUENCING PARTICIPANTS' DECISION ON THE SIMILARITY OF THE PAGES WITH EXISTING ONLINE NEWS Ease of use All participants thought the conforming main page and category page to be easier to use than the non-conforming alternatives. Despite the fact that 19 participants thought the conforming story page would be easier to use, one participant thought that the non-conforming story page would be easier to use than its conforming counterpart. When queried why, the participant explained that because the text was in one block, without any image or headlines interfering with it, it would probably be easier to read than the conforming one. Here again the configuration of the news pieces and the style and position of the menu were the most frequently chosen elements (Figure 109). 247
- 270. Visual conventions and performance over time FIGURE 109 - NUMBER OF TIMES EACH VISUAL ATTRIBUTE OR ELEMENT WAS MENTIONED AS INFLUENCING PARTICIPANTS’ JUDGEMENTS ON THE EASE OF USE OF THE PAGES Perceptions of Conventionality and perception of ease-of-use A Spearman-! test revealed a strong correlation between perceived conventionality and perceived ease of use (!=0.99). This result confirms that the correlation found in web forums earlier in chapter 5 also applies for online news pages. 8.3.4. Discussion Perceptions of conventionality The results show that online news users agreed that the conforming layout was very similar to existing online pages which they are accustomed to use. The fact that none of the participants considered any of the non-conforming pages more similar to existing online news sites indicates that the layout does seem to represent to some degree a non-conformance with the visual conventions of the online news genre. Most interesting was the fact that essentially (a) the configuration between the headline, summary and photo of a news piece, (b) the position of the navigational menu and the fact that it utilises tabs to indicate the news categories and (c) the position of the adverts on the page seem to have been used by participants to decide whether the website conformed to the conventions of the genre. Following these results, further analysis of these particular elements and their attributes was done. 248
- 271. Visual conventions and performance over time Configuration of the elements of news pieces The standard configuration of the components of a single piece of news is to have the headline on top of the summary text and the picture either to the left or to the right of these elements (Figure 110). FIGURE 110- COMMON CONFIGURATION OF THE COMPONENTS OF A PIECE OF NEWS A common variation to this configuration happens when the summary text is wrapped around the picture either to the left or to the right (Figure 111). Even though this type of configuration can be found across all three types of online news pages (main, category and story pages) it is more common in the story page due to the larger quantity of text present. 249
- 272. Visual conventions and performance over time FIGURE 111 - CONFIGURATION OF THE COMPONENTS OF A PIECE OF NEWS WITH THE SUMMARY TEXT WRAPPED AROUND THE PICTURE Yet another type of configuration common to articles in the story page of online news websites also contains text wrapped around the picture but, in this case the headline is located above both the text and the picture (Figure 112). FIGURE 112 - TYPE OF CONFIGURATION OF THE ELEMENTS OF AN ARTICLE IN THE STORY PAGE OF ONLINE NEWS PAGES 250
- 273. Visual conventions and performance over time These observations were used when generating the conforming online news website. Also based on these observations, the configurations used for the news elements in the non-conforming version were determined. The strategy was basically to visually dissociate the components of the news element by either placing them in a different configuration (Figure 113) or separating them (Figure 114) by introducing another element (in this case an advert) between the components of the news article. FIGURE 113 - CONFIGURATIONS OF NEWS PIECES USED IN THE NON-CONFORMING ONLINE NEWS PAGES FIGURE 114 - CONFIGURATION USED IN THE STORY PAGES OF THE NON-CONFORMING ONLINE NEWS WEBSITE WHERE ELEMENTS OF THE ARTICLE WERE DISSOCIATED BY PARTIALLY SEPARATING THEM. 251
- 274. Visual conventions and performance over time Style and placement of the navigational menu The survey results also indicated that the visual style and placement of the navigational menu play an important role in the decision of the participants regarding the level of conformance of the layouts presented with the online news pages they were familiar with. FIGURE 115 - STYLE AND PLACEMENT OF THE NAVIGATIONAL MENU CONSIDERED AS COMMONLY OCCURRING IN ONLINE NEWS SITES Again this result was used to inform the design of the conforming online news page and also to determine the placement and style of the non-conforming menu (Figure 115). In the non-conforming layout the menu was placed on the right side of the screen (Figure 116). Similarly to what was done previously for web forums, the symbolization mode of the menu items was changed from alphanumeric to pictorial by replacing the words with icons. Also the configuration of the menu items was altered from horizontal to vertical. 252
- 275. Visual conventions and performance over time FIGURE 116 - STYLE AND PLACEMENT OF THE NAVIGATIONAL MENU IN THE NON-CONFORMING ONLINE NEWS LAYOUT Adverts Interestingly enough, it seems that not only it is common for adverts to be present in online news pages but there are also certain positions where adverts can be placed which are expected by online news users. Generally speaking on online news pages, adverts seem to be located in two main areas of the page: the top and the right (Figure 117). These results partly agree with those found by (Bernard, 2001a, 2001b) that the most common position for adverts is the top of the page. However, on the second most common position of adverts, results diverge, while Bernard found it to be the centre bottom of the page, in the online news pages analysed, the second most common position for adverts was the right side of the page. The common positions found for the adverts are consistent through main, category and story pages, though the format of the adverts may vary depending on the page on which it is shown. Most adverts consist of long vertical or horizontal image banners either in flash or animated GIFs. However, samples of smaller and shorter banners are present along with some sponsored advertising presented in the form of text and links. 253
- 276. Visual conventions and performance over time FIGURE 117 - POSITIONING AND SHAPES OF ADVERTS COMMONLY OCCURRING ON ONLINE NEWS PAGES Perceptions of ease of use Generally speaking participants used the same visual elements to judge conventionality and ease of use. The most frequently cited elements were again the configuration of the elements of a piece of news, the placement and style of the navigational menu and the positioning and format of the adverts. However, the positioning of the search bar was almost as important to participants in judging ease of use as the positioning of the adverts. This indicates that participants were taking into account this element when judging ease of use which did not occur when they were judging conventionality. This suggests that participants were able to separate which elements they were using to guide their judgments of conventionality and which ones they used for ease of use. Since the search bar has an unique function in aiding search and is commonly used as part of a search strategy it is no wonder why participants used it as a relevant factor in their judgments of ease of use. 254
- 277. Visual conventions and performance over time 8.3.5. Conclusion This survey confirmed that the conforming layout did indeed conform to the expectations of what an online news page should look like. It also provided useful information on which visual elements and attributes mostly influence their judgments on the conventionality and ease of use of an online news page. Knowing the role that visual elements play on the perceptions of users is invaluable information to designers who can use it to estimate to a certain extent, the impact of their layouts on users. Finally, it also confirms the correlation between conventionality and perceived ease of use found in Chapter 5, which suggests that it may be extendable to other web genres. The implications of this correlation will be explained in more detail in chapter 9. 8.4. Experiment 6 – Effects of conventionality on searching for information on online news websites 8.4.1. Objectives This experiment aims to investigate whether the results obtained in experiment 5 (see chapter 7) can be replicated on another web genre. Experiment 5 investigated the effect of non-conformance to the visual conventions of the web forum genre on searching for information. Therefore this experiment will attempt to: ! See if participants using a non-conforming online news website will perform worse than the participants using a conforming online news website. ! Check if the performance of participants using the non-conforming online news page will improve over time. ! Assess the development of the feeling of disorientation of the participants through the experiment. ! Assess if learning occurred amongst participants using the non-conforming news page. ! Find whether participants using a non-conforming online news website will adopt similar strategies to search for information as participants in the conforming site. 255
- 278. Visual conventions and performance over time 8.4.2. Method Material Websites To design the websites, the layouts presented to participants in the preliminary survey were used as a template. To determine which categories will be used in the experimental websites, the eight mainstream UK online news websites were examined, their categories and subcategories were noted. The categories and subcategories which occurred in at least four of the websites were picked to be used in the experimental sites. These categories are listed in Table 48. TABLE 48 - LIST OF COMMONLY OCCURRING ONLINE NEWS CATEGORIES Category Subcategory News UK World Science and Technology Sport Football Cricket Motorsports Rugby Money Property Investments Saving Insurance Lifestyle Food & Drink Health The images and textual content for the conforming and non-conforming website were obtained from the eight mainstream UK online news websites during the course of a week (from 15th to the 21st of April 2009). Adverts were also obtained from existing online news pages and were all animated GIFs. Clicking on adverts was disabled to prevent participants leaving the test pages and going to other websites. The search engine was disabled to force participants to navigate the website using the menu and links available. A different colour scheme to the one shown in the survey samples was used in the final version of the material. This decision was made because the blue colour used for 256
- 279. Visual conventions and performance over time the survey layouts was quite dark and it was believed that a lighter colour would improve contrast and legibility. Non-conforming pages To represent the categories and subcategories graphically in the non-conforming online news page several icons were designed. An attempt was made to make the icons as clear and as representative as possible so that interpretation difficulties could be minimized. The icons chosen are illustrated on Figure 118. FIGURE 118 - ICONS DESIGNED FOR EACH CATEGORY AND SUBCATEGORY OF THE CONFORMING ONLINE NEWS SITE The menu was designed in an ‘accordion’ style vertically in a way that all categories were visible at all times but only the subcategories of the selected category were displayed as can be seen in Figure 119. In the conforming website, the menu also displayed all categories but only the subcategories of the selected category. 257
- 280. Visual conventions and performance over time FIGURE 119 - ACCORDION MENU DESIGNED FOR THE NON-CONFORMING ONLINE NEWS PAGES (A) WITH THE NEWS CATEGORY SELECTED AND (B) WITH THE MONEY CATEGORY SELECTED a. b. Thumbnails of some pages from the finished conforming and non-conforming websites are shown in figures 120 and 121. FIGURE 120 - SAMPLE PAGES OF THE CONFORMING NEWS WEBSITE 258
- 281. Visual conventions and performance over time FIGURE 121 - SAMPLE PAGES OF THE NON-CONFORMING ONLINE NEWS WEBSITE The same screen capture and time measuring techniques used in experiment 5 were also used in this experiment. The pop-up window with the finished button was also used for participants to either indicate they found the answer or for them to quit the current task and move onto the next one. This experimental setup ensured that at every mouse click that a participant made, an image of the screen was saved to a file. The image contained: the browser window, the mouse cursor (with an indication of where the click took place), the stopwatch and the pop-up window (see Figure 122). 259
- 282. Visual conventions and performance over time FIGURE 122 - SAMPLE OF THE EXPERIMENTAL SETTING WITH THE NON-CONFORMING WEBSITE Tasks design There seems to be three main ways in which users acquire information on an online news website: one is searching for information about a well defined target (e.g. finding out what was the score of the Arsenal vs. Manchester United football match); another is searching for information on a target which is not totally defined (e.g. finding out updates on the next general election); and finally the third method is the serendipitous discovery of a topic of interest (e.g. finding out more about a new recent scientific development in the field of genetics). These methods of acquiring information generate in the users different styles of browsing which have been described by Catledge and Pitkow (1995) and were mentioned in Chapter 2: search browsing, general purpose browsing and serendipitous browsing. Since this thesis is concerned with searching for information, for the purposes of this experiment tasks will consist of asking participants to search for information about a well-defined object. Six tasks were defined with the intention of making participants look for information on a well-defined target (Table 49). The answers to all tasks were located in the story pages and there were no direct links to the stories in the main page. This was done to encourage participants to use the navigational structure of the website. 260
- 283. Visual conventions and performance over time TABLE 49 - TASKS DESIGNED FOR THE EXPERIMENT Nº Task 1 How many properties in the UK will be repossessed in 2009? 2 What change is going to happen to Formula One next year? 3 Which country in the EU has the most comfortably sized women? 4 How much have BSkyB shares risen? 5 Which country will host the next Rugby world cup in 2011? 6 How many appearances has the football player Rafael da Silva had for the Manchester United team? In the sixth task participants were asked to verbally talk aloud what they were thinking and doing as they navigated through the websites. The order of tasks 1-5 was randomized per participant per group while task 6 was the final task for each participant in both groups. Although all tasks were designed to have participants searching for information on a specific object, data from the first five tasks were used to assess the first two objectives described in 8.4.1: (a) See if participants using a non-conforming online news website will perform worse than the participants using a conforming online news website and (b) Check if the performance of participants using the non- conforming online news page will improve over time. Data from the sixth task was used to assess the fifth objective described in 8.4.1: Find whether participants using a non-conforming online news website will adopt similar strategies to search for information as participants in the conforming site. Disorientation questionnaire Part of the disorientation questionnaire which was used in Experiment 5 (see Appendix R) was also applied in this experiment. Similarly to before, participants were asked to answer the questionnaire at two different times in the test: in the beginning (after the first task was completed) and at the end of the five initial tasks. The idea here was to check if perceptions of disorientation, ease of use and ease of learning of the participants change over the course of the experiment. To assess the fourth objective (‘Assess if learning occurred amongst participants using the non- conforming news page’) these three questions were monitored for changes in perceptions which also may indicate that participants have learned to use the site. 261
- 284. Visual conventions and performance over time Participants 42 online news users participated in this experiment. All participants confirmed reading online news at least three times a week and have not participated in the survey or in any previous experiments in this thesis. All participants were fluent in English and their academic background varied from completed GCSEs to PhD degree. Similarly to experiment 5 two independent groups of participants were used and participants were randomly allocated to one of the two experimental groups so that each group contained 21 online news users. 8.4.3. Procedure Participants were tested individually on one of the two websites to which they were randomly assigned. Participants were informed that the experiment consisted of looking for answers to questions in an online news website. They were asked to search for information as quickly and as accurately as possible, highlighting the piece of text which best answered the question asked, and then clicking on a ‘finished’ button, which was located in the pop-up window to the left of the browser. Participants were told that they should use the website as they normally would any other in a real situation and that they were allowed to quit on a task if they wanted (by clicking on the finished button). Finally they were informed that at two instances during the experiment they would be asked to answer a number of questions regarding their experience and feelings when using the website. When the participant was ready, the experimenter handed a sheet of paper with the first task to the participant (since task order was randomized, this varied between tasks 1-5). When the participant finished reading the task and looked at the computer screen, the experimenter started the stopwatch. When the participant found the answer or quit the first task, the experimenter paused the time measure. Then the participant was asked to answer the questions from the disorientation questionnaire. Once that was done, a paper with the next task was handed to the participant and once the participant stopped reading the task and looked at the screen, the experimenter resumed the measure of time. Once the participant had either found the answer to the task or given up searching for it the experimenter paused the 262
- 285. Visual conventions and performance over time stopwatch and moved on to the next task. This was repeated until tasks 1-5 had all been attempted. After the participant finished the fifth task, they were once more asked to answer the disorientation questionnaire. After that, the participant was told that for the next task they would still have to search for the answer to the task on the online news page but that this time they were required to think aloud as they searched for it. The participant was told that their verbal protocol was going to be recorded, but the data would remain anonymous. If the participant agreed to be recorded, the experimenter would start the recorder and give the participant the next task. If the participant did not agree to the recording, then the experimenter would take notes of their verbalizations. Time measurement was resumed after the participant read the task and looked at the screen. These procedures were repeated for all participants in the test. It is relevant to note that all participants agreed to be recorded therefore no notes needed to be taken. 8.4.4. Results The results described here, unless specified otherwise, refer to the first 5 tasks participants performed in the test. The results of the 6th task have not been included in the performance analysis since participants were asked to talk as they went through the process of completing the task. Time The group using the conforming website spent on average 90.18 seconds (S.D. = 65.59) on each task while the group which used the non-conforming website spent an average of 135.17 seconds (S.D. = 111.65) on each task. A one-way ANOVA on the data indicated that this difference is significant (F(1,40)=7.78, df=1, p=0.008). 263
- 286. Visual conventions and performance over time FIGURE 123 - MEAN TIME SPENT ON TASKS PER GROUP Analysing the changes in the time spent on tasks per group throughout the experiment it is possible to observe that participants who used the non-conforming news website did spend less time on each task as the test progressed. The group which used the conforming website does seem to show a slight reduction in time spent on tasks after the second task (see Figure 124). However, a Mann-Kendall trend analysis in the conforming group did not indicate any trend (Z=-4, n=5, p=0.208) while in the non-conforming group it indicated a significant down trend (Z=-8, n=5, p=0.042). FIGURE 124 - MEAN TIME SPENT ON TASKS PER GROUP THROUGHOUT THE EXPERIMENT 264
- 287. Visual conventions and performance over time Right Answers Of a total of 105 tasks, participants in the conforming group answered 84 tasks correctly (M=0.80, S.D.=0.23) compared to 71 (M=0.68, S.D.=0.29) correctly completed tasks in the non-conforming group (Figure 125). A one-way ANOVA showed this difference is not significant (F(1,40)=1.98, df=1, p=0.167). FIGURE 125 - MEAN RIGHT ANSWERS PER GROUP Over the course of the experiment, the group using the non-conforming website showed a gradual increase in the number of tasks answered right while the group using the conforming website did not display any variation. A Mann-Kendall trend analysis confirmed this by showing that the non-conforming group indeed had an upward trend in the number of right answers (Z=8, n=5, p=0.042) while the conforming group showed no definite trend (Z=-1, n=5, p=0.475). 265
- 288. Visual conventions and performance over time FIGURE 126 - MEAN NUMBER OF RIGHT ANSWERS PER GROUP THROUGHOUT THE EXPERIMENT RPS rate The number of right answers was combined with time into the RPS rate, which has been used previously in experiments 4 and 5 to evaluate performance. Results show that the group using the conforming site achieved a mean RPS of 1.50 (S.D.=1.38) while the non-conforming group achieved a mean RPS of 1.14 (S.D.=1.34). A one-way ANOVA on the data indicates that the difference is not significant (F(1,40)=2.34, df=1, p=0.134). Looking at how the RPS varied throughout the experiment for each group (Figure 127) it is possible to see that the non-conforming group seems to have had a significant performance improvement (measured by the RPS). In fact a Mann-Kendall trend analysis shows that the non-conforming group experienced a significant upward trend in their performance (Z=10, n=5, p<0.01). Even though the conforming group seems to have had a performance improvement from the 2nd to the 4th task, their performance declines from the 1st to the 2nd task and again from the 4th to the 5th task. A Mann-Kendall trend analysis showed no significant trend for this group (Z=4, n=5, p=0.21). 266
- 289. Visual conventions and performance over time FIGURE 127 - MEAN RPS PER GROUP THROUGHOUT THE EXPERIMENT Disorientation Backtracking Participants using the non-conforming website performed more backtracking actions (Mean=5.76, S.D.=6.87) than participants using the conforming website (Mean=1.67, S.D.=1.53). A one-way ANOVA on the data showed this difference to be significant (F(1,20)=7.17, df=1, p=0.011). Perceived disorientation As mentioned earlier, user perceived disorientation was assessed on two instances during the experiment: one after the 1st task and the other after the 5th task. The first question in the questionnaire (‘Complete the sentence as appropriate: ‘I felt lost/disorientated…’) measured the perceived disorientation of participants. Participants had to choose between five possible answers: 1) Never 2) Sometimes 3) A few times 4) Most of the time 5) Always The median value of question 1 for each group in each instance the test was applied is shown in Table 50. 267
- 290. Visual conventions and performance over time TABLE 50 - MEDIAN VALUES OF THE ANSWERS GIVEN IN TWO INSTANCES DURING THE EXPERIMENT TO THE DISORIENTATION QUESTION After 1st task After 5th task Conforming 2 (Sometimes) 2 (Sometimes) Non-conforming 4 (Most of the time) 3 (A few times) It is clear from looking at the table that while participants who used the conforming website showed no change in their feeling of disorientation, participants in the non- conforming group seem to have felt less disorientated towards the end of the test than when they started. However, the answers also indicate that even though participants in the non-conforming group felt less disorientated towards the end of the test, they still felt more lost than the participants in the conforming group. Perceptions on ease of navigation The third question in the questionnaire examined the perceptions of the participants regarding how easy or difficult to navigate they thought the website they used was. The question stated that ‘the site was easy to navigate’ and participants had to express their agreement or disagreement by choosing one of five options: 1) Strongly disagree 2) Disagree 3) Neither agree nor disagree 4) Agree 5) Strongly agree. Participants in the conforming group showed no variation throughout the experiment in their perceived ease of use of the website on both occasions agreeing with the statement (Median=4, Mode=4). Participants from the non-conforming group disagreed (Median=2, Mode=2) that ‘the site was easy to navigate’ but by the end of the 5th question they agreed (Median=4, Mode=4) with the statement. Navigation Strategy From the task which participants were asked to think aloud it was possible to have an idea of the search strategies that participants were using in each website. The majority of participants using the conforming website (57%) adopted the following strategy to answer the 6th task (Figure 128). 268
- 291. Visual conventions and performance over time FIGURE 128 - STRATEGY 1 - STRATEGY USED BY THE MAJORITY OF THE PARTICIPANTS IN THE CONFORMING AND NON-CONFORMING GROUPS TO ANSWER THE 6TH TASK. The second most adopted strategy by the participants in the conforming group (adopted by 21% of them) was a variant of the previous one (Figure 129) where the only difference was that instead of searching for either the name ‘Rafael’ or the word ‘Manchester United’ in the football category page, the searched only for the name ‘Rafael da Silva’ FIGURE 129 - VARIATION ON STRATEGY 1 THAT WAS THE SECOND MOST USED STRATEGY BY PARTICIPANTS ON BOTH THE CONFORMING AND NON-CONFORMING GROUPS. Coincidentally, the majority of participants in the non-conforming group adopted the same strategies as participants in the conforming group. Most interesting is the similarity in the proportions with 52% of the participants using strategy 1 (see Figure 128) and 21% of the participants using the variation where only the name ‘Rafael da Silva’ was searched (see Figure 129). Learning Perceptions of ease of learning The second question in the questionnaire addressed the perceptions of ease of learning the website. The question asked participants to judge how strongly they agreed or disagreed with the statement: ‘Learning to use the site was easy’. As in the previous question, the possible answers ranged from 1 – Strongly disagree to 5 – Strongly agree. Participants who used the conforming website showed no difference in their perception of ease of learning over the course of the experiment agreeing on both instances (Median=4, Mode=4) that the website ‘was easy to learn’. 269
- 292. Visual conventions and performance over time However, the group using the non-conforming page started the test disagreeing (Median=2, Mode=2) that ‘Learning to use the site was easy’ and ended the test neither agreeing nor disagreeing (Median=3, Mode=4) with the statement. This result shows an evident improvement in their perception of the ease of use of the website which means that by the end of the test they did not think the website was as hard to learn to use than when they started. Furthermore, the difference between the median and the mode indicates that although the middle value for the distribution was 3 (Median) the most frequent choice was 4 (Mode). In fact, examining the results, by the end of the test 10 participants in the non-conforming group agreed with the statement while 5 participants neither agreed nor disagreed and 5 participants disagreed. It is plausible to think that if participants in the non-conforming group spent a little more time using the website they would agree that it was easy to learn to use it. Thinking aloud As explained earlier, in the 6th task participants were requested to think aloud as they were searching for the answer to the question. Analysis of the recording of the participants revealed that some participants in the non-conforming forum indicated verbally having learned what the navigational menu was or the meaning of the icons in the navigational menu. They made evident in their comments that they have at some point during the experiment decoded the symbols on the menu and learned their meaning. Some of their comments are listed below: ! “Go to the symbol of sports on the menu, I know that is the menu now, click there, there are four symbols the 1st is the football category.” ! “That carpet [icon], what is that? I can’t figure this one out… Oh! There is an atom…so it must be science…still don’t get the carpet! ! “Football…I'm gonna look at the trophy thing because now I know that's sports…” ! “Since I have been here before, I can tell that this [icon] is sports. I didn't know that before…Anyway I can see it is sports because it has a logo I can identify.” Only one of the participants in the conforming group has indicated having to learn the navigational menu by commenting “Well, now I know where the sports section 270
- 293. Visual conventions and performance over time is…”. But the participant then explains what may be the origin of the problem by saying: “I haven't quite got used to the menu as it is quite small.” 8.4.5. Discussion Performance Even though participants in the conforming online news site spent less time on each task compared to the participants in the non-conforming site, no significant differences were found between groups in number of right answers or in their RPS ratio. This suggests that overall there was no significant difference in performance between these two groups. Examining what happened to their performance over time, it becomes evident that while the conforming group showed no trends of improvement in any of the three measures (time, right answers and RPS ratio), the group using the non-conforming pages showed a significant downtrend in time, an upward trend in the number of right answers and an upward trend in their RPS ratio. In other words, they spent less time on each task as the experiment progressed, got more correct answers and had an overall performance increase. Disorientation Backtracking Participants in the non-conforming group performed on average around 3 ! times more backtracking actions than participants in the conforming group. Considering that backtracking can be an indicator of how lost a website user is this can be seen as a predictor of how disorientated participants felt. Looking at the development of backtracking over time (Figure 130) it is clear that the group who used the conforming site had a small decrease in the number of backtracking actions executed over time, while the group using the non-conforming website increased the number of backtracking actions up to the third task and then showed a dramatic decrease in the last two tasks. 271
- 294. Visual conventions and performance over time FIGURE 130 - MEAN BACKTRACKING ACTIONS EXECUTED OVER TIME BY PARTICIPANTS IN THE CONFORMING AND NON-CONFORMING GROUPS A Mann-Kendall trend analysis revealed a significant downtrend in the conforming group (Z=-8, n=5, p=0.042) but no significant trend in the non-conforming group (Z=0, n=5, p=0.525). One explanation for the backtracking behaviour of the non- conforming group could be that until the third task participants in this group were lost and after that they strted learning the website. Questionnaire Examining the questionnaire, the answers given confirm the overall backtracking results. Participants who used the conforming online news sites did not feel extremely disorientated having mostly stated that only sometimes they felt lost/disorientated. Furthermore, their feelings of disorientation and ‘lostness’ were not reduced nor increased as they used the website. On the other hand, participants in the non-conforming group started the experiment feeling lost/disorientated ‘most of the time’ and towards the end of the experiment they were feeling lost/disorientated only ‘a few times’. Even though their disorientation feeling eased, they were still comparatively more disorientated than their counterparts in the conforming group. Perceived ease of navigation Here again, participants from the conforming group showed no variation in their perception of how easy to navigate the website was. However, participants in the non-conforming group started the experiment disagreeing that the website was easy 272
- 295. Visual conventions and performance over time to navigate and towards the end of the test they agreed that the site was easy to navigate. Considering that by the end of the test participants in the non-conforming group were still feeling lost/disorientated ‘a few times’ one can not help but wonder why also towards the end they agreed that the website was easy to navigate. One possible explanation for this is that because their website conformed to the structural and functional conventions of the online news genres participants quickly figured out how to navigate through its pages without much problem. By conforming to the structural and functional genre conventions, the website met the expectations of users on the two aspects which can be most influential in the process on navigating through a website. Therefore users could use the navigation strategies and mental processes from genre conforming websites in the non- conforming site without any difficulty. Based on this idea and the concept that some mental processes are triggered by the presence of genre-conforming visual elements (van Dijk & Kintsch, 1983) the following scenario could have happened. When participants started using the website, its non-conformity with the visual conventions of the genre did not give them any cues which signalled them to use the strategies and mental processes associated with the online news genre. But as participants started learning the website, they figured that the strategies and mental processes associated with the online news genre applied to that website as well. As they started applying those strategies and received positive confirmation navigating the difficulty was minimized or even disappeared. This idea is further reinforced by the fact that during the thinking aloud task participants of both groups used the same strategies to locate the answer to the task. The fact that participants in the conforming group were adopting it suggests that this strategy is commonly used by online news users when facing a convention- conforming online news page. The fact that participants in the non-conforming group also adopted allied to the fact that they were all heavy online news users could indicate that the same strategy they would use in a convention-conforming news page was transferred to a website which did not conform to the visual conventions of the genre. 273
- 296. Visual conventions and performance over time Learning Participants who used the conforming online news website agreed that the website was easy to learn. But, while perceptions of ease of learning in the conforming group remained unaltered throughout the experiment, in the non-conforming group they changed. In the beginning of the experiment, participants of this group disagreed that the website was easy to learn, but towards the end they neither agreed nor disagreed that the website was easy to learn. This supports the theory that after some time using the website participants realised that the strategies and mental processes associated with the online news genre were applicable to the non-conforming website. Thus learning to use the website became an easy task. Indications that learning occurred in the non-conforming group were also seen during the ‘think aloud’ phase where a considerable number of participants expressed either implicitly or explicitly having learned the meaning of the icons in the navigational menu. 8.4.6. Conclusion Overall, results from this experiment are in accordance with the results from the ones obtained in the first part of experiment 5 (see Chapter 7). Below a comparison of the results in both experiments is made. Performance In the first part of experiment 5, the non-conforming group performed worse than the conforming group while in experiment 6 the difference was not significant. More interesting, however is the confirmation of the performance improvement of participants using the non-conforming pages in experiment 6. As in the first part of experiment 5, in experiment 6, participants using the non-conforming page showed a significant upward trend in their performance as the test progressed. Also confirming the results in experiment 5, the conforming group showed no significant trend in their performance. Disorientation Similar to what occurred in experiment 5, in experiment 6, participants using the non-conforming news website felt less lost and disorientated after using the website for some time. As in experiment 5, in experiment 6 participants from the conforming 274
- 297. Visual conventions and performance over time group did not report any changes in their feelings of disorientation and ‘lostness’ throughout the test. Also in terms of the use of the back button, on both experiments 5 and 6 the results were the same: the group using the non-conforming website performed significantly more backtracking actions than the group using the conforming online news site. Examining the graph of number of backtracking actions over time (Figure 130) it seems that for the group using the non-conforming site, the third task represented a turning point in the number of times the back button was used. Considering the relation between number of backtracking actions and disorientation it is possible that after the third task participants started feeling less lost and may have been able to orient themselves better within the website. Finally, while in experiment 6 participants in the non-conforming group altered their perceptions of how easy it was to navigate the site over the course of the test, in experiment 5 their perceptions remained unaltered. This could be explained by the different instances where the questionnaire was applied between the two experiments. While in experiment 6, the questionnaire was applied after participants have finished the first task and after the 5th task in experiment 5 the questionnaire was applied after the first part (participants had already carried out 10 tasks) then after the second part (participants had carried out another 5 tasks). It is likely that, in experiment 5, by the time participants had to answer the questionnaire they had already figured out how to navigate the website and their answers reflected that. Because in experiment 6 the questions were asked straight after the first task had been completed the variations in their perceptions on ease of navigating the site were more evident. Learning As expected, in both experiments for participants who used the conforming website perceptions on how easy the website was to learn remained constant. In all cases they agreed that the website was easy to learn. With the participants in the non-conforming group perceptions of ease of learning were not the same between the two experiments. In Experiment 5, perceptions of ease of learning remained unchanged across the test. Participants neither agreed nor 275
- 298. Visual conventions and performance over time disagreed that the website was easy to learn. On the other hand, in experiment 6 participants initially disagreed that the website was easy to learn, but changed their perception towards the end of the test, neither agreeing nor disagreeing with the statement. It is likely that in experiment 5, the same explanation for the lack of difference in perceptions of ease of use of the non-conforming group applies to the lack of difference in their perceptions of ease of leaning. By the time that participants were asked to answer the questionnaire they had already learned to use the website and from then on their perceptions remained constant. However, the data obtained from the ‘think aloud’ phase of the experiment described in this chapter seems to indicate that learning has occurred since participants have learned certain features on the non-conforming layout throughout the experiment. Final remarks The results of the experiment described here have satisfactorily replicated the results found in earlier studies of the web forum genre. This strengthens the confidence that these results can be extended to other website genres. Results from this study on online news genre along with the previous results from web forum genres show that users of a web genre suffer an initial decrease in performance and an increase in disorientation when navigating through a website which violates the visual conventions of the genre in search of information. The results also indicate that performance improves quite rapidly and feelings of disorientation diminish after a very short time using the website. This study however, goes beyond that and shows that heavy users of a particular web genre, can adopt the same strategies they would use in a genre-conforming website in a website which violates the visual genre conventions. However, considering that initially these users were lost and were not completing tasks correctly, it seems that at some point during the process of using the website participants realised that the same strategies used in a convention-conforming site could be applied successfully. What may have caused this realisation as well as if that realization signalled the start of the performance improvement is yet to be investigated. 276
- 299. Chapter 9 Conclusion 9.1. Introduction This thesis investigated the relationship between some of the forces involved in the design process, specifically web document genres and usability. As a strategy for studying and categorizing genres, they were viewed as a set of conventions and these were divided into structural, linguistic, visual and functional conventions. Through a review of genre and usability literature it established that genres are directly linked to the usability principle of consistency. Since usability literature recommends maintaining consistency on websites to improve usability (Nielsen, 1999a; Shneiderman & Plaisant, 2005), conforming to the conventions of a web genre should also improve usability of a website belonging to that particular genre. Literature suggests that not conforming to conventions impairs usability (Nielsen, 1999a) and makes users lost, confused and disorientated (Cappel & Huang, 2006). Research, however, has so far provided conflicting empirical evidence on the effect on user performance of non-conformance with conventions, with results ranging from serious performance hindrance (Vaughan & Dillon, 2006) to no difference in performance (Kalbach & Bosenick, 2006). This thesis suggests that the usage of the term convention in usability literature is quite broad, specially when web genres are involved. It proposes that before any assumptions are made regarding conformance to conventions, it is first necessary to establish which type of convention is being conformed to: structural, linguistic, visual or functional. The thesis then sets out to investigate how not conforming with the visual conventions of a website genre affects the performance and orientation of users. 277
- 300. Conclusion A systematic approach was chosen to investigate which visual attributes would make up the visual conventions of the web forum genre. To do this, three attributes of the visual elements of a web forum were scrutinized: symbolization mode, screen position and mode of configuration. From the outcomes of this investigation, a convention-conforming template was produced and validated by web forum users. The approach proved successful in determining the visual attributes which make up a convention conforming web forum layout. The principle of surveying the three attributes of the visual elements of a web genre was then applied to determine what a convention-conforming layout of the online news genre should look like. Again, the approach was successful in finding a layout which conformed with what users thought a conventional online news page should look like. Finally, a series of experiments were put in place to test the impact of not conforming to the visual conventions of a web genre on its users. Results from the experiments showed that although user performance did indeed suffer in the non-conforming website and users were disoriented, performance soon improved and disorientation diminished. This suggests that not conforming to the visual conventions of a website genre may not be have such dire consequences as suggested in the literature. Considering the continuous growth of the number of websites in the World Wide Web and the fact that many of these websites have frequent changes in layout (e.g. the Amazon-UK website from 2000 to 2005 has made at least one significant layout change a year to its homepage and The Independent website from 1999 to 2006 has made substantial layout changes to their homepage at least once a year*), the results presented here serve as empirical evidence to help designers understand, and consequently better balance some of the forces involved in the design process such as genre conventions and usability. 9.2. Overview of the thesis This thesis began by establishing a framework to serve as the basis of the empirical investigation which was conducted later. To define the framework a literature review was made. Drawing from genre theory, pattern languages and usability, a link between these was established. It was found that further investigation on the effects of genre conventions was needed in order to improve the understanding of the impact of not conforming to conventions on the user of electronic documents. * Source: The Wayback machine Internet archive –www.archive.org/web/web.php 278
- 301. Conclusion Two exploratory studies were then conducted to investigate two aspects of genre conventions which are more relevant to designers: structural and visual conventions. Exploratory study 1 examined the effect of organizing content in an online text using conventional access structures (such as headings and titles) and spacing, on reading comprehension and structured recall. Results from this study indicated that the use of conventional structures (colour-coded headings and spacing) to configure and organise text helps participants to recall the content in an ordered manner. Exploratory study 2 examined the effect of visual appearance on the ability to identify the category of a website. This study showed that most participants managed to identify the category of a website based only on its layout. It also indicated that some visual elements seem to be associated with certain categories of websites. Even though this experiment concentrated on website categories rather than web genres (as some categories used in the study could belong to a single web genre) its results are in accordance with Toms (2001) who found that users of a document genre firstly attempt to identify the genre of a document by its visual appearance. This becomes extremely relevant if the model proposed by Vaughan and Dillon (2006) which suggests that when users of a certain genre of documents are in the presence of certain visual cues associated with that genre a mental models triggered which can help them navigate through the document and find information easily. Based on the results of these two exploratory studies, it was decided that further investigation on visual conventions of web genres was going to be pursued. To investigate the effects of visual conventions of a web genre on its users, two categories of web forums were chosen: for the initial studies web forums were picked and for the final study online news was used. These choices were supported by the characteristics of genres given in Swales (1990): a genre depends on communicative events to occur and on the existence of a discourse community. Because the main purpose of web forums is to facilitate communication between its members, it was considered as a strong candidate for an emergent web genre. Online news websites have on the other hand frequently been cited in literature as a web genre (Åkesson, 2003; Eriksen & Ihlström, 2000; Ihlström & Åkesson, 2004; Vaughan & Dillon, 2006) and so were thought appropriate for the purposes of this thesis. 279
- 302. Conclusion Survey 1 was then carried out to define which visual elements occurred more frequently on web forums, how they were presented and located on the page and also how their components were configured. Based on the results of this survey a set of the most commonly occuring elements was established. Then, a series of templates was designed ranging from the ‘most conventional’ layout (with the most frequently occurring symbolization modes, position and configuration) to the ‘least conventional’ layout (with the least or non occurring symbolization modes, positions and configurations of each element). To verify how the results from Survey 1 conform with the perceptions of web forum users and therefore choose which templates would be used to create the material for future experiments, three experiments were conducted. Experiment 1 assessed which templates were perceived as more conventional by web forum users. Experiment 2 assessed which templates were perceived as easy to use by web forum users. From the results of these two experiments a correlation between perceptions of conventionality and perceptions of ease of use was established. From the results it was also determined which templates would be used to design the future experimental material. Experiments 3 investigated aspects of the finished material. It examined how the users identified the graphic and interactive elements of the non-conforming interface, and assessed whether the conforming and non-conforming materials were perceived as such by web forum users. The results showed which graphic elements were more easily identified and which caused more problems identifying. Participants had no difficulty in identifying the interactive elements of the non-conforming material. Results also confirmed that the conforming material was perceived by web forum users as being more similar to existing web forums than the non-conforming material. These results were used to inform the design of the next experiments. Experiment 4 investigated the effect of not conforming to visual conventions on the performance of users who are familiar with web forums and others who are not familiar with web forums executing different types of task. Results showed that participants familiar with web forums performed better than participants unfamiliar with the genre on both conforming and non-conforming layouts. Thus, no effect of conventionality on performance was detected for either group. An interaction between conventionality and task type was detected and it indicated that there was a greater difference in performance between content and interface tasks for 280
- 303. Conclusion participants in the non-conforming group than for participants in the conforming group. Experiment 5 further investigated the lack of difference in performance of participants familiar with web forums on both the conforming and non-conforming layouts. Results showed that participants who used the non-conforming web forum improved their performance over the course of the experiment. Results also revealed that this performance improvement was carried-over to other websites which followed the same non-conforming principles from the website used earlier. This may be an indication that when faced with a website which does not fully meet the expected visual genre conventions, users quickly create a mental schema for the new website or adapt their existing schema to fit the new visual conventions. Results also confirmed the interaction between conventionality and task type and indicated that perhaps the style of interaction may change as participants become more familiar with a non-conforming web forum. Finally, Experiment 6 investigated if the results obtained in Experiment 5 could be extended to another web genre. In this study online news genre was investigated. The experiment confirmed that the participants using the non-conforming website improved their performance over the course of the experiment. Similar to Experiment 5, in this study participants also felt less disorientated over the course of the test. This experiment also indicated that over a period of time participants in the non- conforming group learned aspects of the interface. This can explain the gradual increase in user performance observed on both studies. Another interesting outcome from Experiment 6 is the fact that by the end of the test both groups of participants were using the same strategy to search for information. This suggests that users navigating through the non-conforming website were using the strategies they apply for information search in a website which conforms with the genre. The fact that results from online news tests were similar to those of web forum tests (i.e. the same performance improvement observed on users of the non-conforming web forum was detected on users of the non-conforming online news website) strengthens the generality of the findings and strongly suggests that these results are extendable to other web genres. This means that breaking away from visual genre conventions on the Web does not cause such catastrophic usability problems as initially thought. 281
- 304. Conclusion 9.3. Contributions 9.3.1. Design As mentioned in the introduction, only in the past decades have genre theories started to consider the visual aspect of document genres as an important factor to be studied and very few have gone deeply into it (Delin, Bateman & Allen, 2002; Bateman & Delin, 2001; Toms, 2001; Waller, 1987a). This thesis contributes to the knowledge of document genres by dividing the conventions that make up a document genre into structural, visual, linguistic and functional conventions, thus allowing more focused investigations to take place. Furthermore, it focuses on the visual conventions, bringing the contribution of the visual appearance of document genres to centre stage. Conventionality and perceived ease of use By focusing on the visual conventions of genres of documents, results from this thesis reinforce the influence of the visual appearance of websites on different types of user perceptions as has already been suggested by some authors (Kurosu & Kashimura, 1995; Michailidou, et al., 2008; Tractinsky, 1997; Tractinsky, et al., 2000). Results from studies conducted for this thesis have shown a strong correlation between the perceived conventionality of a webpage and how easy to use people perceive them to be. For two different web genres (web forums and online news) users perceived pages conforming to the conventions of the genre as easier to use. This result relates to a particular finding from Kurosu and Kashimura (1995) that the familiarity of participants with the interface layout was highly correlated with perceived usability. Participants in their experiment associated the familiar configuration of an ATM keypad with high apparent usability. The same correlation was detected by Tractinsky (1997). Despite the strong correlation, both authors concentrated their analyses on the relation between perceived aesthetics (attractiveness of the layouts) and perceived usability. By doing so, the authors failed to acknowledge that several studies have related familiarity with perceptions of attractiveness in people (Langlois & Roggman, 1990; Langlois, Roggman, & Musselman, 1994; Monin, 2003) and in objects (Halberstadt & Rhodes, 2000). Recent results from a study conducted by Michailidou et al. (2008) found a correlation between familiarity and aesthetics (familiar web pages are perceived as being more aesthetically pleasing than unfamiliar ones). It is then, plausible to consider that perhaps familiarity was one important underlying factor which guided the choice of participants on both Kurosu and Kashimura (1995) and Tractinsky et al. (1997). Since 282
- 305. Conclusion perceptions of conformance to the conventions of a genre require familiarity with documents of that genre, the correlation between perceived conventionality and perceived ease of use support the results described above (Kurosu & Kashimura, 1995; Tractinsky, 2000). Thus, considering that perceptions of ease of use can influence user attitudes towards an interface (Morris & Dillon, 1997), and determine whether or not they will use a website (Hasan & Ahmed, 2007), this perceptual relation between conventionality and ease of use is an important one to be considered in the negotiation process which happens during the design process. When designing a website which does not conform to the visual conventions of a particular genre, designers must be aware that users familiar with the visual conventions of that particular genre will at first glance perceive the website as not so easy to use as another one that conforms with the visual conventions of the genre. This can influence their decision on whether or not to use the website. However, results from Experiment 6 have shown that user perceptions of ease of use of the non-conforming website did change very quickly over time with participants thinking the website was later easier to use than they had initially thought. The change in perceptions of ease of use may have been quick because in this experiment only the visual conventions of the web genre has been altered and it is possible that non-conformance with other types of web genre conventions may have a different impact on user perceptions of ease of use. Nevertheless it is important that designers keep in mind that perceived difficulty of use induced by non-conformance of a website with visual conventions of a genre can be quickly reduced as users interact with the non-conforming website. Assessing visual conventions of a genre The general method used for designing genre-conforming website pages was successfully tried in two distinct web genres (web forums and online news) and can be easily applied to any design process where the need to identify what the visual conventions of a particular document genre are. The process can be separated into four simple stages: 1) Representative samples of documents belonging to the genre are obtained. 2) Visual elements are identified in the samples and the most frequently occurring ones are noted. 3) The attributes for each frequent visual element on each sample are noted. 283
- 306. Conclusion 4) Based on the most common values for the most frequently occurring elements a layout is designed. Even though this process of analysis is a common one amongst designers and is normally a part of the design process the work presented here contributes to the methodology through the systematic approach of dividing the visual characteristics of an element into three attributes (symbolization mode, position and configuration) and examining them for each frequently occurring visual element. Another contribution, particular to web design, is the expansion of the grid system used by Ihlström and Åkesson (2004) for the navigational menu, to include all visual elements of the page, including the ones located outside the browser area and not visible without scrolling. Furthermore, considering that it is important for a designer to investigate a genre before starting a design task (Pemberton, 2000), this method can help to find the visual conventions of a particular website category or genre and consequently help with identifying and classifying existing design solutions for a particular genre so that it can be used to help define a corpus of design patterns, and subsequently a pattern language. Visual attributes and perceptions of conventionality and ease of use Even though literature suggests that website elements be placed in conventional positions where users expect them to be (Bernard, 2001a, 2002; Markum & Hall, 2003) results from Experiments 1 and 2 (described in chapter 5) show that in fact the symbolization mode of the visual elements on a website is a more influent factor on user perceptions of conventionality and ease of use than the positioning of these elements. As such, designers should consider that altering the location of an element from a conventional, expected place to an unexpected position on the screen might not affect the perceptions of conventionality and ease of use as negatively as changing perhaps a textual menu item to an icon will. It is important to add that in some web genres, the configuration of the components of an element may also play an important role in influencing perceptions of conventionality. For example, in the online news genre, even though the symbolization mode of the menu was quite important in helping participants to judge the conventionality of the layout, the configuration of the components of a 284
- 307. Conclusion news piece (headline, summary and picture) also played an important role in guiding their judgments. This suggests that in other genres different attributes may play a role that is just as important or even more important than the symbolization mode of the elements on user perceptions of conventionality and ease of use. Further investigation on this matter can allow for better understanding on the importance of each element attribute in user perceptions of conventionality and ease of use. Task type The interaction between conventionality and task type is relevant as it indicates that the type of task users are executing may influence the effects of conventionality on their performance. Experiments in this thesis examined two types of tasks: retrieving information from textual content in a website (called content tasks for simplicity) and retrieving information from the website interface (called interface tasks). Content tasks are one of the most common which web users consciously perform online. Interface tasks on the other hand, are not often consciously performed but they are an extremely common activity when navigating a website. During the navigation process users are constantly scanning the interface for information which may aid their search (e.g. an indicator that a new post has been placed in a thread) or give feedback on executed actions (e.g. a change on the breadcrumbs of a website to indicate that the user has moved from one page to another). When designing a website therefore, designers and usability engineers must consider the types of task which users will have to execute particularly if the website will not conform to existing visual genre conventions. A clear example is illustrated by the results from Experiment 5 which show that users perform worse obtaining information from non-genre conforming interfaces than from genre conforming ones. This is expected as, if visual conventions are altered, the interface elements may have different screen locations, symbolization modes and configuration that can make it difficult for users to obtain information from the interface. This can in turn hinder user performance and make it difficult to navigate the website. 9.3.2. Usability Genres and usability This thesis contributes towards an understanding of the relationship between genres and usability of web interfaces. The analysis of literature described in Chapter 2 linked genres of web pages and usability through the usability principle of external 285
- 308. Conclusion consistencywhich is achieved when an interface is consistent with other interface designs familiar to a user (Grudin, 1989). Therefore, genre conforming websites are intrinsically externally consistent since they share a set of structural, linguistic, visual and functional conventions. From this association several authors recommend that conventions should always be followed (Bevan, 1997; Nielsen, 1999a; Shneiderman & Plaisant, 2005; UPA, 2005) even when ‘a different design would be better if seen in isolation’ (Nielsen, 1999b). However, as noted by van Schaik and Ling (2009) some aspects of usability are not addressed by designs which conform to the conventions of a genre and therefore this conformity does not guarantees that the design will be usable. From this stems the idea that a website which does not conform with the conventions of a genre may not necessarily be less usable than one which does (Constantine & Lockwood, 2002). It is true that when this happens, the usability principles of consistency are violated, but some authors have pointed out that this is not necessarily a bad thing and in some instances it may even be beneficial (Grudin, 1989; Satzinger, 1991). Structural, linguistic, visual and functional conventions and usability By separating genre conventions into four components: structural, linguistic, visual and functional, this thesis proposes that each of these components should be investigated separately and whenever referring to violation of genre conventions, literature should specify which of the components is being violated. This stems from the idea that non-conformance with each type of convention may impact usability differently. This principle, allied with the outcomes from this thesis can explain the conflicting results found between Vaughan and Dillon (2006) and the studies of Kalbach and Bosenick (2006) and McCarthy et al. (2003). It is possible that because Vaughan and Dillon (2006) violated both structural and visual genre conventions participants took longer to learn to use the non-conforming website while both in Kalbach and Bosenick and in McCarthy et al. (2003) learning happened much faster since only visual conventions were violated (and in both cases only the positioning of the menu). Results from Chapters 6, 7 and 8 have confirmed that non-conformance with the visual conventions of a genre does not seem to cause performance and disorientation 286
- 309. Conclusion problems for very long as participants quickly learn to use the non-conforming website and performance improves. This is illustrated by the lack of performance differences between the users of conforming and non-conforming layouts in the results of the experiment described in chapter 6 and the performance improvement and decreased feeling of disorientation in users of the non-conforming layout in the experiments in chapter 7 and 8. This contributes to the debate on whether external consistency and conformance to the visual conventions of a genre should be maintained at the expense of innovation and even usability (Constantine & Lockwood, 2002). Based on the results presented here, interface designers and systems engineers can make more informed decisions on what is to be gained and what is to be lost on usability by following or not conforming to the visual conventions of a genre which are determinants of external consistency. Task type, conventions and performance Furthermore, user performance seems to vary depending not only on the level of conformity of a website with the visual conventions of its genre but also on the type of task participants are performing. As seen earlier, results from this thesis showed an interaction between the task type and conventionality of the website participants were using. Most interactions indicated that there was a greater difference in performance between content and interface tasks in the non-conforming group than there was for participants using the conforming websites. This indicates the added importance of considering the role that each visual element on the website may have on content retrieval tasks and also on general navigation before deciding whether to make it conform or not with the visual conventions of the genre. Navigation strategy Experiment 6 showed that participants using the non-conforming website applied the same strategy when searching for information as participants using the conforming site. Considering that participants in the conforming group were heavy online news users it follows that this strategy is commonly used in websites which conform with the online news genre. However, the fact that this strategy was also adopted by participants using the non-conforming website suggests that participants thought that the same strategy would yield positive results in that scenario. This reinforces the idea that users of a particular genre have schemas of that genre stored in their memory which, upon encountering certain cues in a document, are 287
- 310. Conclusion triggered and can help usage. It seems that participants using the non-conforming site at some point realised that either the same, or part of the mental schema for online news websites could be successfully applied to the non-conforming website. Perhaps the fact that only visual conventions were altered, while structural, linguistic and functional conventions were maintained allowed for this to happen. This may not happen in non-conformance with other types of convention. Contextual usability Finally, this thesis reinforces the importance of a contextual approach to studies of usability as suggested by some authors (Bevan, et al., 1991; Bevan & MacLeod, 1994; Henninger, 2000; Maguire, 2001; van Schaik & Ling, 2009). A contextual approach to usability considers that the characteristics of the users, tasks and environment are as important as the characteristics of the product in order to determine the usability of a system. Maguire (2001) expands the concept of ‘characteristics of the users’ to include not only the main user of the system, but also the goals of the wider user community. It becomes clear that the understanding of a digital genre and how it affects users from its particular community is relevant if a contextual approach to usability is to be pursued. It follows that, in order to evaluate the usability of a website under a contextual approach, the non-conformance of a digital document with each type of genre convention has to be weighed against the usability since they may impact the performance of users from the genre’s discourse community differently. This thesis takes a step towards improving this understanding since it has shown that users can adapt reasonably quickly to non compliance to the visual conventions of a digital genre. 9.3.3. Final remarks on contributions This work has shown that the non-conformance with the visual conventions of a web genre is not as bad as suggested in usability literature. Therefore, designers should not be inhibited to innovate by applying design solutions that may go against established visual genre conventions. It is important, however, that before decisions of breaking away from visual conventions are made, the impact they will have on user perceptions and attitudes towards the website as well their performance be carefully considered against the improvements that the changes will bring to the design. 288
- 311. Conclusion Conventions are an important part of the design process, but they should never limit or block innovation or usability. People adapt quickly to layout changes, even the most extreme ones (the material used in this thesis is a proof of that). What needs to be considered by designers is what cost will the non-conformance to the visual conventions of the genre have to the user and what benefits it will bring to them after they have adapted to the new design. 9.4. Limitations and suggestions for future research However, this thesis only investigated the search for information when a specific target was defined (search browsing). It may be worth pursuing in other experiments the impact of not conforming to visual genre conventions when users are performing other styles of browsing such as serendipitous browsing and general purpose browsing (Catledge & Pitkow, 1995; Salomon, 1989). It is possible that the impact of not following visual genre conventions on user performance may vary according with the browsing strategy that users are applying. It is possible for example, that serendipitous browsing (a common strategy used by online news readers) may suffer by the configuration changes applied to the news elements in the non-conforming news website. Because serendipitous browsing relies on chance discovery of a topic of interest while browsing the website, the change in the configuration of the news piece elements (by altering the relative position of headline, summary and picture) may cause confusion since users may not identify these components as a single news piece and therefore hinder the location of topics of interest. An interesting result from Experiment 4 was the lack of performance difference between the two groups of unfamiliar users: one using a conforming website and the other using a non-conforming website. A further experiment could investigate if this may be due to both interfaces taking the same time to be learned. Experiment 6 gave some indications that online news users were adopting the same strategies they use on conforming websites on the website which did not conform to the visual conventions of the genre. Two interesting points of further investigation are: 1) to evaluate the development of the strategies used by the non-conforming group over time and 2) see if this transference of strategy happens when other genre conventions are altered. Evaluating how the strategies used by the group navigating a non-conforming website evolve over time can clarify among other things: (a) whether there is a point 289
- 312. Conclusion of realisation that the same strategies used in a genre conforming site apply or (b) whether participants apply the same strategy from the beginning and stick to it until by trial-and-error they learn the interface and the strategy pays off. Investigating whether the transference of strategy happens when the website does not conform with other types of conventions can help predict if the impact of not conforming to certain types of genre conventions will be stronger than others. It is possible that not conforming to structural conventions will demand a greater effort from participants than not conforming to visual conventions, since it may require deeper processing to adapt the existing schema or even require the generation of a new one. This could again help explain the great impact on user performance that the study conducted by Vaughan and Dillon (2006) had. Since in their study both visual and structural conventions have been altered, it is possible that participant had to build a whole new mental schema for the non-conforming website. As seen in Chapter 7, the two parts of Experiment 5 were quite close together in time. This interval may not have been sufficient to establish whether participants would be able to maintain the performance improvement after a longer period of time without interacting with the non-conforming website. Therefore a longitudinal study with a larger time interval between trials is necessary to determine if participants can maintain the performance improvement using the non-conforming interface. If the performance improvement is maintained over a large time interval without practice it could indicate that participants not only have adapted to the non-conforming layout, but that they also have learned it and possibly either generated a new mental schema for this new design or adapted their existing genre schema. On the other hand, if after a large time interval without using the website user performance is again low it would be likely that the adaptation is a temporary thing and that real learning, and perhaps the development or adaptation of a mental schema, may only occur after repeated exposure. Finally, this study concentrated on the visual conventions of a web genre and although some aspects of structural and linguistic conventions of genres have been addressed by Vaughan and Dillon (2006) and Dillon and Schaap (1996), further investigation is needed to determine how the non-conformance with other types of genre conventions (structural, linguistic and functional) can affect the user of electronic documents and more specifically web pages. 290
- 313. Conclusion It is also important to determine the relationship between structural, linguistic, visual and functional conventions of a genre including how much each type of genre convention contributes to perceptions of conventionality and ease of use. This knowledge becomes particularly important in case non-conformance to one type of genre convention makes users perceive a web page as harder to use than non- conformance to the other types of genre conventions. Since, as seen earlier in this chapter, perceptions of ease of use influence user attitudes towards an interface (Morris & Dillon, 1997), and determine whether or not users will use a website (Hasan & Ahmed, 2007) this knowledge can be crucial for the success or failure of a web site. When results from Vaughan and Dillon (2006) are examined under the light of this thesis they seem to indicate that non-conformance with some types of genre conventions may have a greater impact on user performance than others. Therefore, more research is required to identify how much the non-conformance to each type of genre convention influences the performance of users. 291
- 314. Appendix A Texts used in the first Exploratory study 1 which examined conventional access structures How Wine is Made The difference between grape-juice, simple and sweet, and wine with all its qualities of keeping, of maturing and finally of inspiring, is the process of fermentation. In fermentation sugar is transformed into alcohol: grape-juice becomes wine. Fermentation comes naturally to grapes. Wine did not have to wait to be invented. A grape left alone could make wine if its skin were broken. Everything needed to make wine is there already when the grape has ripened on the vine. In the final stages of ripening, yeast cells appear from somewhere - it is not clear where - on the skin of the grape and cling to it. As soon as its skin is broken they pounce on the sugar in its juice, which is their object, and turn it into alcohol. Only a grape has enough sugar for this to happen naturally. So-called wines are made from other fruit, but all need extra sugar. Something like 30 per cent of the juice of a ripe grape is sugar, and this the yeast cells set about converting. Pressing The way the grapes are broken and the container they ferment in are two of the most variable things about wine. On small farms in Chianti all the grapes, black and white, are bundled together in a large opening at the top of a barrel. They still have their stalks, even the mud from the fields on them. They have only been slightly broken up, in the big tub on an ox-cart into which they were gathered, by the farmer with a knotty club. He plunges it up and down a few times to see that a fair proportion of the grapes have been smashed, relying on the heat generated in the barrel when fermentation starts to swell and burst the rest. On a few farms in France, Portugal, Spain and some parts of Italy the grapes are still trodden by bare feet, as they have been since wine was first made. Larger wine-growers now all have presses or mechanical crushers to break the grapes, but even these vary from old hand-operated squeezers to huge revolving horizontal drums. The latest kind have a gigantic rubber balloon inside. When it is inflated it presses the grapes against the slatted sides and forces out their juice without crushing their pips, which contain unwanted and evil-tasting substances. The biggest presses of all are those of Champagne, in which an enormous weight descends by scarcely perceptible degrees on an enclosure containing four tons of grapes at a time. Usually the juice and broken grapes, or for white wine the juice alone, are pumped for fermentation into vats or barrels, traditionally of oak but now more and more of concrete, glass or stainless steel. Sometimes it is left to ferment where it has been crushed or trodden. Fermentation The process of fermentation, which is what happens when the yeast cells get to work, produces turbulent eruptions in the juice. Not only alcohol is made, but carbon dioxide gas is given off in large quantities. Its bubbles keep the mass of broken grapes moving. At 292
- 315. Appendix A the same time the heat given off by the reaction makes it warm. Fermentation goes on until the sugar supply runs out, or until the yeast cells are asphyxiated by the growing concentration of alcohol, whichever happens sooner. Usually it is the sugar which gives out first. The liquid, then, instead of being a solution of sugar and water, is a solution of alcohol in water, with the same small quantities of the acids and oils which give it its peculiar flavour and scent. On average there will be about 10 per cent of alcohol in red wine, a little more-11 or so-in white. There is no sugar left at all and the resulting wine is completely dry. If, on the other hand, the grapes were so ripe that there was still more sugar to ferment when a concentration of 15 per cent alcohol was reached, it would be the yeast cells which would give out first. In a solution of alcohol this strong they grow drowsy and cease to function. They are not dead, but they are completely under the table. When this happens there is still some sugar left unconverted into alcohol and the wine is more or less sweet to the taste. This process is the basic one for the making of all wine and, come to that, of any other alcoholic drink. What is "Great" Wine? Anyone who has high claims for his wine, then, tends to be as specific as possible about it. He uses whatever cover the law of his country allows him-the most specific appellation he is entitled to. He often goes further and bottles it himself, adding his personal guarantee. If these precautions seem over-elaborate, consider what danger there is in more generalized naming. It is the same with food. The statement "Yorkshire pudding is delicious" is of very limited value. The important thing is who cooks it. Ordinary The daily drink of real wine-drinking countries-not those for whom wine is a luxury-is ordinary wine. In France vin ordinairehas a definite connotation. It is not a vague term for anything which is not very exciting. Its price depends on its strength-nothing else. It is almost always red. In Germany it is called Konsumwein, in Spain vino corriente ("current", or running, wine), in Portugal consume. We have no real term of our own for it in English. Ordinaire is the word most commonly used. In America, particularly, "ordinary" sounds more or less insulting. Vin-de-pays Vin de pays-wine of the country-is a cut above ordinaire. Though the wine in itself may be no better it has one added dignity; you know where it comes from. Ordinaire is completely anonymous. It is often a blend of the wine of France, Spain, North Africa, Italy or anywhere where the price per degree of alcohol is low at the time. Vin de pays at its worst is simply ordinaire with a birth-certificate. At its best, on the other hand, it can be one of those excellent regional specialities that are always referred to in a patronizing way as "little". A good hotel almost anywhere in the southern half of France should have quite a good vin du pays (switched from de to du because it is referring to the particular bit of pays in question). The patron gets it by knowing the grower. There is not enough to become a widespread marketable proposition, but it is none the worse for that. Good With "good" wine we are on trickier ground. What does a dealer mean when he says that a picture is good? What do you mean when you say that a dish is good? They are not quite the same thing. "Good" must be allowed to cover all wine from just plain well-made upwards to the top. But at the same time it has a field of its own, between ordinary and fine. Whereas ordinary is not worth tasting with any attention, has no scent or characteristics of its own, good wine begins to be worth thinking about. It is well made and has its own character, its own variation on the general character of its region. In this class come most of the wines of Bordeaux, for example, which are not Classed Growths and, come to that, a few that are; the wines of Burgundy which are known by their village-names alone, not the names of their individual vineyards; the blended German 293
- 316. Appendix A wines such as Liebfraumilch; the Chiantis and Orvietos of Italy - all the wines which carry the name of their type, when that name is one of the world's great wine-regions. Fine There is no general agreement on what is meant by the word "fine", often though it comes into discussions about wine. I would narrow it down to the field above Good and below Great. All wines which tell their full story-their vintage year, the particular vineyard and not just the village where they were grown, often the name of the owner-on their labels should come into this class. Most-though not all-estate-bottled wines deserve to be called fine wines. In France the annual production of fine wines (in the broadest sense) in a typical year is in the region of one seventh of the total (178 million Imperial gallons in a total of 1,170 million in 1971). Great Great wine is a different matter. There is no vineyard in the world which always produces great wine, however great and famous its name. Great wine is the rare and exquisite result of perfect conditions in a perfect vineyard, perfectly handled by the grower and carefully matured afterwards. Where the line comes between fine and great is always a matter for debate. But the word should not be used lightly. A great wine is a work of art, capable of providing aesthetic pleasure of the highest order to anyone who will be attentive. As an everyday drink it is as fitting as Hamlet is for cabaret in a night- club. Only five or six districts in the whole world have shown themselves to be capable of producing wine like this with any regularity. They are Bordeaux, Burgundy, Champagne, the Rhine and Moselle, the Douro river in Portugal and Jerez in Spain. At various times great wine has been made in Hungary, Italy, South Africa, California, Australia, central Portugal and maybe elsewhere. But it remains rare, and grows more and more expensive as more and more people want to taste it. Even a convinced ordinary-wine drinker should taste great wine sometimes. It is easy to get into a rut with wine and to wonder what all the fuss is about when anyone mentions the glory of the best growths. I hope nobody drinks them every day, because it is partly by comparison that their superlative qualities stand out. It is hard to put a finger on the quality of greatness in wine. To me there is always something faintly sweet, but sweet in the sense of sweet-natured rather than sugary, about one. There is a definite tendency to reach a quintessential taste of fresh grapes, as even really fine cognac does although it has been distilled. But one thing they all have in common, whether they are clarets, delicate fino sherries, champagne or hock or burgundy: they provoke discussion. To drink a great wine alone is almost painful. There must be somebody to share the experience with, for it is an experience, and it needs discussing, analysing and gloating over just as a great play does; or, to put it more precisely, a great picture, for it is not dramatic-it is beautiful. 294
- 317. Appendix B Summary task and multiple-choice questions and internet usage questionnaire used in the first exploratory study Part I – Summarization task Summarize the text you have just read. You have 5 minutes to complete this task Part II - Questions 1) Tick the option containing countries where you still find the traditional method of treading grapes by bare feet being used. (You should tick only one option) ( ) Italy, Spain,Portugal, France. ( ) United Kingdom, Australia, South Africa. ( ) United States, Greece, Argentina, Chile. ( ) Greece, Australia, Brazil, Argentina. 2) Tick the option which shows right order of processes necessary to make wine. ( ) Harvesting > Juice extraction > Alcohol production. ( ) Juice extraction > Harvesting > Alcohol production. ( ) Alcohol production > Harvesting > Juice extraction. ( ) Juice extraction > Alcohol production > Harvesting. ( ) Harvesting > Alcohol production > Juice extraction. 3) Number the sentences according to the order they happen during the winemaking process. ( ) The juice and broken grapes are pumped into vats or barrels. ( ) In fermentation sugar is transformed into alcohol: grape-juice becomes wine. ( ) The grapes are slightly broken up by the farmer with a club. ( ) Fermentation goes on until the sugar supply runs out. ( ) In the final stages of ripening, yeast cells appear on the skin of the grape and cling to it. 4) Why is it so important not to crush the grapes’ pips during the process of pressing them? (You should tick only one option) ( ) The carbon dioxide bubbles keep them moving impairing fermentation. ( ) They will generate heat when fermentation starts bursting the barrel. ( ) They contain unwanted evil-tasting substances. ( ) They give off acid oils that might ruin the wine. ( ) They will leave some sugar unconverted into alcohol and the wine will be sweet to the taste. 5) Tick the option in which all the names refer to ordinary wine. (You should tick only one option) ( ) Konsumwein, Consumo, Vino Corriente ( ) Consumo, Popular, Konsumwein ( ) Common, daily, Ordinaire ( ) Popular, daily, Konsumwein 295
- 318. Appendix B 6) Assign (T) for true or (F) for false for each of the sentences below: ( ) Good wine is usually an “ordinaire” wine with a birth certificate. ( ) Usually the yeast cells can work up to a 15% concentration of alcohol ( ) The world’s largest wine presses are located on small farms in Chianti. ( ) Not many regions of the world have been able to produce great wine regularly ( ) Fine wine is the rare and exquisite result of perfect conditions in a perfect vineyard. Part III – Internet usage questionnaire 1) You are: ( ) 17 – 25 years old ( ) 26 – 35 years old ( ) 36 – 45 years old ( ) 46 – 55 years old ( ) 56 – 65 years old ( ) > 65 years old 2) Academic Level: ( ) Elementary School ( ) Graduate ( ) Post Graduate 3) How frequently do you use the internet (tick the answer that applies) ( ) Daily. ( ) Every other day. ( ) At least once a week and less than every other day. ( ) At least once in a fortnight and less than once a week. ( ) At least once a month and less than once in a fortnight. ( ) Never. 4) When you are connected, how long on average do you stay connected? (tick the answer that applies) ( ) < 1 hour ( ) 1 – 3 hours ( ) 3 – 5 hours ( ) > 5 hours 5) Which kinds of online activities you usually perform when connected? (tick all that apply) ( ) Search for information using a search engine ( Google, Lycos, Altavista, etc). ( ) Browse the internet for fun. ( ) Compare price information using a website (NexTag, Froogle, etc). ( ) Send and receive e-mails. ( ) Play a game. ( ) Pay Bills. ( ) Bank Online. ( ) Buy something on an online auction (eBay, Ubid, etc.) ( ) Buy a product from an online store. ( ) Participate in a chat room. ( ) Listen to or download online music. ( ) Join a newsgroup. 296
- 319. Appendix C Elements which helped identifying the website in the second exploratory study Comments made by participants in each category on both passes and the respective keywords assigned. Keywords were based on the words in bold. Category Comments Keyword Web mail Two lists side by side similar to my e-mail list List Sequence of topics organized as a list List Ordered List List to; from; size; date columns To-From Columns 'To', 'From' To-From Columns To-From 'From', 'To', 'Date', square to check on or off Columns/Checkbox Small squares followed by numbers. Checkbox Check box, same format of e-mail sites. Checkbox Square boxes to check/uncheck Checkbox Lines of a table Table Table with lines with messages. Table Use of colour to highlight items that you are supposed to read and what you have read Colours Pictures of letters, lines of text. Envelope Icon elements organization looked like an e-mail interface Online 'News', 'sports', bizarre, looks like a tabloid. Words News 'News' written. Words The word "news" and a picture of the London bomber. Words / Picture Picture of suicide bomber Picture Underlined texts, pictures. Picture and Text Headlines / Picture & Way the pictures and text were arranged, with headlines Text picture of policemen and text next to it Picture and Text A picture of British police with some text beside it. Picture and Text Picture of London and a text. Picture and Text Picture with text next to it. Looks like Hotmail news web site Picture and Text Picture and Text / A picture with text next to it. Headlines Headlines Presented as a list of headlines and therefore, don't look like "online news" Headlines One big headline with a link where you can click to know more Headlines Colours and layout of information. Colours / Layout Layout looks like a newspaper. Layout Has a date, the News where you can click. Date / Links News with links to click Links 297
- 320. Appendix C Category Comments Keyword Search the redundancy of names (Leonardo da Vinci) Repeated names Engine List of names displayed like a result from the search Repeated names Items, many results. Repeated names Same name repeated several times Repeated names 'London' displayed many times. Repeated names list of links ad numbers accompanied by a summary Links and summaries Many links where you can click to know more. Links list of links on one topic Links Picture with a name and a description highlighted Picture & description Tourism or entertainment with the word London. Words Items ordered on the left alternating colours. Ordered Items Search results. Two of them separated from the rest. Layout Search results about London similar appearance to all search engines. Layout e- Piece of advertisement Advertisement commerce Picture of book or DVD cover. E-commerce because a search engine usually doesn't Advertisement / have adverts. Pictures Supermarket online, it has the products displayed. Pictures Pictures displayed as tiles on the page. Pictures Pictures framed, small pictures that resemble movie posters Pictures squares with pictures that resembled DVDs Pictures DVD covers pictures Pictures Shows products pictures that look like book covers Pictures Picture of a movie is shown although the "Summer Holiday" ad distracted me a bit. Pictures DVD picture, price, link 'buy', new releases. The advert mislead on the real business Pictures / Price / of the site. Words Word "what's new" Words 'food and drinks', 'special offers', 'fast food', 'buy one get one free'. Words It’s written on the website "books" but there was no price. There is a shopping cart so Words it’s an e-commerce. Word ‘Carrefour’, many products, search interface. Words 'Carrefour', to make your shopping list. Saw Carrefour written. Words 'Carrefour' word, type of products. Words sentence "shopping online is easy", the section delivery is separated by a tab Words Read the word "shopping". Words Word 'shopping online'. Words Sentence "shopping on line" Words Products’ brands 'Flora', 'Sainsbury' Words Sentence "summer sale" Words Price of products, a line of text. Price List / Pictures and list of books using pictures and text text 'War of the worlds' written next to a picture of the movie. Pictures & text Rectangular picture that looked like a DVD or a book and next to it links Pictures and Links Shopping cart, with a review on the film. Shopping Cart E-commerce…there is a shopping cart…video, DVD Names of similar products (drinks) Shopping Cart Mentioned that it didn't look like a supermarket website. 298
- 321. Appendix C Category Comments Keyword Online the sentence "current account card insurance" Words Banking words "current account" Words Card insurance written on the middle of the screen Words ‘Internet 30hs’, ‘current account card insurance’. Words word "account" Words name of branch transactions were listed Words word 'account' Words Words word 'security' and had some tabs on top that are typical of bank Tabs Words / Bank, there is an advert of card insurance. 'Current Account' written. Advertisement Saw a picture of a credit card. Picture Items ordered on table with dark/light grey colours. Table Online the words "express auctions" Words Auction Saw word 'auctions'. Words Products, 'bids' and 'time left'. Saw someone using something similar before. Words 'bid now', 'auction' written on a tab. Words It reads 'Price', 'offers', 'visits', 'ends on'. 'How to buy' and 'how to sell' on top. Words 'how to buy', 'how to sell', 'safe shopping' Words It’s not very clear that it is an auction site, looks like a search engine. Small pictures Picture & description of products and their descriptions small picture of products and text next to it Picture & description The pictures are small and all of them of the same size, the text is small and looks like Picture & description a description of the product. Many colours, pictures of products. Colours / Pictures Total Frequency count per elements mentioned for each website category. Webmail Search Engine Online news Element Ef Element Ef Element Ef Checkboxes 4 Repeated Names 5 Picture and Text 6 List 3 Links 2 Headlines 4 To-From Columns 3 Layout 2 Words 3 Table 2 Links and Summaries 1 Picture 3 Colours 1 Picture & description 1 Layout 2 Envelope Icon 1 Words 1 Links 2 Ordered items 1 Underlined Text 1 Colours 1 Date 1 e-commerce Online banking Online auction Element Ef Element Ef Element Ef Words 13 Words 9 Words 6 Pictures 9 Tabs 1 Picture & description 3 Advertisement 2 Advertisement 1 Colours 1 Price 2 Picture 1 Pictures 1 Shopping Cart 2 Table 1 List 1 Pictures & description 1 Pictures & Links 1 299
- 322. Appendix D Criteria used for determining the screen position of a visual element for the survey The criteria used in the survey described in Chapter 4 to determine what the position of a visual element of a web forum was in relation to the grid is described below: (a) When more than 50% of the element is contained in a quadrant, its position is assigned to that particular quadrant. (b) When an element occupies three consecutive horizontal or vertical quadrants the position associated with it is the centre quadrant of the respective line or column in which it appears. This usually happens due to two factors: the size of the element and/or its mode of configuration. This can be observed in the menu element (Figure 27) which has a horizontal configuration thus occupying the three top quadrants. This criterion was adopted because when assembling the non-conforming version the same element might have a different configuration. Having all three quadrants assigned to the element could make it problematic to assign a non-conforming position to it. (c) Because of how the grid is constructed, when an element occupies two adjacent horizontal quadrants these always consist of one central quadrant (top centre, left centre, right centre bottom centre) and another outer quadrant (top left, top right, bottom left and bottom right). In this case, the position of the element was assigned to the outermost quadrant. This will avoid a great concentration of elements described as in the centre of the page. In Figure 27 the element ‘Last post’ occupies both the top row and the centre row, but since more than 50% of its area is in the central row, it considered to be located in the central row. But within the central row, it extends itself from the central quadrant to the centre-right quadrant occupying two quadrants at the same time thus being noted as located at the outmost quadrant, the centre-right quadrant. 300
- 323. Appendix D (d) When an element appears in two or more different positions simultaneously on a page, a separate category will be created with all occurring positions and its position will be allocated there. As an example, suppose that one element appears simultaneously in the top left and bottom left of a page. Instead of counting one occurrence in the top left and one occurrence in the bottom left (which would not properly represent how the element is displayed), a ‘top left and bottom left’ category will be created and the element will be counted there. The reasoning behind this can be illustrated with an example: consider an element which appears 9 out of 10 times simultaneously in the top left and bottom left of the screen and one time in the top left. If multi-positions were attributed to each single location, the most frequent position would be considered to be the top left. But in fact, the most frequent layout would be the one where the element appears simultaneously in the two positions. 301
- 324. Appendix E Explanation of elements found in all three levels of a web forum I) Elements found in the Forum index page (first level) of a web forum ! Forum index table – This element is made up by a group of components that are usually placed together in a table. It occupies a large portion of the forum area. ! Menu – This is the element that contains the links related to the user account management and forum administration. ! Logo – The logo of the forum. ! Login field – This element is where the registered users log themselves in the forum. ! Captions – Its function is to explain the codes used within the forum. ! Current time and date – Present the current time and date in the country where the forum is held. ! Users information area – In this area all the information regarding users in the forum are displayed (e.g. birthdays, new users). ! Breadcrumbs – This element works as a locator in the forum space, illustrating to the users not only where they are but also the navigational path that took them there. ! Forum notice board – Here the forum moderators can post announcements and notes to all visitors of the forum. ! Adverts – These are usually links that, when clicked on, take the users to other pages, not always related to the forum. These adverts are usually a source of income to the forum creators. ! Forum search field – Here the user can look for a particular topic within the forum. Within the Forum index table, the following elements were found: ! Topic title – This element is a link to a topic of a specific subject within the forum. ! Topic summary – A brief description of what that topic is about. ! Topic status indicator – Indicates to the user the current situation of a particular topic (e.g. if somebody has posted something since the last visit, if that topic allows posting or not). ! Number of threads in topic – Displays the number of threads that a particular topic contains. ! Number of posts in topic – Shows the total number of postings that a particular topic has. ! Last post in topic – Informs the user when the last post in a topic was made. ! Topic moderator – Indicates which user is responsible for managing a certain topic, most of the times the forum moderators are the ones who manage all topics in a forum but some forums give the responsibility for managing the topic to the user who started it (see 31). 302
- 325. Appendix E ! Number of views in topic – This element shows the total number of views that a particular topic in a forum had. II) Elements found in the Threads page (second level) of a web forum which are not in the forum index page. ! Thread table – Similar to the forum index table, this table contains a list of all threads for the particular topic chosen. ! New thread button – Allows the user to create a new thread. ! Number of thread pages – This element is usually a sequence of links and it appears when the number of threads is too big and consequently the list is too long and has to be divided into pages. When clicked upon it takes the user to the desired page. ! Jump to forum – This element is a list of the topics in the forum and allows the user to quickly move from one topic to another without having to go back to the forum index table. ! Posting rules – Establishes what the user is and is not allowed to do on that particular level of the forum. ! Forum search – Allows the users to search for a specific topic or subject within the whole forum. ! Popular topics – Shows which topics have been most visited and/or have had the biggest number of posts. Within the threads table, the following elements were found: ! Thread status indicator – Similar to the topic status indicator, this indicates to the user the current situation of a particular thread (e.g. if somebody has posted something since the last visit, if a thread allows posting or not). ! Thread name – It is the main title of a thread. ! Thread author – Presents the name of the author of a thread. ! Last post – This element informs the users when the last message was posted in a particular forum. It usually contains the date and time of the post along with the name of the person who wrote it. ! Number of replies – Indicates how many replies a particular thread has had. ! Number of views – Shows the number of views (hits) that a thread has had. ! Emoticons – These are usually small graphic images that are mainly used to demonstrate the mood of the author of a thread. III) Elements found in the Post page (third level) of a web forum which are not in the forum index page nor the threads page. ! Post table – Here is where the members of the forum leave their comments on a specific topic or discussion. ! Next/previous thread button –These buttons allow users to navigate forward and backwards within the threads without having to return to the threads page. ! Reply to post button – This button allows the users to reply to a specific post from a user. ! Thread title – This consists of the title of the thread the user is actually on. ! New post button – This buttons allows users to start a new topic in a thread without having to return to the threads page. ! Table of contents – Indicates in a list format all the postings and replies to a post that a posting area contains. 303
- 326. Appendix E Within the post table, the following elements were found: ! Time & date of post – Displays the time and date a particular post entered the system. ! Post content – This is the actual content that a user has posted. ! Author info – This is and area that contains information about the author such as the date they entered the forum, their nickname, sometimes an avatar (picture or symbol that represents the author) or a link to further author information. ! Quote post button – When clicked on allows the user to include a quote from another post in his post. ! Post title – It is the title of a post, different from the topic title. ! Signature – It consists of a sentence and sometimes a picture that users insert after their posting that works like a trademark or a signature of that particular user. ! User profile button – This button links to the profile of a user that has posted something in a forum. ! Private message button – This button allows users to send messages to another user that has left a post in a forum. ! Post number – Indicates the number of a post within a thread. ! Edit post – This button is only active for the users that have posted something to edit what they have written. ! Author’s website button – This button links to the website of the user that has written a particular post. ! Email author button – This button allows a user to send an e-mail to another user that has written a particular post. 304
- 327. Appendix F Tables with the symbolization mode frequencies of each element appearing on each of the three forum levels The tables illustrated in this appendix show the frequency of each symbolization mode of the web forum visual elements listed in the survey described in Chapter 4. TABLE 51 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE FORUM INDEX PAGE. THE INDENTED ELEMENTS ARE FOUND WITHIN THE INDEX TABLE. Forum index elements Graphic Alphanumeric Hybrid Forum index table 0 1 14 Topic title 0 15 0 Topic summary 0 14 0 Nº of threads 0 13 0 Nº of posts 0 13 0 Last post 0 3 10 Topic moderator 0 5 0 Nº of views 0 1 0 Status indicator 12 0 1 Topic type 0 1 0 Menu 0 11 4 Logo 0 7 5 Login field 0 11 0 Captions 0 0 10 Current time & date 0 10 0 User information area 0 4 6 Breadcrumbs 0 2 7 Forum noticeboard 0 6 1 Adverts 0 3 3 Forum search field 0 2 1 305
- 328. Appendix G TABLE 52 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE THREAD PAGE. THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREAD TABLE. Thread page elements Graphic Alphanumeric Hybrid Thread table 0 1 14 Thread name 0 8 7 Thread author 0 0 14 Thread status 10 0 2 Last Post 0 5 9 Number of replies 0 14 0 Number of views 0 13 0 Emoticons 5 0 0 Menu 0 12 3 New thread button 0 3 11 Number of thread pages 0 9 5 Breadcrumbs 0 9 4 Forum jump 0 11 1 Captions 0 0 11 Logo 0 4 7 Posting rules 0 4 6 Current time & date 0 7 0 Forum search 0 4 1 Login field 0 5 0 Popular topics 0 0 1 TABLE 53 - FREQUENCY OF SYMBOLIZATION MODE OF THE ELEMENTS ON THE POST PAGE. THE INDENTED ELEMENTS ARE FOUND WITHING THE POST AREA. Post page elements Graphic Alphanumeric Hybrid Post area 0 0 15 Time & Date of Post 0 5 10 Post content 0 10 5 Author Information 0 4 11 Quote button 1 3 8 Post title 0 10 2 Signature 1 8 1 User profile button 1 2 4 Private message button 1 2 3 Post number 0 6 0 Edit Post 1 2 1 Author’s website button 0 2 1 E-mail author button 0 2 1 Menu 0 12 2 Next/previous thread button 1 11 2 Breadcrumbs 0 4 9 Reply to post button 0 3 10 Jump to forum 0 12 0 Logo 0 5 6 Posting rules 0 8 0 Thread title(out post area) 0 6 0 Current time & date 0 6 0 Adverts 0 2 4 Login field 0 5 0 New post button 0 1 4 Search topic field 0 3 1 Table of contents 0 0 1 306
- 329. Appendix G Appendix G Tables with the configuration mode frequencies of each element appearing on each of the three forum levels Tables 54, 55 and 56 show the frequency counts of the configurations of the elements across the web forums and the indented elements appearing on each table are the ones contained within the tables (forum index table, threads table) or in the post area. As mentioned earlier on Chapter 4, for the elements which had only one component the abbreviation of the words ‘not applicable’ (N.A.) was used in the tables to indicate that no configuration mode was established for this particular element. TABLE 54- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM INDEX PAGE. THE INDENTED ELEMENTS ARE FOUND WITHIN THE INDEX TABLE. Forum index elements Horizontal Linear interrupted Matrix Index Table 0 0 15 Forum Title 15 0 0 Nº of Threads 13 0 0 Nº of Posts 13 0 0 Topic moderator 4 1 0 Topic summary 1 13 0 Last Post 1 12 0 Topic type 1 0 0 Nº of views N.A. N.A. N.A. Status indicator N.A. N.A. N.A. Menu 12 1 2 Current time & date 10 0 0 Breadcrumbs 9 0 0 Logo 7 0 5 Login field 5 1 5 Adverts 5 0 1 Captions 4 0 5 User info area 1 0 9 Forum search field 1 0 2 Forum noticeboard 0 0 7 307
- 330. Appendix G TABLE 55- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM THREAD PAGE. THE INDENTED ELEMENTS ARE FOUND WITHIN THE THREAD TABLE. Forum thread elements Horizontal Linear interrupted Matrix Thread table N.A. N.A. N.A. Thread author 11 3 0 Thread name 9 6 0 Status indicator N.A. N.A. N.A. Last post 1 13 0 Nº of replies N.A. N.A. N.A. Nº of views N.A. N.A. N.A. Emoticons N.A. N.A. N.A. Nº of thread pages 13 0 1 Menu 12 1 2 New topic button 12 2 0 Breadcrumbs 9 4 0 Current time & date 7 0 0 Forum jump 6 1 5 Logo 6 5 0 Forum search 2 1 2 Captions 0 1 10 Posting rules 0 9 1 Popular topics 0 0 1 Login field 0 1 4 TABLE 56- FREQUENCIES OF CONFIGURATION OF ELEMENTS IN THE FORUM POST PAGE. THE INDENTED ELEMENTS ARE FOUND WITHING THE POST AREA. Post area elements Horizontal Linear interrupted Matrix Post Area 8 7 0 Time & date of post 15 0 0 Post title 12 0 0 Quote button 10 0 2 Profile button 7 0 0 Private message button 6 0 0 Post number 6 0 0 Edit post 4 0 0 Poster website button 3 0 0 email button 3 0 0 Author info 1 10 4 Signature 0 10 0 Post content 0 15 0 Next/previous thread button 13 1 0 Menu 11 1 2 Reply to topic button 11 0 0 Breadcrumbs 9 0 4 Jump to Forum 7 0 5 Thread title(out post area) 6 0 0 Logo 5 2 4 New post button 5 0 0 Current time & date 4 0 0 Adverts 3 0 3 Search topic field 3 1 0 Posting rules 0 8 0 Login field 0 1 4 Table of contents 0 0 1 308
- 331. Appendix H Tables showing the survey 1 results for the most frequently occurring groups of elements Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq Index table 15 Menu 15 Logo 12 Login Field 11 Captions 10 Current Time & Date 10 User Info Area 10 Breadcrumbs 9 Forum Index Page Forum Noticeboard 7 Adverts 6 Forum Search Field 3 Forum Table Elements Forum Title 15 Forum Status Indicator 14 Forum Summary 14 Nº of Threads in Forum 13 Nº of Posts in Forum 13 Last Post in Forum 13 Forum Moderator 5 Nº of Views in Forum 1 Forum Type 1 Least Most frequent frequent 309
- 332. Appendix H Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq Menu 15 Thread Table 15 New Topic Button 14 Nº of Thread Pages 14 Breadcrumbs 13 Thread Status Indic. 12 Forum Jump 12 Captions 11 Logo 11 Threads page Posting Rules 10 Current Time & Date 7 Forum Search 5 Login Field 5 Popular Topics 1 Thread table Elements Thread Name 15 Last Post 14 Thread Author 14 Nº of Replies 14 Nº of Views 13 Emoticons 5 Least Most frequent frequent 310
- 333. Appendix H Element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Freq Post Area 15 next/prev. thread btn 15 Menu 14 Breadcrumbs 13 reply to topic button 13 Logo 11 Jump to Forum 11 Posting Rules 8 Topic title 6 time and date 6 Adverts 6 New post button 5 Login field 5 Post area Search topic field 4 Table of contents 1 Post Area Elements Author Info 15 Post Content 15 Time & Date of Post 15 Post Title 13 Quote Button 12 Signature 10 Profile Button 7 Post Number 6 Private Msg Button 6 Edit Post 4 email Button 3 Poster website button 3 Least Most frequent frequent 311
- 334. Appendix I Templates used in experiments 1 and 2 to assess perceptions of conventionality and ease of use FIGURE 131 - TEMPLATE C11 OF THE FORUM INDEX PAGE, MOST FREQUENTLY CHOSEN AS MORE LIKE OTHER EXISTING FORUMS. 312
- 335. Appendix I FIGURE 132 - TEMPLATE C12 OF THE FORUM INDEX PAGE 313
- 336. Appendix I FIGURE 133 - TEMPLATE C13 OF THE FORUM INDEX PAGE 314
- 337. Appendix I FIGURE 134 - TEMPLATE C14 OF THE FORUM INDEX PAGE 315
- 338. Appendix I FIGURE 135 - TEMPLATE N11 OF THE FORUM INDEX PAGE 316
- 339. Appendix I FIGURE 136 - TEMPLATE N12 OF THE FORUM INDEX PAGE 317
- 340. Appendix I FIGURE 137 - TEMPLATE N13 OF THE FORUM INDEX PAGE 318
- 341. Appendix I FIGURE 138 - TEMPLATE N14 OF THE FORUM INDEX PAGE, CHOSEN THE LEAST NUMBER OF TIMES AS MORE LIKE OTHER WEB FORUMS. 319
- 342. Appendix I FIGURE 139 - TEMPLATE C21 OF THE THREADS PAGE, MOST FREQUENTLY CHOSEN AS MORE LIKE OTHER WEB FORUMS. 320
- 343. Appendix I FIGURE 140 - TEMPLATE C22 OF THE THREADS PAGE 321
- 344. Appendix I FIGURE 141 - TEMPLATE C23 OF THE THREADS PAGE 322
- 345. Appendix I FIGURE 142 - TEMPLATE C24 OF THE THREADS PAGE 323
- 346. Appendix I FIGURE 143 - TEMPLATE N21 OF THE THREADS PAGE 324
- 347. Appendix I FIGURE 144 - TEMPLATE N22 OF THE THREADS PAGE 325
- 348. Appendix I FIGURE 145 - TEMPLATE N23 OF THE THREADS PAGE, CHOSEN THE LEAST NUMBER OF TIMES AS MORE LIKE OTHER WEB FORUMS. 326
- 349. Appendix I FIGURE 146 - TEMPLATE N24 OF THE THREADS PAGE 327
- 350. Appendix I FIGURE 147 - TEMPLATE C31 OF THE POST PAGE, MOST FREQUENTLY CHOSEN AS MORE LIKE OTHER WEB FORUMS. 328
- 351. Appendix I FIGURE 148 - TEMPLATE C32 OF THE POST PAGE 329
- 352. Appendix I FIGURE 149 - TEMPLATE C33 OF THE POST PAGE 330
- 353. Appendix I FIGURE 150 - TEMPLATE C34 OF THE POST PAGE 331
- 354. Appendix I FIGURE 151 - TEMPLATE N31 OF THE POST PAGE 332
- 355. Appendix I FIGURE 152 - TEMPLATE N32 OF THE POST PAGE 333
- 356. Appendix I FIGURE 153 - TEMPLATE N33 OF THE POST PAGE 334
- 357. Appendix I FIGURE 154 - TEMPLATE N34 OF THE POST PAGE, CHOSEN THE LEAST NUMBER OF TIMES AS MORE LIKE OTHER WEB FORUMS. 335
- 358. Appendix J Results from experiment 3, the icon identification test Icon 1 Counts Icon 2 Counts explains something 1 Admin 2 FAQ 3 Encrypt 1 get info 1 Key 2 Help 22 Lock 1 help page 1 Lock thread 1 Question 6 Locked 1 Total 34 Login 2 Options 1 Password 12 Password protected area 1 Password/login 1 Secure payment 1 Icon 3 Counts Secure website 1 ? 2 Security 6 Address 1 Start 1 Edit 4 Total 34 Entry form 1 Handwriting 1 Insert info 1 New post 3 New thread 1 Post 2 Post new message 1 Reply 3 Response 1 Signature 1 Write 6 Write message 1 Write new 1 Write new post 2 Write post 1 Writing 1 Total 34 336
- 359. Appendix J Icon 4 Counts Icon 5 Counts ? 9 Two people 1 Advert 1 Chat 2 Application 1 Comments 1 Birthdays 1 Contact 1 Calendar 11 Discussion 1 Date 1 Discussion forum 1 Dates 1 Forum 2 Font 1 Friend list 1 Gallery 1 Instant messenger 2 Information 1 Man handle 1 News 1 Members 2 Notes 1 Messenger 4 Pictures 1 Msn 8 Spell check 1 Msn/user details 1 Start new thread 1 People 2 Text document 1 PM 1 Total 34 Talk 1 Users 2 Total 34 Icon 6 Counts archives 1 Icon 7 Counts Calendar 30 ? 3 Calendar month 1 contact card 2 date 1 contact info 1 gallery 1 email 3 Total 34 identification 2 identity 2 insert image 1 member info 1 Mini-profile 1 personal page 1 profile 10 profile card 1 user id 1 vcard 2 Video 1 video conference 1 your profile 1 Total 34 337
- 360. Appendix J Icon 8 Counts Icon 9 Counts appointment 2 ? 5 calendar 14 Audio 1 date 2 Call centre 1 date & time 5 Chat 1 events 2 Headset 1 meetings 1 Help 1 Schedule 1 Instant messenger 2 settings 1 Live assistance 1 Time 2 Live support 1 time & calendar 1 Online 1 time & date 2 Online help 1 zoom in 1 Person 1 Total 34 Profile 1 Skype 1 Something audible 1 Speech 1 Spoke word computer 1 Support 5 Icon 10 Counts User 2 ? 5 Voice 3 attachment 1 Voice call 1 Cut 1 Voip 1 download 2 Total 34 e-blog 1 edit 1 exit 1 go to 1 go to document 2 Icon 11 Counts incoming data 1 ? 14 insert 1 archive 1 new post 2 closed topic 1 new thread 2 cotton reel 1 Next Page 3 delete 2 post 1 forum thread 1 print 1 Ink 1 save 3 movie 1 send 1 new thread 1 send file 2 no connection 1 submit 1 Save 1 submit file 1 sewing advice 1 Total 34 tailor 1 thrash 1 thread 5 write a post 1 Total 34 338
- 361. Appendix J Icon 12 Counts Icon 13 Counts ? 8 ? 5 add comment 1 enlarge 1 chat 2 find 1 Comments 4 glasses 2 discuss 1 larger text 1 discussion 3 read 1 important info 1 Search 14 message 1 sight impaired 1 new post 3 accessibility option no connection 1 text larger 1 note in text 1 view 2 Quote 2 watch thread 1 quote 1 Zoom 4 read 1 Total 34 reply 1 talk 1 thread 1 transcript 1 Total 34 339
- 362. Appendix K Results for Experiment 3 which asked participants to identify the interactive elements of the non-conforming forum Part1 Part2 Part3 Part4 Part5 Part6 Username and Change text Password to Associated Threads Selected 1 font and website time access member programs within forum thread colour areas forum entry 2 search links forum link forum status time box 3 Don’t know Don’t know Don’t know Don’t know Don’t know Don’t know more criterion 4 entry information, main page Don’t know put on disc main page dates logon with navigating topics of a post from 5 password and who is online time and date options discussion user username forum entry 6 login toolbar topic/thread links time and date box 7 Don’t know Don’t know Don’t know Don’t know Don’t know Don’t know 8 login menu bar main forum topic online users time and date thread with 9 login section links list of threads key date and stats details functions guide to new 10 login conversation one post time and date e.g. search posts 11 login operators page # text go to close list of thread with 12 sign in menu bar Don’t know date stamp messages details access/actio 13 name/password Don’t know Don’t know Don’t know Don’t know ns quick user particular 14 login forum posts who is online time and date panel post forum 15 login faq forum font colour time comment login and forum current date 16 user menu forum page site details password message link and time login and page of thread of what you are 17 icon bar Don’t know password postings forum allowed to do a single user control list of threads 18 login area thread or Don’t know time panel o posts post page holding 19 login area menu area thread page key stats threads 20 login tools messages a message meta data clock 21 login user settings main content topics links time and date 340
- 363. Appendix K Part1 Part2 Part3 Part4 Part5 Part6 user control current date 22 login boxes all threads single thread users online panel and time access to 23 login section forum thread legend date user panel 24 login menu background thread key time login username forum most recent 25 one thread links copyright & password information threads List of Menu of 26 login forms threads in Thread info Who is online Time and date some kind date order contact results of a one result of 27 login to site Don’t know save details search a search control system time 28 login panel topics list topic footer text info settings 29 user functions login forum body topic key time change 30 sign in Don’t know forum thread time and date colour single piece 31 login navigation content key/legend time of content 32 login select action all posts post copyright info time 33 login members posts thread logout login login and main menu 34 thread list thread listing Don’t know Don’t know password options 341
- 364. Appendix L Text introducing PDAs presented for participants of experiment 4 A Brief introduction to PDAs PDAs or Personal Digital Assistants, also called handhelds or palmtops, have definitely evolved over the years. Not only can they manage your personal information, such as contacts, appointments, and to-do lists, today's devices can also connect to the Internet, act as global positioning system (GPS) devices, and run multimedia software. What's more, manufacturers have combined PDAs with cell phones, multimedia players and other electronic gadgetry. There are many brands of PDAs each with diverse features and caracteristics. The two main groups of PDAs are the ones that are based on the PALM OS operating system and those that are based on the Windows operating system (called Pocket PCs). Within both groups you have the standard PDAs and other PDAs that are combined with mobile phones, these are called smartphones. The greatest examples of this category are the Palm Treo line and the Blackberry but Panasonic, Nokia, Samsung and other big companies also have their own smartphones. PALM POCKET PC OTHER Standard PDA Smartphone Standard PDA Smartphone Standard PDA Standard PDA Tungsten E2 Treo 680 Dell Axim HP iPAQ Apple Newton Sharp Zaurus 342
- 365. Appendix M Tasks designed for the experiment 4 which tested the performance of users familiar and unfamiliar with web forums on conforming and non- conforming websites. Tasks 1. According to the forum member hawklet00, what is needed to use a GPS (Global Positioning System) with the Blackberry PEARL? 2. How many threads with new posts does the ‘Palm General Discussion’ topic have? 3. Which is the oldest post under the Miscellaneous PDAs topic? 4. Which video player is suggested by the member pipesmoker to be used to play video podcast files and display them in a full screen of a Tungsten E (PalmOS)? 5. Which is considered to be the best Newton by the forum member stercusaccidit, the 130, the 2000 or or the 2100? 343
- 366. Appendix N Tasks designed for the pilot study which was conducted in preparation for experiment 5 Tasks A) How many birthdays does the forum have on 11th of October 2007? B) What is the difference between a 1st edition and a 1st trade edition? C) Which forum member wrote the last post on the thread “Trimming pages” under the “Adding value to your books” topic? D) Find the answer for this frequently asked forum question: “How do I prevent my user name from appearing in the online user’s listing”? E) Which thread in the ‘Books wanted’ topic has the last post dated Nov 22? F) What is the price of a third edition of the ‘Compendious Geographical Dictionary’? G) In the forum topic “Help on identification”, how many viewings has the thread “Atlas of lost worlds” had? H) How much is worth a T.S. Eliot signed 1st edition? I) Which thread under the “What I’m reading now” has the greatest number of views? H) Go to the topic in the forum which has the highest number of threads and find who published the earliest copies of ‘Le Petit Prince’ 344
- 367. Appendix O Tasks used on both parts of experiment 5 Part I A) How much is a 1901 American standard Bible worth? B) Identify which company now owns the publisher called Doubleday. C) What is the procedure to add value to a book by removing its musty odors using kitty litter? D) What is the difference between a first edition and a 1st trade edition? E) What is the name of the author of the Non-fiction book ‘Nickel and Dimed: On (Not) Getting by in America’? F) Go to the ‘Books Wanted’ topic and find out which forum member wrote the last post on the thread “looking for 1st ed / 1st printing of Flowers for Algernon” G) Go to the ‘Science Fiction & Fantasy’ topic and find the title of the only thread with new posts. H) How many pages does the ‘General Discussion’ topic have? I) Go to the ‘Classic literature’ topic and find who wrote the book ‘Moon and Sixpence’ J) How many birthdays does the forum have on the calendar date of 11th of October 2007? Part II A) Which brand of keyboard works with the PALM Zire 31? B) Where can you find a SWF player for the Windows Mobile/Pocket PC? C) What can you use instead of a stylus to edit Word or .PDF documents on a Blackberry? D) Go to the forum topic with the highest number of threads and find out which Newton is the best: the 130, 2000 or the 2100. E) Go to the forum topic whose last post was done on 11-18-2005 at 6:53 pm and find the web address where you can get the game Space Quest for the Nokia SERIES 60. F) What is the title of the only closed thread in the ‘General Smartphone Discussion’ topic? 345
- 368. Appendix P Disorientation questionnaire applied on both parts of experiment 5 For each question below, mark the box that most adequately represent how felt while doing the tasks on the web forum you just used. 1) I felt lost / disorientated Never Sometimes A few Most of the Always times time 2) I felt like I was going around in circles Never Sometimes A few Most of the Always times time 3) It was difficult to find a page that I had previously viewed Never Sometimes A few Most of the Always times time 4) Learning to use the site was easy Strongly Disagree disagree Neither Agree Strongly agree nor agree disagree 5) The site was easy to navigate. Strongly Disagree disagree Neither Agree Strongly agree nor agree disagree 346
- 369. Appendix Q Questionnaire used in Experiment 6 1) I felt lost / disorientated Never Sometimes A few Most of the Always times time 2) Learning to use the site was easy Strongly Disagree disagree Neither Agree Strongly agree nor agree disagree 3) The site was easy to navigate. Strongly Disagree disagree Neither Agree Strongly agree nor agree disagree 347
- 370. References Ahuja, J. S., & Webster, J. (2001). Perceived disorientation: an examination of a new measure to assess web design effectiveness. Interacting with Computers, 14, 15-29. Åkesson, M. (2003). Design patterns for the online newspaper genre - A genre analysis of 85 Swedish daily online newspapers. Götenburg University, Götenburg. Albers, M. J., & Kim, L. (2000). User web browsing characteristics using palm handhelds for information retrieval. Paper presented at the Proceedings of IEEE professional communication society international professional communication conference and Proceedings of the 18th annual ACM international conference on Computer documentation: technology & teamwork. Alexander, C. (1999, Sep/Oct). The origins of pattern theory: the future of the theory, and thegeneration of a living world. Software, IEEE, 16, 71-82. Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A Pattern Language: Towns, buildings, construction. New York, United States: Oxfor University Press. Baker, R., Bernard, M., & Riley, S. (2002). Reading online news: A comparison of three presentation formats. Usability News Retrieved 06 January, 2005, from http://psychology.wichita.edu/surl/usabilitynews/42/depth.htm Bateman, J., & Delin, J. (2001). From genre to text critiquing in multimodal documents. Workshop presented at the 4th International Workshop on Multidisciplinary Approaches to Discourse: Improving text: From text structure to text type. MAD 2001, Yttre, Belgium. Beghtol, C. (2001). The concept of genre and its characteristics. Bulletin of the American Society for Information Science and Technology, 27(2). Bernard, M. (2001a). Developing schemas for the location of common web objects. Usability News, 3(1). Bernard, M. (2001b). User expectations for the location of web objects. Paper presented at the CHI '01 extended abstracts on Human factors in computing systems, Seattle, Washington. Bernard, M. (2002). Examining User Expectations for the Location of Common E- Commerce Web Objects. Usability News, 4(1). Retrieved from http://www.surl.org/usabilitynews/41/web_object-ecom.asp Bernard, M., Chaparro, B., & Thomasson, R. (2000). Finding information on the web: Does the amount of whitespace really matter? Usability News, 2(1). Bevan, N. (1997). Usability issues in web site design. Advances in Human Factors Ergonomics, 21(B), 803-806. Bevan, N., Kirakowski, J., & Maissel, J. (1991, September 1-6). What is Usability? Paper presented at the 4th International Conference on HCI, Stuttgart, Germany. 348
- 371. Bevan, N., & MacLeod, M. (1994). Usability measurement in context. Behaviour & Information Technology, 13(1), 132 - 145. Blackler, A. L., Popovic, V., & Mahar, D. P. (2004). Intuitive interaction with complex artefacts. Paper presented at the Futureground - Design research society international conference, Melbourne, Australia. Blattner, M., Sumikawa, D. A., & Greenberg, R. M. (1989). Earcons and icons: Their structure and common design principles. Human Computer Interaction, 4, 11 - 44. Borchers, J. O. (1999). Electronic Books: Definition, Genres, Interaction Design Patterns. Paper presented at the CHI'99 International Conference on Human Factors in Computing Systems. Borchers, J. O. (2001). A pattern approach to interaction design AI & Society, 15(4), 359- 376. Borchers, J. O., Fincher, S., Griffiths, R., Pemberton, L., & Siemon, E. (2001). Usability Pattern Language: Creating A Community. AI & Society, 15(4), 377-385. Broder, A. (2002). A taxonomy of web search. SIGIR Forum, 36(2), 3-10. Burleson, J. (2004). Conducting the Web Designer job interview. Kittrell, NC, United States: Rampant Tech Press. Byrne, M. D., John, B. E., Wehrle, N. S., & Crow, D. C. (1999). The tangled web we wove: A taskonomy of WWW use. Paper presented at the SIGCHI conference on Human factors in computing systems: the CHI is the limit, Pittsburgh, Pennsylvania, United States. Cappel, J. J., & Huang, Z. (2006). An analysis of the use of web design conventions in company websites. Paper presented at the Annual conference of the International Association for Computer Information Systems - The changing role of IS Education, Reno, Nevada, USA. Capra, R. G. (2006). An investigation of finding and refinding information on the Web. Virginia Polytechnic Institute and State University, Blacksburg. Catledge, L. D., & Pitkow, J. E. (1995). Characterizing browsing strategies in the World- Wide Web. Computer Networks and ISDN Systems, 27(6), 1065 - 1073. Chaparro, B., Baker, J. R., Shaikh, A. D., Hull, S., & Brady, L. (2004). Reading online text: A comparison of four white space layouts. Usability News Retrieved 17 December, 2005, from http://psychology.wichita.edu/surl/usabilitynews/62/whitespace.htm Chaparro, B., Shaikh, A. D., & Baker, J. R. (2005). Reading online text with a poor layout: Is performance worse? Usability News Retrieved 27 November, 2005, from http://psychology.wichita.edu/surl/usabilitynews/71/page_setting.html Chawda, B., Craft, B., Cairns, P., Rüger, S., & Heesch, D. (2005). Do “Attractive Things Work Better”? An Exploration of Search Tool Visualisations. Paper presented at the HCI 2005 The bigger picture - 19th British HCI Group Annual Conference, Edinburgh, Scotland, United Kingdom. Conklin, J. (1987). Hypertext: An introduction and survey. Computer, 20(9), 17-42. Constantine, L. L., & Lockwood, L. A. D. (2002). Instructive interaction: Making innovative interfaces self-teaching. User Experience Magazine, 1(3). 349
- 372. Cosio, M. G. d., & Dyson, M. C. (2002). Identifying graphic conventions for genre definition in web sites. Digital Creativity, 13(3), 165-181. Cove, J. F., & Walsh, B. C. (1988). Online text retrieval via browsing. Information Processing and Management: an International Journal, 24(1), 31-37. Crowston, K., & Williams, M. (2000). Reproduced and emergent genres of communication on the World Wide Web. The Information Society, 16(3), 201-216. Danielson, D. R. (2002a). Transitional volatility in web navigation. Stanford university, Stanford. Danielson, D. R. (2002b). Web navigation and the behavioural effects of constantly visible site maps. Interacting with Computers, 14, 601-618. Delin, J., Bateman, J., & Allen, P. (2002). A Model of genre in document layout. Information Design Journal, 11(1), 54 - 66. DeVigal, A. (2003). Putting the eyetrack study to good use Retrieved 10 December, 2005, from http://www.poynter.org/content/content_view.asp?id=38357 Dewhurst, S. A., & Hitch, G. J. (1999). Cognitive effort and recollective experience in recognition memory. Memory, 7(2), 129 - 146. Dickson, S., Simmons, D. C., & Kameenui, E. J. (1995). Text organization and its relation to reading comprehension: A synthesis of the research. (Tech. Rep. No. 17). Eugene, OR: University of Oregon, National Center to Improve the Tools of Educators. Dillon, A. (1991). Readers' models of text structures. International Journal of man-machine studies, 35(6), 913-925. Dillon, A. (2000). Spatial-semantics: How users derive shape from information space. Journal of the American Society for Information Science, 51(6), 521-528. Dillon, A. (2002). Writing as design: hypermedia and the shape of information space. In R. Bromme & E. Stahl (Eds.), Writing Hypertext and Learning - Conceptual and Empirical Approaches: Elsevier. Dillon, A. (2004). Shape: information as a structured space. In A. Dillon (Ed.), Designing Usable Electronic Text (2 ed., pp. 214). Boca Raton, Fla. London: Routledge. Dillon, A., & Gushrowski, B. (2000). Genres and the web: Is the personal home page the first uniquely digital genre? Journal of the American Society for Information Science, 51(2), 202-205. Dillon, A., & Schaap, D. (1996). Expertise and the perception of shape in information. Journal of the American Society for Information Science, 47(10), 786-788. Dillon, A., & Vaughan, M. (1997). It's the journey and the destination: Shape and the emergent property of genre in evaluating digital documents. New Review of Multimedia and Hypermedia, 3, 91-106. DomainTools.com (2008). Domain Counts & Internet Statistics Retrieved 27 November, 2008, from http://www.domaintools.com/internet-statistics/ Dontcheva, M., Drucker, S. M., Salesin, D., & Cohen, M. F. (2007). Changes in Webpage structure over time. Washington, United States: University of Washington. 350
- 373. Dyson, M. C., & Gregory, J. (2002). Typographic cueing on screen. Visible Language, 36(3), 326-346. Dyson, M. C., & Haselgrove, M. (2000). The effects of reading speed and reading patterns on the understanding of text read from screen. Journal of Research in Reading, 23(2), 210- 223. Dyson, M. C., & Haselgrove, M. (2001). The influence of reading speed and line length on the effectiveness of reading from screen. Journal of Human-Computer Studies, 54, 585-612. Eason, K. D. (1984). Towards the experimental study of usability. Behaviour & Information Technology, 3(2), 133 - 143. Ehret, B. D. (2002). Learning where to look: location learning in graphical user interfaces. Paper presented at the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, Minneapolis, Minnesota, USA. Eriksen, L. B., & Ihlström, C. (2000). Evolution of the web news genre - the slow move beyond the print metaphor. Paper presented at the 33rd International Conference on System Sciences, Hawaii. Eveland, W. P., Cortese, J., Park, H., & Dunwoody, S. (2004). How web site organization influences free recall, factual knowledge, and knowledge structure density. Human Communication Research, 30(2), 208-233. Fayard, A.-L., & DeSanctis, G. (2006). Evolution of an Online Forum for Knowledge Management Professionals: A Language Game Analysis. Journal of Computer-Mediated Communication, 10(4). Fayard, A.-L., DeSanctis, G., & Roach, M. (2004). The language games of online forums. Paper presented at the Sixty-third Annual meeting of the Academy of Management, New Orleans, United States. Frye, N. (1957). Anatomy of Criticism: Four Essays (10th edition ed.). Princeton: Princeton University Press. Fu, L., & Salvendy, G. (2002). The contribution of apparent and inherent usability to a user's satisfaction in a searching and browsing task on the Web. Ergonomics, 45(6), 415 - 424. Galletta, D., Henry, R. M., McCoy, S., & Polak, P. (2006). When the wait isn't so bad: the interacting effects of website delay, familiarity and breadth. Information Systems Research, 17(1), 20 - 37. Gefen, D. (2000). E-commerce: the role of familiarity and trust. Omega - The International Journal of Management Science, 28(6), 725-737. Gery, G. (1995). Attributes and behaviors of performance-centered systems. Performance Improvement Quarterly, 8(1), 47-93. Gordon, M., & Pathak, P. (1999). Finding information on the World Wide Web: the retrieval effectiveness of search engines. Information Processing and Management: an International Journal, 35(2), 141-180. Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what designers think. Communications of the ACM, 28(3), 300-311. 351
- 374. Granlund, Å., Lafrenière, D., & Carr, D. A. (2001, August 5-10). A Pattern-Supported Approach to the User Interface Design Process. Paper presented at the HCI International 2001 9th International Conference on Human-Computer Interaction, New Orleans, Louisiana, United States. Grier, R. A., Kortum, P., & Miller, J. T. (2007). How users view web pages: An exploration of cognitive and perceptual mechanisms. In P. Zaphiris & S. Kurniawan (Eds.), Human Computer Interaction Research in Web Design and Evaluation (1 ed., pp. 22-41). London: IGI - Idea Group Inc. Griffiths, R., Pemberton, L., Borchers, J., & Stork, A. (2000). Pattern languages for interaction design: building momentum. Paper presented at the CHI '00 extended abstracts on Human factors in computing systems. Grudin, J. (1989). The case against user interface consistency. Commun. ACM, 32(10), 1164- 1173. Gwizdka, J., & Spence, I. (2007). Implicit measures of lostness and success in web navigation. Interacting with computers, 19, 357-369. Halberstadt, J., & Rhodes, G. (2000). The Attractiveness of Nonface Averages: Implications for an Evolutionary Explanation of the Attractiveness of Average Faces. Psychological Science, 11(4), 285-289. Hartley, J. (1978). Designing instructional text (3rd ed.). London: Kogan Page Limited. Hartley, J. (1987). Typography and executive control processes in reading. In B. K. Britton & S. M. Glynn (Eds.), Executive control processes in reading (pp. 57-79). New Jersey: Lawrence Erlbaum Associates, Inc. Hartley, J. (1994). Designing instructional text (3rd ed.). London, United Kingdom: Kogan Page Limited. Hartley, J. (2004). Designing instructional and informational text. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 917-947). New Jersey: Lawrence Erlbaum Associates, Inc. Hartley, J., & Jonassen, D. H. (1985). The role of headings in printed and electronic text. In D. H. Jonassen (Ed.), The Technology of text : principles for structuring, designing, and displaying text (Vol. 2, pp. 237-263). Englewood Cliffs, N.J.: Educational Technology Publications. Hartley, J., & Trueman, M. (1985). A research strategy for text designers: The role of headings. Instructional Science, 14(2), 99-155. Hasan, B., & Ahmed, M. U. (2007). Effects of interface style on user perceptions and behavioral intention to use computer systems. Computers in Human Behavior, 23(6), 3025. Henninger, S. (2000). A methodology and tools for applying context-specific usability guidelines to interface design. Interacting with Computers, 12(3), 225. Herder, E. (2003). Revisitation patterns and disorientation. Paper presented at the German Workshop on Adaptivity and User Modeling in Interactive Systems: ABIS 2003, Karlsruhe. Herder, E., & Dijk, B. v. (2003). From Browsing Behavior to Usability Matters. Workshop on Human Information Processing and Web Navigation, HCI 2003 Retrieved January, 2008, 352
- 375. from http://www.tm.tno.nl/pals/documents/from_browsing_behavior_to_usability_matte rs.pdf Herring, S. C., Scheidt, L. A., Bonus, S., & Wright, E. (2004). Bridging the Gap: A Genre Analysis of Weblogs. Paper presented at the Proceedings of the 37th Annual Hawaii International Conference on System Sciences, 2004, Hawaii, USA. Hirsch, S. G. (1997). How do children find information on different types of tasks ? Children's use of the Science Library Catalog. Library trends, 45(4), 725-745. Holscher, C., & Strube, G. (2000). Web search behavior of Internet experts and newbies. Computer Networks, 33(1-6), 337. Ihlström, C., & Åkesson, M. (2004). Genre characteristics - a front page analysis of 85 Swedish online newspapers. Paper presented at the 37th. Hawaii international conference on systems science, Hawaii. ISO/IEC 9241-11 (1998). Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11:Guidance on Usability. Geneva, Switzerland: International Organization for Standardization. ISO/IEC (1998). ISO 9241-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11:Guidance on Usability. Geneva, Switzerland: International Organization for Standardization. Jenkins, C., Corritore, C. L., & Wiedenbeck, S. (2003). Patterns of information seeking on the web: A qualitative study of domain expertise and web expertise. Information Technology and Society, 1(3), 64-89. Jonassen, D. H., Beissner, K., & Yacci, M. (1993). Structural knowledge: Techniques for representing, conveying and acquiring structural knowledge (1st ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Kalbach, J., & Bosenick, T. (2006). Web page layout: A comparison between left- and right-justified site navigation menus. Journal of Digital Information, 4(Vol. 4). Retrieved from http://journals.tdl.org/jodi/rt/printerFriendly/jodi-111/93 Kelly, D., & Cool, C. (2002). The Effects of Topic Familiarity on Information Search Behaviour. Paper presented at the 2nd ACM/IEEE-CS joint conference on Digital libraries, Portland, Oregon. Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences (3rd. ed.). Pacific Grove, California: Brooks/Cole. Kurosu, M., & Kashimura, K. (1995). Apparent usability vs. inherent usability: experimental analysis on the determinants of the apparent usability. Paper presented at the Conference companion on Human factors in computing systems. from file:///E:/_UK/PhD/Artigos/printed/KUROSU%20%26%20KASHIMURA%20- %20Apparent%20usability%20vs.%20Inherent%20usability.pdf Langlois, J. H., & Roggman, L. A. (1990). Attractive faces are only average. Psychological Science, 1(2), 115-121. Langlois, J. H., Roggman, L. A., & Musselman, L. (1994). What Is Average and What Is Not Average About Attractive Faces? Psychological Science, 5(4), 214-220. 353
- 376. Larson, K., & Czerwinski, M. (1998). Web page design: implications of memory, structure and scent for information retrieval. Paper presented at the SIGCHI conference on Human factors in computing systems, Los Angeles, California, United States. Lazonder, A. W., Biemans, H. J. A., & Wopereis, I. G. J. H. (2000). Differences between novice and experienced users in searchin information on the World Wide Web. Journal of the American Society for Information Science, 51(6), 576-581. Lehtonen, T., Benamar, S., Laamanen, V., Luoma, I., Ruotsalainen, O., Salonen, J., et al. (2006). Towards user-friendly mobile browsing. Paper presented at the Proceedings of the 2nd international workshop on Advanced architectures and algorithms for internet delivery and applications. Leichty, G., & Esrock, S. (2001). Change and response on the corporate web site. American Communication Journal, (5 (1)). Retrieved from http://www.acjournal.org/holdings/vol5/iss1/articles/leichtyesrock.htm Lidwell, W., Holden, K., & Butler, J. (2003). Universal Principles of Design. Gloucester: Rockport Publishers, Inc. Lim, K., Benbasat, I., & Todd, P. A. (1996). An experimental Investigation of the interactive effects of interface style, instructions and task familiarity on user performance. ACM TRANSACTIONS ON COMPUTER HUMAN INTERACTION, 3(1), 1-37. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological review, 95(4), 492-527. Loines, R. (1991). Giving readers what they want: What the research tells us about readable documents (Vol. 2). Orlando, Florida. Lorch, R. F., Jr., & Lorch, E. P. (1996). Effects of headings on text recall and summarization. Contemporary Educational Psychology, 21, 261-278. Love, H. (1985). Manuscript versus Print in the Transmission of English Literature, 1600- 1700. The Bibliographical Society of Australia and New Zealand Bulletin, 9(3), 95-106. MacKay, B., Watters, C., & Duffy, J. (2004). Web Page Transformation When Switching Devices Lecture Notes in Computer Science (Vol. 3160/2004, pp. 228-239). Berlin: Springer. Maguire, M. (2001). Context of Use within usability activities. International Journal of Human-Computer Studies, 55(4), 453. Marchionini, G. (1995). Information seeking in electronic environments (3rd edition ed.). Cambridge: Cambridge University Press. Marchionini, G., & Shneiderman, B. (1988). Finding facts vs. browsing knowledge in hypertext systems. Computer, 21(1), 70-80. Markum, J., & Hall, R. H. (2003). E-commerce web objects: Importance and expected placement. Laboratory for Information Technology Evaluation Technical Report Retrieved 23 January, 2008, from http://campus.umr.edu/lite/tech_reports/LITE-2003-02.pdf McCarthy, J. D., Sasse, A. M., & Riegelsberger, J. (2003, 8th - 12th September). Could I have the Menu Please? An Eye Tracking Study of Design Conventions. Paper presented at the HCI 2003 - Designing for Society, Bath, United Kingdom. 354
- 377. McDonald, S., & Stevenson, R. J. (1998). Navigation in hyperspace: An evaluation of the effects of navigational tools and subject matter expertise on browsing and information retrieval in hypertext. Interacting with Computers, 10(2), 129. McDougall, S. J. P., & Curry, M. B. (2004, 16 - 17 September). More than just a picture: Icon representation in context. Paper presented at the First International Workshop on Coping with Complexity, Bath, United Kingdom. McDougall, S. J. P., Curry, M. B., & De Brujin, O. (1999). Measuring Symbol and icon characteristics: norms for concretness, complexity, meaningfulness, familiarity and semantic distance for 239 symbols. Behavior research methods, instruments & computers, 31(3), 487-519. McEneaney, J. (2001). Graphic and numerical methods to assess navigation in hypertext. International Journal of Human-Computer Studies, 55, 761-786. McGann, R. (2005). Online banking increased 47 percent since 2002 Retrieved 30 november, 2005, from http://www.clickz.com/stats/sectors/finance/article.php/3481976#table Michailidou, E., Harper, S., & Bechhofer, S. (2008, September 22-24). Visual Complexity and Aesthetic Perception of Web Pages. Paper presented at the SIGDOC'08 - 26th ACM International Conference on Design of Communication, Lisbon, Portugal. Milic-Frayling, N., Jones, R., Rodden, K., Smyth, G., Blackwell, A., & Sommerer, R. (2004). Smartback: supporting users in back navigation. Paper presented at the 13th international conference on World Wide Web, New York, NY, USA. Miller, C. R. (1984). Genre as a social action. Quarterly journal of speech, 70(2), 151-167. Miniwatts Marketing Group (n.d.). Internet usage statistics - the Internet big picture Retrieved 27 November, 2008, from http://www.internetworldstats.com/stats.htm Monin, B. (2003). The warm glow heuristic: When liking leads to familiarity. Journal of Personality and Social Psychology, 85(6), 1035-1045. Morris, M. G., & Dillon, A. (1997). How user perceptions influence software use. Software, IEEE, 14(4), 58. Moyes, J. (1994). When users do and don't rely on icon shape. Paper presented at the Conference on Human Factors in Computing Systems' 94, Boston, Massachusetts, United States. Moyes, J. (1995). Putting icons in context: the influence of contextual information on the usability of icons. University of Glasgow, Glasgow. Moyes, J., & Jordan, P. W. (1993). Icon design and its effect on guessability, learnability, and experienced user performance. Paper presented at the People and Computers VIII: HCI 93, Loughborough, United Kingdom. Muter, P., & Kruk, R. S. (1984). Reading of continuous text on video screen. Human Factors, 26(3), 339-345. Muter, P., Kruk, R. S., Buttigieg, M. A., & Kang, T. J. (1988). Reader-controlled computerized presentation of text. Human Factors, 30(4), 473-486. 355
- 378. Nielsen, J. (1990). The art of navigating through hypertext. Communications of the ACM, 33(3), 296-310. Nielsen, J. (1993). Usability engineering. Boston ; London: Academic Press. Nielsen, J. (1994). Ten Usability Heuristics. The Alertbox:Current issues in web usability Retrieved 14 October, 2008, from http://www.useit.com/papers/heuristic/heuristic_list.html Nielsen, J. (1999a). Do interface standards stifle design creativity? The Alertbox: Current issues in Web usability Retrieved 30 June, 2007, from http://useit.com/alertbox/990822.html Nielsen, J. (1999b, 2004). When bad design elements become the standard. The Alertbox:Current issues in web usability Retrieved January, 2008, from http://www.useit.com/alertbox/991114.html Norman, D. (2005, July + August). Human-centered design considered harmful. Interactions, 12, 14-19. OPA (2005, October 2005). Internet activity index Retrieved 30 November, 2005, from http://www.online-publishers.org/?pg=activity Østerlund, C. S. (2006). Combining genres: How practice matters. Paper presented at the 39th Hawaii International Conference on System Sciences, Hawaii. Otter, M., & Johnson, H. (2000). Lost in hyperspace: metrics and mental models. Interacting with computers, 13, 1-40. Oulasvirta, A., Kärkkäinen, L., & Laarni, J. (2005). Expectations and memory in link search. Computer in Human Behaviour, 21, 773-789. Ozok, A. A., & Salvendy, G. (2004). Twenty guidelines for the design of Web-based interfaces with consistent language. Computers in Human Behavior, 20(2), 149. Patel, M. R. (2003). Examining web design conventions across site types. Usability News, (5.1). Retrieved from http://psychology.wichita.edu/surl/usabilitynews/51/guest.htm Patel, S. C., Drury, C. G., & Shalin, V. L. (1998). Effectiveness of expert semantic knowledge as a navigational aid within hypertext. Behaviour and Information Technology, 17(6), 313-324. Pearson, R., & van Schaik, P. (2003). The effect of spatial layout of and link colour in web pages on performance in a visual search task and an interactive search task. International Journal of Human-Computer Studies, 59(3), 327. Peels, A. J. H., Janssen, N. J. M., & Nawijn, W. (1985). Document architecture and text formatting. ACM Trans. Inf. Syst., 3(4), 347-369. Pemberton, L. (2000). Genre as a structuring concept for interaction design pattern languages. HCI-SIG Workshop on patterns Retrieved 30 October, 2008, from http://www.cmis.brighton.ac.uk/staff/lp22/genredraft.html Pollard, M. (2008). E-commerce and ICT Activity - 2007 e-commerce Survey of Business. from http://www.statistics.gov.uk/pdfdir/ecom1108.pdf. 356
- 379. Rosenberg, J. (1996). The structure of hypertext activity. Paper presented at the Seventh ACM conference on hypertext, Bethseda, Maryland, United States. Rumpradit, C., & Donnell, M. L. (1999). Navigational Cues on User Interface Design to Produce Better Information Seeking on the World Wide Web. Paper presented at the Thirty- second Annual Hawaii International Conference on System Sciences, Hawaii, USA. Salomon, G. (1989). Designing casual-user hypertext: the CHI'89 InfoBooth. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, Seattle, Washington, United States. Santini, M. (2005). Genres In Formation? An Exploratory Study of Web Pages using Cluster Analysis. Paper presented at the CLUK - 8th Annual Computational Linguistics Research Colloquium, Manchester. Santini, M. (2007). Characterizing Genres of Web Pages: Genre Hybridism and Individualization. Paper presented at the HICSS 2007 - 40th Annual Hawaii International Conference on System Sciences. Satizinger, J. W. (1991). User interface consistency across end-user application programs: effect on learning and satisfaction. Claremont Graduate School. Satzinger, J. W. (1991). User interface consistency across end-user application programs: effect on learning and satisfaction. Claremont Graduate School. Satzinger, J. W., & Olfman, L. (1998). User interface consistency across end-user applications: the effects on mental models. J. Manage. Inf. Syst., 14(4), 167-193. Scheidt, L., & Wright, E. (2004). Common visual design elements of weblogs. Into the Blogosphere: Rethoric, community and culture of weblogs Retrieved June, 2007, from http://blog.lib.umn.edu/blogosphere/common_visual.html Schmid-Isler, S. (2000). The language of digital genres - a semiotic investigation of style and iconology on the World Wide Web. Paper presented at the 33rd Hawaii International Conference on System Sciences, Maui, Hawaii. Schoon, P., & Cafolla, R. (2002). World Wide Web hypertext linkage patterns. Journal of Educational Multimedia and Hypermedia, 11(2), 117-139. Schultz, L. D., & Spyridakis, J. D. (2004). The effects of heading frequency on comprehension of online information: A study of two populations. Technical communication, 51(4), 504-516. Scott, B. M., & Schwartz, N. H. (2007). Navigational spatial displays: The role of metacognition as cognitive load. Learning and instruction, 17, 89-105. Seffah, A., & Metzker, E. (2004). The obstacles and myths of usability and software engineering. Commun. ACM, 47(12), 71-76. Selvidge, P. R., Chaparro, B. S., & Bender, G. T. (2002). The world wide wait: effects of delays on user performance. International Journal of Industrial Ergonomics, 29, 15-20. Shackel, B. (1981). The concept of usability. Paper presented at the IBM Software and Information Usability Symposium, Poughkeepsie, New York, USA. 357
- 380. Shepherd, M., & Watters, C. (1999). The Functionality Attribute of Cybergenres. Paper presented at the Proceedings of the Thirty-Second Annual Hawaii International Conference on System Sciences-Volume 2 - Volume 2. Shneiderman, B., & Plaisant, C. (2005). Designing the user interface: Strategies for effective Human-Computer Interaction (4th. ed.): Addison-Wesley, Ltd. Smith, P. A. (1996). Towards a practical measure of hypertext usability. Interacting with Computers, 8(4), 365-381. Stamboltzis, A., & Pumfrey, P. (2000). Reading across Genres: a Review of Literature. Support for Learning, 15(2), 58-61. Stanoevska-Slabeva, K., Hombrecher, A., Handschuh, S., & Schmid, B. (1998, 29-30 October). Efficient information retrieval: Tools for knowledge management. Paper presented at the Proceedings of the second european conference on practical aspects of knowledge management, Basel, Switzerland. Swales, J. M. (1990). Genre analysis: English in academic and research settings. Cambridge: Cambridge University Press. Tabatabai, D., & Shore, B. M. (2005). How experts and novices search the web. Library & Information Science Research, 27, 222-248. Tan, B., Foo, S., & Hui, S. C. (2001). Web information monitoring: an analysis of Web page updates. Online Information Review, 25(1), 6-19. Tauscher, L., & Greenberg, S. (1996). Design Guidelines for Effective WWW History Mechanisms. Paper presented at the Microsoft Workshop, Designing for the Web: Empirical Studies. Tauscher, L., & Greenberg, S. (1997). Revisitation patterns in World Wide Web navigation. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Atlanta, United States. Tidwell, J. (1999). Common Ground: A pattern language for Human-Computer Interface design Retrieved 18 November, 2008, from http://www.mit.edu/~jtidwell/common_ground_onefile.html Toms, E. G. (2001). Recognizing Digital Genres. Bulletin - American Society for Information Science. Tractinsky, N. (1997). Aesthetics and apparent usability: empirically assessing cultural and methodological issues. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems. Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127-145. Twyman, M. (1979). A schema for the study of graphic language. In P. A. Kolers, M. E. Wrolstad & H. Bouma (Eds.), Processing of Visible Language (Vol. 1). New York: Plenum Press. Twyman, M. (1982). The graphic presentation of language. Information design journal, 3(1), 2-22. 358
- 381. UPA (2005). Principles for usable design. Usability Body of Knowledge Retrieved 21 October, 2008, from http://www.usabilitybok.org/design/p287 Usborne, N. (2003). Web users crave familiarity. Excess Voice Retrieved 30 June, 2008, from http://www.excessvoice.com/usborne-articles7.htm van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York, United States: Academic Press, Inc. van Schaik, P., & Ling, J. (2001). Design Parameters in Web Pages: Frame Location and Differential Background Contrast in Visual Search Performance. International Journal of Cognitive Ergonomics, 5(4), 459 - 471. van Schaik, P., & Ling, J. (2009). The role of context in perceptions of the aesthetics of web pages over time. International Journal of Human-Computer Studies, 67, 79-89. van Welie, M., van der Veer, G., & Eliëns, A. (1999). Breaking down usability. Paper presented at the INTERACT, 1999, Edinburgh. Vaughan, M., & Dillon, A. (1998). The Role of Genre in Shaping our Understanding of Digital Documents. Paper presented at the Annual Meeting of the American Society for Information Science, Pittsburgh, PA, United States. Vaughan, M., & Dillon, A. (2006). Why structure and genre matter for users of digital information: A longitudinal experiment with readers of a web-based newspaper. International Journal of Human-Computer studies, 64, 502-526. Wallace, P. M. (1999). The Psychology of the Internet (1 ed.). New York: Cambridge University Press. Waller, R. (1979). Typographic access structures for educational texts. In P. A. Kolers, M. E. Wrolstad & H. Bouma (Eds.), Processing of visible language (Vol. 1, pp. 175 - 187). New York: Plenum Press. Waller, R. (1987a). The typographic contribution to language: towards a model of typographic genres and their underlying structures. University of Reading, Reading. Waller, R. (1987b). Typography and reading strategy. In B. K. Britton & S. M. Glynn (Eds.), Executive Control Processes in Reading (pp. 81-106). Hillsdale, New Jersey: L. Erlbaum Association. Waller, R. (1999). Making connections: typography, layout and language. Paper presented at the 1999 Autumn Symposium, American Association for Artificial Intelligence, Orlando, Florida. Walton, M., & Vukovic, V. (2003, March and April of 2003). Cultures, literacy and the web: Dimensions of information scent. Interactions, 10, 64-71. Whiteside, J., Jones, S., Levy, P. S., & Wixon, D. (1985). User performance with command, menu, and iconic interfaces. Paper presented at the Conference on Human Factors in Computing Systems, San Francisco, California, USA. Whittlesea, B. W. A. (1993). Illusions of familiarity. Journal of experimental psychology. Learning, memory, and cognition, 19(6), 1235-1253. Wildemuth, B. M. (2004). The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science and Technology, 55(3). 359
- 382. Williams, T. R., & Spyridakis, J. H. (1992). Visual discriminability of headings in text. Transactions on Professional Communication, 35(2), 64-70. Xu, G., Cockburn, A., & McKenzie, B. (2001). Lost on the web: An introduction to web navigation research. Paper presented at the 4th New Zealand Computer Science Research Students Conference, Christchurch, New Zealand. Yates, J., & Orlikowski, W. J. (1992). Genres of Organizational Communication: A Structurational Approach to Studying Communication and Media. Academy of Management Review, 17(2), 299. Yates, J., & Orlikowski, W. J. (2002). Genre Systems: Structuring Interaction through Communicative Norms. The Journal of business communication, 39(1), 13. Yates, J., Orlikowski, W. J., & Rennecker, J. (1997). Collaborative Genres for Collaboration: Genre Systems in Digital Media. Paper presented at the Proceedings of the Thirtieth Hawaii International Conference on System Sciences HICSS-97. Yates, S., & Sumner, T. (1997). Digital Genres and the New Burden of Fixity. Paper presented at the Proceedings of the 30th Hawaii International Conference on System Sciences: Digital Documents - Volume 6. Yatim, N. F. M. (2002). A combination measurement for studying disorientation. Paper presented at the 35th Hawaii International Conference on System Sciences, Hawaii. Yonelinas, A. P. (2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of memory and language, 46(3), 441-517. Zhang, J., & Ghorbani, A. A. (2004, 13-15 October 2004). Familiarity and Trust: Measuring Familiarity with a Web Site. Paper presented at the Proceedings of the 2nd Annual Conference on Privacy, Trust and Security (PST'04), New Brunswick, Canada. 360