Testing Content Effectiveness - Penn State 2015
3 likes1,685 views
The document discusses the effectiveness of web content and how it impacts user engagement and conversions. It emphasizes the importance of analyzing web traffic, defining clear goals, and understanding user behavior through metrics like page views, session duration, and event tracking. Additionally, it highlights the need for qualitative assessments to gauge user impressions and the overall effectiveness of the site's content.
![https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-
a32c0b3353d5dff3aba41c5a2a7dcd33.png
#psuweb @misb
How to ask…
After completing some information seeking tasks
Impressions about the people
Describe what kind of [customer, student, etc.]
you think is a good fit for this [organization,
product, program, etc.]
Impressions about the institution
What 3 adjectives or phrases would you use to
describe [organization or company] based on
your visit to this website?](https://image.slidesharecdn.com/testingcontentpennstate2015-150623212152-lva1-app6892/85/Testing-Content-Effectiveness-Penn-State-2015-33-320.jpg)
![https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-
a32c0b3353d5dff3aba41c5a2a7dcd33.png
#psuweb @misb
How to ask…
Specific task-based scenario
Part 1. You want to find out [this
amorphous topic]. What information do
you look for on this website?
Part 2. How do you evaluate if this
[program/product/service] fits your
needs?](https://image.slidesharecdn.com/testingcontentpennstate2015-150623212152-lva1-app6892/85/Testing-Content-Effectiveness-Penn-State-2015-44-320.jpg)
![https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-
a32c0b3353d5dff3aba41c5a2a7dcd33.png
#psuweb @misb
How to ask…
General browsing scenario
Part 1. You’d like to check out
[organization].
Think about what qualities you look for in
[that type of organization].
Part 2. Exploring this website, how do
you find out if [this organization] is a
good fit for you?](https://image.slidesharecdn.com/testingcontentpennstate2015-150623212152-lva1-app6892/85/Testing-Content-Effectiveness-Penn-State-2015-45-320.jpg)
![https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-
a32c0b3353d5dff3aba41c5a2a7dcd33.png
#psuweb @misb
How to ask…
Same question as before…
Part 1. You want to find out [this]. What
information do you look for on this
website?
Part 2. How do you evaluate if this
[college or program] fits your needs?
But repeat with 3–4 other competitors.
Randomize or place your site
somewhere in the middle.](https://image.slidesharecdn.com/testingcontentpennstate2015-150623212152-lva1-app6892/85/Testing-Content-Effectiveness-Penn-State-2015-51-320.jpg)
![https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot-
a32c0b3353d5dff3aba41c5a2a7dcd33.png
#psuweb @misb
How to ask…
Follow-up with some ranking
questions…
Which [organization or company]
seemed the most [welcoming, innovative,
goal adjective] based on their website?
Why?
Ask up to 3 of these.](https://image.slidesharecdn.com/testingcontentpennstate2015-150623212152-lva1-app6892/85/Testing-Content-Effectiveness-Penn-State-2015-52-320.jpg)
Testing Content Effectiveness - Penn State 2015
- 1. It’s What’s Inside That Counts. Testing Content Effectiveness #psuweb @misb
- 2. Hello! My name is… Melissa Beaver Director of User Experience NewCity, insidenewcity.com #psuweb @misb
- 3. Is our content doing what we think it’s doing? #psuweb @misb
- 4. Is anyone using/reading it? #psuweb @misb CrazyEgg Heatmap
- 5. Wrong place? #psuweb @misb Source: Think Education with Google, “Education Trends through the Eyes of Your Customer (April 2012), http://www.thinkwithgoogle.com/research-studies/education-trends-through-the-eyes-of-your-customer.html
- 6. Web content is free! #psuweb @misbPhoto courtesy Jesslee Cuizon, Flickr.
- 7. The Hidden Cost #psuweb @misb Editing Writing 1/2 –1 hour 1–3 hours 2–5 hours 10–20 hours to copy edit to edit or rewrite existing content to research & write original content to research & write 1 feature article
- 9. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Every page has a job to do Support conversions Answer user questions Leave users with the intended impression
- 11. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Where do I look? Identify outliers Examine top entrances Look at recurring content types Map to stakeholder & project goals
- 12. #psuweb @misb Establish your baseline ‣ Unique sessions, users, pageviews ‣ Traffic sources ‣ Bounce rate ‣ Session duration ‣ Entrances
- 13. Make fair comparisons #psuweb @misb
- 14. #psuweb @misb Define clear goals This report is base Page path level 4 All Sessions From New York From Not US All Sessions Explorer Pageviews Unique Pageviews Avg. Time on Page All Sessions 15,355 % of Total: 1.97% (780,172) 13,081 % of Total: 2.25% (581,869) 00:01 Site Avg: 00:01:05 (1 From New York 553 % of Total: 0.07% (780,172) 467 % of Total: 0.08% (581,869) 00:01 Site Avg: 00:01:05 (29 From Not US 1,915 % of Total: 0.25% (780,172) 1,647 % of Total: 0.28% (581,869) 00:01 Site Avg: 00:01:05 (21 1. /index.aspx 14,225 (92.64%) 12,116 (92.62%) 00 524 (94.76%) 439 (94.00%) 00 1,744 (91.07%) 1,493 (90.65%) 00 2. /naturalsciences.aspx 433 (2.82%) 369 (2.82%) 00 + Pageviews ( )All Sessions Pageviews ( )From New York Pageviews ( )From Not US Jun 15 Jun 29 Jul 13 500500500 1,0001,0001,000 All Sessions 1.97% From New York 0.07% From Not US 0.25% Increase… ‣ Quality? ‣ Quantity? ‣ Awareness/Visibility? For… ‣ Which programs? ‣ Which regions? ‣ Which audiences?
- 15. #psuweb @misb Be specific By Site Section/Page ‣ Navigation paths ‣ Page elements ‣ Conversions ‣ Unique pageviews by site section ‣ Bounce rate ‣ Time on page ‣ Scrolling Segment by… ‣ location ‣ device ‣ traffic source ‣ referrer ‣ custom segments
- 16. Content groups #psuweb @misb Rows 1 8 of 8 365,569 % of Total: 100.00% (365,569) 365,569 % of Total: 100.00% (365,569) 1. 96,900 26.51% 2. 77,120 21.10% 3. 63,282 17.31% 4. 50,873 13.92% 5. 30,970 8.47% 6. 30,829 8.43% 7. 8,819 2.41% 8. 6,776 1.85% Pageviews Jun 29 Jul 6 Jul 13 Jul 20 Jul 27 Aug 3 Aug 10 Aug 17 5,0005,0005,000 10,00010,00010,000 Main site sections (Content Group) Pageviews Pageviews Contribution to total: Pageviews Resources 26.5% 21.1% 8.4% 8.5% 13.9% 17.3% Homepage Research About Work & Study Sanger directory Search results (not set) © 2014 Google • Navigate to Behavior » Site Content » All Pages for an overview. (Also available in other Site Content reports.) • Select “Main site sections (Content Group)” in the “Content Grouping:” drop down, just to the right of the “Primary Dimension:” field, in between the explorer graph and the data table.
- 17. Segments #psuweb @misb • Available in most reports, at the top underneath the gray bar, series of boxes showing the applied segments. Click “+ Add Segment” • To define a new segment, click the red “+ New Segment” button. • To share your custom segment with other analytics users, select “Share” from the “Actions” menu.
- 18. Entrances by traffic type Segments #psuweb @misb
- 19. Tracking impact after big news story in national media #psuweb @misb
- 20. Any increase in traffic? #psuweb @misb Sessions Unique Pageviews
- 21. Other impacts? #psuweb @misb • Navigate to: Audience » Mobile » Overview. • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change to line graph view (far right tab in chart). • Change options to: “Sessions” for y, “Bounce Rate” for color.
- 22. Mobile & tablet traffic changed #psuweb @misb • Navigate to: Audience » Mobile » Overview. • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change to line graph view (far right tab in chart). • Change options to: “Sessions” for y, “Bounce Rate” for color. • Uncheck desktop.
- 23. Where did mobile traffic enter? #psuweb @misb • Navigate to: Behavior » Site Content » All Pages. • Change segment if desired (Mobile in this example). • In visualization: Explorer Tab » Motion Chart View (upper right corner). • Change view options to y=“Unique Pageviews”, Color = “Bounce Rate, Size = “Entrances” • Drag timeline (at bottom) to appropriate date.
- 24. … and where did they go? #psuweb @misb Previous Page Path Pageviews % Pageviews 19 82.61% 2 8.70% 1 4.35% 1 4.35% Next Page Path Pageviews % Pageviews 5 38.46% 1 7.69% 1 7.69% 1 7.69% 1 7.69% 1 7.69% 1 7.69% 1 7.69% 1 7.69% Entrances Jun 25, 2014 Sep 24, 2014: 71.14% Previous Pages Jun 25, 2014 Sep 24, 2014: 28.86% Exits Jun 25, 2014 Sep 24, 2014: 77.85% Next Pages Jun 25, 2014 Sep 24, 2014: 22.15% Navigation Summary Group pages by: Current Selection: /about/press/2014/140801.html Show rows: 10 Pageviews July 2014 August 2014 September 2014 303030 606060 Ungrouped / /about/press/ /about/people/ /about/press/2014/140721.html / /about/engagement/ /about/history/ /about/how/ /about/press/ /about/press/2014/ /about/press/2014/140730.html /about/press/2014/140805.html /workstudy/ © 2014 Google • Navigate to Behavior » Site Content » All Pages. • Change segment if desired; (mobile in this example). • In visualization, click “Navigation Summary” tab. • Change “Current Selection:” to page of interest.
- 25. Did they click on the accordion, click on the tabs, watch a video, interact with forms … ? #psuweb @misb Event Category Rows 1 12 of 12 Total Events Unique Events Event Value Avg. Value 201,402 % of Total: 100.00% (201,402) 51,269 % of Total: 100.00% (51,269) 11,647,434 % of Total: 100.00% (11,647,434) 57.83 Site Avg: 57.83 (0.00%) 1. Reading 75,421 (37.45%) 10,590 (20.66%) 11,613,031 (99.70%) 153.98 2. Accordion 63,538 (31.55%) 12,629 (24.63%) 0 (0.00%) 0.00 3. External Link 19,508 (9.69%) 14,991 (29.24%) 0 (0.00%) 0.00 4. Interest Form 13,460 (6.68%) 1,121 (2.19%) 4,727 (0.04%) 0.35 5. Engagement Module 11,324 (5.62%) 8,506 (16.59%) 0 (0.00%) 0.00 6. Event Registration Form 10,803 (5.36%) 1,853 (3.61%) 7,412 (0.06%) 0.69 7. Video 6,157 (3.06%) 631 (1.23%) 0 (0.00%) 0.00 8. Download 632 (0.31%) 595 (1.16%) 0 (0.00%) 0.00 9. Mailto 263 (0.13%) 195 (0.38%) 0 (0.00%) 0.00 10. Application 173 (0.09%) 67 (0.13%) 22,264 (0.19%) 128.69 11. Campus Visit Registration Form 110 (0.05%) 79 (0.15%) 0 (0.00%) 0.00 12. Error 13 (0.01%) 12 (0.02%) 0 (0.00%) 0.00 Total Events Pages / Session Jul 8 Jul 15 Jul 22 Jul 29 5,0005,0005,000 10,00010,00010,000 555 101010 Google Analytics Events
- 26. Is anyone watching? #psuweb @misb
- 27. But are users really watching it? #psuweb @misb Jul 5, 2014 Aug 4, 2014Top Events ALL » EVENT CATEGORY: Video Event Action Explorer Event Total Events Unique Events Event Value Avg. Value 6,157 % of Total: 3.06% (201,402) 631 % of Total: 1.23% (51,269) 0 % of Total: 0.00% (11,647,434) 0.00 Site Avg: 57.83 (100.00%) 1. Start 919 (14.93%) 627 (13.85%) 0 (0.00%) 0.00 2. 1% 794 (12.90%) 561 (12.39%) 0 (0.00%) 0.00 3. 5% 725 (11.78%) 523 (11.55%) 0 (0.00%) 0.00 4. 10% 704 (11.43%) 513 (11.33%) 0 (0.00%) 0.00 5. 20% 663 (10.77%) 490 (10.82%) 0 (0.00%) 0.00 6. 33% 622 (10.10%) 462 (10.21%) 0 (0.00%) 0.00 7. 50% 563 (9.14%) 433 (9.56%) 0 (0.00%) 0.00 8. 67% 504 (8.19%) 393 (8.68%) 0 (0.00%) 0.00 9. 90% 437 (7.10%) 345 (7.62%) 0 (0.00%) 0.00 10. Ended 226 (3.67%) 180 (3.98%) 0 (0.00%) 0.00 + Add Segment Total Events Pages / Session Jul 8 Jul 15 Jul 22 Jul 29 200200200 400400400 151515 303030 All Sessions 1.23%
- 28. Where are users clicking? #psuweb @misb CrazyEgg Heatmaps Note; Recommend cross- referencing data with Google Analytics. Relative sizes of clickable elements influence intensity. E.g., buttons with a proportionally smaller clickable area can seem more clicked than a slideshow or hero image with a large clickable area.
- 29. Did they scroll that far? #psuweb @misb CrazyEgg Scrollmaps
- 31. #psuweb @misb Perceptions Is our website making the right impression?
- 32. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Use to… See how you might be influencing users’ impressions Figure out why traffic is good but conversions are poor Include a user’s voice in design decisions Inform disagreement among stakeholders
- 33. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb How to ask… After completing some information seeking tasks Impressions about the people Describe what kind of [customer, student, etc.] you think is a good fit for this [organization, product, program, etc.] Impressions about the institution What 3 adjectives or phrases would you use to describe [organization or company] based on your visit to this website?
- 34. What students are a good fit? #psuweb @misb
- 35. “The polar opposite of me - i.e., someone who loves the outdoors and wants to be part of a tight knit community…” Describe what kind of student you think would be a good fit for this college. #psuweb @misb
- 36. Which design makes the right impression? #psuweb @misb “Seems clearer, less like a business.” “This one feels more academic”
- 37. Which design makes our institution feel like the right fit? #psuweb @misb
- 38. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Impressive “Top 3%—that’s pretty awesome.” “I now know that the Science Center is part of the University of Texas. I didn’t even notice that on the others... I did see UT but wasn’t sure what that stood for. Nice to know who they are affiliated with.”
- 39. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Too much research “He looks very scientific... I don’t get a health care impression... Seems more focused on research aspect—doesn’t make me want to choose this.”
- 40. #psuweb @misb What is a Health Science Center? Google Consumer Survey
- 41. #psuweb @misb What does UT Health Science Center do? ‣ “It provides healthcare, is a medical school, and research center, and these 3 aspects combined make for an excellent healthcare experience.” ‣ “It’s a teaching medical center that offers all sorts of resources—medical and dental care, a learning environment, and research facilities.”
- 42. #psuweb @misb Rating Helpfulness Which content do users find helpful, useful?
- 43. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Use to… Determine if content answers questions completely Gain insight into how your users answer fuzzy questions that don’t always have clear answers Understand placement in the user’s journey
- 44. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb How to ask… Specific task-based scenario Part 1. You want to find out [this amorphous topic]. What information do you look for on this website? Part 2. How do you evaluate if this [program/product/service] fits your needs?
- 45. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb How to ask… General browsing scenario Part 1. You’d like to check out [organization]. Think about what qualities you look for in [that type of organization]. Part 2. Exploring this website, how do you find out if [this organization] is a good fit for you?
- 46. “So, what did they do, exactly?” Unmoderated User Testing #psuweb @misb
- 47. I want examples… #psuweb @misb
- 48. Receptiveness #psuweb @misb “Eyesore.” Seems made up, “fake.” “Impressive…Meeting their patient’ needs—not just making money.”
- 49. #psuweb @misb Comparison Which one do users “pick” and why?
- 50. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Use to… See how your site compares to your peers or competitors in terms of usefulness (not just look and feel) Observe more realistic behavior Understand less tangible influences
- 51. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb How to ask… Same question as before… Part 1. You want to find out [this]. What information do you look for on this website? Part 2. How do you evaluate if this [college or program] fits your needs? But repeat with 3–4 other competitors. Randomize or place your site somewhere in the middle.
- 52. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb How to ask… Follow-up with some ranking questions… Which [organization or company] seemed the most [welcoming, innovative, goal adjective] based on their website? Why? Ask up to 3 of these.
- 56. Specialist content could be more helpful than you may think #psuweb @misb Show me what kind of surgeon he is
- 57. “Love/hate relationship with pictures because they were helpful but disturbing.” Unmoderated User Test #psuweb @misb
- 59. Experiment Image courtesy BigdogLHR, Flickr.
- 60. https://d14tj3xbwh88sx.cloudfront.net/assets/snapscore-screenshot- a32c0b3353d5dff3aba41c5a2a7dcd33.png #psuweb @misb Some tools in your lab kit UserTesting.com (More qualitative) ‣ Try peek.com for a free, lightweight version Try My UI Typeform VerifyApp Testing Suite Optimal Workshop Crazy Egg Heap Analytics Google Analytics Try Amazon’s Mechanical Turk to recruit participants inexpensively.
- 61. Albert Einstein “No amount of experimentation can ever prove me right; a single experiment can prove me wrong. #psuweb @misbImage Universiteit Leiden, Flickr.
- 62. I don’t know. Let’s test it. Said by you #psuweb @misb
- 63. #psuweb @misb Thank you! @misb What do you think about the future of digital communications in higher education? http://bit.ly/future-he