Sense-making with open-ended text
2 likes686 views
This document outlines the process of sense-making with open-ended text, focusing on how to define problems, analyze qualitative data, and extract stories using various frameworks. It discusses different types of text, analytical methods like grounded theory and narrative analysis, and emphasizes the importance of linking insights to business outcomes. The presentation is part of a series aimed at enhancing understanding and communication of qualitative research results.
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Prompted and Individual
What are the most important factors to consider for the future of
sampling? [GRIT 2017 Q1 / 2 Survey]
• “Quality of sample. More clients need to understand just how much of a
problem this issue is.”
• ”Allowing for diversity among those in the sample. I want a sample that
is representaKve of my target audience”
• “Accuracy”
• “Ability to catch bots on online surveys”](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-11-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Linked Within One Person
Which agency is Innova-ve? Why do you say that?
“Lucid” “Ability to leverage plahorm to connect buyers
and sellers”
“System 1 formerly Brain Juicer” “Send out a lot informaKon about what they're
doing with webinars and case studies”
“Sawtooth” “Cukng edge technology and knowledge in
specialized field of market research”
“Happy Thinking People” “Screening and integraKng current trends into a
holisKc research approach”
[GRIT 2017 Q1 / 2 Survey]](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-13-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Linked Between People
[John] I think Saucony are the best running shoes at the moment.
[Sally] Me too!
[Pete] Yes, every Cme
[Adele] They are nice, but I think they are too expensive.
[John] I think they are worth it, they feel more comfortable
and supporCve, don’t even have to break them in.
[Sally] Yes, definitely worth it
4 people discussing
running shoes
All 4 like Saucony, but 1
person thinks they are
‘too expensive’
Humans have no
problem with ‘Me too!’
and ‘Yes, every Kme’
being about Saucony – if
the discourse is present](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-14-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Conversa-on Analysis
Q. What did you take into account when you decided to buy this new
technology?
What did we... we looked at cost, we looked at reliability and we sort of,
we compared a few different types, talked to some people that had
them.
Q. When you say you talked to some people who were they?
Some dental colleagues. There's a couple of internet sites that we talked
to some people... people had tried out some that didn't work very well.
Q. So in terms of materials either prevenKve materials or restoraKve
materials; what do you take in account when you decide which one to
adopt?
Well, that's a good quesKon. I don't know. I suppose we [laughs] look at
reliability. I suppose I've been looking at literature involved in it so I quite
like my own liSle research about that, because I don't really trust the
research that comes with the product and once again what other
denKsts are using and what they've been using and they're happy with.
I'm finding the internet, some of those internet forums are actually quite
good for new products.
Conversa-on Analysis
Pauses/Repairs/Disconnects:
Person is portraying that they are
not confident.
Restructured answer
“Well, that’s a good quesKon.” –
Indicates the quesKon was not a
good quesKon, deals with it by
saying ‘Don’t know’ and then
proceeds to answer what he/she
thinks the quesKoner is hoping to
learn.
From an example of Grounded Theory
www.biomedcentral.com/imedia/4037816045634649/supp3.doc](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-22-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Discourse Analysis
Q. What did you take into account when you decided to buy this new
technology?
What did we... we looked at cost, we looked at reliability and we sort of,
we compared a few different types, talked to some people that had them.
Q. When you say you talked to some people who were they?
Some dental colleagues. There's a couple of internet sites that we talked to
some people... people had tried out some that didn't work very well.
Q. So in terms of materials either prevenKve materials or restoraKve
materials; what do you take in account when you decide which one to
adopt?
Well, that's a good quesKon. I don't know. I suppose we [laughs] look at
reliability. I suppose I've been looking at literature involved in it so I quite
like my own liSle research about that, because I don't really trust the
research that comes with the product and once again what other denKsts
are using and what they've been using and they're happy with. I'm finding
the internet, some of those internet forums are actually quite good for new
products.
DA - Foo-ng
The role the denKst is filling?
Somebody who is not confident,
and who is doubhul about the
sources available to him/her.](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-23-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Discourse Analysis
Q. What did you take into account when you decided to buy this new
technology?
What did we... we looked at cost, we looked at reliability and we sort of, we
compared a few different types, talked to some people that had them.
Q. When you say you talked to some people who were they?
Some dental colleagues. There's a couple of internet sites that we talked to
some people... people had tried out some that didn't work very well.
Q. So in terms of materials either prevenKve materials or restoraKve
materials; what do you take in account when you decide which one to adopt?
Well, that's a good quesKon. I don't know. I suppose we [laughs] look at
reliability. I suppose I've been looking at literature involved in it so I quite like
my own liSle research about that, because I don't really trust the research
that comes with the product and once again what other denKsts are using
and what they've been using and they're happy with. I'm finding the internet,
some of those internet forums are actually quite good for new products.
DA – Repe--on
Reliability & “Internet sites”
No repeKKon of cost. Cost is a
‘preferred response’ – it is used
and discarded.](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-24-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Discourse Analysis
Q. What did you take into account when you decided to buy this new
technology?
What did we... we looked at cost, we looked at reliability and we sort of, we
compared a few different types, talked to some people that had them.
Q. When you say you talked to some people who were they?
Some dental colleagues. There's a couple of internet sites that we talked to
some people... people had tried out some that didn't work very well.
Q. So in terms of materials either prevenKve materials or restoraKve
materials; what do you take in account when you decide which one to
adopt?
Well, that's a good quesKon. I don't know. I suppose we [laughs] look at
reliability. I suppose I've been looking at literature involved in it so I quite
like my own liSle research about that, because I don't really trust the
research that comes with the product and once again what other denKsts
are using and what they've been using and they're happy with. I'm finding
the internet, some of those internet forums are actually quite good for new
products.
DA – Evalua-ve terms
I quite like my own liSle
research
I don’t really trust the research
that comes with the product
Some of those internet forums
are actually quite good for new
products](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-25-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
DA Thoughts
Q. What did you take into account when you decided to buy this new
technology?
What did we... we looked at cost, we looked at reliability and we sort of,
we compared a few different types, talked to some people that had them.
Q. When you say you talked to some people who were they?
Some dental colleagues. There's a couple of internet sites that we talked
to some people... people had tried out some that didn't work very well.
Q. So in terms of materials either prevenKve materials or restoraKve
materials; what do you take in account when you decide which one to
adopt?
Well, that's a good quesKon. I don't know. I suppose we [laughs] look at
reliability. I suppose I've been looking at literature involved in it so I quite
like my own liSle research about that, because I don't really trust the
research that comes with the product and once again what other denKsts
are using and what they've been using and they're happy with. I'm finding
the internet, some of those internet forums are actually quite good for
new products.
The story?
The denKst lacks confidence, he/she
menKons cost, but comes back to the
topic of reliability.
He/she distrusts the research from
the manufacturers, so tries to do his/
her own research, by connecKng
with people who have used the new
products, via internet forums
Sales Recommenda-on
Connect this type of denKst with
happy users. Encourage reliability
tesKmonials and SM posts.](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-26-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Holis-c Coding
HT The Research Business 1990s
Wendy Gordon, Colleen Ryan, Rebecca Wynberg et al
[GRIT 2017 Q1 / 2 Survey]](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-29-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Word Frequency and Stemming
[GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-33-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Word Clouds
[GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-34-320.jpg)
![Finding and Communica-ng the Story 2017 – Lesson 3 of 4
Sense-making with Open-ended Text - Ray Poynter
Word Trees
[GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]](https://image.slidesharecdn.com/findingthestory-sense-makingwithopen-endedtext-l32017-170908100803/85/Sense-making-with-open-ended-text-35-320.jpg)
Sense-making with open-ended text
- 2. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Agenda 1. The context for open-ended text 2. Finding stories with a frameworks approach 3. Linked and Independent text 4. Methods of analysing qual 5. CAQDA 6. Top Tips
- 3. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Series Schedule – 2017 • How to see the meaning in numbers 31 May • How to communicate numerical informaKon 5 July • Sense-making with open-ended text 7 September • Think, Feel, Do PresentaKons that change behaviour 9 November
- 4. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Open-ended Text Yes – Survey verbaKms – Text from online discussions/groups that you did not moderate – SM & blog posts – LeSers and emails from customers Not today – Images – Video – Other artefacts – Speech – Face/body – Your own notes/journals
- 5. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The Frameworks Approach 1. Define and frame the problem 2. Establish what is already known – And, what is believed/expected 3. Organise the data to be analysed 4. Apply systemaKc analysis processes 5. Extract and create the story
- 6. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Who is the project for? _________________ What is the business issue/problem that is being addressed? __________________________________________________ What does the business want to do, once it has addressed this issue? ______________________________________________________ What do we already know? Item Held by: Descrip-on 1 ______ ______ ______________ 2 ______ ______ ______________ 3 ______ ______ ______________ Assump-ons and predic-ons Who What 1. ______ ______ 2. ______ ______ Simplified
- 7. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Define the Problem A problem defined is a problem half-solved Talk to the client (ideally including end-client) – What is the background to the research? – Why is it happening now? – What does success look like? – What acKons depend on the findings? – What do people believe? What do they expect the results to be?
- 8. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Use the business ques-on as a lens • Insights need to link to outcomes and acKon • What is happening? • Why is it happening? • What changes: – Can I make? – Can I encourage others to make?
- 10. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Standalone or Linked Text? • Prompted and individual • Unprompted and individual • Linked within one person • Linked between people
- 11. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Prompted and Individual What are the most important factors to consider for the future of sampling? [GRIT 2017 Q1 / 2 Survey] • “Quality of sample. More clients need to understand just how much of a problem this issue is.” • ”Allowing for diversity among those in the sample. I want a sample that is representaKve of my target audience” • “Accuracy” • “Ability to catch bots on online surveys”
- 13. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Linked Within One Person Which agency is Innova-ve? Why do you say that? “Lucid” “Ability to leverage plahorm to connect buyers and sellers” “System 1 formerly Brain Juicer” “Send out a lot informaKon about what they're doing with webinars and case studies” “Sawtooth” “Cukng edge technology and knowledge in specialized field of market research” “Happy Thinking People” “Screening and integraKng current trends into a holisKc research approach” [GRIT 2017 Q1 / 2 Survey]
- 14. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Linked Between People [John] I think Saucony are the best running shoes at the moment. [Sally] Me too! [Pete] Yes, every Cme [Adele] They are nice, but I think they are too expensive. [John] I think they are worth it, they feel more comfortable and supporCve, don’t even have to break them in. [Sally] Yes, definitely worth it 4 people discussing running shoes All 4 like Saucony, but 1 person thinks they are ‘too expensive’ Humans have no problem with ‘Me too!’ and ‘Yes, every Kme’ being about Saucony – if the discourse is present
- 16. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The TripAdvisor Paradigm • What are the themes, elements and concepts? • Why have they wriSen it? • What sort of person are they? • What were their expectaKons? • What ‘role’ are they ‘performing’ • What elements can I triangulate? • Where are the strong feelings, beliefs, comments
- 17. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Academic versus Commercial Analysis of Qualita-ve Data Many qual techniques are used by both academic & commercial researchers , e.g. conversaKon analysis, grounded theory, etc But! – Timelines vary, commercial one day to one week, academic can be months (or years) – Success can vary, commercial = beSer business decision, academic = advancing knowledge – Purity of methodology, academic more pure, commercial more pragmaKc (oben means using hybrids)
- 18. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Approaches • Grounded Theory – created by Glaser & Strauss in the 1960s adopts a formal approach to coding the data, linking the codes into concepts, linking these into categories, and creaKng an overarching structure. Tends to require plenty of Kme. Tries to ignore exisKng theories – increasing sensiKvity to the content of the data. InducKve approach, general theories from specific observaKons. • Abduc-ve Analysis – compares the data with the theories and expectaKons, idenKfy the non-expected and leap (abduct) from these observaKons to a new theory that is sufficient and probably. • Content Analysis – is popular both with tradiKonal researchers and those seeking to computerise some or all of qualitaKve analysis. As with other approaches, the data is coded and categorised, but in content analysis the frequency of codes and categories and the frequency of links between them is taken into greater account that with most other methods. The use of ‘’counKng’ increases the importance of sampling when using content analysis.
- 19. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Approaches • Narra-ve Analysis – focuses on the enKre text, not subdivided components. Enter the text (coding/memoing), interpreKng, verifying (e.g. alternaKve explanaKons), represenKng (write the plot of the story), illustraKng (e.g. finding quotes, drawing diagrams). • Conversa-on Analysis – is one form of Discourse Analysis, CA, ConversaKon Analysis, was developed from the work of Harvey Sacks’ work in the 1960s & 1970s. CA looks at how people speak, the paSerns they use, how they create meaning, for example: turn-taking, repairs, dispreferred responses. ConversaKon analysis pays less aSenKon to what people say than the way they say it. • Thema-c Analysis – the focus is to generate themes from the data. In parKcular paSerns (e.g. codes and categories) are idenKfied in the early data (e.g. the first interviews or focus groups) and then used as tools to analyse subsequent data. One difference between themaKc and grounded theories is that grounded theory seeks to create a broader theory, themaKc analysis tends to be happy to create a narraKve to explain the data.
- 27. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Elements • Saturated analysis – keep going unKl you stop finding new/useful things • Structure – find/create an architecture to what you find • Make notes of what you find, linking back to the data, highlighKng examples • Look to support AND break hypotheses
- 28. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Computers ＆ Qualita-ve Analysis • Scissors & coloured pens è Word, Excel etc • CAQDAS – Computer Aided QualitaKve Data Analysis Sobware, e.g. Nvivo • Text analyKcs, from word clouds to Leximancer • Social Media analysis, e.g. Brandwatch & Radian 6 • Coding sobware, e.g. Ascribe • Photos and Video organising, e.g. Google Photos and Living Lens Your organisaKon’s Framework should specify the tools to be used, storage protocols, and approaches to things like memos, tags, and notes.
- 31. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Holis-c Coding • Random order of parKcipants • Sort by who is most innovaKve brand • Sort by country, and/or client/supplier, and/or number of employees • Sort by whether samples gekng beSer or worse
- 40. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Leximancer • Starts with automated opKons • Looks at words in context • Creates concepts and themes • The user can then drill down hSps://info.leximancer.com/
- 47. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Finding the Story • Use the client’s quesKon as the lens • Tag, code, memo the material as you analyse • Challenge what is known/believed • Find the main story • Find the relevant excepKons/differences • Create an overall structure, the plot • Is it good news or bad news?
- 48. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Top Tips 1. Define the problem before starKng the analysis (before starKng the research is best) 2. Understand what qualitaKve analysis is, use the paradigm of understanding TripAdvisor 3. Use the business quesKon as a lens to find the main story, the relevant excepKons 4. Create a story with a clear architecture 5. Learn how to use at least one of the computer aided tools