R in the Humanities: Text Analysis
Download as PPTX, PDF0 likes232 views
This document introduces a course on using R for text analysis, covering its basic programming functionalities and applications in computational text analysis. Participants will learn to use RStudio for analyzing long-form texts, cleaning data, and visualizing results through two sessions. Various resources, packages, and a corpus of public domain texts for analysis are also provided.
![The Basics (1/2)
Calculating
• 10 + 2 (spaces optional)
• 10 – 2
• 10 * 2
• 10 / 2
Strings and Things
• 1:50
• print(“Hello world!!”)
• [variable name] <- c(1, 2, 3)
• [variable name][2]
Meme: https://knowyourmeme.com/memes/math-lady-confused-lady](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-13-320.jpg)
![The Basics (2/2)
• Data types: character, numeric, integer, logical, complex
• Data structures: vector, list, matrix, data frame, factors
• Keep notes using #
• Need help?
• ?____________
• help()
• install.packages(“[name of package]”)
Meme: https://www.reddit.com/r/ProgrammerHumor/comments/8w54mx/code_comments_be_like](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-14-320.jpg)
![First, set your working directory: Session > Set Working Directory > Choose Directory > [folder]
install.packages(“tm”)
library(tm)
getwd()
texts <- Corpus(DirSource(“[path to working directory]”)
writeLines(as.character(texts[[4]])
?tm_map
getTransformations()
texts1 <- tm_map(texts, removePunctuation)
texts2 <- tm_map(texts1, removeNumbers)
texts3 <- tm_map(texts2, content_transformer(tolower))
texts4 <- tm_map(texts3, removeWords, stopwords(“english”))
texts_final <- tm_map(texts4, stripWhitespace)
writeLines(as.character(texts_final[[4]])
dtm <- DocumentTermMatrix(texts_final)](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-18-320.jpg)
![First, set your working directory: Session > Set Working Directory > Choose Directory > [folder]
install.packages(“tm”)
library(tm)
getwd()
texts <- Corpus(DirSource(“[path to working directory]”)
writeLines(as.character(texts[[4]])
?tm_map
getTransformations()
texts1 <- tm_map(texts, removePunctuation)
texts2 <- tm_map(texts1, removeNumbers)
texts3 <- tm_map(texts2, content_transformer(tolower))
texts4 <- tm_map(texts3, removeWords, stopwords(“english”))
texts_final <- tm_map(texts4, stripWhitespace)
writeLines(as.character(texts_final[[4]])
dtm <- DocumentTermMatrix(texts_final)](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-25-320.jpg)
![Getting word frequencies and associations:
freq <- colSums(as.matrix(dtm))
freq[1:10]
freq_d <- sort(freq, decreasing=TRUE)
freq_d[1:10]
findFreqTerms(dtm, lowfreq=100)
findAssocs(dtm, "women", 0.95)
?findAssocs](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-26-320.jpg)
![Making a bar chart (and then making it look nice):
barplot(freq_d[1:10])
?barplot
install.packages("RColorBrewer")
library(RColorBrewer)
?RColorBrewer
display.brewer.all|)
cols <- brewer.pal(8, "Spectral")
barplot(freq_d[1:10], col=cols, main="My Cool Plot", xlab="Word", ylab="Instances")](https://image.slidesharecdn.com/n8rfortextanalysisslides-210602143925/85/R-in-the-Humanities-Text-Analysis-27-320.jpg)
R in the Humanities: Text Analysis
- 1. R in the Humanities: Text Analysis Dr Leah Henrickson Lecturer in Digital Media School of Media and Communication University of Leeds L.R.Henrickson@leeds.ac.uk twitter.com/leahhenrickson
- 2. Who am I? • A Lecturer in Digital Media • A book historian • A digital humanist • Canadian 🍁 L.R.Henrickson@leeds.ac.uk twitter.com/leahhenrickson
- 4. Session 1: Gettin’ to Grips with R CC Image: https://en.wikipedia.org/wiki/File:Piratey,_vector_version.svg
- 5. Overview This course is a gentle introduction to R for text analysis. Over the course of two sessions you will be taught the basics of the powerful programming language before being provided with hands-on experience analysing long-form text in the RStudio development environment. By the end of the course, you will be able to: • Navigate the RStudio development environment • Prepare long-form prose texts for computational analysis using R • Conduct basic computational analyses of long-form prose texts • Construct and explain visualisations of computed results • Critically apply computational text analysis to complement other analytical methods To complete this course you will need to install: • R version 3.6 or higher (download at https://www.r-project.org) • RStudio Desktop: Open Source Edition 1.2 or higher (download at https://www.rstudio.com/products/rstudio)
- 6. Session 1 Agenda 1. What are R and RStudio? 2. What can R help you do? 3. A quick note about Computational Literary Analysis 4. Getting started with R 5. Cleaning text CC Image: https://pixabay.com/photos/dog-laptop-computer-glasses-2983021
- 7. What are R and RStudio? R is: • a programming language • a software environment • a really fancy calculator • free/open source Download: https://cran.r-project.org/mirrors.html RStudio is: • an integrated development environment (IDE) • a great way to make your coding experiences easier, more colourful, and more fun! Download: https://www.rstudio.com/products/rstudio/download
- 8. What can R help you do? • Count words • Find linguistic patterns within and across texts • Compare texts • Make pretty pictures But it’s still up to you to explain results. Also, is R always the most appropriate tool? CC Image: https://pixabay.com/photos/letters-tiles-word-game-crossword-4938486
- 9. A quick note about Computational Literary Analysis (CLS) CLS has a long history (for example, Father Robert Busa, ~1940s), but has been criticised for: • Misinterpretation of statistical data (Da) • Unchecked enthusiasm for technological ‘hype’ (Kirsch) • Turning literature into data and neglecting reception of works (Marche) Da, Nan Z. “The Computational Case against Computational Literary Studies.” Critical Inquiry, vol. 45, 2019, pp. 601-639. Kirsch, Adam. “Technology Is Taking Over English Departments.” The New Republic, 2014, https://newrepublic.com/article/117428/limits-digital-humanities-adam-kirsch. Accessed 21 December 2020. Marche, Stephen. “Literature Is not Data: Against Digital Humanities.” The Los Angeles Review of Books, 2012, https://lareviewofbooks.org/article/literature-is-not-data-against-digital-humanities. Accessed 21 December 2020. CC Image: https://melissaterras.org/2013/10/15/for-ada-lovelace-day-father-busas-female-punch-card-operatives
- 12. Terminal (write your script) Console (run your script) Environment (your data) Everything else!
- 13. The Basics (1/2) Calculating • 10 + 2 (spaces optional) • 10 – 2 • 10 * 2 • 10 / 2 Strings and Things • 1:50 • print(“Hello world!!”) • [variable name] <- c(1, 2, 3) • [variable name][2] Meme: https://knowyourmeme.com/memes/math-lady-confused-lady
- 14. The Basics (2/2) • Data types: character, numeric, integer, logical, complex • Data structures: vector, list, matrix, data frame, factors • Keep notes using # • Need help? • ?____________ • help() • install.packages(“[name of package]”) Meme: https://www.reddit.com/r/ProgrammerHumor/comments/8w54mx/code_comments_be_like
- 15. Tools > Global Options > Appearance (You will need to restart RStudio to apply these changes).
- 16. Let’s clean some text! CC Image: https://thenounproject.com/term/cleaning/199037
- 17. You can use whatever corpus you’d like for this course. However, I have prepared a corpus of six texts for you. You may download the corpus at http://tinyurl.com/n8texts. This corpus includes six public domain texts (1870-1914) about the women’s suffrage movement in the United States and the United Kingdom: • debate: Debate on Woman Suffrage in the Senate of the United States (https://www.gutenberg.org/ebooks/11114) • femalesuffrage: Female Suffrage: A Letter to the Christian Women of America, Susan Fenimore Cooper (https://www.gutenberg.org/ebooks/2157) • myownstory: My Own Story, Emmeline Pankhurst (https://www.gutenberg.org/ebooks/34856) • republic: Woman and the Republic, Helen Kendrick Johnson (https://www.gutenberg.org/ebooks/7300) • unexpurgated: The Unexpurgated Case Against Woman Suffrage, Almroth Wright (https://www.gutenberg.org/ebooks/5183)
- 18. First, set your working directory: Session > Set Working Directory > Choose Directory > [folder] install.packages(“tm”) library(tm) getwd() texts <- Corpus(DirSource(“[path to working directory]”) writeLines(as.character(texts[[4]]) ?tm_map getTransformations() texts1 <- tm_map(texts, removePunctuation) texts2 <- tm_map(texts1, removeNumbers) texts3 <- tm_map(texts2, content_transformer(tolower)) texts4 <- tm_map(texts3, removeWords, stopwords(“english”)) texts_final <- tm_map(texts4, stripWhitespace) writeLines(as.character(texts_final[[4]]) dtm <- DocumentTermMatrix(texts_final)
- 19. Help me! (1/3) R Communities #rstats (Twitter): https://twitter.com/hashtag/rstats Forwards: https://forwards.github.io R-Bloggers: https://www.r-bloggers.com R-Ladies: https://rladies.org r/rstats: https://www.reddit.com/r/rstats RStudio Community: https://community.rstudio.com Stack Overflow: https://stackoverflow.com/questions/tagged/r
- 20. Help me! (2/3) R Resources Matthew Jockers, Text Analysis with R for Students of Literature (New York: Springer, 2014) https://www.matthewjockers.net/text-analysis-with-r-for-students-of-literature/ LinkedIn Learning: R: https://www.linkedin.com/learning/topics/r Emmanuel Paradis, R for Beginners (2005): https://cran.r-project.org/doc/contrib/Paradis-rdebuts_en.pdf Emma Rand, ‘Reproducible Analyses in R’, N8 CIR (2020): https://n8cir.org.uk/events/event-resource/analyses-r W. N. Venables, D. M. Smith, and the R Core Team, An Introduction to R (2021): https://cran.r-project.org/doc/manuals/r- release/R-intro.pdf
- 21. Help me! (3/3) R Packages for Text Analysis corpustools (tokenised text analysis): https://cran.r-project.org/web/packages/corpustools gutenbergr (searching/downloading Project Gutenberg): https://cran.r-project.org/web/packages/gutenbergr quanteda (quantitative text analysis): https://cran.r-project.org/web/packages/quanteda/index.html stylo (stylometry): https://cran.r-project.org/web/packages/stylo syuzhet (sentiment analysis): https://cran.r-project.org/web/packages/syuzhet/index.html tidytext (a bit of everything!): https://cran.r-project.org/web/packages/tidytext tm (text mining – what we’ve done here): https://cran.r-project.org/web/packages/tm/index.html
- 22. If you’re interested in stylometry specifically… The Digital Humanities Summer Institute is offering its annual ‘Stylometry with R’ workshop FREE and ASYNCHRONOUSLY this year (14-18 June 2021)! Details and registration at https://dhsi.org/dhsi-2021-online-edition/dhsi-2021-online- edition-workshops.
- 23. Session 2: Charts, Clouds, and Confidence Image: https://pixabay.com/illustrations/rainbow-cloud-sunset-colorful-sky-5389074/
- 24. Session 2 Agenda 1. Any questions from last week? 2. Review of last week’s session (i.e. cleaning text) 3. Counting words 4. Plotting results 5. Making word clouds 6. Wrapping up CC Images: https://thenounproject.com/term/graph/21394; https://thenounproject.com/term/word-cloud/195993
- 25. First, set your working directory: Session > Set Working Directory > Choose Directory > [folder] install.packages(“tm”) library(tm) getwd() texts <- Corpus(DirSource(“[path to working directory]”) writeLines(as.character(texts[[4]]) ?tm_map getTransformations() texts1 <- tm_map(texts, removePunctuation) texts2 <- tm_map(texts1, removeNumbers) texts3 <- tm_map(texts2, content_transformer(tolower)) texts4 <- tm_map(texts3, removeWords, stopwords(“english”)) texts_final <- tm_map(texts4, stripWhitespace) writeLines(as.character(texts_final[[4]]) dtm <- DocumentTermMatrix(texts_final)
- 26. Getting word frequencies and associations: freq <- colSums(as.matrix(dtm)) freq[1:10] freq_d <- sort(freq, decreasing=TRUE) freq_d[1:10] findFreqTerms(dtm, lowfreq=100) findAssocs(dtm, "women", 0.95) ?findAssocs
- 27. Making a bar chart (and then making it look nice): barplot(freq_d[1:10]) ?barplot install.packages("RColorBrewer") library(RColorBrewer) ?RColorBrewer display.brewer.all|) cols <- brewer.pal(8, "Spectral") barplot(freq_d[1:10], col=cols, main="My Cool Plot", xlab="Word", ylab="Instances")
- 28. Making a word cloud (and then making it look nice): install.packages("wordcloud") library(wordcloud) matrix <- as.matrix(dtm) words <- sort(colSums(matrix), decreasing=TRUE) df <- data.frame(word=names(words), freq=words) ?data.frame wordcloud(words=df$word, freq=df$freq, max.words=100, random.order=FALSE, col=cols) ?wordcloud
- 29. Help me! (1/3) R Communities #rstats (Twitter): https://twitter.com/hashtag/rstats Forwards: https://forwards.github.io R-Bloggers: https://www.r-bloggers.com R-Ladies: https://rladies.org r/rstats: https://www.reddit.com/r/rstats RStudio Community: https://community.rstudio.com Stack Overflow: https://stackoverflow.com/questions/tagged/r
- 30. Help me! (2/3) R Resources Matthew Jockers, Text Analysis with R for Students of Literature (New York: Springer, 2014) https://www.matthewjockers.net/text-analysis-with-r-for-students-of-literature/ LinkedIn Learning: R: https://www.linkedin.com/learning/topics/r Emmanuel Paradis, R for Beginners (2005): https://cran.r-project.org/doc/contrib/Paradis-rdebuts_en.pdf Emma Rand, ‘Reproducible Analyses in R’, N8 CIR (2020): https://n8cir.org.uk/events/event-resource/analyses-r W. N. Venables, D. M. Smith, and the R Core Team, An Introduction to R (2021): https://cran.r-project.org/doc/manuals/r- release/R-intro.pdf
- 31. Help me! (3/3) R Packages for Text Analysis corpustools (tokenised text analysis): https://cran.r-project.org/web/packages/corpustools gutenbergr (searching/downloading Project Gutenberg): https://cran.r-project.org/web/packages/gutenbergr quanteda (quantitative text analysis): https://cran.r-project.org/web/packages/quanteda/index.html stylo (stylometry): https://cran.r-project.org/web/packages/stylo syuzhet (sentiment analysis): https://cran.r-project.org/web/packages/syuzhet/index.html tidytext (a bit of everything!): https://cran.r-project.org/web/packages/tidytext tm (text mining – what we’ve done here): https://cran.r-project.org/web/packages/tm/index.html
- 32. If you’re interested in stylometry specifically… The Digital Humanities Summer Institute is offering its annual ‘Stylometry with R’ workshop FREE and ASYNCHRONOUSLY this year (14-18 June 2021)! Details and registration at https://dhsi.org/dhsi-2021-online-edition/dhsi-2021-online- edition-workshops.
- 33. Thank you! Dr Leah Henrickson Lecturer in Digital Media School of Media and Communication University of Leeds L.R.Henrickson@leeds.ac.uk twitter.com/leahhenrickson