SchemaCMD - An XML-based storage schema for the compilation of mixed-source CMD corpora
Download as ODP, PDF0 likes589 views
The document discusses issues related to classifying digital genres, particularly blogs, emphasizing the variability in communicative purposes and community roles. It introduces a web-based corpus tool for analyzing blog content and details the structure and methodology implemented for indexing and analyzing language data. Additionally, it highlights the challenges of genre labels and suggests the potential of using meta-data to better classify diverse digital genres.
![SchemaCMD An XML-based storage schema for the compilation of mixed-source CMD corpora Cornelius Puschmann University of Düsseldorf [email_address] Towards a Reference Corpus of Web Genres University of Birmingham 27 July 2007](https://image.slidesharecdn.com/schemacmd-an-xmlbased-storage-schema-for-the-compilation-of-mixedsource-cmd-corpora589/85/SchemaCMD-An-XML-based-storage-schema-for-the-compilation-of-mixed-source-CMD-corpora-1-320.jpg)
![A sample RSS 2.0 feed <?xml version="1.0" encoding="utf-8"?> <rss version="2.0"> <channel> <title>Yahoo! Search Blog</title> <link>http://www.ysearchblog.com/</link> <description>A look inside the world of search from the people at Yahoo!</description> <language>en</language> <copyright>Copyright 2007</copyright> <lastBuildDate>Thu, 19 Jul 2007 12:32:38 -0800</lastBuildDate> <generator>http://www.sixapart.com/movabletype/?v=3.2ysb5-20051201</generator> <docs>http://blogs.law.harvard.edu/tech/rss</docs> <item> <title>Weather Report: Yahoo! Search update</title> <description><![CDATA[<p>We've been rolling out some changes to our fresh web data and crawling, indexing and ranking...]]></description> <link>http://www.ysearchblog.com/archives/000470.html</link> <guid>http://www.ysearchblog.com/archives/000470.html</guid> <category>Weather Report</category> <pubDate>Thu, 19 Jul 2007 12:32:38 -0800</pubDate> </item> <item> ...](https://image.slidesharecdn.com/schemacmd-an-xmlbased-storage-schema-for-the-compilation-of-mixedsource-cmd-corpora589/85/SchemaCMD-An-XML-based-storage-schema-for-the-compilation-of-mixed-source-CMD-corpora-10-320.jpg)
![Example: high f-score (press release) The Toshiba Portege R400 is a Windows Vista-inspired signature mobile PC that incorporates innovative connectivity and display technologies to provide timely access to e-mail and appointments via Active Notifications and is built on Windows SideShow™ technology. [...] http://www.microsoft.com/presspass/press/2007/jan07/01-07CES2007PR.mspx high noun frequency high adjective frequency more nominal than verbal often relate complex information often describe future events/potentiality](https://image.slidesharecdn.com/schemacmd-an-xmlbased-storage-schema-for-the-compilation-of-mixedsource-cmd-corpora589/85/SchemaCMD-An-XML-based-storage-schema-for-the-compilation-of-mixed-source-CMD-corpora-17-320.jpg)
![Example: low f-score (blog entry) OK, OK, I'm partly at fault here. But, hear me out. Last year at Gnomedex I had my son demonstrate Second Life up on stage while I was hosting a panel discussion. Someone from Linden Labs (the folks who make Second Life), Beth Goza (she now works at Microsoft), saw that, and told me and my son to knock it off. People under 18 aren't allowed in Second Life. So, what did I do? I just told Patrick never to go into Second Life and I didn't go back into Second Life either. [...] http://scobleizer.com/2007/02/18/second-life-has-my-credit-card-and-wont-let-go/ high frequency of personal pronouns more verbal than nominal often describe past events, personal impressions, feelings](https://image.slidesharecdn.com/schemacmd-an-xmlbased-storage-schema-for-the-compilation-of-mixedsource-cmd-corpora589/85/SchemaCMD-An-XML-based-storage-schema-for-the-compilation-of-mixed-source-CMD-corpora-18-320.jpg)
![SchemaCMD An XML-based storage schema for the compilation of mixed-source CMD corpora Cornelius Puschmann University of Düsseldorf [email_address] Towards a Reference Corpus of Web Genres University of Birmingham 27 July 2007](https://image.slidesharecdn.com/schemacmd-an-xmlbased-storage-schema-for-the-compilation-of-mixedsource-cmd-corpora589/85/SchemaCMD-An-XML-based-storage-schema-for-the-compilation-of-mixed-source-CMD-corpora-24-320.jpg)
SchemaCMD - An XML-based storage schema for the compilation of mixed-source CMD corpora
- 1. SchemaCMD An XML-based storage schema for the compilation of mixed-source CMD corpora Cornelius Puschmann University of Düsseldorf [email_address] Towards a Reference Corpus of Web Genres University of Birmingham 27 July 2007
- 2. Contents of this presentation Problems of classifying digital genres Granularity and meta-data in blogs An example: the corporate web log corpus Variation and genre Individuated CL?
- 3. A) Problems of classifying digital genres
- 4. Approaches to genre and text typology Genre (abstract meta-concept) “a class of communicative events with a shared set of communicative purposes” (Swales) “socially recognized types of communicative actions that are habitually enacted by members of a community to realize particular social purposes” (Yates & Orlikowski) Text typology (concrete instantiations) “Linguistic features, their co-occurrence and relative distribution in a text” (Biber, paraphrase)
- 5. A faceted classification scheme (Herring 2007) Situational factors (8) Participation structure Participant characteristics Purpose ... Medium factors (10) Synchronicity Message transmission Persistence of transcript ... = discourse community and communicative purpose = creation and presentation
- 6. Aspects of digital genres concrete abstract
- 7. Are blogs a genre? discourse community and communicative purpose are both highly variable aspects of digital genres creation and presentation are relative stable aspects and the result of design choices made by software developers
- 8. B) Granularity and meta-data in blogs
- 9. Blog content syndication blog content is usually available via web feeds RSS 0.91, 0.92, 1.0, 2.0 Atom 1.0 RSS and Atom are based on XML (and can thus be extended) RSS is mostly for blogs, Atom is for content syndication in general
- 10. A sample RSS 2.0 feed <?xml version="1.0" encoding="utf-8"?> <rss version="2.0"> <channel> <title>Yahoo! Search Blog</title> <link>http://www.ysearchblog.com/</link> <description>A look inside the world of search from the people at Yahoo!</description> <language>en</language> <copyright>Copyright 2007</copyright> <lastBuildDate>Thu, 19 Jul 2007 12:32:38 -0800</lastBuildDate> <generator>http://www.sixapart.com/movabletype/?v=3.2ysb5-20051201</generator> <docs>http://blogs.law.harvard.edu/tech/rss</docs> <item> <title>Weather Report: Yahoo! Search update</title> <description><![CDATA[<p>We've been rolling out some changes to our fresh web data and crawling, indexing and ranking...]]></description> <link>http://www.ysearchblog.com/archives/000470.html</link> <guid>http://www.ysearchblog.com/archives/000470.html</guid> <category>Weather Report</category> <pubDate>Thu, 19 Jul 2007 12:32:38 -0800</pubDate> </item> <item> ...
- 11. C) An example: the corporate web log corpus
- 12. The corpus tool web-based, runs on Apache, PHP and MySQL a researcher can point the tool to a blog (or any source exposing a web feed) which is then indexed indexing is initiated manually, but this could be automated with a cron job exploration of sources could also be automated (e.g. by using blogger.com's “random blog” feature, or by implementing the Google Data API)
- 13. MySQL data structure - sources - BNC top 100 words - sub-types of corp. blogs - blogs, press eds., ... - n-grams (not computed due to cost) - POS frequencies by post - post data (via RSS/Atom) - additional post statistics - tokens (depends on types) - types (string + POS)
- 14. Corpus data feeds are used to retrieve, store and analyze language data implemented TreeTagger for automated POS annotation 161 sources (133 corporate blogs, 18 personal, 1 political*, 1 technical**) 3 press editorial sections (New York Times, Washington Post, LA Times) 5 press release sections (Microsoft, GM, Sun, Oracle, McDonald's) 29,528 posts 7,821,317 tokens Note: a much bigger corpus could easily be built using the tool, provided there is access to the right amount of computational resources.
- 15. D) Variation and genre
- 16. Measuring text formality via f-scores F-score (Heylighen & Dewaele) a metric to quantify the level of formality in a text, where formality is specifically defined as context-independence f = 0.5 * ((N + ADJ + PRP + DET) - (PN + V + ADV + ITJ) + 100)
- 17. Example: high f-score (press release) The Toshiba Portege R400 is a Windows Vista-inspired signature mobile PC that incorporates innovative connectivity and display technologies to provide timely access to e-mail and appointments via Active Notifications and is built on Windows SideShow™ technology. [...] http://www.microsoft.com/presspass/press/2007/jan07/01-07CES2007PR.mspx high noun frequency high adjective frequency more nominal than verbal often relate complex information often describe future events/potentiality
- 18. Example: low f-score (blog entry) OK, OK, I'm partly at fault here. But, hear me out. Last year at Gnomedex I had my son demonstrate Second Life up on stage while I was hosting a panel discussion. Someone from Linden Labs (the folks who make Second Life), Beth Goza (she now works at Microsoft), saw that, and told me and my son to knock it off. People under 18 aren't allowed in Second Life. So, what did I do? I just told Patrick never to go into Second Life and I didn't go back into Second Life either. [...] http://scobleizer.com/2007/02/18/second-life-has-my-credit-card-and-wont-let-go/ high frequency of personal pronouns more verbal than nominal often describe past events, personal impressions, feelings
- 19. F-score over time for two sources light blue = Jonathan Schwartz (blog); dark blue = New York Times (editorials)
- 20. F-score and standard deviation for all sources x-axis = stdev; y-axis = f-score; dot size = number of posts editorials press releases
- 22. Observations while some (established) genres are highly conventionalized (and therefore distinctive) in regards to their style and content , blogs are not taking into account that community and purpose in blogs are highly diversified, it is doubtful whether this will change variability – in all four aspects – may be a constitutive feature of blogs O1: digital genre labels are difficult to assign using traditional categories O2: however, the availability of meta-data ( author, time of creation, tags ) could solve the problem of a permanently unstable genre ecology O3: perhaps we should consider building web-data corpora that are speaker-diversified in addition to being register-diversified ?
- 24. SchemaCMD An XML-based storage schema for the compilation of mixed-source CMD corpora Cornelius Puschmann University of Düsseldorf [email_address] Towards a Reference Corpus of Web Genres University of Birmingham 27 July 2007