A Method for Analysing Large-Scale UGC Data for Tourism: Application to the Case of Catalonia

Editor's Notes

• #6: Catalonia is not an Anglophone region, and therefore the problems related to character codification beyond ASCII 127 should be considered and, specifically, those related to existing accent marks in destination and tourist attraction factor names.
• #7: The first thing after selecting the methodology is to The problem is that blogs come from diverse sources and websites do not have homogeneous structures, which makes it impossible to automatize the process of downloading, classification and refinement, as intended in this study. WEBSITES WITH A CRITERION: at least 100 entries about Catalonia
• #8: The first thing after selecting the methodology is to The problem is that blogs come from diverse sources and websites do not have homogeneous structures, which makes it impossible to automatize the process of downloading, classification and refinement, as intended in this study. WEBSITES WITH A CRITERION: at least 100 entries about Catalonia
• #9: HTTrack Website Copier (www.httrack.com). The first step to download data is to navigate the selected websites manually to identify the initial pages, that is to say, those containing hyperlinks which lead to the individual blogs and OTR pages, and save their complete URLs. A Level 0 filter only downloads the page indicated by the initial URL, a Level 1 filter, downloads that page and all the resources directly linked to it, etc; 2. The file type filter allows to download, for example, only HTML files, and the remaining files (multimedia, PDF, etc.) will only be visualized if an Internet connection is available; this system is ideal to analyse the textual content of diaries saving space in the local disk; 3. The URL filters allow to act at any part of it (protocol, server, domain subdirectories or folders, filename and file type); and 4. The content filter is the least efficient because it is necessary to download the page to assess whether or not it contains the chain of key characters, while with URL filters only the pages of interest are downloaded (Figure 2). For example, in the case of TB, it is sufficient to place an inclusive folder filter: /Catalonia/, with no level limit, because the server has a hierarchical territorial structure of folders to store the files. Conversely, in the case of TA, all the files of interest contain the word Catalonia, those which have hyperlinks which lead to OTRs start with Attraction, and those of the same OTRs start with ShowUserReview; therefore, a couple of inclusive filename filters are enough: Attraction*Catalonia and ShowUserReview*Catalonia. To understand the importance of the filters in this case, we ought to bear in mind that TA reached the figure of more than 170,000,000 reviews and opinions, and all its webpages are linked at different levels by hyperlinks.
• #10: Geography: TAcode;VTcode;Destination;Brand g187496;c1;Catalonia;unCla g187497;430de;Barcelona;Barna g494960;402fe;Lloret-de-Mar;cBrav LANGUAGE DETECTION: plain text  HTML As Text Once the CSV files are ready, a batch programme (Marine-Roig, 2013: Annex A3) is run for each website, which goes through all files, extracts internal data such as the date of the diary and the name of the destination, eliminates entries without narrative content (more than 70,000 OTRs in the case of TA), changes the format of such dates to yyyymmdd, creates new territorial directories, and transfers the diary to the destination folder already with its articulated name to facilitate future classifications. Finally, the two-character ISO 639-1 codes are introduced in the name of the files, after the date (Figure 3).
• #11: Eliminating noise. The original HTML format should be preserved in order to be able to weight keywords and key phrases according to their potential impact SITE CONTENT ANALYZER, frequency, sitewide-density, weight