Arcomem training Specifying Crawls Beginners

Editor's Notes

• #2: {"16":"Netarchivesuite (http://netarchive.dk/suite/) developed by the two national deposit libraries in Denmark, The Royal Library and The State and University Library\nto plan, schedule and run web harvests for selective and broad crawl\nbuilt-in bit preservation functionality\nWeb curator tool: http://webcurator.sourceforge.net\nOpen-source workflow management application for selective web archiving developped by the National Library of New Zealand and the British Library, initiated by the International Internet Preservation Consortium\nArchive-it http://www.archive-it.org/\nA subscription service by Internet Archive to build and preserve collections: allows to\nharvest, catalog, manage and browse archived collections\nArcomem crawler cokpit\n","33":"On the top of the page, a progression bar gives an estimation of crawl progress until completion. It is a ratio between seen and unseen URL recorded by the crawler. Seen URLs are all the URLs which have been already crawled. Unseen are the URLs, which have been discovered but are waiting to be crawled \n","22":"http://www.arcomem.eu/wp-content/uploads/2012/05/D5_2.pdf\n","28":"For each campaign, the archivist can select which SMC, he wants to focus on (blogs, video, discussion) and he does the same for the API crawler (Facebook, Twitter, Flickr, YouTube…).\n","6":"There is several protocol : \nMai protocol as \nPOP3 (post office protocol version 3)\nSMTP (simple mail transfer protocol\nDNS Domain name service\nDHCP Dynamic Host configuration \nFTP File transfer Protocole\nIMAP Internet Message Access Protocole\n","13":"A lot of data are stored in DB hidden to search engine like google are not available for such engine,\nmoreover many pages are created dynamicaly to answer to queries so hey do not existbefor user requested information. \nThis enorme reservoir \nhttp://www.dailymotion.com/video/x9udyo_the-virtual-private-library-and-dee_news\n","8":"URI Uniform Resource Identifier (URI) is a string of characters used to identify a name or a resource on the Internet.\n","25":"A crawl is guided by the crawl specifications defined by the user. The crawl specification contains URLs to start the discovery from seeds, keywords to look for in web pages, social web sites APIs to query (and with which keywords) and Social Media Categories (SMC) to focus the crawl on. The seeds get fetched, and the corresponding content and the social sites API query responses are inserted into the document store. The insertion triggers the online analysis process. The Web resources and the links extracted from them are analyzed and scored by the Online Analysis Modules. The links get sent to the crawler’s URL queue, where their score is used to determine the order in which they should be crawled, thereby guiding the crawler. The newly crawled content gets written to the document store, completing the loop. On top of the prototype, a UI allows the user to target topics to archive and offers some analyses of collected data.\n","31":". The information is organized following different topics: general description of the campaign, metadata, current status, crawl activity, statistics and analysis \n","20":"http://www.arcomem.eu/wp-content/uploads/2012/05/D5_2.pdf\n","4":"To find an information online, I have to know is address. \nLe système de nom de domaine (Domain Name System - DNS) aide les utilisateurs à naviguer sur Internet. Chaque ordinateur relié à Internet a une adresse unique appelée “adresse IP” (adresse de protocole Internet). Étant donné que les adresses IP (qui sont des séries de chiffres) sont difficiles à mémoriser, le DNS permet d’utiliser à la place une série de lettres familières (le “nom de domaine”). Par exemple, au lieu de taper “192.0.34.163,” vous pouvez taper “www.icann.org.”\n","10":"On line information heterogeneous\nthere is copy online \n"}