Neeraj Agrawal

Data presented on commerce sites runs into thousands of pages, and is typically delivered from multiple back-end sources. This makes it difficult to identify incorrect, anomalous, or interesting data such as $9.99 air fares, missing links, drastic changes in prices and addition of new products or promotions. We describe a system that monitors Web sites automatically and generates various types of reports so that the content of the site can be monitored and the quality maintained. The solution… Show more

Data presented on commerce sites runs into thousands of pages, and is typically delivered from multiple back-end sources. This makes it difficult to identify incorrect, anomalous, or interesting data such as $9.99 air fares, missing links, drastic changes in prices and addition of new products or promotions. We describe a system that monitors Web sites automatically and generates various types of reports so that the content of the site can be monitored and the quality maintained. The solution designed and implemented by us consists of a site crawler that crawls dynamic pages, an information miner that learns to extract useful information from the pages based on examples provided by the user, and a reporter that can be configured by the user to answer specific queries. The tool can also be used for identifying price trends and new products or promotions at competitor sites. A pilot run of the tool has been successfully completed at the ibm.com site. Show less

Many enterprises are beginning to exploit the vast amount
of data available on the Web, to streamline their business
processes and gain advantage over their competitors. However, building text analytic applications that provide such
vital business information, is very hard. Further, there are
several functionalities that are common across many text
analytic applications. In this paper, we provide a platform
called TAP (Text Analytic Platform), that provides several
tools… Show more

Many enterprises are beginning to exploit the vast amount
of data available on the Web, to streamline their business
processes and gain advantage over their competitors. However, building text analytic applications that provide such
vital business information, is very hard. Further, there are
several functionalities that are common across many text
analytic applications. In this paper, we provide a platform
called TAP (Text Analytic Platform), that provides several
tools and services that are used commonly across many text
analytic applications. TAP could be used by business enterprises to build text analytic applications rapidly. It uses
WebFountain to gather the application-specific data and
provide other tools that help in developing and deploying
application-specific miners. Show less

Structural information (such as layout and look-and-feel) has been extensively used in the literatuce for extraction of interesting or relevant data, efficient storage, and query optimization. Traditionally, tree models (such as DOM trees) have been used to represent structural information, especially in the case of HTML and XML documents. However, computation of structural similarity between documents based on the tree model is computationally expensive. In this paper, we propose an… Show more

Structural information (such as layout and look-and-feel) has been extensively used in the literatuce for extraction of interesting or relevant data, efficient storage, and query optimization. Traditionally, tree models (such as DOM trees) have been used to represent structural information, especially in the case of HTML and XML documents. However, computation of structural similarity between documents based on the tree model is computationally expensive. In this paper, we propose an alternative scheme for representing the structural information of documents based on the paths contained in the corresponding tree model. Since the model includes partial information about parents, children and siblings, it allows us to define a new family of meaningful (and at the same time computationally simple) structural similarity measures. Our experimental results based on the SIGMOD XML data set as well as HTML document collections from ibm.com, dell.com, and amazon.com show that the representation is powerful enough to produce good clusters of structurally similar pages Show less