Lawry-Daniel.doc
- 1. Clarkson Honors Program Thesis Proposal Altering the AdaBoost algorithm to produce a new boosting method yielding more accurate results under the same number of repetitions. April 5, 2000 Daniel Lawry Professor Christino Tamon, Advisor Topics: Boosting is a method used implicitly to improve the accuracy of learning algorithms. Boosting's roots lie in a theoretical framework for studying machine learning called "PAC" learning model. The creators of this model: Kearns and Valiant, presented the hypothesis that a "weak" learning algorithm, an algorithm which produces results slightly better than random guessing, in the PAC model can be boosted, increasing the weak learning algorithm's accuracy and creating a "strong" learning algorithm. Currently, a boosting algorithm called AdaBoost produces the desired increase in accuracy given a weak learning algorithm. The AdaBoost method utilizes the weak learning method it is given and a training set (xl,yl),..., (xm,ym) where xi belongs to a domain X and each label yi is in some label set Y. AdaBoost then calls the weak learning algorithm repeatedly in a series of T rounds giving weights to the training sets and updating the weights of these sets each repetition by utilizing data from the last run of the weak predictor and current weights. These weights will increase or decrease as they are run through the method, yielding more accurate results based on these new weights each time the training set is run through the weak learning algorithm. It is believed that by eliminating the last k runs of the weak learning algorithm where k < t where t is the number of times the weak learning algorithm has been used so far will force this method to produce more accurate results with the same amount of repetitions. The elimination of the last k runs forces the current run's data to draw on a smaller set of output from the weak hypothesis repetitions. By doing this, the hope is that the algorithm will place more of an emphasis on the runs left in the hypothesis repetitions forcing it to have to become more accurate faster. The parameters to investigate include the appropriate value of k based on the weak learning algorithm and the number of repetitions, T, of that algorithm. The investigation will also involve developing this new boosting method and testing it against the AdaBoost method. Methodology: The new boosting method will be developed and constructed for testing purposes using the C programming language. Likewise the AdaBoost method will be constructed using the C programming
- 2. language. A formula to maximize the value of k using the number of repetitions, T, and the efficiency of the weak learning method will be formed. This formula will then be tested in conjuncture with the new boosting method and variations on the variable k. Once the optimal value of k is achieved, the two methods will be run on the same training sets and the resulting data will be compared as to see which method yields more accurate results.