4.LOCAL DYNAMIC NEIGHBORHOOD BASED OUTLIER DETECTION APPROACH AND ITS FRAMEWORK FOR LARGE-SCALE DATASETS.docx
Download as DOCX, PDF0 likes55 views
The document introduces a new local outlier detection algorithm, LDNOD, designed for large-scale datasets, utilizing a stable neighborhood strategy to improve detection quality and efficiency. It addresses the limitations of traditional algorithms, which often yield low accuracy and efficiency, and combines the proposed algorithm with a k-means framework. Experimental results indicate that LDNOD delivers superior detection outcomes while efficiently managing larger datasets.
![VENKAT PROJECTS
Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110
Website: www.venkatjavaprojects.com What‘s app: +91 9966499110
EXITING SYSTEM :
The local outlier factor (LOF) [14] is the most well-known local outlier detection algorithm and
introduce the idea of local outlier first. LOF can be considered as a ratio of local densities, and a
higher LOF value indicates more likely of a local outlier. Based on the idea of LOF, many
variants of it have been proposed, such as connectivity-based outlier factor (COF) [15], local
correlation integral (LOCI) [16], influenced outlierness (INFLO) [17], local outlier probability
(LoOP) [18], local distance-based outlier factor(LDOF) [19], and so on. COF is similar to LOF,
but the former estimates the local density of a data record using a set based nearest trail (SBN-
trail) approach. Compared to LOF, COF indicates how far away a data instance shifts from a
pattern. A interesting contribution of LOCI is the LOCI plot, which summarizes a wealth of
information about the data in the vicinity of the point, and provides an intuitive understanding of
why specific data points should be recognized as outliers. When a dataset contains clusters with
different densities and they are close to each other, some traditional methods, such as LOF,
would fail to score the points at the borders of the clusters. In INFLO,both the kNN and Reverse
k-nearest neighbors (RkNN) [20,21] are combined to compute the outlier score. By using this
strategy, outliers between clusters with different densities would be detected more accurately. To
address the issue of threshold selection, LoOP uses a useful idea of outputting an anomaly
probability instead of a outlier score for a data point. Due to implicit data patterns and parameter
setting issues, existing outlier detection algorithms are ineffective on scattered real-world
datasets. Zhang et al. proposed a novel LDOF method to measure the outlier scores of objects in
scattered datasets, which uses the relative location of an instance to its kNN neighbors to
determine the degree to which the instance deviates from its neighborhood. These methods
mentioned above usually use kNN or RkNN to measure their neighborhoods. In recent years,
Zhu et al. proposed natural neighbors (NaN) [22] and natural outlier factor (NOF) [23] to
improve the robustness of local detection. The NaN is designed by integrating kNN and RkNN
when the stable searching state is satisfied, and the NOF algorithm can detection outliers without
parameter k.
DISADVANTAGES OF EXISTING SYSTEM :
1) Less accuracy
2)low Efficiency](https://image.slidesharecdn.com/4-220715105421-2dfa1713/85/4-LOCAL-DYNAMIC-NEIGHBORHOOD-BASED-OUTLIER-DETECTION-APPROACH-AND-ITS-FRAMEWORK-FOR-LARGE-SCALE-DATASETS-docx-2-320.jpg)
4.LOCAL DYNAMIC NEIGHBORHOOD BASED OUTLIER DETECTION APPROACH AND ITS FRAMEWORK FOR LARGE-SCALE DATASETS.docx
- 1. VENKAT PROJECTS Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110 Website: www.venkatjavaprojects.com What‘s app: +91 9966499110 Local Dynamic Neighborhood Based Outlier Detection Approach and its Framework for Large-Scale Datasets ABSTARCT : Local outlier detection is a hot area and great challenge in data mining, especially for large-scale datasets. On the one hand, traditional algorithms often achieve low-quality detection results and are sensitive to neighborhood size. On the other hand, they are infeasible for large-scale datasets due to at least O(N2 ) time and space complexity. In light of these, we propose a new local outlier detection algorithm, which is designed based on a new stable neighborhood strategy- dynamic references nearest neighbors (DRNN). Meanwhile, we present a new detection framework by combining the proposed approach and k-mean for large-scale datasets. Experimental results demonstrate that the proposed algorithm can produce higher quality and robust detection results compared to several classic methods. Meanwhile, the new detection framework is able to significantly improve detecting efficiency without sacrificing accuracy.
- 2. VENKAT PROJECTS Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110 Website: www.venkatjavaprojects.com What‘s app: +91 9966499110 EXITING SYSTEM : The local outlier factor (LOF) [14] is the most well-known local outlier detection algorithm and introduce the idea of local outlier first. LOF can be considered as a ratio of local densities, and a higher LOF value indicates more likely of a local outlier. Based on the idea of LOF, many variants of it have been proposed, such as connectivity-based outlier factor (COF) [15], local correlation integral (LOCI) [16], influenced outlierness (INFLO) [17], local outlier probability (LoOP) [18], local distance-based outlier factor(LDOF) [19], and so on. COF is similar to LOF, but the former estimates the local density of a data record using a set based nearest trail (SBN- trail) approach. Compared to LOF, COF indicates how far away a data instance shifts from a pattern. A interesting contribution of LOCI is the LOCI plot, which summarizes a wealth of information about the data in the vicinity of the point, and provides an intuitive understanding of why specific data points should be recognized as outliers. When a dataset contains clusters with different densities and they are close to each other, some traditional methods, such as LOF, would fail to score the points at the borders of the clusters. In INFLO,both the kNN and Reverse k-nearest neighbors (RkNN) [20,21] are combined to compute the outlier score. By using this strategy, outliers between clusters with different densities would be detected more accurately. To address the issue of threshold selection, LoOP uses a useful idea of outputting an anomaly probability instead of a outlier score for a data point. Due to implicit data patterns and parameter setting issues, existing outlier detection algorithms are ineffective on scattered real-world datasets. Zhang et al. proposed a novel LDOF method to measure the outlier scores of objects in scattered datasets, which uses the relative location of an instance to its kNN neighbors to determine the degree to which the instance deviates from its neighborhood. These methods mentioned above usually use kNN or RkNN to measure their neighborhoods. In recent years, Zhu et al. proposed natural neighbors (NaN) [22] and natural outlier factor (NOF) [23] to improve the robustness of local detection. The NaN is designed by integrating kNN and RkNN when the stable searching state is satisfied, and the NOF algorithm can detection outliers without parameter k. DISADVANTAGES OF EXISTING SYSTEM : 1) Less accuracy 2)low Efficiency
- 3. VENKAT PROJECTS Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110 Website: www.venkatjavaprojects.com What‘s app: +91 9966499110 PROPOSED SYSTEM: In this section, the proposed algorithm (LDNOD) and its framework (LDNOD-km) are introduced in details. The LDNOD can produce high-quality and robust detection results. Meanwhile, the detection framework by integrating LDNOD with k-means can handle larger- scale datasets efficiently without sacrificing accuracy. ADVANTAGES OF PROPOSED SYSTEM : 1) High accuracy 2)High efficiency
- 4. VENKAT PROJECTS Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110 Website: www.venkatjavaprojects.com What‘s app: +91 9966499110 SYSTEM ARCHITECTURE :
- 5. VENKAT PROJECTS Email:venkatjavaprojects@gmail.com Mobile No: +91 9966499110 Website: www.venkatjavaprojects.com What‘s app: +91 9966499110 HARDWARE & SOFTWARE REQUIREMENTS: HARD REQUIRMENTS : System : i3 or above Ram : 4GB Ram. Hard disk : 40GB SOFTWARE REQUIRMENTS : Operating system : Windows Coding Language : python