Distributed in memory processing of all k nearest neighbor queries
0 likes70 views
The document discusses Spitfire, a distributed algorithm designed for efficient all k nearest neighbor (aknn) computations in real-time applications involving large datasets. It contrasts traditional centralized methods with its scalable framework, emphasizing a fast load-balanced partitioning and effective replication-set selection to improve performance. Evaluation shows that Spitfire significantly reduces communication and response times compared to existing state-of-the-art aknn algorithms.
Related topics:1 of 1
Download to read offline
More Related Content
What's hot (14)
Ad
Viewers also liked (15)
Ad
Similar to Distributed in memory processing of all k nearest neighbor queries (20)
More from ieeepondy (20)
Distributed in memory processing of all k nearest neighbor queries
- 1. Distributed In-Memory Processing of All k Nearest Neighbor Queries Abstract: A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. Most traditional techniques for computing AkNN queries are centralized, lacking both scalability and efficiency. Only recently, distributed techniques for shared-nothing cloud infrastructures have been proposed to achieve scalability for large datasets. These batch-oriented algorithms are sub- optimal due to inefficient data space partitioning and data replication among processing units. In this paper, we present Spitfire, a distributed algorithm that provides a scalable and high-performance AkNN processing framework. Our proposed algorithm deploys a fast load-balanced partitioning scheme along with an efficient replication-set selection algorithm, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three other state-of-the-art distributed AkNN algorithms executed in distributed main-memory.