Scalable and adaptive data replica placement for geo distributed cloud storages
Download as DOCX, PDF0 likes76 views
This document discusses scalable and adaptive data replica placement schemes for geo-distributed cloud storage systems. It proposes using overlapping community discovery and adjustment to determine where to place data replicas in a distributed way. An offline scheme uses average read/write rates to find an initial replica placement solution. An online scheme then adaptively handles bursty requests without completely overriding the existing placement. Performance evaluations using real-world data show the approaches reduce inter-node traffic and system overhead compared to traditional centralized placement schemes.
![Venkat Java Projects
Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com
Email:venkatjavaprojects@gmail.com
Scalable and Adaptive Data Replica Placement for Geo-Distributed
Cloud Storages
Abstract:
In geo-distributed cloud storage systems, data replication has been widely used to serve the ever
more users around the world for high data reliability and availability. How to optimize the data
replica placement has become one of the fundamental problems to reduce the inter-node traffic
and the system overhead of accessing associated data items. In the big data era, traditional
solutions may face the challenges of long running time and large overheads to handle the
increasing scale of data items with time-varying user requests. Therefore, novel offline
community discovery and online community adjustment schemes are proposed to solve the
replica placement problem in a scalable and adaptive way. The offline scheme can find a replica
placement solution based on the average read/write rates for a certain period of time. The
scalability can be achieved as 1) the computation complexity is linear to the amount of data items
and 2) the data-node communities can evolve in parallel for a distributed replica placement.
Furthermore, the online scheme is adaptive to handle the bursty data requests, without the need
to completely override the existing replica placement. Driven by realworld data traces, extensive
performance evaluations demonstrate the effectiveness of our design to handle large-scale
datasets.
Index Terms—Geo-distributed storage system, data replica placement, scalability, adaptivity,
community discovery
Existing System:
Apart from the inter-node traffic, the storage locations of data replicas may also affect the system
overhead of accessing associated data items [4], [5]. It is worth noting that users may request
multiple data items in one transaction. For example, in online analytical processing (OLAP)
systems, a query may be executed by accessing multiple data blocks [6]. The system overhead
could be reduced if fewer storage nodes are involved to handle such a request. The reason is that
a certain overhead, e.g., the establishment of TCP connections, will be introduced if the read
request is dispatched to a storage node. In short, data replica placement reduces the system
overhead by placing associated data items together in the same storage location. With the
increasing number of data items, how to choose the proper number and storage locations of data
replicas becomes a critical issue.](https://image.slidesharecdn.com/99-200930134852/85/Scalable-and-adaptive-data-replica-placement-for-geo-distributed-cloud-storages-1-320.jpg)
![Venkat Java Projects
Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com
Email:venkatjavaprojects@gmail.com
Various data replica placement schemes have been proposed to seek optimal data storage
locations, which are typically implemented in a centralized/offline way: At every distributed
storage node handling the user requests, the data access logs are captured. Then, a central
controller is deployed to collect all logs and analyze the request frequency of each data item. The
extracted information is fed into the replica placement algorithms, e.g., mathematical
programming [8] and graph partitioning [5], [7], [9], which finally output the storage locations of
data replicas. These centralized/offline schemes can iteratively approximate the optimal solutions
with high accuracy.
Proposed System
In this paper, based on the overlapping community discovery and adjustment, we design scalable
and adaptive data replica placement schemes in geo-distributed cloud storage systems. A data-
node community is defined as the group of a storage node and all data items placed at it, which
should have more internal data access requests than external ones. Therefore, a more compact
community structure means more data requests are served locally with lower system overhead
and less inter-node traffic. Unlike traditional centralized placement schemes, communities can
evolve to decide whether each data replica should be placed at the node in a parallel and adaptive
way. The scalability of our design can be achieved by this distributed implementation along with
the computation complexity linear to the amount. of data items. Our major contributions in this
paper include:
A novel distributed overlapping community discovery scheme is proposed to solve the data
replica placement problem in a scalable way. This offline scheme can find a replica placement
solution based on the average read/write rates for a certain time period.
Guided by the offline scheme, an online community adjustment scheme is proposed to
adaptively handle the bursty requests.
The worst-case performance guarantees of the proposed schemes are provided via theoretical
analysis.
Extensive evaluation results driven by real-world data traces show the superiority of our design
over the state-of-the-art replica placement methods.
SYSTEM REQUIREMENTS:
HARDWARE REQUIREMENTS:
• PROCESSOR : I3.](https://image.slidesharecdn.com/99-200930134852/85/Scalable-and-adaptive-data-replica-placement-for-geo-distributed-cloud-storages-2-320.jpg)
Scalable and adaptive data replica placement for geo distributed cloud storages
- 1. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com Scalable and Adaptive Data Replica Placement for Geo-Distributed Cloud Storages Abstract: In geo-distributed cloud storage systems, data replication has been widely used to serve the ever more users around the world for high data reliability and availability. How to optimize the data replica placement has become one of the fundamental problems to reduce the inter-node traffic and the system overhead of accessing associated data items. In the big data era, traditional solutions may face the challenges of long running time and large overheads to handle the increasing scale of data items with time-varying user requests. Therefore, novel offline community discovery and online community adjustment schemes are proposed to solve the replica placement problem in a scalable and adaptive way. The offline scheme can find a replica placement solution based on the average read/write rates for a certain period of time. The scalability can be achieved as 1) the computation complexity is linear to the amount of data items and 2) the data-node communities can evolve in parallel for a distributed replica placement. Furthermore, the online scheme is adaptive to handle the bursty data requests, without the need to completely override the existing replica placement. Driven by realworld data traces, extensive performance evaluations demonstrate the effectiveness of our design to handle large-scale datasets. Index Terms—Geo-distributed storage system, data replica placement, scalability, adaptivity, community discovery Existing System: Apart from the inter-node traffic, the storage locations of data replicas may also affect the system overhead of accessing associated data items [4], [5]. It is worth noting that users may request multiple data items in one transaction. For example, in online analytical processing (OLAP) systems, a query may be executed by accessing multiple data blocks [6]. The system overhead could be reduced if fewer storage nodes are involved to handle such a request. The reason is that a certain overhead, e.g., the establishment of TCP connections, will be introduced if the read request is dispatched to a storage node. In short, data replica placement reduces the system overhead by placing associated data items together in the same storage location. With the increasing number of data items, how to choose the proper number and storage locations of data replicas becomes a critical issue.
- 2. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com Various data replica placement schemes have been proposed to seek optimal data storage locations, which are typically implemented in a centralized/offline way: At every distributed storage node handling the user requests, the data access logs are captured. Then, a central controller is deployed to collect all logs and analyze the request frequency of each data item. The extracted information is fed into the replica placement algorithms, e.g., mathematical programming [8] and graph partitioning [5], [7], [9], which finally output the storage locations of data replicas. These centralized/offline schemes can iteratively approximate the optimal solutions with high accuracy. Proposed System In this paper, based on the overlapping community discovery and adjustment, we design scalable and adaptive data replica placement schemes in geo-distributed cloud storage systems. A data- node community is defined as the group of a storage node and all data items placed at it, which should have more internal data access requests than external ones. Therefore, a more compact community structure means more data requests are served locally with lower system overhead and less inter-node traffic. Unlike traditional centralized placement schemes, communities can evolve to decide whether each data replica should be placed at the node in a parallel and adaptive way. The scalability of our design can be achieved by this distributed implementation along with the computation complexity linear to the amount. of data items. Our major contributions in this paper include: A novel distributed overlapping community discovery scheme is proposed to solve the data replica placement problem in a scalable way. This offline scheme can find a replica placement solution based on the average read/write rates for a certain time period. Guided by the offline scheme, an online community adjustment scheme is proposed to adaptively handle the bursty requests. The worst-case performance guarantees of the proposed schemes are provided via theoretical analysis. Extensive evaluation results driven by real-world data traces show the superiority of our design over the state-of-the-art replica placement methods. SYSTEM REQUIREMENTS: HARDWARE REQUIREMENTS: • PROCESSOR : I3.
- 3. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com • Hard Disk : 40 GB. • Ram : 2 GB. SOFTWARE REQUIREMENTS: • Operating system : Windows. • Coding Language : JAVA/J2EE • Data Base : MYSQL • IDE :Netbeans8.1