Accessing Data Anywhere with Unified Namespace
1 like2,386 views
Jiri Simsa presented on Alluxio (formerly Tachyon), which provides a unified namespace for data access across multiple storage systems. Alluxio allows applications to access different storage systems like HDFS and S3 through the same interface. It provides a transparent namespace that maps Alluxio objects to the underlying storage, preserving file paths. Alluxio benefits applications by future-proofing them to new storage systems and enabling new workloads by facilitating data sharing across storage through its unified namespace and API.
Accessing Data Anywhere with Unified Namespace
- 1. Alluxio (formerly Tachyon): Accessing Data Anywhere with Unified Namespace Jiri Simsa June 15, 2016 @ Alluxio Meetup (hosted by Intel)
- 2. About Me • Software Engineer @ Alluxio, Inc. • PMC Member and Maintainer of Alluxio Open Source Project • Ph.D. from Carnegie Mellon University (Parallel Data Lab) • Worked at Google before joining Alluxio • Twitter: @jsimsa, Github: jsimsa 2
- 3. Outline • Motivation • Unified Namespace • Use Cases • Demo 3
- 7. Alluxio Benefits • Future-proofing your applications –applications can communicate with different storage systems, both existing and new, using the same namespace and interface –seamless integration between applications and new storage systems enables faster innovation • Enabling new workloads –one-time effort to enable an application to access many different types of storage systems and a storage system to be accessed by many different types of applications 7
- 8. Outline • Motivation • Unified Namespace • Use Cases • Demo 8
- 9. Unified Namespace an abstraction that makes it possible for applications to access different storage systems through the same interface 9
- 10. Transparent Naming •Operations over persisted Alluxio objects mapped transparently to underlying storage •Alluxio paths are preserved in storage layer Alluxio Storage System (HDFS, S3, …) alluxio://host:port/ Data Users Reports Sales Alice Bob hdfs://host:port/ Data Users Reports Sales Alice Bob 10
- 11. Multiple Storage Systems •Unified namespace for multiple data sources •Sharing of data across storage systems •API for on-the-fly mounting / unmounting Alluxio Storage System A alluxio://host:port/ Data Users Alice Bob hdfs://host:port/ Users Alice Bob Storage System B s3://host/bucket Reports Sales Reports Sales 11
- 12. Outline • Motivation • Unified Namespace • Use Cases • Demo 12
- 13. Multiple Storage / Compute 13
- 16. Outline • Motivation • Unified Namespace • Use Cases • Demo 16
- 17. Resources • Alluxio Project: http://www.alluxio.org • Development: https://github.com/Alluxio/alluxio • Meet Friends: http://www.meetup.com/Alluxio • Alluxio, Inc.: http://www.alluxio.com • Contact us: info@alluxio.com 17
- 18. Backup Slides 18