52 nfs

MapR NFS
7/6/2012

© 2012 MapR Technologies Storage Service Architecture 1

MapR NFS
Agenda
• Before NFS
• Exercise: Using hadoop fs
• MapR NFS Overview
• NFS Mounting Models
• Exercise: Mount the Cluster to Itself and Copy a File


MapR NFS
Objectives
At the end of this module you will be able to:
• Describe the tools that are used with HDFS and their limitations
• List the features and benefits of MapR Direct Access NFS
• Identify several use cases for NFS on Hadoop
• Describe the different models for mounting NFS volumes with MapR


Before NFS


Before NFS
 Getting work done without MapR NFS
– Example: mining web logs for meaningful data

• First: get the logs into the cluster
– Do you stop writing to the logs?
– How long does it take?
– What do you do in the meantime?

hadoop fs -put, copyfromlocal

distcp

Flume


Before NFS
 Getting work done without MapR NFS
– Example: mining web logs for meaningful data

• Next: getting answers from the data
– What tools do you use to access data in your cluster?
– Who is using these tools?

hadoop fs

copytolocal


Exercise:
hadoop fs


MapR NFS Overview


Introducing Direct Access NFS
 Drag and drop
 Open files and apps
 Watch live data come in
 Fully read/write
 Multiple concurrent reads/writes
 Easy to get data in/out
 Same namespace paths as Hadoop
 Mount with
-o tcp,ver=3,nolock


Getting Data into the Cluster with NFS
 Simply mount your cluster filesystem locally
– /mapr/my.cluster.com/myvolume/…
 Redirect (or copy) logs to cluster filesystem

log data


Using NFS with Clients
 Mount your cluster filesystem locally
– /mapr/my.cluster.com/myvolume/…
 Access all of the data in your cluster with standard applications
– Double-click a file (spreadsheet, report, etc.)
• edit, save, etc.
– Run a perl script
– grep, sed, awk, shell scripts, etc.
 Use Hadoop apps (Hive, Pig, Hbase, etc.)


Using NFS with Clients
 Data is constantly changing
 How do I view the data at a point in time?
– Easy: take a snapshot
– cd to the snapshot volume
– View the data of interest
– Note: overhead for taking a snapshot: none to minimal
• The only write overhead is the data that changed
– Recall snapshot architecture


Using NFS with Legacy Applications
 What if I have legacy applications that work with my data?
– Log analysis, reporting tools, etc.
 Mount the cluster where your tools run
– Take a snapshot if necessary
– Then use your app just as before


NFS Mounting Models


NFS Mounting Models

 Export to the world
– NFS gateway runs on selected gateway hosts
 Local server
– NFS server runs on local host
– Enables local compression and check summing
 Export to self
– NFS gateway runs on all data nodes, mounted from localhost


Export to the World

NFS
NFS
Server
NFS
Server
NFS
Server
NFS Server
Client


Local Server

Application

NFS
Server
Client

Cluster
Nodes


Universal Export to Self

Cluster Nodes

Application

NFS
Cluster Server
Node


Nodes are Identical

Application
Application
NFS
NFS
Cluster Server
Node Cluster Server
Node

Application

NFS
Cluster Server
Node


Exercise:
Mount the Cluster to
Itself and Copy a File


Questions


52 nfs

More Related Content

What's hot (20)

Similar to 52 nfs (20)

More from mapr-academy (9)

Recently uploaded (20)

52 nfs