Developing apps and_integrating_with_gluster_fs_-_libgfapi
2 likes632 views
This document discusses developing applications and integrating with GlusterFS. It describes using the libgfapi library to access GlusterFS volumes, providing improved speed over FUSE. The document outlines the Python and C APIs for libgfapi and provides an example of using the Python API. It also discusses rapidly prototyping GlusterFS translators using the Python Glupy library and provides a simple "hello world" translator example.
![Helloworld translator demo
[Run through the helloworld demo here]](https://image.slidesharecdn.com/ixo6uskvq7yitfilhrnd-signature-1e22e3c02f143bdbaade1033e41083e3186e10cee46774b782105433d60d093c-poli-160517072541/85/Developing-apps-and_integrating_with_gluster_fs_-_libgfapi-15-320.jpg)
Developing apps and_integrating_with_gluster_fs_-_libgfapi
- 1. Developing Apps and Integrating with GlusterFS Justin Clift, GlusterFS Integration Open Source and Standards team @ Red Hat Gluster Community Day London 2013
- 2. #whoami ● Previously an experienced SysAdmin (Solaris and Linux) for many, many years ● Now works on GlusterFS integration ● Mostly worked on Mission Critical systems in corporate/enterprise environments (eg. Telco, banking, Insurance) ● Has been helping build Open Source Communities after hours for many years (PostgreSQL, OpenOffice back in the day) ● Joined Red Hat mid 2010 jclift@redhat.com
- 3. September 2013 - Niels de Vos Agenda ● Options for Gluster Application development ● What is Libgfapi? ● Pros & Cons ● Libgfapi example ● Rapid Prototyping GlusterFS Translators ● Translator example ● Contact Me
- 4. The old way: access through FUSE
- 5. libgfapi: access through library
- 6. Pros: Why use Libgfapi? ● Improved speed and latency ● Eliminate the FUSE and kernel VFS layer from GlusterFS volume access ● Examples: – qemu storage layer – Samba VFS plugin
- 7. Cons: Downsides ● Existing applications need to be extended – Not a drop in replacement for POSIX based filesystem access ● Client side API only available for C and Python so far
- 8. Python and C API ● The C API is fairly comprehensive – 87 callable C functions. Eg: ● glfs_creat() ● glfs_open() ● glfs_close() ● glfs_seekdir() ● glfs_stat() ● glfs_chmod() ● The Python API is only 19 functions so far – Just the most used ones, more coming
- 9. Python API – Volume operations ● set_logging(path, level) ● mount() ● creat(path, flags, mode) ● getxattr(path, key, maxlen) ● listxattr(path) ● lstat(path) ● mkdir(path) ● open(path, flags) ● opendir(path) ● rename(old_path, new_path) ● rmdir(path) ● setxattr(path, key, value, vlen) ● unlink(path)
- 10. Python API – File operations ● fsync() ● read(buflen, flags) ● read_buffer(buf, flags) ● write(data, flags) ● fallocate(mode, offset, len) ● discard(offset, len)
- 11. Libgfapi example Example Python code: # Import Python libgfapi library (required) from gluster import gfapi # Mount the volume vol = gfapi.Volume('f19laptop', 'playground') vol.mount() # Create a file import os myfile = vol.creat('somefile.txt', os.O_RDWR, 0o640)
- 12. Libgfapi example (continued) # Write to the file bytes_written = myfile.write('This is some data') print bytes_written # Seek back to the start of the file myfile.seek(0, os.SEEK_SET) # Read from the file file_data = myfile.read(5) print file_data # Rename the file vol.rename('somefile.txt', 'someotherfile.txt') # Delete the file vol.unlink('someotherfile.txt')
- 13. Rapid Prototyping GlusterFS Translators ● Uses the Python Gluster “Glupy” library ● Easy to write translators, compared to C ● Not as fast to execute as C based translators, but can still fire off over 1,000 times per second
- 14. Trivial “helloworld” example import sys from gluster_glupy import * class xlator (Translator): def __init__(self, c_this): Translator.__init__(self, c_this) def lookup_fop(self, frame, this, loc, xdata): print "Python xlator: Hello!" dl.wind_lookup(frame, POINTER(xlator_t)(), loc, xdata) return 0 def lookup_cbk(self, frame, cookie, this, op_ret, op_errno, inode, buf, xdata, postparent): print "Python xlator: Hello again!" dl.unwind_lookup(frame, cookie, this, op_ret, op_errno, inode, buf, xdata, postparent) return 0
- 15. Helloworld translator demo [Run through the helloworld demo here]
- 16. Real world example ● GlusterFlow translator: https://forge.gluster.org/glusterflow/glusterflow/blobs/master/json/glus terflowclient.py ● Fires off 1,600 times per second on my laptop in a F19 VM ● Can be re-written in C if that's not fast enough
- 17. Questions? ● User questions and experiences – gluster-users@gluster.org – #gluster on Freenode IRC ● Developer questions and discussions – gluster-devel@nongnu.org – #gluster-dev on Freenode IRC ● Announcements mailing list – announce@gluster.org
- 18. Contact me ● Happy for people to email me about any of this stuff: Justin Clift ● Work email: jclift@redhat.com ● Personal email: justin@postgresql.org ● Twitter: @realjustinclift