Accessing gluster ufo_-_eco_willson
Download as ODP, PDF0 likes397 views
This document provides an introduction to accessing Gluster volumes through various methods like traditional filesystem mounts, NFS/CIFS, and the Unified File and Object (UFO) feature. It discusses the benefits of using UFO to access data simultaneously via both REST API and filesystem. It also includes examples of common commands for authenticating and interacting with objects and containers when using UFO with Gluster.
Accessing gluster ufo_-_eco_willson
- 1. Introduction to accessing Gluster volumes Eco Willson Gluster Communities
- 2. A bit about me Systems administration and DevOps background. Joined Gluster in September of 2010 Current role is a technical liaison to GlusterFS upstream via the Open Source and Standards organization (OSAS) within Red Hat.
- 3. Fun by example! (cont.) 6) Upload a new picture curl -v -X PUT -T CoolLinuxConfPic.jpg -H 'X-Auth-Token: AUTH_tk3b615e3c5ada4e06b2354ed10a129ecb' -k https://127.0.0.1:443/v1/AUTH_test/conference_pictures/pic8675309.jpg 7) Tag new picture with metadata curl -v -X POST -H 'X-Auth-Token: AUTH_tk3b615e3c5ada4e06b2354ed10a129ecb' -k https://127.0.0.1:443/v1/AUTH_test/conference_pictures -H 'X-Container- Meta-Demo-Venue: LinuxCon EU 2012' -H 'X-Container-Meta-Creator: Eco' -H 'X- Container-Meta-Creator-Taskmaster: John Mark Walker' 8) Ask the intern to verify the backups worked correctly since you need to step out and the boss will ask while you are away
- 4. Overview of Gluster philosphy Your data should be yours, always We enable a framework of methods to allow data access any way that suits you This includes * Traditional filesystem mounts * Mount via NFS or CIFS * Access via REST using our Unified File and Object (UFO) feature * xlator framework for times when you need data to work better for the particular needs of your environment or application
- 5. Accessing using traditional filesystem mounts Requires the Gluster packages to be installed on the client Including _netdev option in fstab will help avoid accidentally filling up / partition It is recommended that your mount points are clearly separate from your export folders on the Gluster servers Consider the confusion that can occur if you have the brick mounted on /mnt/export/gluster, and the client mount on /mnt/gluster...
- 6. Accessing via NFS/CIFS Gluster has it's own NFS server inline. Due to this, it is not recommended to run native NFS on the Gluster servers No additional Gluster packages need to be installed, although you may need to add additional packages (e.g. nfs-common in ubuntu or rpcbind) Many distributions use NFS by default now. Due to this, use the -o vers=3 option when mounting Include the _netdev option in fstab as you would with regular NFS Don't forget that automatic failover does not occur when not using the Gluster native client (but can still be achieved via carp or CTDB)
- 7. Accessing via Unified File and Object Additional setup is required vs the standard Gluster setup See Kalebs' post on Gluster.org: http://www.gluster.org/2012/09/howto-using-ufo-swift-a-quick-and-dirty-setup- guide/ Allows simultaneous access to your data from both developers and data consumers (users and applications)
- 8. Why use UFO on Gluster? No 5GB limit on transfer size Takes advantage of the Gluster features you have already set up in your volume (e.g., replication, automatic fail over and self-heal) Global access to the same data, using the same namespace, at the same time Ease of use and management versus other clustering solutions
- 9. How does simultaneous access help me? By being able to access the same data via both the REST API and traditional file system access methods, you can work on the same data via the method you choose, and using multiple methods, simultaneously, on the same data. This allows you to use the newer cloud based storage without having to sacrifice functionality for older programs that may be dependent on direct access to the filesystem, for example, using anti-virus scanners on uploaded files. This also means the changes I make via the filesystem, or via UFO, are being done against the same exact object.
- 10. What do I need to test UFO with GlusterFS? At least two nodes with Gluster installed already. For testing, virtual machines are fine. Some knowledge of the REST API or cURL to be able to address the Gluster volume and files within it
- 11. What do I need to test UFO with GlusterFS? At least two nodes with Gluster installed already. For testing, virtual machines are fine. Some knowledge of the REST API or cURL to be able to address the Gluster volume and files within it
- 12. Common Commands Create an authentication token curl -v -H 'X-Storage-User: gv0:admin' -H 'X-Storage-Pass:admin' -k https://127.0.0.1:443/auth/v1.0/ Create a new container curl -v -X PUT -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test/pictures -k Display existing containers curl -v -X GET -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test -k
- 13. Common Commands Create a new object in a container curl -v -X PUT -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test/pictures/dog -H 'Content- Length: 0' -k List all objects in a container curl -v -X GET-H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test/images -k
- 14. Common Commands Delete a container curl -v -X DELETE -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test/pictures/dog -H 'Content- Length: 0' -k Delete an object in a container curl -v -X DELETE -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://example.storage.com:443/v1/AUTH_test/ pictures/cat -k
- 15. Some things to remember for Gluster with SWIFT When we reference “account”, we are referring to the gluster volume When adding users, the second field is where you specify the Gluster volume. In the following string: user_test_tester = testing .admin Replace “test” with the Gluster volume A container lives within a Gluster volume, and can be thought of as a logically separated space to execute commands The “Container Server” handles listing objects The “Account Server” handles listing the Gluster volumes (remember 'accounts' from above)
- 16. Fun by example! 1) Get an auth token: curl -v -H 'X-Storage-User: ufo:test' -H 'X-Storage-Pass: test' -k https://127.0.0.1:443/auth/v1.0 2) List containers in a Gluster volume named “test” curl -v -X GET -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://127.0.0.1:443/v1/AUTH_test -k 3) Create a new container to for uploading more pictures curl -v -X PUT -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://127.0.0.1:443/v1/AUTH_test/conference_pictures -k
- 17. Fun by example! (cont.) 4) List new container curl -v -X GET -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' https://127.0.0.1:443/v1/AUTH_test -k 5) Haze the intern by telling them to back up the container. Make sure to point out how critical the data is. Do not ruin the fun by letting them know you backed it up properly already (using the Gluster volume already mounted on their machine) sudo rm -rf /mnt/ufo/halloween_pix/*
- 18. Thanks! eco@redhat.com www.gluster.org Come chat with us at #gluster on freenode IRC Joe Julians' HTML UFO Demo https://github.com/joejulian/ufopilot