Implementing a Fileserver with Nginx and Lua

Implementing a Fileserver
with Nginx and Lua
techtalk@ferret
Andrii Gakhov
27.02.2017

Current Architecture
Restful API server that communicates with clients using
application/json
Every client is authenticated by a dedicated 3rd party
authentication server and authorized by the API server
End users get benefits from the API via client-side
frontend application

The Problem
At some point, end users request the ability to manage files
using the same client-side application and, consequently,
same API.
allow to process multipart/form-data requests (that will be
proxied from the form on the client-side application)
extract and handle file metadata
provide file storage and access

The Solution
Obviously, such functionality is out of the scope for the API
and the natural decision is to split it across 2
applications:
API is used for meta file management
Fileserver takes care about actual files upload/download
Instead of using Flask/Django etc., we have decided to
implements such functionality in Nginx with Lua scripting

Nginx and Lua
Nginx is a free, open-source, high-performance HTTP server
and reverse proxy, as well as an IMAP/POP3 proxy server.
Lua is a powerful, efficient, lightweight, embeddable
scripting language. It supports procedural programming,
object-oriented programming, functional programming,
data-driven programming, and data description.
The Lua module for Nginx ngx_http_lua_module embeds Lua into
Nginx and by leveraging Nginx's subrequests, allows the
integration of Lua threads into the Nginx event model.

Nginx and Lua
Nginx object is available in Lua as ngx and server variables
are accessible as ngx.var.{name}, requested GET arguments -
as ngx.var.arg_{name}, but unfortunately POST variables Nginx
doesn't parse into variables automatically.
The various *_by_lua_* configuration directives serve as
gateways to the Lua API within the nginx.conf. We use
*_by_lua_block to embed Lua code into Nginx as the most
notable way to do so.

Nginx and Lua
header_filter_by_lua_block - uses Lua code to define an
output header filter (e.g. for overriding or adding a
response header).
access_by_lua_block - acts as an access phase handler and
executes Lua code for every request. as with other access
phase handlers, access_by_lua will not run in subrequests.
content_by_lua_block - acts as a "content handler" and
executes Lua code for every request.

Architecture / Upload
location = /_upload {
limit_except POST { deny all; }
if ($http_content_type !~ "multipart/form-data") {
return 406;
}
lua_need_request_body off;
client_body_buffer_size 200M;
client_max_body_size 200M;
default_type 'application/json';
...
}

Architecture / Download
location ~ ^/_download/(.*)$ {
set $path /$1;
default_type 'application/json';
limit_except GET { deny all; }
...
}

User Authentication
function authenticate_user(access_token)
local params = {
method = ngx.HTTP_GET,
args = {
access_token = access_token
}
}
local res = ngx.location.capture(
"/_auth/access/check", params)
if not res or res.status ~= 200 then
return nil
end
return cjson.decode(res.body)
end
access_by_lua_block {
local cjson = require("cjson")
local access_token = ngx.var.arg_access_token
if not access_token then
return_http_forbidden("Forbidden")
end
-- authenticate user
local credentials = authenticate_user(access_token)
-- if user can't be resolved, return 403 Forbidden
if not credentials or not credentials.data.user.id
then
return_http_forbidden("Forbidden")
end
}

Upload
-- Extract POST variables in a streaming way
-- and store file object in a temporary
-- file (form_data.file.filepath)
-- All.other form variables are in form_data
-- (we expect them to be small)
local helpers = require("utils")
form_data, err = helpers.streaming_multipart_form()
if form_data == nil then
return_http_internal_server_error(err)
end
data = {
status = "not_ready",
title = form.title.value,
filename = form.file.filename,
}
local params = {
method = ngx.HTTP_POST,
args = {access_token = access_token},
body = cjson.encode(data)
}
local res = ngx.location.capture("/_api/v1/file", params)
if not res then
return_http_bad_gateway("No metadata")
end
local create_metadata_resp = res.body
if res and res.status ~= 201 then
ngx.status = res.status
ngx.print(create_metadata_resp)
ngx.exit(res.status)
end
local file_metadata = cjson.decode(create_metadata_resp)
Important part to make it work is to specify
client_body_buffer_size equal to
client_max_body_size.
Otherwise, if the client body is bigger than
client_body_buffer_size, the nginx variable
$request_body will be empty.

Upload
local filex = require("pl.file")
local file_fullpath = "/storage/files/" ..
file_metadata.hash .. file_metadata.path
-- ensure that subdirectories exist (if not, create)
-- store tmp file to its permanent position
is_moved, err = filex.move(
form.file.fullpath, file_fullpath)
-- make it available for download
if is_moved then
local params = {
method = ngx.HTTP_PUT,
body = cjson.encode({status = "ready"})
}
ngx.location.capture(
"/_api/v1/file/" .. file_metadata.id, params)
end
-- provide some headers with metadata
ngx.header["X-File-ID"] = file_metadata.id
ngx.status = ngx.HTTP_CREATED
ngx.print(create_metadata_resp)
ngx.exit(ngx.HTTP_CREATED)
Since we need to manage access to the file, we
use access policy of files’ metadata instead of the
files themselves.
We create metadata in any case (“register the
file”), but allow to download it only if file has status
“ready”, meaning it was successfully created at the
specified location.

Download
local search_by_path = {
filters = {
path = ngx.var.path,
status = "ready"
},
size = 1
}
local params = {
method = ngx.HTTP_POST,
body = cjson.encode(search_by_path)
}
local res = ngx.location.capture(
"/_api/v1/search/files", params)
if not res then
return_http_bad_gateway("Search error")
end
local found = cjson.decode(res.body)
if found.total < 1 then
return_http_not_found("File Not Found")
end
local file_metadata = found.results[1]
ngx.header["X-File-Name"] = file_metadata.name
ngx.header["X-File-Path"] = file_metadata.path
ngx.header["X-File-ID"] = file_metadata.id
ngx.req.set_uri(
"/" .. file_metadata.hash .. file_metadata.path)
ngx.exec("@download_file", download_params)
As soon as the file has been found, its metadata
provides us with all necessary information about
the location and we are ready to respond it to the
user.
Sometimes people recommend to read such file in
Lua and return it to the user with ngx.print that is a
bad idea for big files (Lua virtual machine will just
crash).

Download
location @download_file {
internal;
root /storage/files/;
try_files $uri =404;
header_filter_by_lua_block {
ngx.header["Cache-Control"] = "no-cache"
ngx.header["Content-Disposition"] = "attachment; filename="" .. ngx.header["X-File-Name"] .. """
}
}
The @download_file location is quite simple, but additionally we want to play with response headers to provide
a real filename for download (on our filesystem all files are stored with unique generated names).
It is an internal named location (to prevent unauthorized access) that just serves requested static files from the
desired directory.

How to Use
Upload
curl -XPOST https://files.example.com/_upload?access_token={SOME_TOKEN}
--form file=@/tmp/file.pdf
--form title="Example title"
-H "Content-Type: multipart/form-data"
Download
curl -XGET https://files.example.com/_download/723533/2338342189083057604.pdf?access_token={SOME_TOKEN}

Thank you
Read it in the web:
https://www.datacrucis.com/research/implementing-api-based-
fileserver-with-nginx-and-lua.html

Implementing a Fileserver with Nginx and Lua

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Implementing a Fileserver with Nginx and Lua (20)

More from Andrii Gakhov (20)

Recently uploaded (20)

Implementing a Fileserver with Nginx and Lua