Scylla Summit 2022: ScyllaDB Embraces Wasm

ScyllaDB Embraces Wasm
Piotr Sarna
Principal Software Engineer @ScyllaDB

Piotr Sarna
■ software engineer keen on open-source projects, C++ and Rust
■ used to develop a distributed ﬁle system (LizardFS)
■ wrote a few patches for the Linux kernel
■ graduated from University of Warsaw with MSc in Computer Science
■ maintainer of the Scylla Rust Driver project
Principal Software Engineer @ScyllaDB

WebAssembly
Binary format for expressing executable code, executed on a stack-based virtual
machine. Designed to be:
■ portable
■ easily embeddable
■ eﬃcient
WebAssembly is binary, but it also speciﬁes a standard human-readable format:
WAT (WebAssembly Text Format).

Runtime of choice: Wasmtime
A variety of WebAssembly engines are available for embedding into C++ projects
■ Wasmtime
• implemented in Rust
• WebAssembly only
• lightweight (esp. compared to v8)
• has bindings for C/C++
• native support for yielding
■ v8
• implemented in C++
• supports javascript too
• a heavy dependency
• no direct support for yielding
the execution to reduce
latency

Runtime of choice: Wasmtime
For an initial implementation, we chose Wasmtime and its C++ bindings -
libwasmtime.
The next step is to get rid of the bindings due to its incomplete feature set,
and instead write the UDF support in Rust and compile it directly into Scylla.

How to code in WebAssembly?
Option 1: by hand (for Lisp enthusiasts)
(module
(func $fib (param $n i64) (result i64)
(if
(i64.lt_s (local.get $n) (i64.const 2))
(return (local.get $n))
)
(i64.add
(call $fib (i64.sub (local.get $n) (i64.const 1)))
)
)
(export "fib" (func $fib))
)

Option 2: write in C, compile with clang
int fib(int n) {
if (n < 2) {
return n;
}
return fib(n - 1) + fib(n - 2);
}
clang -O2 --target=wasm32 --no-standard-libraries -Wl,--export-all -Wl,--no-entry
fib.c -o fib.wasm
wasm2wat fib.wasm > fib.wat

Option 3: Rust!
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn fib(n: i32) -> i32 {
if n < 2 {
n
} else {
fib(n - 1) + fib(n - 2)
}
}
rustup target add wasm32-unknown-unknown
cargo build --target wasm32-unknown-unknown
wasm2wat target/wasm32-unknown-unknown/debug/fib.wasm > fib.wat

Option 4: AssemblyScript
export function fib(n: i32): i32 {
if (n < 2) {
return n
}
return fib(n - 1) + fib(n - 2)
}
asc fib.ts --textFile fib.wat --optimize
source: https://www.assemblyscript.org/introduction.html

User-deﬁned functions
User-deﬁned functions are a CQL feature that allows applying a custom function
to the query result rows.
cassandra@cqlsh:ks> SELECT id, inv(id), mult(id, inv(id)) FROM t;
id | ks.inv(id) | ks.mult(id, ks.inv(id))
----+------------+-------------------------
7 | 0.142857 | 1
1 | 1 | 1
0 | Infinity | NaN
4 | 0.25 | 1
(4 rows)

User-deﬁned aggregates
A powerful tool for combining functions into accumulators, which aggregate
results from single rows.
cassandra@cqlsh:ks> SELECT * FROM words;
word
------------
monkey
rhinoceros
dog
(3 rows)
cassandra@cqlsh:ks> SELECT avg_length(word) FROM words;
ks.avg_length(word)
-----------------------------------------------
The average string length is 6.3333333333333!
(1 rows)
CREATE FUNCTION accumulate_len(acc tuple<bigint,bigint>, a text)
RETURNS NULL ON NULL INPUT
RETURNS tuple<bigint,bigint>
LANGUAGE lua as 'return {acc[1] + 1, acc[2] + #a}';
CREATE OR REPLACE FUNCTION present(res tuple<bigint,bigint>)
RETURNS NULL ON NULL INPUT
RETURNS text
LANGUAGE lua as
'return "The average string length is " .. res[2]/res[1] .. "!"';
CREATE OR REPLACE AGGREGATE avg_length(text)
SFUNC accumulate_len
STYPE tuple<bigint,bigint>
FINALFUNC present INITCOND (0,0);

User-deﬁned aggregates
Possible scenarios for user-deﬁned aggregates:
■ gathering statistical data: variance, standard deviation, percentiles, etc.
■ combining multiple rows into a new format, e.g. JSON or XML
■ custom predicates, e.g. "return 10 highest values"
■ you name it!

UDF coded with Wasm
Creating a user-deﬁned function with Wasm is as easy as providing its source code
represented in WebAssembly Text Format:
CREATE FUNCTION fib(input bigint) RETURNS NULL ON NULL INPUT RETURNS
bigint
LANGUAGE xwasm AS
'(module
(func $fib (param $n i64) (result i64)
(if
(i64.lt_s (local.get $n) (i64.const 2))
(return (local.get $n))
)
(i64.add
)
)
(export "fib" (func $fib))
)';
cassandra@cqlsh:ks> SELECT n, fib(n) FROM numbers;
n | ks.fib(n)
---+-----------
1 | 1
2 | 1
3 | 2
4 | 3
5 | 5
6 | 8
7 | 13
8 | 21
9 | 34
(9 rows)

UDF coded with Wasm
The interface for expressing CQL types, return values, NULL values and many more
details are thoroughly explained in a public design doc:
https://github.com/scylladb/scylla/blob/master/docs/design-notes/wasm.md

Try it out!
Support for Wasm-based user-defined functions and
user-defined aggregates is already available
in experimental mode.
Enable it for testing today by adding these entries
to your scylla.yaml configuration file:
enable_user_defined_functions: true
experimental_features:
- udf
Scylla currently supports Lua and Wasm for
user-defined functions.

Thank you!
Stay in touch
Piotr Sarna
sarna@scylladb.com

Scylla Summit 2022: ScyllaDB Embraces Wasm

More Related Content

What's hot (20)

Similar to Scylla Summit 2022: ScyllaDB Embraces Wasm (20)

More from ScyllaDB (20)

Recently uploaded (20)

Scylla Summit 2022: ScyllaDB Embraces Wasm