Shared memory and multithreading in Node.js - Timur Shemsedinov - JSFest'19

Timur Shemsedinov
5-6 APRIL
Introduction to multithreading concurrent programming in
Node.js with worker_threads, SharedArrayBuffer and Atomics
Shared memory and
multithreading in Node.js

Shared memory and multithreading in Node.js
Process
- Separate memory
- Separate resources
- Separate environment
Process / Thread
Thread
- Shared memory space
- Shared resources
- Common environment
Memory mapped files

Node.js
- child_process
- cluster
- worker_threads
--experimental-worker
- Atomics
- SharedArrayBuffer
Node.js 10.5 ... 11.7

process
process
JavaScript
thread
V8 libuv
node.js
JavaScript
Shared memory and Message passing
thread
V8 libuv
node.js
JavaScript
thread
V8 libuv
node.js
IPC

Node.js: worker_threads
Not supported:
- process.abort()
- process.chdir(name)
- process.initgroups(...)
- trace_events module
- IPC from parent
- process.setegid(id)
- process.seteuid(id)
- process.setgid(id)
- process.setgroups(...)
- process.setuid(id)

Read-only in Worker threads:
- process.env
- process.title
- process.umask([mask])

Different behavior:
- process.exit([code]) stops thread not process
- process.memoryUsage() rss for entire process
- Each thread has an independent async_hooks
- Signals will not be delivered through process.on

Node.js: worker_threads API
const threads = require('worker_threads');
const { Worker, isMainThread } = threads;
if (isMainThread) {
const worker =
new Worker(__filename, { workerData: {} });
} else {
const { parentPort } = threads;
}

Node.js: MessagePort API
worker.on('message', (...args) => {});
worker.on('error', err => {});
worker.on('exit', code => {});
worker.postMessage('Hello there!');
parentPort.postMessage('Hello there!');
parentPort.on('message', (...args) => {});

class Point {
constructor(buffer, offset) {
this.data = new Int8Array(buffer, offset, 2);
}
get x() { return this.data[0]; }
set x(value) { this.data[0] = value; }
...
Wrap Shared Memory with OOP

const buffer = new SharedArrayBuffer(64);
const point = new Point(buffer, 4);
point.x += 10;
point.y = 7;
const { x } = point;
Wrap Shared Memory with OOP

- Parallel Programming
- Asynchronous Programming
- Actor Model
- Transactional Memory
- Coroutines
Concurrent Computing

Parallel Programming
- Race condition
- Critical section
- Deadlock
- Livelock
- Starvation

Int8Array, Uint8Array,
Int16Array, Uint16Array,
Int32Array, Uint32Array
Atomics and SharedArrayBuffer

prev = Atomics.add(array, index, value)
prev = Atomics.sub(array, index, value)
prev = Atomics.and(array, index, value)
prev = Atomics.or(array, index, value)
prev = Atomics.xor(array, index, value)
ES2017 Atomics

function add(array, index, value) {
1 const prev = array[index];
2 const sum = prev + value;
3 array[index] = sum;
4 return prev;
}
Atomics.add(array, index, value)

stored = Atomics.store(array, index, value)
value = Atomics.load(array, index)
prev = Atomics.exchange(array, index, value)
prev = Atomics.compareExchange(
array, index, expected, replacement
)
ES2017 Atomics (CAS)

woken = Atomics.notify(array, index, count)
res = Atomics.wait(array, index, value, [timeout])
res <"ok" | "not-equal" | "timed-out">
array <Int32Array>
ES2017 Atomics (notify/wait)

Synchronization primitives
- Semaphore
- BinarySemaphore
- CountingSemaphore
- Condition variable
- SpinLock
- Mutex
- TimedMutex
- SharedMutex
- RecursiveMutex
- Monitor
- Barrier

- thread safe data structures
- lock-free data structures
- wait-free algorithms
- conflict-free data structures
Parallel Solutions

class Mutex {
constructor(shared, offset = 0)
enter()
leave()
}
https://github.com/HowProgrammingWorks/Mutex
Mutex

mutex.enter();
// do something
// with shared resources
// or data structures
mutex.leave();
Mutex usage

constructor(shared, offset = 0) {
this.lock = new Int32Array(shared, offset, 1);
this.owner = false;
}
const threads = require('worker_threads');
const { workerData } = threads;
const mutex1 = new Mutex(workerData, offset);
Mutex constructor

enter() {
let prev = Atomics.exchange(this.lock, 0, LOCKED);
while (prev !== UNLOCKED) {
Atomics.wait(this.lock, 0, LOCKED);
prev = Atomics.exchange(this.lock, 0, LOCKED);
}
this.owner = true;
} Example: 6-blocking.js
Mutex.enter with Atomics.wait

leave() {
if (!this.owner) return;
Atomics.store(this.lock, 0, UNLOCKED);
Atomics.notify(this.lock, 0, 1);
this.owner = false;
}
Mutex.leave with Atomics.notify

mutex.enter(() => {
// do something
mutex.leave();
});
Async Mutex usage

await mutex.enter();
// do something
mutex.leave();
Async Mutex usage

enter() {
return new Promise(resolve => {
while (true) {
if (prev === UNLOCKED) break;
}
this.owner = true;
resolve();
});
}
Spinlock

enter() {
const tryEnter = () => {
if (prev === UNLOCKED) {
this.owner = true;
resolve();
} else {
setTimeout(tryEnter, 0);
}
};
tryEnter();
});
}
Spinlock with setTimeout

constructor(messagePort, shared, offset = 0) {
this.port = messagePort;
this.lock = new Int32Array(shared, offset, 1);
this.owner = false;
this.trying = false;
this.resolve = null;
if (messagePort) {
messagePort.on('message', kind => {
if (kind === 'leave' && this.trying) this.tryEnter();
});
} Example: 8-async.js
}
Asynchronous Mutex

enter() {
this.resolve = resolve;
this.trying = true;
this.tryEnter();
});
}
Asynchronous Mutex.enter

tryEnter() {
if (!this.resolve) return;
if (prev === UNLOCKED) {
this.owner = true;
this.trying = false;
this.resolve();
this.resolve = null;
}
}
Asynchronous Mutex.tryEnter

- Low-level structures
e.g. Register, Counter, Buffer, Array, Lists...
- Abstract structures
e.g. Queue, Graph, Polyline, etc.
- Subject-domain classes
e.g. Sensors, Payment, Biometric data, etc.
Thread safe data structures

locks.request('resource', opt, async lock => {
if (lock) {
// critical section for `resource`
// will be released after return
}
});
https://wicg.github.io/web-locks/
Web Locks API

- https://github.com/nodejs/node/issues/22702
Open
- https://github.com/nodejs/node/pull/22719
Closed
Web Locks for Node.js

- Web Locks requires memory + messaging
- Multiple locks.request() calls requires queue
- Mutex.queue: Array<Lock> // resource
- Lock { name, mode, callback } // request
- Need await locks.request()
- Options: { mode, ifAvailable, steal, signal }
Web Locks API Implementation

locks.request('resource', lock => new Promise(
(resolve, reject) => {
// you can store or pass resolve and
// reject here as callbacks
}
));
https://wicg.github.io/web-locks/
Web Locks: Promise or thenable

- Passing handles between workers
- Native add-ons (with certain conditions)
- Debug
- Experimental
https://github.com/nodejs/node/issues/22940
To be solved

https://github.com/HowProgrammingWorks/Semaphore
https://github.com/HowProgrammingWorks/Mutex
https://github.com/metarhia/metasync
https://wicg.github.io/web-locks
https://nodejs.org/api/worker_threads.html
https://developer.mozilla.org/en-US/docs/Web/
JavaScript/Reference/Global_Objects/Atomics
Links

https://github.com/tshemsedinov
https://www.youtube.com/TimurShemsedinov
timur.shemsedinov@gmail.com
Thanks

Shared memory and multithreading in Node.js - Timur Shemsedinov - JSFest'19

More Related Content

What's hot (20)

Similar to Shared memory and multithreading in Node.js - Timur Shemsedinov - JSFest'19 (20)

More from Timur Shemsedinov (20)

Recently uploaded (20)

Shared memory and multithreading in Node.js - Timur Shemsedinov - JSFest'19