SlideShare a Scribd company logo

Infinit filesystem, Reactor reloaded

Infinit, profile picture
Infinit

The document details the Infinit Filesystem, a distributed filesystem designed for Byzantine environments that aggregates multiple computers into a unified storage solution, emphasizing its lack of administrative authority and resilience to network failures. It also describes the Reactor library in C++, which supports coroutines and enables safe concurrent programming through a structured API for creating and managing threads. Additionally, it discusses various threading patterns, operations to exploit multicore systems, and background processing techniques for efficiency.

Software
1 of 63
Downloaded 20 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
Infinit filesystem Reactor reloaded mefyl quentin.hocquet@infinit.io Version 1.2-5-g4a755e6
Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage.
Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking).
Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking). • Distributed: no computer as any specific authority or role. ◦ Availability: no SPOF, network failure resilient. ◦ No admin. As in, no janitor and no tyran. ◦ Scalability flexibility.
Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking). • Distributed: no computer as any specific authority or role. ◦ Availability: no SPOF, network failure resilient. ◦ No admin. As in, no janitor and no tyran. ◦ Scalability flexibility. • Byzantine: you do not need to trust other computers in any way. ◦ No admins. As in, no omniscient god. ◦ Support untrusted networks, both faulty and malicious peers.
Infinit architecture
Infinit filesystem, Reactor reloaded
Coroutines in a nutshell
Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO
Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK
Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK System Coroutines OK OK OK KO KO
Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK System Coroutines OK OK OK KO KO Stackless Coroutines Meh OK OK KO OK
Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK OK OK System Coroutines OK OK OK OK KO Stackless Coroutines Meh OK OK OK OK
Reactor in a nutshell Reactor is a C++ library providing coroutines support, enabling simple and safe imperative style concurrency.
Reactor in a nutshell Reactor is a C++ library providing coroutines support, enabling simple and safe imperative style concurrency. while (true) { auto socket = tcp_server.accept(); new Thread([socket] { try { while (true) socket->write(socket->read_until("n")); } catch (reactor::network::Error const&) {} }); }
Sugar for common pattern Coroutines are mostly used in three patterns:
Sugar for common pattern Coroutines are mostly used in three patterns: • The entirely autonomous detached thread. • The background thread tied to an object. • The parallel flow threads tied to the stack.
Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable)
Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread)
Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now()
Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now() Nota bene: • Don't destroy an unfinished thread. Wait for it or terminate it.
Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now() Nota bene: • Don't destroy an unfinished thread. Wait for it or terminate it. • Exceptions escaping a thread terminate the whole scheduler.
Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { }); // Will execute until all threads are done. sched.run();
Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { reactor::Thread t("t", [] { print("world"); }); print("hello"); }); // Will execute until all threads are done. sched.run();
Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { reactor::Thread t("t", [] { print("world"); }); print("hello"); reactor::wait(t); }); // Will execute until all threads are done. sched.run();
The detached thread The detached thread is a global background operation whose lifetime is tied to the program only.
The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p));
The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. new reactor::Thread( [] { for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p)); });
The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. new reactor::Thread( [] { for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p)); }, reactor::Thread::auto_dispose = true);
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b) { this->_blocks->put(b); } void _flush() { while (true) this->_store(this->_blocks.get()); } };
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); std::unique_ptr<reactor::Thread> _flush_thread; Async() : _flush_thread(new reactor::Thread( [this] { this->_flush(); })) {} };
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); std::unique_ptr<reactor::Thread> _flush_thread; Async(); ~Async() { this->_flush_thread->terminate_now(); } };
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); Thread::unique_ptr<reactor::Thread> _flush_thread; Async(); };
The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime struct Terminator : public std::default_delete<reactor::Thread> { void operator ()(reactor::Thread* t) { bool disposed = t->_auto_dispose; if (t) t->terminate_now(); if (!disposed) std::default_delete<reactor::Thread>:: operator()(t); } }; typedef std::unique_ptr<reactor::Thread, Terminator> unique_ptr;
The parallel flow threads Parallel flow threads are used to make the local flow concurrent, like parallel control statements. auto data = reactor::http::get("...");; reactor::http::put("http://infinit.sh/...", data); reactor::network::TCPSocket s("hostname"); s.write(data); print("sent!"); GET HTTP PUT TCP PUT SENT
The parallel flow threads auto data = reactor::http::get("..."); reactor::Thread http_put( [&] { reactor::http::put("http://infinit.sh/...", data); }); reactor::Thread tcp_put( [&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait({http_put, tcp_put}); print("sent!"); GET SENT HTTP PUT TCP PUT
The parallel flow threads auto data = reactor::http::get("..."); std::exception_ptr exn; reactor::Thread http_put([&] { try { reactor::http::put("http://...", data); } catch (...) { exn = std::current_exception(); } }); reactor::Thread tcp_put([&] { try { reactor::network::TCPSocket s("hostname"); s.write(data); } catch (...) { exn = std::current_exception(); } }); reactor::wait({http_put, tcp_put}); if (exn) std::rethrow_exception(exn); print("sent!");
The parallel flow threads auto data = reactor::http::get("..."); std::exception_ptr exn; reactor::Thread http_put([&] { try { reactor::http::put("http://...", data); } catch (reactor::Terminate const&) { throw; } catch (...) { exn = std::current_exception(); } }); reactor::Thread tcp_put([&] { try { reactor::network::TCPSocket s("hostname"); s.write(data); } catch (reactor::Terminate const&) { throw; } catch (...) { exn = std::current_exception(); } }); reactor::wait({http_put, tcp_put}); if (exn) std::rethrow_exception(exn); print("sent!");
The parallel flow threads auto data = reactor::http::get("..."); reactor::Scope scope; scope.run([&] { reactor::http::put("http://infinit.sh/...", data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait(scope); print("sent!"); GET SENT HTTP PUT TCP PUT
Futures auto data = reactor::http::get("..."); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); r.write(data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait(scope); print("sent!");
Futures elle::Buffer data; reactor::Thread fetcher( [&] { data = reactor::http::get("..."); }); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); reactor::wait(fetcher); r.write(data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); reactor::wait(fetcher); s.write(data); }); reactor::wait(scope); print("sent!");
Futures reactor::Future<elle::Buffer> data( [&] { data = reactor::http::get("..."); }); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); r.write(data.get()); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data.get()); }); reactor::wait(scope); print("sent!");
Futures Scope + futures SENT GET HTTP PUT TCP PUT Scope GET SENT HTTP PUT TCP PUT Serial GET HTTP PUT TCP PUT SENT
Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope);
Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope); Concurrent iteration: std::vector<Host> hosts; reactor::concurrent_for( hosts, [] (Host h) { h->send(block); });
Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope); Concurrent iteration: std::vector<Host> hosts; reactor::concurrent_for( hosts, [] (Host h) { h->send(block); }); • Also available: reactor::concurrent_break(). • As for a concurrent continue, that's just return.
CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ?
CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ? • Idea 1: schedule coroutines in parallel ◦ Pro: absolute best parallelism. ◦ Cons: race conditions.
CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ? • Idea 1: schedule coroutines in parallel ◦ Pro: absolute best parallelism. ◦ Cons: race conditions. • Idea 2: isolate CPU bound code in "monads" and run it in background. ◦ Pro: localized race conditions. ◦ Cons: only manually parallelized code exploits multiple cores.
reactor::background reactor::background(callable): run callable in a system thread and return its result.
reactor::background reactor::background(callable): run callable in a system thread and return its result. Behind the scene: • A boost::asio_service whose run method is called in as many threads as there are cores. • reactor::background posts the action to Asio and waits for completion.
reactor::background reactor::background(callable): run callable in a system thread and return its result. Behind the scene: • A boost::asio_service whose run method is called in as many threads as there are cores. • reactor::background posts the action to Asio and waits for completion. Block::seal(elle::Buffer data) { this->_data = reactor::background( [data=std::move(data)] { return encrypt(data); }); }
reactor::background No fiasco rules of thumb: be pure. • No side effects. • Take all bindings by value. • Return any result by value.
reactor::background No fiasco rules of thumb: be pure. • No side effects. • Take all bindings by value. • Return any result by value. std::vector<int> ints; int i = 0; // Do not: reactor::background([&] {ints.push_back(i+1);}); reactor::background([&] {ints.push_back(i+2);}); // Do: ints.push_back(reactor::background([i] {return i+1;}); ints.push_back(reactor::background([i] {return i+2;});
Background pools Infinit generates a lot of symmetric keys. Let other syscall go through during generation with reactor::background: Block::Block() : _block_keys(reactor::background( [] { return generate_keys(); })) {}
Background pools Infinit generates a lot of symmetric keys. Let other syscall go through during generation with reactor::background: Block::Block() : _block_keys(reactor::background( [] { return generate_keys(); })) {} Problem: because of usual on-disk filesystems, operations are often sequential. $ time for i in $(seq 128); touch $i; done The whole operation will still be delayed by 128 * generation_time.
Background pools Solution: pregenerate keys in a background pool. reactor::Channel<KeyPair> keys; keys.max_size(64); reactor::Thread keys_generator([&] { reactor::Scope keys; for (int i = 0; i < NCORES; ++i) scope.run([] { while (true) keys.put(reactor::background( [] { return generate_keys(); })); }); reactor::wait(scope); }); Block::Block() : _block_keys(keys.get()) {}
Generators Fetching a block with paxos: • Ask the overlay for the replication_factor owners of the block. • Fetch at least replication_factor / 2 + 1 versions. • Paxos the result
Generators Fetching a block with paxos: DHT::fetch(Address addr) { auto hosts = this->_overlay->lookup(addr, 3); std::vector<Block> blocks; reactor::concurrent_for( hosts, [&] (Host host) { blocks.push_back(host->fetch(addr)); if (blocks.size() >= 2) reactor::concurrent_break(); }); return some_paxos_magic(blocks); }
Generators reactor::Generator<Host> Kelips::lookup(Address addr, factor f) { return reactor::Generator<Host>( [=] (reactor::Generator<Host>::Yield const& yield) { some_udp_socket.write(...); while (f--) yield(kelips_stuff(some_udp_socket.read())); }); }
Generators Behind the scene: template <typename T> class Generator { reactor::Channel<T> _values; Generator(Callable call) : _thread([this] { call([this] (T e) {this->_values.put(e);}); }) {} // Iterator interface calling _values.get() }; • Similar to python generators • Except generation starts right away and is not tied to iteration
Questions?

More Related Content

PDF
Docker volume plugins and Infinit – Mini-conf Docker Paris #1
Infinit
 
PDF
Highly concurrent yet natural programming
Infinit
 
PDF
Persistent storage tailored for containers #dockersummit
Infinit
 
PDF
XPDS14: Xen 4.5 Roadmap - Konrad Wilk, Oracle
The Linux Foundation
 
PPTX
Container Torture: Run any binary, in any container
Docker, Inc.
 
PDF
OSv at Cassandra Summit
Don Marti
 
PPTX
Journey to Stability: Petabyte Ceph Cluster in OpenStack Cloud
Patrick McGarry
 
PDF
How Networking works with Data Science
HungWei Chiu
 
Docker volume plugins and Infinit – Mini-conf Docker Paris #1
Infinit
 
Highly concurrent yet natural programming
Infinit
 
Persistent storage tailored for containers #dockersummit
Infinit
 
XPDS14: Xen 4.5 Roadmap - Konrad Wilk, Oracle
The Linux Foundation
 
Container Torture: Run any binary, in any container
Docker, Inc.
 
OSv at Cassandra Summit
Don Marti
 
Journey to Stability: Petabyte Ceph Cluster in OpenStack Cloud
Patrick McGarry
 
How Networking works with Data Science
HungWei Chiu
 

What's hot (20)

PDF
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
The Linux Foundation
 
PDF
Introduction to Docker & CoreOS - Symfony User Group Cologne
D
 
PDF
Redis persistence in practice
Eugene Fidelin
 
PPTX
Cgroups, namespaces and beyond: what are containers made from?
Docker, Inc.
 
PDF
Ceph Day San Jose - HA NAS with CephFS
Ceph Community
 
PDF
Docker Architecture (v1.3)
rajdeep
 
PDF
Ceph Day San Jose - From Zero to Ceph in One Minute
Ceph Community
 
PDF
Solr on Docker - the Good, the Bad and the Ugly
Sematext Group, Inc.
 
PDF
2015 open storage workshop ceph software defined storage
Andrew Underwood
 
PDF
Docker.io
Ladislav Prskavec
 
PPTX
Ceph Day Bring Ceph To Enterprise
Alex Lau
 
PPTX
Getting Started with Docker
Geeta Vinnakota
 
ODP
Testing Wi-Fi with OSS Tools
All Things Open
 
PDF
FreeBSD: The Next 10 Years (MeetBSD 2014)
iXsystems
 
PPTX
LXC
Wu Fan-Cheng
 
PDF
iSCSI Target Support for Ceph
Ceph Community
 
PDF
Docker: the road ahead
shykes
 
PDF
Container-relevant Upstream Kernel Developments
Docker, Inc.
 
PDF
XPDS14: Efficient Interdomain Transmission of Performance Data - John Else, C...
The Linux Foundation
 
PDF
Consuming Cinder from Docker
John Griffith
 
XPDS14 - Scaling Xen's Aggregate Storage Performance - Felipe Franciosi, Citrix
The Linux Foundation
 
Introduction to Docker & CoreOS - Symfony User Group Cologne
D
 
Redis persistence in practice
Eugene Fidelin
 
Cgroups, namespaces and beyond: what are containers made from?
Docker, Inc.
 
Ceph Day San Jose - HA NAS with CephFS
Ceph Community
 
Docker Architecture (v1.3)
rajdeep
 
Ceph Day San Jose - From Zero to Ceph in One Minute
Ceph Community
 
Solr on Docker - the Good, the Bad and the Ugly
Sematext Group, Inc.
 
2015 open storage workshop ceph software defined storage
Andrew Underwood
 
Docker.io
Ladislav Prskavec
 
Ceph Day Bring Ceph To Enterprise
Alex Lau
 
Getting Started with Docker
Geeta Vinnakota
 
Testing Wi-Fi with OSS Tools
All Things Open
 
FreeBSD: The Next 10 Years (MeetBSD 2014)
iXsystems
 
LXC
Wu Fan-Cheng
 
iSCSI Target Support for Ceph
Ceph Community
 
Docker: the road ahead
shykes
 
Container-relevant Upstream Kernel Developments
Docker, Inc.
 
XPDS14: Efficient Interdomain Transmission of Performance Data - John Else, C...
The Linux Foundation
 
Consuming Cinder from Docker
John Griffith
 
Ad

Viewers also liked (7)

PDF
Writing a Jenkins / Hudson plugin
Anthony Dahanne
 
PPTX
Job DSL Plugin for Jenkins
Niels Bech Nielsen
 
PDF
The Job DSL Plugin: Introduction & What’s New
Daniel Spilker
 
PDF
Deploying Spring Boot applications with Docker (east bay cloud meetup dec 2014)
Chris Richardson
 
PDF
Infinit: Modern Storage Platform for Container Environments
Docker, Inc.
 
PDF
Persistent storage tailored for containers
Docker, Inc.
 
PPTX
Reactive Programming in Java 8 with Rx-Java
Kasun Indrasiri
 
Writing a Jenkins / Hudson plugin
Anthony Dahanne
 
Job DSL Plugin for Jenkins
Niels Bech Nielsen
 
The Job DSL Plugin: Introduction & What’s New
Daniel Spilker
 
Deploying Spring Boot applications with Docker (east bay cloud meetup dec 2014)
Chris Richardson
 
Infinit: Modern Storage Platform for Container Environments
Docker, Inc.
 
Persistent storage tailored for containers
Docker, Inc.
 
Reactive Programming in Java 8 with Rx-Java
Kasun Indrasiri
 
Ad

Similar to Infinit filesystem, Reactor reloaded (20)

PDF
Communicating State Machines
srirammalhar
 
PDF
Eventdriven I/O - A hands on introduction
Marc Seeger
 
PDF
C++ CoreHard Autumn 2018. Concurrency and Parallelism in C++17 and C++20/23 -...
corehard_by
 
PDF
threads (1).pdfmjlkjfwjgliwiufuaiusyroayr
abhinandpk2405
 
PDF
Processes and threads
Krasimir Berov (Красимир Беров)
 
PPT
Exploiting Multicore CPUs Now: Scalability and Reliability for Off-the-shelf ...
Emery Berger
 
ODP
Multithreading 101
Tim Penhey
 
PPT
Google: Cluster computing and MapReduce: Introduction to Distributed System D...
tugrulh
 
PDF
Concurrency
Sri Prasanna
 
PPTX
OS Module-2.pptx
bleh23
 
PPTX
Operating Systems R20 Unit 2.pptx
Prudhvi668506
 
PDF
Concurrency, Parallelism And IO
Piyush Katariya
 
PDF
Thread
Mohd Arif
 
PDF
Threads operating system slides easy understand
shamsulhuda34
 
PPT
Introduction & Parellelization on large scale clusters
Sri Prasanna
 
PPT
Operating Systems - "Chapter 4: Multithreaded Programming"
Ra'Fat Al-Msie'deen
 
PPTX
Networking threads
Nilesh Pawar
 
PDF
Pthread
Gopi Saiteja
 
PPTX
Operating systems - Introduction to Threads
Dr. Chandrakant Divate
 
PPTX
Os
DeepaR42
 
Communicating State Machines
srirammalhar
 
Eventdriven I/O - A hands on introduction
Marc Seeger
 
C++ CoreHard Autumn 2018. Concurrency and Parallelism in C++17 and C++20/23 -...
corehard_by
 
threads (1).pdfmjlkjfwjgliwiufuaiusyroayr
abhinandpk2405
 
Processes and threads
Krasimir Berov (Красимир Беров)
 
Exploiting Multicore CPUs Now: Scalability and Reliability for Off-the-shelf ...
Emery Berger
 
Multithreading 101
Tim Penhey
 
Google: Cluster computing and MapReduce: Introduction to Distributed System D...
tugrulh
 
Concurrency
Sri Prasanna
 
OS Module-2.pptx
bleh23
 
Operating Systems R20 Unit 2.pptx
Prudhvi668506
 
Concurrency, Parallelism And IO
Piyush Katariya
 
Thread
Mohd Arif
 
Threads operating system slides easy understand
shamsulhuda34
 
Introduction & Parellelization on large scale clusters
Sri Prasanna
 
Operating Systems - "Chapter 4: Multithreaded Programming"
Ra'Fat Al-Msie'deen
 
Networking threads
Nilesh Pawar
 
Pthread
Gopi Saiteja
 
Operating systems - Introduction to Threads
Dr. Chandrakant Divate
 
Os
DeepaR42
 

Recently uploaded (20)

PDF
iTop VPN 6.5.0 Crack + License Key 2025 (Premium Version)
youngle786g
 
PPTX
AMADEUS TRAVEL AGENT SOFTWARE | AMADEUS TICKETING SYSTEM
philipnathen82
 
PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
deepakkhaitan3
 
PDF
How AI/LLM recommend to you ? GDG meetup 16 Aug by Fariman Guliev
HusseinMalikMammadli
 
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
PDF
AutoCAD Professional Crack 2025 With License Key
youngle786g
 
PPTX
Weekly report ppt - harsh dattuprasad patel.pptx
bikerslovers123
 
PDF
Digital Systems & Binary Numbers (comprehensive )
fyfhqfhtv2
 
PDF
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
PDF
Tally Prime Crack Download New Version 5.1 [2025] (License Key Free
itskinga12
 
PDF
17 Powerful Integrations Your Next-Gen MLM Software Needs
ventaforcemlmsoftwar
 
PDF
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
PDF
Salesforce Agentforce AI Implementation.pdf
Robert Mark
 
PDF
Complete Guide to Website Development in Malaysia for SMEs
Asian Business Services & Technologies
 
PDF
Product Update: Alluxio AI 3.7 Now with Sub-Millisecond Latency
Alluxio, Inc.
 
DOCX
Greta — No-Code AI for Building Full-Stack Web & Mobile Apps
niloyislam1187
 
PDF
CapCut Video Editor 6.8.1 Crack for PC Latest Download (Fully Activated) 2025
itskinga12
 
PPTX
Monitoring Stack: Grafana, Loki & Promtail
GhanshyamSwami3
 
PDF
How to Make Money in the Metaverse_ Top Strategies for Beginners.pdf
Herond Labs
 
PPTX
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 
iTop VPN 6.5.0 Crack + License Key 2025 (Premium Version)
youngle786g
 
AMADEUS TRAVEL AGENT SOFTWARE | AMADEUS TICKETING SYSTEM
philipnathen82
 
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
deepakkhaitan3
 
How AI/LLM recommend to you ? GDG meetup 16 Aug by Fariman Guliev
HusseinMalikMammadli
 
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
AutoCAD Professional Crack 2025 With License Key
youngle786g
 
Weekly report ppt - harsh dattuprasad patel.pptx
bikerslovers123
 
Digital Systems & Binary Numbers (comprehensive )
fyfhqfhtv2
 
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
Tally Prime Crack Download New Version 5.1 [2025] (License Key Free
itskinga12
 
17 Powerful Integrations Your Next-Gen MLM Software Needs
ventaforcemlmsoftwar
 
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
Salesforce Agentforce AI Implementation.pdf
Robert Mark
 
Complete Guide to Website Development in Malaysia for SMEs
Asian Business Services & Technologies
 
Product Update: Alluxio AI 3.7 Now with Sub-Millisecond Latency
Alluxio, Inc.
 
Greta — No-Code AI for Building Full-Stack Web & Mobile Apps
niloyislam1187
 
CapCut Video Editor 6.8.1 Crack for PC Latest Download (Fully Activated) 2025
itskinga12
 
Monitoring Stack: Grafana, Loki & Promtail
GhanshyamSwami3
 
How to Make Money in the Metaverse_ Top Strategies for Beginners.pdf
Herond Labs
 
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 

Infinit filesystem, Reactor reloaded

  • 2. Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage.
  • 3. Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking).
  • 4. Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking). • Distributed: no computer as any specific authority or role. ◦ Availability: no SPOF, network failure resilient. ◦ No admin. As in, no janitor and no tyran. ◦ Scalability flexibility.
  • 5. Infinit filesystem Distributed filesystem in byzantine environment: aggregate multiple computers into one storage. • Filesystem in the POSIX sense of the term. ◦ Works with any client, unmodified. ◦ Fine grained semantic (e.g: video streaming). ◦ Well featured (e.g: file locking). • Distributed: no computer as any specific authority or role. ◦ Availability: no SPOF, network failure resilient. ◦ No admin. As in, no janitor and no tyran. ◦ Scalability flexibility. • Byzantine: you do not need to trust other computers in any way. ◦ No admins. As in, no omniscient god. ◦ Support untrusted networks, both faulty and malicious peers.
  • 8. Coroutines in a nutshell
  • 9. Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO
  • 10. Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK
  • 11. Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK System Coroutines OK OK OK KO KO
  • 12. Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK KO OK System Coroutines OK OK OK KO KO Stackless Coroutines Meh OK OK KO OK
  • 13. Coroutines in a nutshell Intelligible Race conditions Scale Multi core Stack memory System threads OK KO KO OK KO Event-based KO OK OK OK OK System Coroutines OK OK OK OK KO Stackless Coroutines Meh OK OK OK OK
  • 14. Reactor in a nutshell Reactor is a C++ library providing coroutines support, enabling simple and safe imperative style concurrency.
  • 15. Reactor in a nutshell Reactor is a C++ library providing coroutines support, enabling simple and safe imperative style concurrency. while (true) { auto socket = tcp_server.accept(); new Thread([socket] { try { while (true) socket->write(socket->read_until("n")); } catch (reactor::network::Error const&) {} }); }
  • 16. Sugar for common pattern Coroutines are mostly used in three patterns:
  • 17. Sugar for common pattern Coroutines are mostly used in three patterns: • The entirely autonomous detached thread. • The background thread tied to an object. • The parallel flow threads tied to the stack.
  • 18. Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable)
  • 19. Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread)
  • 20. Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now()
  • 21. Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now() Nota bene: • Don't destroy an unfinished thread. Wait for it or terminate it.
  • 22. Basic API for threads Three core calls: • Create a thread that will run callable concurrently: reactor::Thread("name", callable) • Wait until the a thread finishes: reactor::wait(thread) • Terminate a thread: thread->terminate_now() Nota bene: • Don't destroy an unfinished thread. Wait for it or terminate it. • Exceptions escaping a thread terminate the whole scheduler.
  • 23. Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { }); // Will execute until all threads are done. sched.run();
  • 24. Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { reactor::Thread t("t", [] { print("world"); }); print("hello"); }); // Will execute until all threads are done. sched.run();
  • 25. Basic API for reactor reactor::Scheduler sched; reactor::Thread main( sched, "main", [] { reactor::Thread t("t", [] { print("world"); }); print("hello"); reactor::wait(t); }); // Will execute until all threads are done. sched.run();
  • 26. The detached thread The detached thread is a global background operation whose lifetime is tied to the program only.
  • 27. The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p));
  • 28. The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. new reactor::Thread( [] { for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p)); });
  • 29. The detached thread The detached thread is a global background operation whose lifetime is tied to the program only. E.g. uploading crash reports on startup. new reactor::Thread( [] { for (auto p: list_directory(reports_dir)) reactor::http::put("infinit.sh/reports", ifstream(p)); }, reactor::Thread::auto_dispose = true);
  • 30. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime
  • 31. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b) { this->_blocks->put(b); } void _flush() { while (true) this->_store(this->_blocks.get()); } };
  • 32. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); std::unique_ptr<reactor::Thread> _flush_thread; Async() : _flush_thread(new reactor::Thread( [this] { this->_flush(); })) {} };
  • 33. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); std::unique_ptr<reactor::Thread> _flush_thread; Async(); ~Async() { this->_flush_thread->terminate_now(); } };
  • 34. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime class Async { reactor::Channel<Block> _blocks; void store(Block b); void _flush(); Thread::unique_ptr<reactor::Thread> _flush_thread; Async(); };
  • 35. The background thread tied to an object The background thread performs a concurrent operation for an object and is tied to its lifetime struct Terminator : public std::default_delete<reactor::Thread> { void operator ()(reactor::Thread* t) { bool disposed = t->_auto_dispose; if (t) t->terminate_now(); if (!disposed) std::default_delete<reactor::Thread>:: operator()(t); } }; typedef std::unique_ptr<reactor::Thread, Terminator> unique_ptr;
  • 36. The parallel flow threads Parallel flow threads are used to make the local flow concurrent, like parallel control statements. auto data = reactor::http::get("...");; reactor::http::put("http://infinit.sh/...", data); reactor::network::TCPSocket s("hostname"); s.write(data); print("sent!"); GET HTTP PUT TCP PUT SENT
  • 37. The parallel flow threads auto data = reactor::http::get("..."); reactor::Thread http_put( [&] { reactor::http::put("http://infinit.sh/...", data); }); reactor::Thread tcp_put( [&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait({http_put, tcp_put}); print("sent!"); GET SENT HTTP PUT TCP PUT
  • 38. The parallel flow threads auto data = reactor::http::get("..."); std::exception_ptr exn; reactor::Thread http_put([&] { try { reactor::http::put("http://...", data); } catch (...) { exn = std::current_exception(); } }); reactor::Thread tcp_put([&] { try { reactor::network::TCPSocket s("hostname"); s.write(data); } catch (...) { exn = std::current_exception(); } }); reactor::wait({http_put, tcp_put}); if (exn) std::rethrow_exception(exn); print("sent!");
  • 39. The parallel flow threads auto data = reactor::http::get("..."); std::exception_ptr exn; reactor::Thread http_put([&] { try { reactor::http::put("http://...", data); } catch (reactor::Terminate const&) { throw; } catch (...) { exn = std::current_exception(); } }); reactor::Thread tcp_put([&] { try { reactor::network::TCPSocket s("hostname"); s.write(data); } catch (reactor::Terminate const&) { throw; } catch (...) { exn = std::current_exception(); } }); reactor::wait({http_put, tcp_put}); if (exn) std::rethrow_exception(exn); print("sent!");
  • 40. The parallel flow threads auto data = reactor::http::get("..."); reactor::Scope scope; scope.run([&] { reactor::http::put("http://infinit.sh/...", data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait(scope); print("sent!"); GET SENT HTTP PUT TCP PUT
  • 41. Futures auto data = reactor::http::get("..."); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); r.write(data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data); }); reactor::wait(scope); print("sent!");
  • 42. Futures elle::Buffer data; reactor::Thread fetcher( [&] { data = reactor::http::get("..."); }); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); reactor::wait(fetcher); r.write(data); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); reactor::wait(fetcher); s.write(data); }); reactor::wait(scope); print("sent!");
  • 43. Futures reactor::Future<elle::Buffer> data( [&] { data = reactor::http::get("..."); }); reactor::Scope scope; scope.run([&] { reactor::http::Request r("http://infinit.sh/..."); r.write(data.get()); }); scope.run([&] { reactor::network::TCPSocket s("hostname"); s.write(data.get()); }); reactor::wait(scope); print("sent!");
  • 44. Futures Scope + futures SENT GET HTTP PUT TCP PUT Scope GET SENT HTTP PUT TCP PUT Serial GET HTTP PUT TCP PUT SENT
  • 45. Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope);
  • 46. Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope); Concurrent iteration: std::vector<Host> hosts; reactor::concurrent_for( hosts, [] (Host h) { h->send(block); });
  • 47. Concurrent iteration The final touch. std::vector<Host> hosts; reactor::Scope scope; for (auto const& host: hosts) scope.run([] (Host h) { h->send(block); }); reactor::wait(scope); Concurrent iteration: std::vector<Host> hosts; reactor::concurrent_for( hosts, [] (Host h) { h->send(block); }); • Also available: reactor::concurrent_break(). • As for a concurrent continue, that's just return.
  • 48. CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ?
  • 49. CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ? • Idea 1: schedule coroutines in parallel ◦ Pro: absolute best parallelism. ◦ Cons: race conditions.
  • 50. CPU bound operations In the cases we are CPU bound, how to exploit multicore since coroutines are concurrent but not parallel ? • Idea 1: schedule coroutines in parallel ◦ Pro: absolute best parallelism. ◦ Cons: race conditions. • Idea 2: isolate CPU bound code in "monads" and run it in background. ◦ Pro: localized race conditions. ◦ Cons: only manually parallelized code exploits multiple cores.
  • 51. reactor::background reactor::background(callable): run callable in a system thread and return its result.
  • 52. reactor::background reactor::background(callable): run callable in a system thread and return its result. Behind the scene: • A boost::asio_service whose run method is called in as many threads as there are cores. • reactor::background posts the action to Asio and waits for completion.
  • 53. reactor::background reactor::background(callable): run callable in a system thread and return its result. Behind the scene: • A boost::asio_service whose run method is called in as many threads as there are cores. • reactor::background posts the action to Asio and waits for completion. Block::seal(elle::Buffer data) { this->_data = reactor::background( [data=std::move(data)] { return encrypt(data); }); }
  • 54. reactor::background No fiasco rules of thumb: be pure. • No side effects. • Take all bindings by value. • Return any result by value.
  • 55. reactor::background No fiasco rules of thumb: be pure. • No side effects. • Take all bindings by value. • Return any result by value. std::vector<int> ints; int i = 0; // Do not: reactor::background([&] {ints.push_back(i+1);}); reactor::background([&] {ints.push_back(i+2);}); // Do: ints.push_back(reactor::background([i] {return i+1;}); ints.push_back(reactor::background([i] {return i+2;});
  • 56. Background pools Infinit generates a lot of symmetric keys. Let other syscall go through during generation with reactor::background: Block::Block() : _block_keys(reactor::background( [] { return generate_keys(); })) {}
  • 57. Background pools Infinit generates a lot of symmetric keys. Let other syscall go through during generation with reactor::background: Block::Block() : _block_keys(reactor::background( [] { return generate_keys(); })) {} Problem: because of usual on-disk filesystems, operations are often sequential. $ time for i in $(seq 128); touch $i; done The whole operation will still be delayed by 128 * generation_time.
  • 58. Background pools Solution: pregenerate keys in a background pool. reactor::Channel<KeyPair> keys; keys.max_size(64); reactor::Thread keys_generator([&] { reactor::Scope keys; for (int i = 0; i < NCORES; ++i) scope.run([] { while (true) keys.put(reactor::background( [] { return generate_keys(); })); }); reactor::wait(scope); }); Block::Block() : _block_keys(keys.get()) {}
  • 59. Generators Fetching a block with paxos: • Ask the overlay for the replication_factor owners of the block. • Fetch at least replication_factor / 2 + 1 versions. • Paxos the result
  • 60. Generators Fetching a block with paxos: DHT::fetch(Address addr) { auto hosts = this->_overlay->lookup(addr, 3); std::vector<Block> blocks; reactor::concurrent_for( hosts, [&] (Host host) { blocks.push_back(host->fetch(addr)); if (blocks.size() >= 2) reactor::concurrent_break(); }); return some_paxos_magic(blocks); }
  • 61. Generators reactor::Generator<Host> Kelips::lookup(Address addr, factor f) { return reactor::Generator<Host>( [=] (reactor::Generator<Host>::Yield const& yield) { some_udp_socket.write(...); while (f--) yield(kelips_stuff(some_udp_socket.read())); }); }
  • 62. Generators Behind the scene: template <typename T> class Generator { reactor::Channel<T> _values; Generator(Callable call) : _thread([this] { call([this] (T e) {this->_values.put(e);}); }) {} // Iterator interface calling _values.get() }; • Similar to python generators • Except generation starts right away and is not tied to iteration