PPT - 2022 - DBOS - a DBMS-oriented Operating System.pdf
- 1. DBOS: a DBMS-oriented Operating System Paper by: Skiadopoulos, Athinagoras, et al. Presentation by: Sahil Naphade Paper by: Skiadopoulos, Athinagoras, et al. Presentation by: Sahil Naphade DBOS: a DBMS-oriented Operating System
- 2. Inspiration Challenges with current OS: • Scale • Cloud proliferation • Parallel computation • Heterogenous HW • New applications • New programming model • Age • Provenance Original Idea: 2020 Partial Execution: 2022
- 3. Proposal (2020) • A new OS with a data-centric architecture All states: Data structures -> DB tables (everything-is-a-file -> everything-is-a- table) State transitions -> use transactions All operations performed as queries Leverage DBMS for all of possible capabilities Separate data from computations OS states represented as uniform data model
- 4. What are the benefits? • Performance Optimization • Security • Virtualization + Containerization • Geographic distribution • (Sophisticated) file management • Better scheduling • Improved state management
- 5. DBOS stack Img. Credit: DBOS (VLDB-2022) https://doi.org/10.14778/3485450.3485454 Layer 4: User space • Distributed applications • Serverless model Layer 3: OS Functionality • Task scheduling, Distributed FS, IPC Layer 2: DBMS • High-performance, multi-node, main-memory T-DB • NoSQL can also be used • Manages own memory Layer 1: Microkernel • No sophisticated Memory Mgmt
- 7. Implementation time! Prototype in 3 stages 1. Straw Possible to provide reasonable performance for 3 operations? a. Task scheduling b. Providing a Filesystem c. Supporting IPC Building the prototype with Layer 1: Linux Layer 2: RDBMS (VoltDB) Layer 3: Coding by hand Layer 4: Test programs Image credits: Google search
- 8. DBMS Straw • Why VoltDB? Parallel, high-performance, multi-node, transactional (+ One more reason) Tables are hashed on a user-specified key across nodes Serializability and transactional failover User-defined DBMS procedures, which are compiled and optimized • As expected, highest performance is obtained when Data in a Single partition User task + data partition -> On the same node • A task and worker Task (p_key#, task_id, worker_id, other_fields) Worker (p_key#, worked_id, unused_capacity)
- 9. DBMS Straw - Scheduling
- 10. DBMS Straw - IPC • Compare against TCP/IP and gRPC • Ping-pong benchmark • A message: Message(sender_id, receiver_id#, message_id, data) • Replicated Message table • In-order delivery with message_id, exactly-once • Limitations: • Periodic Polling -> Mitigate with Triggers (VoltDB does not support!, But Postgres does)
- 11. DBMS Straw – IPC Performance • DBOS achieves 24%–49% lower throughput and 1.3 – 2.5× higher median latency compared to gRPC • DBOS achieves 4–9.5× lower performance than TCP/IP. • Can be further optimized! VoltDB uses TCP/IP as msg substrate Next -> Run bare-bones data transport Next-> Eliminate polling in DBOS Still competitive enough! (Against gRPC)
- 12. DBMS Straw – Filesystems • Transactional and multi-node filesystem • Two filesystems supported 1. Stores data for the user on a single partition – partitioned on user_name 2. Partitioned on block_no • No need of “open” and “close”
- 13. DBMS Straw – Filesystems Perf No need of directory traversal – only single insert. Same for delete!
- 14. DBMS Straw – Filesystems Perf
- 15. Implementation time! Prototype Stages 2. Wood Can OS functions be readily and compactly coded in SQL? Can FS, Scheduling and IPC implementation work well? Installing prototype in Linux is User space Processes -> collection of short-running tasks assembled in a graph Currently On-going! Image credits: Google search
- 16. Implementation time! Prototype Stages 3. Brick Beg, borrow, steal, or implement a micro-kernel Not yet started! Image credits: Google search
- 18. Thank you!