Scalar DL Technical Overview
0 likes1,711 views
The document provides an overview of Scalar DL, a tamper-evident and scalable database system. Some key points: 1. Scalar DL allows detection of Byzantine faults as long as the number of administrative domains exceeds the number of faulty domains. It can linearly scale performance and availability. 2. The system is database-agnostic and cloud-agnostic, supporting databases like Cassandra and clouds like AWS. 3. The architecture uses ledgers, ordering, and auditors. Ordering extracts parallelism while maintaining determinism. Auditors verify proofs of execution without trusting the ledger. 4. Benchmark results show Scalar DL outperforms Hyperledger Fabric for the Smallbank workload on Amazon
![© 2021Scalar, inc.
Confidential
Scalar DL: Ledger
5
User (Client)
Java
Contract
Signature
Private Key
Scalar DL network
function invoke() {
if (accounts[0].data.balance < args.val) {
throw new Error(“not enough balance”);
}
accounts[0].data.balance -= args.val;
accounts[1].data.balance += args.val;
results = { … };
}
Payment Contract
Request :
(contract, args, sig)
Asset ID Age Data (before) Data (after) Sigs Func (ref) Args Hash
A 1 { } { balance = 100,
…}
charge { val = 100 }
B 1 { }
{ balance = 200,
…} charge { val = 200 }
A 2 { A: {balance = 100},
B: {balance = 200} …}
{ balance = 90,
…}
payment { val = 10 }
SN = Func (SN-1 , Args)
Deterministic & TE TE
If S0 is TE ⇒ SN is TE
Public Key
B 2 { A: {balance = 100},
B: {balance = 200} …}
{ balance = 210,
…} payment { val = 10 }
H(A1)
H(B1)
State
hash
chain
Tamper Evidence (TE) of ledger entries:
Arguments
{ accounts = [“A”, “B”], val = 10, …}
* includes other accounts data
Argument](https://image.slidesharecdn.com/20211201scalardloverview-211207020435/85/Scalar-DL-Technical-Overview-5-320.jpg)
Scalar DL Technical Overview
- 1. Scalar DL Technical Overview 1 Dec, 2021 1
- 2. © 2021Scalar, inc. Confidential Scalar DL: Tamper-Evident and Scalable Database System 2
- 3. © 2021Scalar, inc. Confidential Key Characteristics in Scalar DL 3 Tamper-evident Scalable Correct Database-agnostic Cloud-agnostic Byzantine faults can be detected as long as N > f Performance and availability can be linearly scaled No intermediate states are visible, and data is always up-to-date No dependency on particular database products No dependency on particular public clouds N: # of administrative domains in a database system, f: # of faulty domains Cassandra, Amazon DynamoDB, Azure Cosmos DB, and JDBC-supported major RDBMSs (MySQL, PostgreSQL, Oracle Database) are supported. AWS, Azure, EKS, and AKS are supported. ACID-compliant and strong consistent database system (i.e., strict serializable) DVi = DV / N・RF (RF: Replication Factor << N)
- 4. © 2021Scalar, inc. Confidential Scalar DL System Architecture 4 Pattern 1 Ledger Pattern 2 Pattern 3 Client SDK Ledger Ordering Client SDK Proofs Ledger Client SDK Smart Contract (signed) Private keys Public keys Auditor
- 5. © 2021Scalar, inc. Confidential Scalar DL: Ledger 5 User (Client) Java Contract Signature Private Key Scalar DL network function invoke() { if (accounts[0].data.balance < args.val) { throw new Error(“not enough balance”); } accounts[0].data.balance -= args.val; accounts[1].data.balance += args.val; results = { … }; } Payment Contract Request : (contract, args, sig) Asset ID Age Data (before) Data (after) Sigs Func (ref) Args Hash A 1 { } { balance = 100, …} charge { val = 100 } B 1 { } { balance = 200, …} charge { val = 200 } A 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 90, …} payment { val = 10 } SN = Func (SN-1 , Args) Deterministic & TE TE If S0 is TE ⇒ SN is TE Public Key B 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 210, …} payment { val = 10 } H(A1) H(B1) State hash chain Tamper Evidence (TE) of ledger entries: Arguments { accounts = [“A”, “B”], val = 10, …} * includes other accounts data Argument
- 6. © 2021Scalar, inc. Confidential Scalar DL: Auditor 6 Auditor managed by a different organization Client Execute Asset Proof Request Proof Asset Proof T Ledger Compare to detect tampering Auditor manages the proofs of execution to recompute assets without trusting the integrity of data given from Ledger (1 patented, 1 filed)
- 7. © 2021Scalar, inc. Confidential Scalar DL: Ordering 7 7 Client SDK Scalar DL Ordering Private key Smart Contract (signed) Extracts parallelism by utilizing partial order property of contracts execution without violating determinism (2 patented) Execute contracts independently … Compare to detect tampering
- 8. © 2021Scalar, inc. Confidential Key Difference between Public Blockchains and Scalar DL 8 Public Blockchains (Ethereum …) Scalar DL • History diverges in normal cases – It diverges even if no malicious activities ⇒ Impossible to guarantee finality • History diverges only in abnormal cases – If it diverges, it is caused by malicious activities ⇒ Possible to guarantee finality IEEE ICDCS Workshop’16
- 9. © 2021Scalar, inc. Confidential Key Difference between Private Blockchains (BFT SMR*) and Scalar DL 9 Private Blockchains / BFT SMR (HL fabric, Tendermint…) Scalar DL Data is totally ordered ⇒ Hard to parallelize/scale Data is partially ordered ⇒ Easy to parallelize/scale Sequential processing is required Parallelizable * Private blockchains are thought to be variants of techniques called Byzantine-fault tolerance state machine replication (BFT SMR)
- 10. © 2021Scalar, inc. Confidential Unique Features 1 : ACID Nested Contract Execution • Multiple contracts can be executed atomically 10 Contract 1 Transaction Tamper-evident ledger Contract 2 Contract 3 Atomic execution
- 11. © 2021Scalar, inc. Confidential Unique Features 2 : User-defined Function (patented) • Function is a business logic for creating deletable and easy to search records • Remedy the downsides of ledger structure and tamper-evidence property 11 Mutable database Function Contract Transaction Tamper-evident ledger Atomic execution Business logic Evidence that business logic is executed
- 12. © 2021Scalar, inc. Confidential Benefits of User-defined Function 12 App Not deletable Not searchable flexibly App Not tamper-evident Blockchain DB App DB Blockchain Inconsistent App Scalar DL log log Mutable records Tamper-evident records Tamper-evident Deletable Searchable flexibly Consistent
- 13. © 2021Scalar, inc. Confidential Benchmark Results with Smallbank workload • Fabric 1.4.11: 2 orgs (each has 3 peers) + 3-node Raft orderers • Scalar DL: 2 orgs (each has 3 C* node w/ RF=3) + 3 Scalar DL nodes 13 i3.2xlarge for each peer and database node
- 14. © 2021Scalar, inc. Confidential Scalability 14 Scalar DL achieves near-linear scalability
- 15. © 2021Scalar, inc. Confidential Verification Results • Scalar DL has been heavily tested with Jepsen and our destructive tools – Note that Jepsen tests are created and conducted by Scalar – It has passed both tests for a long time – See https://github.com/scalar-labs/scalar-jepsen for more detail • TLA+ formal verification is also passing 15 Jepsen Passed
- 16. © 2021Scalar, inc. Confidential Use Cases : Where to use Scalar DL 16 • Data integrity is required – Data can not be altered or deleted maliciously • Scalability is required – Many requests from many end-users or devices • There is a main organization – And there are a few auditors • Byzantine fault detection is acceptable Digital evidence / Database forensics Our focuses: