Fault Tolerant Message Passing Distributed Systems An Algorithmic Approach Michel Raynal

Fault Tolerant Message Passing Distributed
Systems An Algorithmic Approach Michel Raynal
download
https://textbookfull.com/product/fault-tolerant-message-passing-
distributed-systems-an-algorithmic-approach-michel-raynal/
Download more ebook instantly today - get yours now at textbookfull.com

We believe these products will be a great fit for you. Click
the link to download now, or visit textbookfull.com
to discover even more!
Adaptive and Fault-Tolerant Control of Underactuated
Nonlinear Systems 1st Edition Jiangshuai Huang
https://textbookfull.com/product/adaptive-and-fault-tolerant-
control-of-underactuated-nonlinear-systems-1st-edition-
jiangshuai-huang/
Advanced methods for fault diagnosis and fault-tolerant
control Steven X. Ding
https://textbookfull.com/product/advanced-methods-for-fault-
diagnosis-and-fault-tolerant-control-steven-x-ding/
Intelligent Video Surveillance Systems: An Algorithmic
Approach First Edition Maheshkumar H. Kolekar
https://textbookfull.com/product/intelligent-video-surveillance-
systems-an-algorithmic-approach-first-edition-maheshkumar-h-
kolekar/
Fault-Tolerant Systems 2nd Edition Koren D.Sc.
Electrical Engineering Israel Institute Of Technology
Haifa
https://textbookfull.com/product/fault-tolerant-systems-2nd-
edition-koren-d-sc-electrical-engineering-israel-institute-of-
technology-haifa/

Robust Integration of Model-Based Fault Estimation and
Fault-Tolerant Control Jianglin Lan
https://textbookfull.com/product/robust-integration-of-model-
based-fault-estimation-and-fault-tolerant-control-jianglin-lan/
Robust and Fault Tolerant Control Neural Network Based
Solutions Krzysztof Patan
https://textbookfull.com/product/robust-and-fault-tolerant-
control-neural-network-based-solutions-krzysztof-patan/
Computational Network Science An Algorithmic Approach
1st Edition Hexmoor
https://textbookfull.com/product/computational-network-science-
an-algorithmic-approach-1st-edition-hexmoor/
Advances in Gain-Scheduling and Fault Tolerant Control
Techniques 1st Edition Damiano Rotondo (Auth.)
https://textbookfull.com/product/advances-in-gain-scheduling-and-
fault-tolerant-control-techniques-1st-edition-damiano-rotondo-
auth/
Bio-Inspired Fault-Tolerant Algorithms for Network-on-
Chip 1st Edition Muhammad Athar Javed Sethi (Author)
https://textbookfull.com/product/bio-inspired-fault-tolerant-
algorithms-for-network-on-chip-1st-edition-muhammad-athar-javed-
sethi-author/

Michel Raynal
Fault-Tolerant
Message-Passing
Distributed
Systems
An Algorithmic Approach

Fault-Tolerant Message-Passing Distributed Systems

Michel Raynal
Fault-Tolerant
Message-Passing
Distributed Systems
An Algorithmic Approach

Michel Raynal
IRISA-ISTIC Université de Rennes 1
Institut Universitaire de France
Rennes, France
Parts of this work are based on the books “Fault-Tolerant Agreement in Synchronous Message-
Passing Systems” and “Communication and Agreement Abstractions for Fault-Tolerant Asynchro-
nous Distributed Systems”, author Michel Raynal, © 2010 Morgan & Claypool Publishers (www.
morganclaypool.com). Used with permission.
ISBN 978-3-319-94140-0 ISBN 978-3-319-94141-7 (eBook)
https://doi.org/10.1007/978-3-319-94141-7
© Springer Nature Switzerland AG 2018
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, express or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Library of Congress Control Number: 2018953101

Preface
La recherche du temps perdu passait par le Web. [...]
La mémoire était devenue inépuisable, mais la profondeur du temps [...] avait disparu.
On était dans un présent inﬁni.
In Les années (2008), Annie Ernaux (1940)
Sed nos immensum spatiis confecimus aequor,
Et iam tempus equum fumentia solvere colla.1
In Georgica, Liber II, 541-542, Publius Virgilius (70 BC–19 BC)
Je suis arrivé au jour où je ne me souviens plus quand j’ai cessé d’être immortel.
In Livro de Crónicas, António Lobo Antunes (1942)
C’est une chose étrange à la ﬁn que le monde
Un jour je m’en irai sans en avoir tout dit.
In Les yeux et la mémoire (1954), chant II, Louis Aragon (1897–1982)
Tout garder, c’est tout détruire.
Jacques Derrida (1930–2004)
1
French: Mais j’ai déjà fourni une vaste carrière, il est temps de dételer les chevaux tout fumants.
English: But now I have traveled a very long way, and the time has come to unyoke my steaming horses.
v

What is distributed computing? Distributed computing was born in the late 1970s when researchers
and practitioners started taking into account the intrinsic characteristic of physically distributed sys-
tems. The field then emerged as a specialized research area distinct from networking, operating sys-
tems, and parallel computing.
Distributed computing arises when one has to solve a problem in terms of distributed entities
(usually called processors, nodes, processes, actors, agents, sensors, peers, etc.) such that each entity
has only a partial knowledge of the many parameters involved in the problem that has to be solved.
While parallel computing and real-time computing can be characterized, respectively, by the terms
efficiency and on-time computing, distributed computing can be characterized by the term uncertainty.
This uncertainty is created by asynchrony, multiplicity of control flows, absence of shared memory
and global time, failure, dynamicity, mobility, etc. Mastering one form or another of uncertainty is
pervasive in all distributed computing problems. A main difficulty in designing distributed algorithms
comes from the fact that no entity cooperating in the achievement of a common goal can have an
instantaneous knowledge of the current state of the other entities, it can only know their past local
states.
Although distributed algorithms are often made up of a few lines, their behavior can be difficult
to understand and their properties hard to state and prove. Hence, distributed computing is not only
a fundamental topic but also a challenging topic where simplicity, elegance, and beauty are first-class
citizens.
Why this book? In the book “Distributed algorithms for message-passing systems” (Springer, 2013),
I addressed distributed computing in failure-free message-passing systems, where the computing enti-
ties (processes) have to cooperate in the presence of asynchrony. Differently, in my book “Concurrent
programming: algorithms, principles and foundations” (Springer, 2013), I addressed distributed com-
puting where the computing entities (processes) communicate through a read/write shared memory
(e.g., multicore), and the main adversary lies in the net effect of asynchrony and process crashes
(unexpected definitive stops).
The present book considers synchronous and asynchronous message-passing systems, where pro-
cesses can commit crash failures, or Byzantine failures (arbitrary behavior). Its aim is to present in a
comprehensive way basic notions, concepts and algorithms in the context of these systems. The main
difficulty comes from the uncertainty created by the adversaries managing the environment (mainly
asynchrony and failures), which, by its very nature, is not under the control of the system.
A quick look at the content of the book The book is composed of four parts, the first two are on
communication abstractions, the other two on agreement abstractions. Those are the most important
abstractions distributed applications rely on in asynchronous and synchronous message-passing sys-
tems where processes may crash, or commit Byzantine failures. The book addresses what can be done
and what cannot be done in the presence of such adversaries. It consequently presents both impossi-
bility results and distributed algorithms. All impossibility results are proved, and all algorithms are
described in a simple algorithmic notation and proved correct.
• Parts on communication abstractions.
– Part I is on the reliable broadcast abstraction.
Preface
vi

– Part II is on the construction of read/write registers.
• Parts on agreement.
– Part III is on agreement in synchronous systems.
– Part IV is on agreement in asynchronous systems.
On the presentation style When known, the names of the authors of a theorem, or of an algorithm,
are indicated together with the date of the associated publication. Moreover, each chapter has a bib-
liographical section, where a short historical perspective and references related to that chapter are
given.
Each chapter terminates with a few exercises and problems, whose solutions can be found in the
article cited at the end of the corresponding exercise/problem.
From a vocabulary point of view, the following terms are used: an object implements an abstrac-
tion, defined by a set of properties, which allows a problem to be solved. Moreover, each algorithm
is first presented intuitively with words, and then proved correct. Understanding an algorithm is a
two-step process:
• First have a good intuition of its underlying principles, and its possible behaviors. This is nec-
essary, but remains informal.
• Then prove the algorithm is correct in the model it was designed for. The proof consists in a
logical reasoning, based on the properties provided by (i) the underlying model, and (ii) the
statements (code) of the algorithm. More precisely, each property defining the abstraction the
algorithm is assumed to implement must be satisfied in all its executions.
Only when these two steps have been done, can we say that we understand the algorithm.
Audience This book has been written primarily for people who are not familiar with the topic and
the concepts that are presented. These include mainly:
• Senior-level undergraduate students and graduate students in informatics or computing engineer-
ing, who are interested in the principles and algorithmic foundations of fault-tolerant distributed
computing.
• Practitioners and engineers who want to be aware of the state-of-the-art concepts, basic princi-
ples, mechanisms, and techniques encountered in fault-tolerant distributed computing.
Prerequisites for this book include undergraduate courses on algorithms, basic knowledge on operat-
ing systems, and notions on concurrency in failure-free distributed computing. One-semester courses,
based on this book, are suggested in the section titled “How to Use This Book” in the Afterword.
Origin of the book and acknowledgments This book has two complementary origins:
• The first is a set of lectures for undergraduate and graduate courses on distributed computing I
gave at the University of Rennes (France), the Hong Kong Polytechnic University, and, as an
invited professor, at several universities all over the world.
Hence, I want to thank the numerous students for their questions that, in one way or another,
contributed to this book.
• The second is the two monographs I wrote in 2010, on fault-tolerant distributed computing,
titled “Communication and agreement abstractions for fault-tolerant asynchronous distributed
Preface vii

systems”, and “Fault-tolerant agreement in synchronous distributed systems”. Parts of them
appear in this book, after having been revised, corrected, and improved.
Hence, I want to thank Morgan & Claypool, and more particularly Diane Cerra, for their per-
mission to reuse parts of this work.
I also want to thank my colleagues (in no particular order) A. Mostéfaoui, D. Imbs, S. Rajsbaum,
V. Gramoli, C. Delporte, H. Fauconnier, F. Taı̈ani, M. Perrin, A. Castañeda, M. Larrea, and Z. Bouzid,
with whom I collaborated in the recent past years. I also thank the Polytechnic University of Hong
Kong (PolyU), and more particularly Professor Jiannong Cao, for hosting me while I was writing parts
of this book. My thanks also to Ronan Nugent (Springer) for his support and his help in putting it all
together.
Last but not least (and maybe most importantly), I thank all the researchers whose results are pre-
sented in this book. Without their work, this book would not exist. (Finally, since I typeset the entire
text myself – L
A
TEX2 for the text and xﬁg for ﬁgures – any typesetting or technical errors that remain
are my responsibility.)
Professor Michel Raynal
Academia Europaea
Institut Universitaire de France
Professor IRISA-ISTIC, Université de Rennes 1, France
Chair Professor, Hong Kong Polytechnic University
June–December 2017
Rennes, Saint-Grégoire, Douelle, Saint-Philibert, Hong Kong,
Vienna (DISC’17), Washington D.C. (PODC’17), Mexico City (UNAM)
Preface
viii

Contents
I Introductory Chapter 1
1 A Few Deﬁnitions and Two Introductory Examples 3
1.1 A Few Deﬁnitions Related to Distributed Computing . . . . . . . . . . . . . . . . . . . 3
1.2 Example 1: Common Decision Despite Message Losses . . . . . . . . . . . . . . . . . 7
1.2.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.2 Trying to Solve the Problem: Attempt 1 . . . . . . . . . . . . . . . . . . . . . 9
1.2.3 Trying to Solve the Problem: Attempt 2 . . . . . . . . . . . . . . . . . . . . . 9
1.2.4 An Impossibility Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.5 A Coordination Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Example 2:
Computing a Global Function Despite a Message Adversary . . . . . . . . . . . . . . . 11
1.3.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.2 The Notion of a Message Adversary . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.3 The TREE-AD Message Adversary . . . . . . . . . . . . . . . . . . . . . . . 13
1.3.4 From Message Adversary to Process Mobility . . . . . . . . . . . . . . . . . . 15
1.4 Main Distributed Computing Models Used in This Book . . . . . . . . . . . . . . . . . 16
1.5 Distributed Computing Versus Parallel Computing . . . . . . . . . . . . . . . . . . . . 17
1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.8 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
II The Reliable Broadcast Communication Abstraction 21
2 Reliable Broadcast in the Presence of Process Crash Failures 23
2.1 Uniform Reliable Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.1 From Best Effort to Guaranteed Reliability . . . . . . . . . . . . . . . . . . . 23
2.1.2 Uniform Reliable Broadcast (URB-broadcast) . . . . . . . . . . . . . . . . . . 24
2.1.3 Building the URB-broadcast Abstraction in CAMPn,t[∅] . . . . . . . . . . . . 25
2.2 Adding Quality of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.1 “First In, First Out” (FIFO) Message Delivery . . . . . . . . . . . . . . . . . . 27
2.2.2 “Causal Order” (CO) Message Delivery . . . . . . . . . . . . . . . . . . . . . 29
2.2.3 From FIFO-broadcast to CO-broadcast . . . . . . . . . . . . . . . . . . . . . 31
2.2.4 From URB-broadcast to CO-broadcast: Capturing Causal Past in a Vector . . . 34
2.2.5 The Total Order Broadcast Abstraction Requires More . . . . . . . . . . . . . 38
2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
ix

x Contents
3 Reliable Broadcast in the Presence of Process Crashes and Unreliable Channels 41
3.1 A System Model with Unreliable Channels . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1.1 Fairness Notions for Channels . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1.2 Fair Channel (FC) and Fair Lossy Channel . . . . . . . . . . . . . . . . . . . 42
3.1.3 Reliable Channel in the Presence of Process Crashes . . . . . . . . . . . . . . 43
3.1.4 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2 URB-broadcast in CAMPn,t[- FC] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.1 URB-broadcast in CAMPn,t[- FC, t n/2] . . . . . . . . . . . . . . . . . . 45
3.3 Failure Detectors: an Approach to Circumvent Impossibilities . . . . . . . . . . . . . . 47
3.3.1 The Concept of a Failure Detector . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3.2 Formal Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4 URB-broadcast in CAMPn,t[- FC] Enriched with a Failure Detector . . . . . . . . . . 49
3.4.1 Definition of the Failure Detector Class Θ . . . . . . . . . . . . . . . . . . . . 49
3.4.2 Solving URB-broadcast in CAMPn,t[- FC, Θ] . . . . . . . . . . . . . . . . . 50
3.4.3 Building a Failure Detector Θ in CAMPn,t[- FC, t n/2] . . . . . . . . . . 50
3.4.4 The Fundamental Added Value Supplied by a Failure Detector . . . . . . . . . 51
3.5 Quiescent Uniform Reliable Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.5.1 The Quiescence Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.5.2 Quiescent URB-broadcast Based on a Perfect Failure Detector . . . . . . . . . 52
3.5.3 The Class HB of Heartbeat Failure Detectors . . . . . . . . . . . . . . . . . . 54
3.5.4 Quiescent URB-broadcast in CAMPn,t[- FC, Θ, HB] . . . . . . . . . . . . . 56
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4 Reliable Broadcast in the Presence of Byzantine Processes 61
4.1 Byzantine Processes and Properties of the Model BAMPn,t[t n/3] . . . . . . . . . 61
4.2 The No-Duplicity Broadcast Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2.3 A No-Duplicity Broadcast Algorithm . . . . . . . . . . . . . . . . . . . . . . 63
4.3 The Byzantine Reliable Broadcast Abstraction . . . . . . . . . . . . . . . . . . . . . . 65
4.4 An Optimal Byzantine Reliable Broadcast Algorithm . . . . . . . . . . . . . . . . . . 66
4.4.1 A Byzantine Reliable Broadcast Algorithm for BAMPn,t[t n/3] . . . . . . 66
4.4.2 Correctness Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.3 Benefiting from Message Asynchrony . . . . . . . . . . . . . . . . . . . . . . 68
4.5 Time and Message-Efficient Byzantine Reliable Broadcast . . . . . . . . . . . . . . . . 69
4.5.1 A Message-Efficient Byzantine Reliable Broadcast Algorithm . . . . . . . . . 70
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
III The Read/Write Register Communication Abstraction 75
5 The Read/Write Register Abstraction 77
5.1 The Read/Write Register Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.1.1 Concurrent Objects and Registers . . . . . . . . . . . . . . . . . . . . . . . . 77

Contents xi
5.1.2 The Notion of a Regular Register . . . . . . . . . . . . . . . . . . . . . . . . 78
5.1.3 Registers Defined from a Sequential Specification . . . . . . . . . . . . . . . . 79
5.2 A Formal Approach to Atomicity and Sequential Consistency . . . . . . . . . . . . . . 81
5.2.1 Processes, Operations, and Events . . . . . . . . . . . . . . . . . . . . . . . . 81
5.2.2 Histories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.2.3 A Formal Definition of Atomicity . . . . . . . . . . . . . . . . . . . . . . . . 84
5.2.4 A Formal Definition of Sequential Consistency . . . . . . . . . . . . . . . . . 84
5.3 Composability of Consistency Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3.1 What Is Composability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3.2 Atomicity Is Composable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3.3 Sequential Consistency Is Not Composable . . . . . . . . . . . . . . . . . . . 87
5.4 Bounds on the Implementation of Strong Consistency Conditions . . . . . . . . . . . . 88
5.4.1 Upper Bound on t for Atomicity . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.4.2 Upper Bound on t for Sequential Consistency . . . . . . . . . . . . . . . . . . 89
5.4.3 Lower Bounds on the Durations of Read and Write Operations . . . . . . . . . 90
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6 Building Read/Write Registers
Despite Asynchrony and Less than Half of Processes Crash (t n/2) 95
6.1 A Structural View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.2 Building an SWMR Regular Read/Write Register in CAMPn,t[t n/2] . . . . . . . . 96
6.2.1 Problem Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.2.2 Implementing an SWMR Regular Register in CAMPn,t[t n/2] . . . . . . . 97
6.2.3 Proof of the SWMR Regular Register Construction . . . . . . . . . . . . . . . 99
6.3 From an SWMR Regular Register to an SWMR Atomic Register . . . . . . . . . . . . 100
6.3.1 Why the Previous Algorithm Does Not Ensure Atomicity . . . . . . . . . . . . 100
6.3.2 From Regularity to Atomicity . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4 From SWMR Atomic Register to MWMR Atomic Register . . . . . . . . . . . . . . . 101
6.4.1 Replacing Sequence Numbers by Timestamps . . . . . . . . . . . . . . . . . . 101
6.4.2 Construction of an MWMR Atomic Register . . . . . . . . . . . . . . . . . . 102
6.4.3 Proof of the MWMR Atomic Register Construction . . . . . . . . . . . . . . . 102
6.5 Implementing Sequentially Consistent Registers . . . . . . . . . . . . . . . . . . . . . 105
6.5.1 How to Address the Non-composability of Sequential Consistency . . . . . . . 105
6.5.2 Algorithms Based on a Total Order Broadcast Abstraction . . . . . . . . . . . 105
6.5.3 A TO-broadcast-based Algorithm with Local (Fast) Read Operations . . . . . 106
6.5.4 A TO-broadcast-based Algorithm with Local (Fast) Write Operations . . . . . 107
6.5.5 An Algorithm Based on Logical Time . . . . . . . . . . . . . . . . . . . . . . 108
6.5.6 Proof of the Logical Time-based Algorithm . . . . . . . . . . . . . . . . . . . 112
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7 Circumventing the t n/2 Read/Write Register Impossibility:
the Failure Detector Approach 119
7.1 The Class Σ of Quorum Failure Detectors . . . . . . . . . . . . . . . . . . . . . . . . 119
7.1.1 Definition of the Class of Quorum Failure Detectors . . . . . . . . . . . . . . 119
7.1.2 Implementing a Failure Detector Σ When t n/2 . . . . . . . . . . . . . . . 120
7.1.3 A Σ-based Construction of an SWSR Atomic Register . . . . . . . . . . . . . 121

xii Contents
7.2 Σ Is the Weakest Failure Detector to Build an Atomic Register . . . . . . . . . . . . . 122
7.2.1 What Does “Weakest Failure Detector Class” Mean . . . . . . . . . . . . . . . 122
7.2.2 The Extraction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.2.3 Correctness of the Extraction Algorithm . . . . . . . . . . . . . . . . . . . . . 124
7.3 Comparing the Failure Detectors Classes Θ and Σ . . . . . . . . . . . . . . . . . . . . 125
7.4 Atomic Register Abstraction vs URB-broadcast Abstraction . . . . . . . . . . . . . . . 126
7.4.1 From Atomic Registers to URB-broadcast . . . . . . . . . . . . . . . . . . . . 126
7.4.2 Atomic Registers Are Strictly Stronger than URB-broadcast . . . . . . . . . . 127
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.7 Exercise and Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8 A Broadcast Abstraction
Suited to the Family of Read/Write Implementable Objects 131
8.1 The SCD-broadcast Communication Abstraction . . . . . . . . . . . . . . . . . . . . . 132
8.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.1.2 Implementing SCD-broadcast in CAMPn,t[t n/2] . . . . . . . . . . . . . . 133
8.1.3 Cost and Proof of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.1.4 An SCD-broadcast-based Communication Pattern . . . . . . . . . . . . . . . . 139
8.2 From SCD-broadcast to an MWMR Register . . . . . . . . . . . . . . . . . . . . . . . 139
8.2.1 Building an MWMR Atomic Register in CAMPn,t[SCD-broadcast] . . . . . . 139
8.2.2 Cost and Proof of Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.2.3 From Atomicity to Sequential Consistency . . . . . . . . . . . . . . . . . . . 142
8.2.4 From MWMR Registers to an Atomic Snapshot Object . . . . . . . . . . . . . 143
8.3 From SCD-broadcast to an Atomic Counter . . . . . . . . . . . . . . . . . . . . . . . . 144
8.3.1 Counter Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
8.3.2 Implementation of an Atomic Counter Object . . . . . . . . . . . . . . . . . . 145
8.3.3 Implementation of a Sequentially Consistent Counter Object . . . . . . . . . . 146
8.4 From SCD-broadcast to Lattice Agreement . . . . . . . . . . . . . . . . . . . . . . . . 147
8.4.1 The Lattice Agreement Task . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.4.2 Lattice Agreement from SCD-broadcast . . . . . . . . . . . . . . . . . . . . . 148
8.5 From SWMR Atomic Registers to SCD-broadcast . . . . . . . . . . . . . . . . . . . . 148
8.5.1 From Snapshot to SCD-broadcast . . . . . . . . . . . . . . . . . . . . . . . . 148
8.5.2 Proof of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
9 Atomic Read/Write Registers in the Presence of Byzantine Processes 155
9.1 Atomic Read/Write Registers in the Presence of Byzantine Processes . . . . . . . . . . 155
9.1.1 Why SWMR (and Not MWMR) Atomic Registers? . . . . . . . . . . . . . . . 155
9.1.2 Reminder on Possible Behaviors of a Byzantine Process . . . . . . . . . . . . 155
9.1.3 SWMR Atomic Registers Despite Byzantine Processes: Definition . . . . . . . 156
9.2 An Impossibility Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
9.3 Reminder on Byzantine Reliable Broadcast . . . . . . . . . . . . . . . . . . . . . . . . 159
9.3.1 Specification of Multi-shot Reliable Broadcast . . . . . . . . . . . . . . . . . 159
9.3.2 An Algorithm for Multi-shot Byzantine Reliable Broadcast . . . . . . . . . . . 159
9.4 Construction of SWMR Atomic Registers in BAMPn,t[t n/3] . . . . . . . . . . . . 161
9.4.1 Description of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
9.4.2 Comparison with the Crash Failure Model . . . . . . . . . . . . . . . . . . . . 163

Contents xiii
9.5 Proof of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
9.5.1 Preliminary Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
9.5.2 Proof of the Termination Properties . . . . . . . . . . . . . . . . . . . . . . . 164
9.5.3 Proof of the Consistency (Atomicity) Properties . . . . . . . . . . . . . . . . . 165
9.5.4 Piecing Together the Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . 166
9.6 Building Objects on Top of SWMR Byzantine Registers . . . . . . . . . . . . . . . . . 166
9.6.1 One-shot Write-snapshot Object . . . . . . . . . . . . . . . . . . . . . . . . . 166
9.6.2 Correct-only Agreement Object . . . . . . . . . . . . . . . . . . . . . . . . . 167
9.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
IV Agreement in Synchronous Systems 171
10 Consensus and Interactive Consistency
in Synchronous Systems Prone to Process Crash Failures 173
10.1 Consensus in the Crash Failure Model . . . . . . . . . . . . . . . . . . . . . . . . . . 173
10.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
10.1.2 A Simple (Unfair) Consensus Algorithm . . . . . . . . . . . . . . . . . . . . 174
10.1.3 A Simple (Fair) Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . . 175
10.2 Interactive Consistency (Vector Consensus) . . . . . . . . . . . . . . . . . . . . . . . . 177
10.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
10.2.2 A Simple Example of Use: Build Atomic Rounds . . . . . . . . . . . . . . . . 178
10.2.3 An Interactive Consistency Algorithm . . . . . . . . . . . . . . . . . . . . . . 178
10.2.5 A Convergence Point of View . . . . . . . . . . . . . . . . . . . . . . . . . . 181
10.3 Lower Bound on the Number of Rounds . . . . . . . . . . . . . . . . . . . . . . . . . 181
10.3.1 Preliminary Assumptions and Definitions . . . . . . . . . . . . . . . . . . . . 182
10.3.2 The (t + 1) Lower Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
10.3.3 Proof of the Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
10.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
11 Expediting Decision
in Synchronous Systems with Process Crash Failures 189
11.1 Early Deciding and Stopping Interactive Consistency . . . . . . . . . . . . . . . . . . . 189
11.1.1 Early Deciding vs Early Stopping . . . . . . . . . . . . . . . . . . . . . . . . 189
11.1.2 An Early Decision Predicate . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
11.1.3 An Early Deciding and Stopping Algorithm . . . . . . . . . . . . . . . . . . . 191
11.1.5 On Early Decision Predicates . . . . . . . . . . . . . . . . . . . . . . . . . . 194
11.1.6 Early Deciding and Stopping Consensus . . . . . . . . . . . . . . . . . . . . . 195
11.2 An Unbeatable Binary Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . . . 196
11.2.1 A Knowledge-Based Unbeatable Predicate . . . . . . . . . . . . . . . . . . . 196
11.2.2 PREF0() with Respect to DIFF() . . . . . . . . . . . . . . . . . . . . . . . . 197
11.2.3 An Algorithm Based on the Predicate PREF0(): CGM . . . . . . . . . . . . . 197
11.2.4 On the Unbeatability of the Predicate PREF0() . . . . . . . . . . . . . . . . . 200
11.3 The Synchronous Condition-based Approach . . . . . . . . . . . . . . . . . . . . . . . 200

xiv Contents
11.3.1 The Condition-based Approach in Synchronous Systems . . . . . . . . . . . . 200
11.3.2 Legality and Maximality of a Condition . . . . . . . . . . . . . . . . . . . . . 201
11.3.3 Hierarchy of Legal Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 203
11.3.4 Local View of an Input Vector . . . . . . . . . . . . . . . . . . . . . . . . . . 204
11.3.5 A Synchronous Condition-based Consensus Algorithm . . . . . . . . . . . . . 204
11.4 Using a Global Clock and a Fast Failure Detector . . . . . . . . . . . . . . . . . . . . . 207
11.4.1 Fast Perfect Failure Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . 207
11.4.2 Enriching the Synchronous Model to Benefit from a Fast Failure Detector . . . 208
11.4.3 A Simple Consensus Algorithm Based on a Fast Failure Detector . . . . . . . 208
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
12 Consensus Variants: Simultaneous Consensus and k-Set Agreement 215
12.1 Simultaneous Consensus: Definition and Its Difficulty . . . . . . . . . . . . . . . . . . 215
12.1.1 Definition of Simultaneous Consensus . . . . . . . . . . . . . . . . . . . . . . 215
12.1.2 Difficulty Early Deciding Before (t + 1) Rounds . . . . . . . . . . . . . . . . 216
12.1.3 Failure Pattern, Failure Discovery, and Waste . . . . . . . . . . . . . . . . . . 216
12.1.4 A Clean Round and the Horizon of a Round . . . . . . . . . . . . . . . . . . . 217
12.2 An Optimal Simultaneous Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . 218
12.2.1 An Optimal Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
12.3 The k-Set Agreement Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12.3.2 A Simple Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
12.4 Early Deciding and Stopping k-Set Agreement . . . . . . . . . . . . . . . . . . . . . . 224
12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
13 Non-blocking Atomic Commitment
in Synchronous Systems with Process Crash Failures 231
13.1 The Non-blocking Atomic Commitment (NBAC) Abstraction . . . . . . . . . . . . . . 231
13.1.1 Definition of Non-blocking Atomic Commitment . . . . . . . . . . . . . . . . 231
13.1.2 A Simple Non-blocking Atomic Commitment Algorithm . . . . . . . . . . . . 232
13.2 Fast Commit and Fast Abort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
13.2.1 Looking for Efficient Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 233
13.3 Weak Fast Commit and Weak Fast Abort . . . . . . . . . . . . . . . . . . . . . . . . . 236
13.4 Fast Commit and Weak Fast Abort Are Compatible . . . . . . . . . . . . . . . . . . . 236
13.4.1 A Fast Commit and Weak Fast Abort Algorithm . . . . . . . . . . . . . . . . 236
13.5 Other Non-blocking Atomic Commitment Algorithms . . . . . . . . . . . . . . . . . . 241
13.5.1 Fast Abort and Weak Fast Commit . . . . . . . . . . . . . . . . . . . . . . . . 241
13.5.2 The Case t ≤ 2 (System Model CSMPn,t[1 ≤ t 3 ≤ n]) . . . . . . . . . . . 242
13.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

Contents xv
14 Consensus in Synchronous Systems Prone to Byzantine Process Failures 245
14.1 Agreement Despite Byzantine Processes . . . . . . . . . . . . . . . . . . . . . . . . . 246
14.1.1 On the Agreement and Validity Properties . . . . . . . . . . . . . . . . . . . . 246
14.1.2 A Consensus Definition for the Byzantine Failure Model . . . . . . . . . . . . 246
14.1.3 An Interactive Consistency Definition for the Byzantine Failure Model . . . . 247
14.1.4 The Byzantine General Agreement Abstraction . . . . . . . . . . . . . . . . . 247
14.2 Interactive Consistency for Four Processes Despite One Byzantine Process . . . . . . . 247
14.2.1 An Algorithm for n = 4 and t = 1 . . . . . . . . . . . . . . . . . . . . . . . . 247
14.3 An Upper Bound on the Number of Byzantine Processes . . . . . . . . . . . . . . . . . 249
14.4 A Byzantine Consensus Algorithm for BSMPn,t[t n/3] . . . . . . . . . . . . . . . . 251
14.4.1 Base Data Structure: a Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
14.4.2 EIG Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
14.4.3 Example of an Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
14.4.4 Proof of the EIG Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
14.5 A Simple Consensus Algorithm with Constant Message Size . . . . . . . . . . . . . . 257
14.5.1 Features of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
14.5.2 Presentation of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 257
14.5.3 Proof and Properties of the Algorithm . . . . . . . . . . . . . . . . . . . . . . 258
14.6 From Binary to Multivalued Byzantine Consensus . . . . . . . . . . . . . . . . . . . . 259
14.6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
14.6.2 A Reduction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14.6.3 Proof of the Multivalued to Binary Reduction . . . . . . . . . . . . . . . . . . 261
14.6.4 An Interesting Property of the Construction . . . . . . . . . . . . . . . . . . . 263
14.7 Enriching the Synchronous Model with Message Authentication . . . . . . . . . . . . . 263
14.7.1 Synchronous Model with Signed Messages . . . . . . . . . . . . . . . . . . . 263
14.7.2 The Gain Obtained from Signatures . . . . . . . . . . . . . . . . . . . . . . . 264
14.7.3 A Synchronous Signature-Based Consensus Algorithm . . . . . . . . . . . . . 264
14.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
V Agreement in Asynchronous Systems 269
15 Implementable Agreement Abstractions
Despite Asynchrony and a Minority of Process Crashes 271
15.1 The Renaming Agreement Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . 271
15.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
15.1.2 A Fundamental Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
15.1.3 The Stacking Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
15.1.4 A Snapshot-based Implementation of Renaming . . . . . . . . . . . . . . . . 274
15.2 The Approximate Agreement Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . 276
15.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
15.2.2 A Read/Write-based Implementation of Approximate Agreement . . . . . . . 277

xvi Contents
15.3 The Safe Agreement Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
15.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
15.3.2 A Direct Implementation of Safe Agreement in CAMPn,t[t n/2] . . . . . . 280
15.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
16 Consensus:
Power and Implementability Limit in Crash-Prone Asynchronous Systems 287
16.1 The Total Order Broadcast Communication Abstraction . . . . . . . . . . . . . . . . . 287
16.1.1 Total Order Broadcast: Definition . . . . . . . . . . . . . . . . . . . . . . . . 287
16.1.2 A Map of Communication Abstractions . . . . . . . . . . . . . . . . . . . . . 288
16.2 From Consensus to TO-broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
16.2.1 Structure of the Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
16.2.2 Description of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
16.3 Consensus and TO-broadcast Are Equivalent . . . . . . . . . . . . . . . . . . . . . . . 292
16.4 The State Machine Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
16.4.1 State Machine Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
16.4.2 Sequentially-Defined Abstractions (Objects) . . . . . . . . . . . . . . . . . . 294
16.5 A Simple Consensus-based Universal Construction . . . . . . . . . . . . . . . . . . . . 295
16.6 Agreement vs Mutual Exclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
16.7 Ledger Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
16.7.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
16.7.2 Implementation of a Ledger in CAMPn,t[TO-broadcast] . . . . . . . . . . . . 299
16.8 Consensus Impossibility in the Presence of Crashes and Asynchrony . . . . . . . . . . 300
16.8.1 The Intuition That Underlies the Impossibility . . . . . . . . . . . . . . . . . . 300
16.8.2 Refining the Definition of CAMPn,t[∅] . . . . . . . . . . . . . . . . . . . . . 301
16.8.3 Notion of Valence of a Global State . . . . . . . . . . . . . . . . . . . . . . . 303
16.8.4 Consensus Is Impossible in CAMPn,1[∅] . . . . . . . . . . . . . . . . . . . . 304
16.9 The Frontier Between Read/Write Registers and Consensus . . . . . . . . . . . . . . . 309
16.9.1 The Main Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
16.9.2 The Notion of Consensus Number in Read/Write Systems . . . . . . . . . . . 310
16.9.3 An Illustration of Herlihy’s Hierarchy . . . . . . . . . . . . . . . . . . . . . . 310
16.9.4 The Consensus Number of a Ledger . . . . . . . . . . . . . . . . . . . . . . . 313
16.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
17 Implementing Consensus in Enriched Crash-Prone Asynchronous Systems 317
17.1 Enriching an Asynchronous System to Implement Consensus . . . . . . . . . . . . . . 317
17.2 A Message Scheduling Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
17.2.1 Message Scheduling (MS) Assumption . . . . . . . . . . . . . . . . . . . . . 318
17.2.2 A Binary Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 318
17.2.4 Additional Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
17.3 Enriching CAMPn,t[∅] with a Perpetual Failure Detector . . . . . . . . . . . . . . . . 321
17.3.1 Enriching CAMPn,t[∅] with a Perfect Failure Detector . . . . . . . . . . . . . 321

Contents xvii
17.4 Enriching CAMPn,t[t n/2] with an Eventual Leader . . . . . . . . . . . . . . . . . 323
17.4.1 The Weakest Failure Detector to Implement Consensus . . . . . . . . . . . . . 323
17.4.2 Implementing Consensus in CAMPn,t[t n/2, Ω] . . . . . . . . . . . . . . 324
17.4.4 Consensus Versus Eventual Leader Failure Detector . . . . . . . . . . . . . . 329
17.4.5 Notions of Indulgence and Zero-degradation . . . . . . . . . . . . . . . . . . 329
17.4.6 Saving Broadcast Instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
17.5 Enriching CAMPn,t[t n/2] with Randomization . . . . . . . . . . . . . . . . . . . 330
17.5.1 Asynchronous Randomized Models . . . . . . . . . . . . . . . . . . . . . . . 330
17.5.2 Randomized Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
17.5.3 Randomized Binary Consensus in CAMPn,t[t n/2, LC] . . . . . . . . . . . 331
17.5.4 Randomized Binary Consensus in CAMPn,t[t n/2, CC] . . . . . . . . . . . 334
17.6 Enriching CAMPn,t[t n/2] with a Hybrid Approach . . . . . . . . . . . . . . . . . 337
17.6.1 The Hybrid Approach: Failure Detector and Randomization . . . . . . . . . . 337
17.6.2 A Hybrid Binary Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . 338
17.7 A Paxos-inspired Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 339
17.7.1 The Alpha Communication Abstraction . . . . . . . . . . . . . . . . . . . . . 340
17.7.2 Consensus Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
17.7.3 An Implementation of Alpha in CAMPn,t[t n/2] . . . . . . . . . . . . . . 341
17.8 From Binary to Multivalued Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . 344
17.8.2 Proof of the Reduction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 345
17.9 Consensus in One Communication Step . . . . . . . . . . . . . . . . . . . . . . . . . . 346
17.9.1 Aim and Model Assumption on t . . . . . . . . . . . . . . . . . . . . . . . . 346
17.9.2 A One Communication Step Algorithm . . . . . . . . . . . . . . . . . . . . . 346
17.9.3 Proof of the Early Deciding Algorithm . . . . . . . . . . . . . . . . . . . . . 347
17.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
18 Implementing Oracles
in Asynchronous Systems with Process Crash Failures 353
18.1 The Two Facets of Failure Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
18.1.1 The Programming Point of View: Modular Building Block . . . . . . . . . . . 354
18.1.2 The Computability Point of View: Abstraction Ranking . . . . . . . . . . . . 354
18.2 Ω in CAMPn,t[∅]: a Direct Impossibility Proof . . . . . . . . . . . . . . . . . . . . . . 355
18.3 Constructing a Perfect Failure Detector (Class P) . . . . . . . . . . . . . . . . . . . . 356
18.3.1 Reminder: Definition of the Class P of Perfect Failure Detectors . . . . . . . . 356
18.3.2 Use of an Underlying Synchronous System . . . . . . . . . . . . . . . . . . . 357
18.3.3 Applications Generating a Fair Communication Pattern . . . . . . . . . . . . . 358
18.3.4 The Theta Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
18.4 Constructing an Eventually Perfect Failure Detector (Class 3P) . . . . . . . . . . . . . 361
18.4.1 Reminder: Definition of an Eventually Perfect Failure Detector . . . . . . . . 361
18.4.2 From Perpetual to Eventual Properties . . . . . . . . . . . . . . . . . . . . . . 361
18.4.3 Eventually Synchronous Systems . . . . . . . . . . . . . . . . . . . . . . . . 361
18.5 On the Efficient Monitoring of a Process by Another Process . . . . . . . . . . . . . . 363
18.5.1 Motivation and System Model . . . . . . . . . . . . . . . . . . . . . . . . . . 363
18.5.2 A Monitoring Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
18.6 An Adaptive Monitoring-based Algorithm Building 3P . . . . . . . . . . . . . . . . . 366
18.6.1 Motivation and Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

xviii Contents
18.6.2 A Monitoring-Based Adaptive Algorithm for the Failure Detector Class 3P . . 366
18.6.3 Proof the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
18.7 From the t-Source Assumption to an Ω Eventual Leader . . . . . . . . . . . . . . . . . 369
18.7.1 The 3t-Source Assumption and the Model CAMPn,t[3t-SOURCE] . . . . . 369
18.7.2 Electing an Eventual Leader in CAMPn,t[3t-SOURCE] . . . . . . . . . . . . 370
18.8 Electing an Eventual Leader in CAMPn,t[3t-MS PAT] . . . . . . . . . . . . . . . . . 372
18.8.1 A Query/Response Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
18.8.2 Electing an Eventual Leader in CAMPn,t[3t-MS PAT] . . . . . . . . . . . . 374
18.9 Building Ω in a Hybrid Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
18.10 Construction of a Biased Common Coin from Local Coins . . . . . . . . . . . . . . . . 377
18.10.1 Definition of a Biased Common Coin . . . . . . . . . . . . . . . . . . . . . . 377
18.10.2 The CORE Communication Abstraction . . . . . . . . . . . . . . . . . . . . . 377
18.10.3 Construction of a Common Coin with a Constant Bias . . . . . . . . . . . . . 380
18.10.4 On the Use of a Biased Common Coin . . . . . . . . . . . . . . . . . . . . . . 381
18.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
18.12 Bibliographic notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
19 Implementing Consensus in Enriched Byzantine Asynchronous Systems 385
19.1 Definition Reminder and Two Observations . . . . . . . . . . . . . . . . . . . . . . . . 385
19.1.1 Definition of Byzantine Consensus (Reminder) . . . . . . . . . . . . . . . . . 385
19.1.2 Why Not to Use an Eventual Leader . . . . . . . . . . . . . . . . . . . . . . . 386
19.1.3 On the Weakest Synchrony Assumption for Byzantine Consensus . . . . . . . 386
19.2 Binary Byzantine Consensus from a Message Scheduling Assumption . . . . . . . . . 387
19.2.1 A Message Scheduling Assumption . . . . . . . . . . . . . . . . . . . . . . . 387
19.2.2 A Binary Byzantine Consensus Algorithm . . . . . . . . . . . . . . . . . . . . 387
19.2.4 Additional Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
19.3 An Optimal Randomized Binary Byzantine Consensus Algorithm . . . . . . . . . . . . 389
19.3.1 The Binary-Value Broadcast Abstraction . . . . . . . . . . . . . . . . . . . . 389
19.3.2 A Binary Randomized Consensus Algorithm . . . . . . . . . . . . . . . . . . 391
19.3.3 Proof of the BV-Based Binary Byzantine Consensus Algorithm . . . . . . . . 393
19.3.4 From Decision to Decision and Termination . . . . . . . . . . . . . . . . . . . 395
19.4 From Binary to Multivalued Byzantine Consensus . . . . . . . . . . . . . . . . . . . . 396
19.4.2 Proof of the Reduction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 398
19.5 From Binary to No-intrusion Multivalued Byzantine Consensus . . . . . . . . . . . . . 399
19.5.1 The Validated Byzantine Broadcast Abstraction . . . . . . . . . . . . . . . . . 399
19.5.2 An Algorithm Implementing VBB-broadcast . . . . . . . . . . . . . . . . . . 399
19.5.3 Proof of the VBB-broadcast Algorithm . . . . . . . . . . . . . . . . . . . . . 401
19.5.4 A VBB-Based Multivalued to Binary Byzantine Consensus Reduction . . . . . 402
19.5.5 Proof of the VBB-Based Reduction Algorithm . . . . . . . . . . . . . . . . . 403
19.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
19.7 Appendix:
Proof-of-Work (PoW) Seen as Eventual Byzantine Agreement . . . . . . . . . . . . . . 405

Contents xix
VI Appendix 409
20 Quorum, Signatures, and Overlays 411
20.1 Quorum Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
20.1.1 Deﬁnitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
20.1.2 Examples of Use of a Quorum System . . . . . . . . . . . . . . . . . . . . . . 412
20.1.3 A Few Classical Quorums . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
20.1.4 Quorum Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
20.2 Digital Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
20.2.1 Cipher, Keys, and Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
20.2.2 How to Build a Secret Key: Difﬁe-Hellman’s Algorithm . . . . . . . . . . . . 416
20.2.3 How to Build a Public Key: Rivest-Shamir-Adleman’s (RSA) Algorithm . . . 417
20.2.4 How to Share a Secret: Shamir’s Algorithm . . . . . . . . . . . . . . . . . . . 417
20.3 Overlay Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
20.3.1 On Regular Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
20.3.2 Hypercube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
20.3.3 de Bruijn Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
20.3.4 Kautz Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
20.3.5 Undirected de Bruijn and Kautz Graphs . . . . . . . . . . . . . . . . . . . . . 422
Afterword 425
Bibliography 431
Index 453

Notation
Symbols
skip, no-op empty statement
process program in action
n number of processes
correct (or non-faulty) process process that does not fail during an execution
faulty process process that fails during an execution
t upper bound on the number of faulty of processes
f actual number of faulty of processes
pi process whose index (or identity) is i
idi identity of process pi (very often idi = i)
τ time instant (from an external observer point of view)
[1..m] set {1, ..., m}
AA[1..m] array with m entries (vector)
equal(a, I) occurrence number of a in the vector (or multiset) I
a, b pair with elements a and b
a, b, c triple with elements a, b, and c
XX small capital letters: message type (message tag)
xxi italics lower-case letters: local variable of process pi
xxi ← v assignment of value v to xxi
XX abstract variable known only by an external observer
xxr
i , XXr
values of xxi, XX at the end of round r
m1; ...; mq sequence of messages
ai[1..s] array of size s (local to process pi)
for each i ∈ {1, ..., m} do statements end for order irrelevant
for each i from 1 to m do statements end for order relevant
wait (P) while ¬P do no-op end while
return (v) returns v and terminates the operation invocation
% blablabla % comments
; sequentiality operator between two statements
⊕ concatenation
empty sequence (list)
|σ| size of the sequence σ
The notation broadcast TYPE(m), where TYPE is a message type and m a message content, is used
as a shortcut for “for each j ∈ {1, · · · , n} do send TYPE(m) to pj end for”. Hence, if it is not faulty
during its execution, pi sends the message TYPE(m) to each process, including itself. Otherwise there
is no guarantee on the reception of TYPE(m).
(In Chap. 1 only, j ∈ {1, · · · , n} is replaced by j ∈ neighborsi .)
xxi

Acronyms (1)
SWMR single-writer/multi-reader register
MWSR multi-writer/single-reader register
SWMR single-writer/multi-reader register
CAMP Crash asynchronous message-passing
CSMP Crash synchronous message-passing
BAMP Byzantine asynchronous message-passing
BSMP Byzantine synchronous message-passing
EIG Exponential information gathering
RB Reliable broadcast
URB Uniform reliable broadcast
ND No-duplicity broadcast
BRB Byzantine reliable broadcast
BV Byzantine binary value broadcast
VBB Validated Byzantine broadcast
CC Consensus in the process crash model
BC Consensus in the Byzantine process model
SA Set-agreement
BBC Byzantine binary consensus
ICC Interactive consistency (vector consensus), crash model
SC Simultaneous (synchronous) consensus
CORE CORE-broadcast
CC-property Crash consensus property
BC-property Byzantine consensus property
xxii Notation

Acronyms (2)
CO Causal order
FIFO First in ﬁrst out
TO Total order
SCD Set-constrained delivery
FC Fair channel
CRDT Conﬂict-free replicated data type
MS PAT Message pattern
ADV Adversary
FD Failure detector
HB Heartbeat
MS PAT Message pattern
SO Send omission
GO General omission
MS Message scheduling assumption
LC Local coin
CC Common coin
BCCB Binary common coin with bias
GST Global stabilization time
xxiii
Notation

List of Figures and Algorithms
1.1 Basic structure of distributed computing . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Three graph types of particular interest . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Synchronous execution (left) vs. asynchronous execution (right) . . . . . . . . . . . 5
1.4 Algorithm structure of a common decision-making process . . . . . . . . . . . . . . 8
1.5 A simple distributed computing framework . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Examples of graphs produced by a message adversary . . . . . . . . . . . . . . . . 13
1.7 Distributed computation in SMPn[TREE-AD] (code for pi) . . . . . . . . . . . . . 14
1.8 The property limiting the power of a TREE-AD message adversary . . . . . . . . . 14
1.9 Process mobility can be captured by a message adversary in synchronous systems . . 16
1.10 Sequential or parallel computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 An example of the uniform reliable broadcast delivery guarantees . . . . . . . . . . 25
2.2 URB-broadcast: architectural view . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 Uniform reliable broadcast in CAMPn,t[∅] (code for pi) . . . . . . . . . . . . . . . 26
2.4 From URB to FIFO-URB and CO-URB in CAMPn,t[∅] . . . . . . . . . . . . . . . 27
2.5 An example of FIFO-URB message delivery . . . . . . . . . . . . . . . . . . . . . 28
2.6 FIFO-URB uniform reliable broadcast: architecture view . . . . . . . . . . . . . . . 28
2.7 FIFO-URB message delivery in ASn,t[∅] (code for pi) . . . . . . . . . . . . . . . . 29
2.8 An example of CO message delivery . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.9 A simple URB-based CO-broadcast construction in CAMPn,t[∅] (code for pi) . . . 31
2.10 From FIFO-URB to CO-URB message delivery in ASn,t[∅] (code for pi) . . . . . . 32
2.11 How the sequence of messages im causal pasti is built . . . . . . . . . . . . . . . 32
2.12 From URB to CO message delivery in ASn,t[∅] (code for pi) . . . . . . . . . . . . . 35
2.13 How vectors are used to construct the CO-broadcast abstraction . . . . . . . . . . . 36
2.14 Proof of the CO-delivery property (second construction) . . . . . . . . . . . . . . . 37
2.15 Total order message delivery requires cooperation . . . . . . . . . . . . . . . . . . 38
2.16 Broadcast of lifetime-constrained messages . . . . . . . . . . . . . . . . . . . . . . 40
3.1 Uniform reliable broadcast in CAMPn,t[- FC, t n/2] (code for pi) . . . . . . . . 45
3.2 Building Θ in CAMPn,t[- FC, t n/2] (code for pi) . . . . . . . . . . . . . . . . 50
3.3 Quiescent uniform reliable broadcast in CAMPn,t[- FC, Θ, P] (code for pi) . . . . 53
3.4 Quiescent uniform reliable broadcast in CAMPn,t[- FC, Θ, HB] (code for pi) . . . 56
3.5 An example of a network with fair paths . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1 Implementing ND-broadcast in BAMPn,t[t n/3] . . . . . . . . . . . . . . . . . 64
4.2 An example of ND-broadcast with a Byzantine sender . . . . . . . . . . . . . . . . 65
4.3 Implementing BRB-broadcast in BAMPn,t[t n/3] . . . . . . . . . . . . . . . . . 67
4.4 Beneﬁting from message asynchrony . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.5 Exploiting message asynchrony . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.6 Communication-efﬁcient Byzantine BRB-broadcast in BAMPn,t[t n/5] . . . . . 70
xxv

xxvi List of Figures and Algorithms
5.1 Possible behaviors of a regular register . . . . . . . . . . . . . . . . . . . . . . . . 78
5.2 A regular register has no sequential speciﬁcation . . . . . . . . . . . . . . . . . . . 79
5.3 Behavior of an atomic register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.4 Behavior of a sequentially consistent register . . . . . . . . . . . . . . . . . . . . . 81
5.5 Example of a history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.6 Partial order on the operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.7 Developing op1 →H op2 →X op3 →H op4 . . . . . . . . . . . . . . . . . . . . . 86
5.8 The execution of the register R is sequentially consistent . . . . . . . . . . . . . . . 87
5.9 The execution of the register R is sequentially consistent . . . . . . . . . . . . . . 87
5.10 An execution involving the registers R and R . . . . . . . . . . . . . . . . . . . . . 87
5.11 There is no atomic register algorithm in CAMPn,t[∅] . . . . . . . . . . . . . . . . . 88
5.12 There is no algorithm for two sequentially consistent registers in CAMPn,t[t ≥ n/2] 89
5.13 Tradeoff duration(read) + duration(write) ≥ δ . . . . . . . . . . . . . . . . . . . 91
5.14 duration(write) ≥ u/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.1 Building a read/write memory on top of CAMPn,t[t ≤ n/2] . . . . . . . . . . . . . 96
6.2 An algorithm that constructs an SWMR regular register in CAMPn,t[t n/2] . . . 98
6.3 Regularity is not atomicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4 SWMR register: from regularity to atomicity . . . . . . . . . . . . . . . . . . . . . 101
6.5 Construction of an atomic MWMR register in CAMPn,t[t n/2] (code for any pi) 103
6.6 Fast read algorithm implementing sequential consistency (code for pi) . . . . . . . . 106
6.7 Beneﬁting from TO-broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.8 Fast write algorithm implementing sequential consistency (code for pi) . . . . . . . 108
6.9 Fast enqueue algorithm implementing a sequentially consistent queue (code for pi) . 108
6.10 Construction of a sequentially consistent MWMR register in CAMPn,t[t n/2]
(code for pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.11 Message exchange pattern for a write operation . . . . . . . . . . . . . . . . . . . . 110
6.12 First message exchange pattern for a read operation . . . . . . . . . . . . . . . . . . 111
6.13 Logical time vs. physical time for write operations . . . . . . . . . . . . . . . . . . 112
6.14 An execution
Hd |X in which resp(op1) Hd |X inv(read2) . . . . . . . . . . . . 113
7.1 Building a failure detector of the class Σ in CAMPn,t[t n/2] . . . . . . . . . . . 120
7.2 An algorithm for an atomic SWSR register in CAMPn,t[Σ] . . . . . . . . . . . . . 121
7.3 Extracting Σ from a register D-based algorithm A . . . . . . . . . . . . . . . . . . 122
7.4 Extracting Σ from a failure detector-based register algorithm A (code for pi) . . . . 124
7.5 From atomic registers to URB-broadcast (code for pi) . . . . . . . . . . . . . . . . 127
7.6 From the failure detector class Σ to the URB abstraction (1 ≤ t n) . . . . . . . . 128
7.7 Two examples of the hybrid communication model . . . . . . . . . . . . . . . . . . 129
8.1 An implementation of SCD-broadcast in CAMPn,t[t n/2] (code for pi) . . . . . 134
8.2 Message pattern introduced in Lemma 16 . . . . . . . . . . . . . . . . . . . . . . . 137
8.3 SCD-broadcast-based communication pattern (code for pi) . . . . . . . . . . . . . . 139
8.4 Construction of an MWMR atomic register in CAMPn,t[SCD-broadcast] (code for
pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
8.5 Construction of an MWMR sequentially consistent register in CAMPn,t[SCD-broadcast]
(code for pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8.6 Example of a run of an MWMR atomic snapshot object . . . . . . . . . . . . . . . 143
8.7 Construction of an MWMR atomic snapshot object in CAMPn,t[SCD-broadcast] . . 144
8.8 Construction of an atomic counter in CAMPn,t[SCD-broadcast] (code for pi) . . . . 145

List of Figures and Algorithms xxvii
8.9 Construction of a sequentially consistent counter in CAMPn,t[SCD-broadcast] (code
for pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.10 Solving lattice agreement in CAMPn,t[SCD-broadcast] (code for pi) . . . . . . . . 148
8.11 An implementation of SCD-broadcast on top of snapshot objects (code for pi) . . . . 149
9.1 Execution E1 (impossibility of an SWMR register in BAMPn,t[t ≥ n/3]) . . . . . 157
9.4 Reliable broadcast with sequence numbers in BAMPn,t[t n/3] (code for pi) . . . 160
9.5 Atomic SWMR Registers in BAMPn,t[t n/3] (code for pi) . . . . . . . . . . . . 162
9.6 One-shot write-snapshot in BAMPn,t[t n/3] (code for pi) . . . . . . . . . . . . . 167
9.7 Correct-only agreement in BAMPn,t[t n/(w + 1)] . . . . . . . . . . . . . . . . 168
10.1 A simple (unfair) t-resilient consensus algorithm in CSMPn,t[∅] (code for pi) . . . . 175
10.2 A simple (fair) t-resilient consensus algorithm in CSMPn,t[∅] (code for pi) . . . . . 176
10.3 The second case of the agreement property (with t = 3 crashes) . . . . . . . . . . . 177
10.4 A t-resilient interactive consistency algorithm in CSMPn,t[∅] (code for pi) . . . . . 179
10.5 Three possible one-round extensions from Et−1 . . . . . . . . . . . . . . . . . . . . 183
10.6 Extending the k-round execution Ek . . . . . . . . . . . . . . . . . . . . . . . . . . 184
10.7 Extending two (k + 1)-round executions . . . . . . . . . . . . . . . . . . . . . . . 185
10.8 Extending again two (k + 1)-round executions . . . . . . . . . . . . . . . . . . . . 185
11.1 Early decision predicate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
11.2 An early deciding t-resilient interactive consistency algorithm (code for pi) . . . . . 192
11.3 Early stopping synchronous consensus (code for pi, t n) . . . . . . . . . . . . . . 195
11.4 The early decision predicate revealed0(i, r) in action . . . . . . . . . . . . . . . . . 197
11.5 Local graphs of p2, p3, and p4 at the end of round r = 1 . . . . . . . . . . . . . . . 198
11.6 Local graphs of p3 and p4 at the end of round r = 2 . . . . . . . . . . . . . . . . . 198
11.7 CGM : Early deciding synchronous consensus based on PREF0() (code for pi, t n)199
11.8 Hierarchy of classes of conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
11.9 A condition-based consensus algorithm (code for pi) . . . . . . . . . . . . . . . . . 205
11.10 Synchronous consensus with a fast failure detector (code for pi) . . . . . . . . . . . 209
11.11 Relevant dates for process pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
11.12 Early deciding synchronous consensus with a fast failure detector (code for pi) . . . 211
11.13 The pattern used in the proof of the CC-agreement property . . . . . . . . . . . . . 211
12.1 Clean round vs failure-free round . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
12.2 Existence of a clean round . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
12.3 Optimal simultaneous consensus in the system model CSMPn,t[∅] (code for pi) . . . 219
12.4 Computing the current horizon value . . . . . . . . . . . . . . . . . . . . . . . . . 219
12.5 A simple k-set agreement algorithm for the model CSMPn,t[∅] (code for pi) . . . . 223
12.6 Early stopping synchronous k-set agreement (code for pi, t n) . . . . . . . . . . . 224
12.7 The differential predicate PREF(i, r) for k-set agreement . . . . . . . . . . . . . . 224
12.8 A condition-based simultaneous consensus algorithm (code for pi) . . . . . . . . . . 228
12.9 A simple k-set agreement algorithm for the model CSMPn,t[SO] (code for pi) . . . 229
13.1 A consensus-based NBAC algorithm in CSMPn,t[∅] (code for pi) . . . . . . . . . . 232
13.2 Impossibility of having both fast commit and fast abort when t ≥ 3 (E3) . . . . . . . 234
13.3 Impossibility of having both fast commit and fast abort when t ≥ 3 (E4, E5) . . . . 235
13.4 Fast commit and weak fast abort NBAC in CSMPn,t[3 ≤ t n] (code for pi) . . . . 237
13.5 Fast abort and weak fast commit NBAC in CSMPn,t[3 ≤ t n] (code for pi) . . . . 242

xxviii List of Figures and Algorithms
13.6 Fast commit and fast abort NBAC in the system model CSMPn,t[t ≤ 2] (code for pi) 243
14.1 Interactive consistency for four processes despite one Byzantine process (code for pi)248
14.2 Proof of the interactive consistency algorithm in BSMPn,t[t = 1, n = 4] . . . . . . 249
14.3 Communication graph (left) and behavior of the t Byzantine processes (right) . . . . 251
14.4 EIG tree for n = 4 and t = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
14.5 Byzantine EIG consensus algorithm for BSMPn,t[t n/3] . . . . . . . . . . . . . 253
14.6 EIG trees of the correct processes at the end of the ﬁrst round . . . . . . . . . . . . 254
14.7 EIG tree tree2 at the end of the second round . . . . . . . . . . . . . . . . . . . . . 255
14.8 Constant message size Byzantine consensus in BSMPn,t[t n/4] . . . . . . . . . . 258
14.9 From binary to multivalued Byzantine consensus in BSMPn,t[t n/3] (code for pi) 260
14.10 Proof of Property PR2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
14.11 Deterministic vs non-deterministic scenarios . . . . . . . . . . . . . . . . . . . . . 263
14.12 A Byzantine signature-based consensus algorithm in BSMPn,t[SIG; t n/2]
(code for pi) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
15.1 Stacking of abstraction layers for distributed renaming in CAMPn,t[t n/2] . . . . 273
15.2 A simple snapshot-based size-adaptive (2p − 1)-renaming algorithm (code for pi) . 274
15.3 A simple snapshot-based approximate algorithm (code for pi) . . . . . . . . . . . . 277
15.4 What is captured by Lemma 62 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
15.5 Safe agreement in CAMPn,t[t n/2] (code for process pi) . . . . . . . . . . . . . 281
16.1 Adding total order message delivery to various URB abstractions . . . . . . . . . . 288
16.2 Adding total order message delivery to the URB abstraction . . . . . . . . . . . . . 289
16.3 Building the TO-broadcast abstraction in CAMPn,t[CONS] (code for pi) . . . . . . 290
16.4 Building the consensus abstraction in CAMPn,t[TO-broadcast] (code for pi) . . . . 293
16.5 A TO-broadcast-based universal construction (code for pi) . . . . . . . . . . . . . . 295
16.6 A state machine does not allow us to retrieve the past . . . . . . . . . . . . . . . . . 298
16.7 Building the consensus abstraction in CAMPn,t[LEDGER] (code for pi) . . . . . . 298
16.8 A TO-broadcast-based ledger construction (code for pi) . . . . . . . . . . . . . . . 299
16.9 Synchrony rules out uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
16.10 To wait or not to wait in presence of asynchrony and failures? . . . . . . . . . . . . 301
16.11 Bivalent vs univalent global states . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
16.12 There is a bivalent initial conﬁguration . . . . . . . . . . . . . . . . . . . . . . . . 305
16.13 Illustrating the sets S1 and S2 used in Lemma 70 . . . . . . . . . . . . . . . . . . . 306
16.14 Σ2 contains 0-valent and 1-valent global states . . . . . . . . . . . . . . . . . . . . 307
16.15 Valence contradiction when i = i . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
16.16 Valence contradiction when i = i . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
16.17 k-sliding window register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
16.18 Solving consensus for k processes from a k-sliding window (code for pi) . . . . . . 311
16.19 Schedule illustration: case 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
16.20 Schedule illustration: case 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
16.21 Building the TO-broadcast abstraction in CAMPn,t[- FC, CONS] (code for pi) . . . 316
17.1 Binary consensus in CAMPn,t[t n/2, MS] (code for pi) . . . . . . . . . . . . . 319
17.2 A coordinator-based consensus algorithm for CAMPn,t[P] (code for pi) . . . . . . 322
17.3 Ω is a consensus computability lower bound . . . . . . . . . . . . . . . . . . . . . . 325
17.4 An algorithm implementing consensus in CAMPn,t[t n/2, Ω] (code for pi) . . . 326
17.5 The second phase for ASn,t[t n/3, Ω] (code for pi) . . . . . . . . . . . . . . . . 330
17.6 A randomized binary consensus algorithm for CAMPn,t[t n/2, LC] (code for pi) 332
17.7 What is broken by a random oracle . . . . . . . . . . . . . . . . . . . . . . . . . . 333

Other documents randomly have
different content

Fault Tolerant Message Passing Distributed Systems An Algorithmic Approach Michel Raynal

The text on this page is estimated to be only 48.22%
accurate
ee P i em. 2 erite LCAT a fv See 2 T yes ay Vif a SE hap e
Lg2 vedd e Bye he fier, Led E A Csr. Creek Ze teu Fag er T De RE
gerer anm | — B euuc e CM: fe Cone indi. evol Zuang E sy f ge 44 n
e m sme : H 0] og ofi TH usd pe) Vn A m a - Hart (Ld echey E tua
pehe fiela ‘ ` e E i allii navi Ze, aun ab, pP le itt gni pre es dy e aeh
on; A it, KE one es €. c4 2— d op. Ire E a) 1 dS ie Ke) 7 Ka i Z dra
ZR P rEZ L- Quint. f Unik Äe A en GC Sei. X ~ dito. Yung 1h lu d.
alone tot laette; faeh Br _ a LI Ge Ev? Uber me, CE tay Fea. A Amp
La E Ios pz Jn ef Itt, Lo g? Pda Lon iu — OMBRE / Kr Ba niot ka gro
deu e la fy, xb YS NEN Ka E e Las 2 du Xa» PLU ANT ikuwe. 2, GL.
INE LI PA PEE d | d if onn r E aid O aen. EA, CVM ei AEN, fl yas 2n.
Let QV te7 720 p. / A d ; pt. Uo HIA Aad 35 is EA CIAN LD 2 Done
fe» rt 44. GE ep EE A time — / d d du 7? e A ea. n FARE. c. b et ea y
o a A p Iapan, P e poe LL uo PUY EI em EP FPP elles, 21427 A. d
Ave t pause : EL ete Daa Aemen? Ser BIO 2 ys Let Ga SMM eal y m
T » AAA Tu MAPS COTES D ; J EC 277 id to feret , exc C URP 7 patur
sel phat d

accurate
cht e AAL f H x dëi d bh Më es feed bp m LS mE. E | d wi
` AER uL aden, an. dee ZC EN BE d Pu P AR ur iin ieee $t m ^ mi SR
Ed th ote, ear Alard A Lorre Pu Congr. 2 Jf ytd Du n cn Da eene? ^a
ef dese Mh Sé et e EAS egene A £A etas I OIL A: 70. He unm. ? zz
et a» Sea Ll- gë rT, SP eg: Zarit e e Ze SE LL. IH CUA J t ie te
mouine GE DUM Ze «uy AV ber Zen 24 E NP vr Pc Y Oh OP MR ed 2
an: SÉ c y e£, KAI 4 Y xy Deas fe ‘ j ^ 7. O y. (id o e hon: ingen e Ye
y? AA ep e Acai) Vs Corg” er MAN Jd e EE a we po. Seen ds yn
geste tle PT Am daer o d D 5 P d hon d urit e EES A en. here), Peak
» in eek Fas pin. F urei tikis eL- KARE dr es AG LNs Ge See Zare 7 €
ette. Laven? T GE GE Vendée elle Caren? fem thigh te. ble SEL A. LE
CS vv AEA Tor Hm Let. ; MN ka 1 t ` id cach ein tL NER NA ITE A f ?
eeneg ref al ltt 7 Laha C8 SE ek ET e777 a? Ct] P y, Ae ht. Seige Me
Ad vie Queer on Saz Fave, et Deri en ENER genee? LEE uo e | BESS
eu LE SE Lorre eT lua dro fe € D y REKA iini d Ji ved se (OA reset E
Labi art, VA PI ECH E P | Exp a 92'. bs porum € sento Be. A E a?
ete - € terne eri SS 1j E a eee” Pt. ée pet e feet p^ : KI fe D : : 4
2k n a y HUN $ AATE o d

accurate
e sø ERU. FY m MNA NRI 7 wn CNN M Al ^ e A TUE ij
A D sg- eres : K d Dé LO etus a: V fete os PL E. tr ev, der ere PEE
tuhkres . » l P v Rau ort ee hwre, Lf He, yore e eve cte» x Be? jl d E
FEI à ee dud L£4! ES th O Fz £z» fy? 7 A AN JI: ag ` V2] Dtr ef
opie A RE p netos d d e, a lod EE Ln 4e 3 l dO ew. Dera E UT. SE TN
dean SECH E as o PE E ey tedetepave? i i | puri pow De E Q7 i
Min mee Dea? ZI ael 1 7 Kai E — 70 ITE eE pz fev’ TR e pt? CL es)
sn tern le M e RM dura. D P mans Ja z € Tie Aa cans CS amip ia
Meinen Dee, erai arg pec e^ £e» Con dg tg pend EP 777 EPFL Aid
7 peer f Lë WIA KÉ ahr sfer e à eepr ye dle Bees OFZ. e Ly tn thives
Nip CH e TEEN en ge e7 Kandha Ge, Q 22 fae gri aL ey de prertls xe
Pe Ae e AN d inn EG Fn Ee Aon Son eisen PTR ZE EE e Bn ui AE iia
sia fv Does Pe ee eae. vs ee Nr ovato | Mu Co 277 771. £ net ? Dev
ae 2» iw o p 2f p Ln PE o Cav ét Jors pee Doar Dae Blagg et LP?
fa» E P pL A adver d RAD EJA Tayy Ferres. ry feud Aen: #71 ter, [ete
E ny at le] pies EU by Lis —— d 7 Am, ae dag fern the? Dee pert
ER DARHT i Cu MR 40 (y CX E DM. «21 Se brad le ibe Cirpi eje tag
ys hey S pe o WE ZUA FA e En Qus eere rg Lj aene g rue due 7 SA
Ei g s, ^ m EA gëscht SZ Set a ae aa se Fi e ttn ILI Atesan a: 8 ~
m P PUES Si À } P

accurate
«92 (A usns. M de np 2 Zu Vilis. jig: EX i ju Mulan 2^ e UC
eon Po cc go even ce Ad Dok ) E Geld 042 A eg out list atu Peer Ns a
! | DU e ur gunst lete ^ ep et COR ee Hho vA putate m Coe anam
OO M ft tg cer cumt, E Pn du ytd P M te eie L2 ADLA Due y rto Ot Se
a pupa t Ave? awe ecc , xo doo, ete AN em tle Dat en, CF m ved aie
Vereen a. Kobo n Py eye) ge Ora? HA Diva) jepewiddes Vous, Car M
Sek ZOHI T EP l er entT— Je Ps bs 3 ATY nolas, brute, Jone Jeu Pru
C urute? Qu nort eu ocmem e) Zeg l Ce aunes rc A I: L cops Jed
mme WË P M E D, 2 He TEL 222 acd Zi e fa Aye aii Avete V au gro
on e but cc? next. Mes iM your ft eit. mL zu it Areg EE ee S See ef Èr
2 don pii Lesap déer I AAI CAE. Bee Wane, Ye guu: 4? Ju Marie, Xe
fi AE OSS ME va dip i det E dasz VAM e; PPPE 22-27 T; e fent NS Aa
Bien, / ; Varia, €^ ctt277 Pr E e4 vec ou 722 2 Zefa; Fe hain c7 Lc
are oet AO PP doten 6 Pedanarkdr, «£t. Ca anto n P Du ^Qepf le
pow A Cur . egenen tg tt, wae ‘Sa pri y fplors? Vo P NEUE iin d
Fant op 7/1

accurate
p ken iS ray € ER mies ence i E D. 2 ST deer? ? Ge Me M
ene 1 E ee Sore moa Oia de A feu LEJ AA e» e R Jh p^ ue E pur
Maes oie a Oe f 2? priat Sen 0. eege ACE EE PP ie MIU ee i / H Á a
TC [^ n n - - d $ K f

accurate
n. 3 d.i. aiiis m — VUES Sté (3 m r^ jj Le 2527» med et
net- gen, N d 313 nme ec Bez ef P. Sale» Tei aya Lo phe, A EE ap
no tye? , peers est ghia » Darr” Lt Votatcley ) ana 6*7 E d feo b m
Sorter ef Dawa Cee («4 AS D) AP a ! D / Ld d S sie CA— Ce 4f deur’,
amide a, Dh ! 621 P7 o e. ez c£. et Vuo far en cy tn ecu e 2 ` Cory
er, n A dive. lea den et D. kimen n I et Kvutegs e dae D lenfa 24224-
ME, ei dÉ far i Chet eur PS A ote A M 7 Y Cot oF; Qu. sone As ete et,
et Ido LLL | | SC KS, Ail Wes 47 6 a 3 tn Rad i A Ke, et 87 Cer E, be:
he : y Len legar? Sar e Aw Lewy’ €. pM P A Piper ue Ds. 2e P H Zt?
heut ek ne fair Anh “we AE Lm A a cmos Do we po ss ert! g oidean?
oo ae LS Z we: the Wer Ge ete. ^ eg £11 e, p DILE I Qut ed I- Aut
x» M -.2 oa, Sack ce te py a feur LL. 3672. £. OP a7 get e £ Étti
Daduna D x Lit le ————— Bad od 47-63 es Ti et Bela of hn. yat er
| iA | xe fem 2g Piece pero EEN a eL- e E Ce E A vorn es LA PA Cer,
C4 ! Er fut dite 2 Gl om pe ia ] KE Lo A R A gt Pad e Pea ai be ee
Hise Jern Ds e 1 d n. tT: e ws ERN twy

accurate
snol gew Vmes ne, et ef pao Deren À QE mU dius Jed) y E
ba e CS CR E. Cars Shs Db Ott Qe ef cct a Qe? Er f Car 2o tore i em
Miis č Eu — ey y ae a P J1 tk fur LI £72 yv Aert red y dad prne dëss
en etrr eill enve; €43 ^» 7 : : ' 77 Zog lid f 47; CIL ue ro. eB fene b
eM. ee c AI aptis if e HX P : FT LPM €! CA AJA of hte” fa A g pz A. oz
Gë Cor pw MP E Lp ute 04 Ge e f E OH eT— ay Mo ee ue E E —
Dieja ae 2A) VFL Dege vgs TAA uU Oe ee? Bc QUU A PARA S bora
stikt 9g. Hh des Le [org WM Ge eek In V Ure HL v Je» EE D See mo
D P Sas CAAO LV cg — eeh D epa ey e ANDA: sa a Se robos SE ange
1 fii ge be ps amm eu z p vd a 2 e? an tet E “EA Seta e. Vito PEL a
NES E, VA. a eg 1229 d rz 2 uw? AEG MALO vi aV o voy LE Ey in
Pape. eva 77 See yaar Qe ys £7 O ouav. Gan PD, MP P 2 a OY et P
ex I vn eher, fer per asi e. he Arrr gt d ea 22 a. RC ct 3 EE EE Py :
Zz. 4 iu LIF? 2 ? Pu ad ean, eL. lagen c CS 4 e wes 3 seat

accurate
ER V tdi fen T ^o d Font, CH VEA os uA aperte GE: Pat 2
mate ale es) _ ua» der d pt atten ae six f £72 Cep v peel forte C 1tY
aF- Li NT NN A £ Gees COVE. ra 7, WE ae D y Lo £x crete prar Qn ei
pad h? Ie. tie TS dt ath BES y ars d Ee api GOS $6 paty 72 DEA x4
£c Ma pute lan ep) 0 ld e d 2 ei em it pefea en e ve faa uae Dig kc
m See or LL ea e X 7 3 VEA e “arr bs y, Leg putt. à Eco P Je f^
COA. ertet) 4 ju? ge. Gë 7 Z7 J Forty feni Nue M NR T Cau v6 dep La
xat e #2 vy d dl y Cer feet IE we ^ Cc Kas FF? fais e 4— V OAM
onm FLEE f : A eas RAZ ey WA e PAD] E P wae 2t) ut Loe “We 2
dio pA GIEL? e C4 Arcot? a ete T gn pf y DL p ezz E T LA A A pud a
Lo un fr fines: | ete Kë: LOK? pen Ge i Jane x y 4:71 De E d ef Cu £t
7 640307 B Care. a) di^ ec Se ag GC | yy s2? KE 92€ o Ge Laph een
E: / £ * Tá ¢ es E 7. 3 gung nd Sea ëch ESPZ qu. i a he feit GOA
finer iat fC st- xL c s y get Meer ) e£» IZ LM c4 Il esz e A 6 8 P PA a
: ; BEY e TERTE Chet ee Se i Pra Y ` } ` n zu. d: » 1^ EN cid i Sai
M : ` i * i d Nen nn vm E / IA

accurate
Ca / 210 bv JAM. Mo fa «e Ze Jf E a Pa er SE, sd à b po zs
E f s A Lëtz A8! wo» thes p pr Mao da ur jb on ed, la / . ` EE SG f
es224 ave? 2 TN Corp tocesz t QA € € té m CA fap grand Ae cv
ety d 25 A EN LA Z/ ct Zuge 2 e. Voy? U21? LID 442 P Seere fortes
— | 3 ROPE inl LEMS eiie gay Jesas fortea gan FI aduser p } A J F `
n. e e 1 JA wy Crd di Arr? a e £O )L-EOo OF th? Ore ——— —— a!
SE 1 C» E: 4 Am 72 AIVE. dy) Ate E IJI A GALAN GAALEN (4 Ji 7 euy
f Ge eelle Ft parara K d Ow fiet — Rao KD an faito P yu ct fe», ef-
Iela fair et ler— — Vi n Ce ed Gd AO, eL o rð puse (o E mi, Jr `. d qi
s Eed Pune gap Pon v PA et ge een ^ P A a^ a 4 N Z e eil ot La une.
VOL aa Bil igh Z defe ay í À (eua EIn lo CAM en ATF, e PH neo e
pAMLL à os f* 2 ef. EJN ez 4, 2F Po : S e eJ» e nia y DE | i y CEET
pte? KAI HE Ku Cx, f. SR ye) rE, Lo e 2e? S et ae? J Teo Bes CLI af
R4 cd bigs Are, 20 | Je prt dy add o. 224 —n4 4 E Byers citt- Aare j
2 | e ^ J V4 2 du SP nos. CA LA Ke 5 (EK Lë fir £ Ne 4A? e s AT. ee.
» Hes ec 2c... 7 ^ 4 LY c—— € 77422 fem tr ov age dud GE p edt C
edt adire sne d. aaf, ET E. P i ` f gee Chee? hon ec. BP atiis cere
habe Sa i ~ 2 E 5 ay at. [U H Ot e IH ee C es E SE aq. AR , 4 Le $ A.
S ah y Biches at- Ya SPI LIN OD pL EA ey, e) T

accurate
‘ay NAP. TE h d Kë (CAM is SEN aoe, M ANN iu 2 ta lites IA
2, as à protect pot! 31 er o eż- Jadeo pus Mr UAE E TE Lk D ee a ae
ie oO De femme Pomme, eda Bb ner ier ect tees Ynes Cre gur d den
€ Ge Ch p Jt e Gut en ea aro 74 A. / yu ; Live yd na, pA Wahl) gv.
piu pts ctt e Kären ^ ! Sane? dou g ent gunn Li E ei e all L y OLIP
AA? ? f ya i fzx E Ce Cle Mn yer EP E, ^ Aaen «t Ae Cunt’ eege, er
en CE ye E vere Fi tire Ouid ereny eL et Ee? A La z nr du pagent,
edet fer z7 t Tr don t et- ct t6 Atm fae- ce fe Yip ^ ES e , Zen Le. Á
^if. EE ver au». rad 05 did go eL rcped. pra eller ar Ja yr pP s chon;
Lo dais Souter e PA VR PLA nte en nc jas DE iil | P: Date (e Ae fA a
SE Zen EY m At e E E tet face £42. eet Auf you go aet LÁ Pm [astris
fei of Eo X da gia feres m ehose Vie T uve d PIV A heye? Je du er.
Lr 23732: 73172 ders MEME. 3 ej ! LC ; n d uh 4 Aeon, EJI (cv do e,
rie M c AP re reat 7 : f f ape ae eJ ai Fe E Awer ame — SCH Kye?
Jela K EA €I Dë as e à ff A Navy ae

accurate
Se WOES ke 00 000 Zä mU TN i aie | p C-O 727 £27 er UP
ef C1 FA ege T . be oe: CO 1227 ery Cn A. yoa eX e. IC d E4CX 7
de e D. iA A, Së EE: 400 Lhe Ad dei id à the Ca Kets etait tex XH a
Cine? TRES Teda fUr 722-3, ee A. Jes SECH Saw? DF DAR VS Vlt e
o ( á eg eA. LA CL? eA 35,2998: Zee » aM Voy Le CLE, 2 A mit P Gë
A itl Ce Pi a7 Marge CAS r NA í der ^ DAVIA ole GF Kn PA fran ir n ai
EA ud 2 foe fao CLF SCH CRTC 7 o Kor? Y ACO? tp yore Dx igen d Ld
tt o : P, drew Oz Yoo; s Ne. FIN ei LET, De AME à 2 OME Te d Yo GLY
EG FE A FLOF decret i Mp a fra DU SS «x Newent hen Fast ce s uc o
© Pine a e go eel eves

accurate
N.ETUUS C NA a pe ee a A PTS ahs nee On CUTE TIR iy H
n DONE MTM WEERNMO 0D. PST TT WT e t ae” bs f7 Ripe | v 3
bye 09 LARES, ed Duel n Dp eA e pio) UA Bit e e E neg Teua GE IPI:
Pres got dr re I Ase b pfi eno J POND, Ge AË HE EVİ ED EO ul cuo
ese oer tnt 2 ev) et ta Mrs SOS opil srona faur Qin ia es ges cr ca gn
on gnarus, Latte Aer | Lo Za? ME. Se $ed P e Kanger Jara Ls IGI
ALAKI EU e€2£—€72. elle» Y. AJ V a SE b dl eL depen gal WA ec €4
—9 ai aen EIL effe 3 I, CFL 144 NEE EE erte Ji P drot ed eteo tt
petis” Era. y. E Pe Jon tn, PF o. Bs A 5 224- dora UL tae tfr — 1. E Pi
lo p= 2e Vo: LAI Curr 447 EC F: ` Darid Son Dages oy ZE oper e Se
ER lapit ug P4 Dew? 4 s Yes Coo 7, EEC ike: rq em ey jo Fd ba ( € eA
eU 722 AN zieht. dans fet free} c2 OTI Jet hn eh on Ue ug ^ 42V
422 ée | Deter MA / f / fj? ain A y pen Dei feo A ret Prop de Keyed
yea f ee AS Bg f FLOP F- emer Foe Yd CF fa spe trita get E adresza a
Je. OA fur IGT IIT’! 9e SE vi BE ee. VOUS. it d un Ge de tg? PLEI
Aad I Ea ’ toads CLU d One ANVE eee he Qua X Qu ncs Peco AAV ES
8 , ZEN Bé Zant ge ! tVow? De ev prt As, t n Les Hager d H 4 REX: d
T

accurate
ae | € Hä KS äer A ^u A e Sr, l VU. 3 M S ts A A be d Mec
ef , Paan : SI WE. S TS = Wand, A - MS, 4 Ht y ^ 4$ k i um N an e
en tU E Cor fu queen toit qu 2 j É: PAE or gav ene DS cus Vo Prius
Odi Z enr Qn. | fere breuxs 5 f CAIFE dei Lh ague faut IOT Pe p d
and V e e ero Geet Kost tani Clery Le Meras ered, 3 Mou i dean.
largas de muee Duce io Mae 4 E E där 726 iguey D. dle De D. TB. s
arty p» dy US e oc A £f e Taro ETE E, Dif asie es bs 1 ef. int fain De
KE Vara las putag wc Bre | les L4 67 P a p Yes ge sue ezut-- devez)
eA eg E Cc the ey duy a ster Ee eene | up bond aeta Bar nt Ze har
Zeie fet, | ei EE 47 444 «— s eaves Cv 22272) Je tar NL Lell iine d f
C Le fas £-2one eque T : his Et ndo d a) eum PO fred 7 PRA P od qd
LUN EM ganh s ull ern tan pid ge A oe ‘ J ereenn Inor? E gor eis Die
fio Fone $ Via 5 Z7 Les La god ty geen? ter Ee E | fn Meg Dads ey et
YE pur fyd teat aret: Cien, i CAV Jid PT H low? Yee fe, esr? MAGNO
Let Soy ON CH Le? e ux» fir UM e. M *, 72 OLE- EE AN EN er | efef-
Sr. Latur gas m Ph NS ait dn Sana fidir ae o Cuu Cezt- Loe» PEE

Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!
textbookfull.com

Fault Tolerant Message Passing Distributed Systems An Algorithmic Approach Michel Raynal

More Related Content

Similar to Fault Tolerant Message Passing Distributed Systems An Algorithmic Approach Michel Raynal (20)

Recently uploaded (20)

Fault Tolerant Message Passing Distributed Systems An Algorithmic Approach Michel Raynal