Thinking Outside the Synchronisation Quadrant

Thinking Outside
the Synchronisation
Quadrant
@KevlinHenney

Architecture represents the
significant design decisions
that shape a system, where
significant is measured by
cost of change.
Grady Booch

Architecture is the art
of how to waste space.
Philip Johnson

Architecture is the art
of how to waste time.

Mutable
Immutable
Unshared Shared
Unshared mutable
data needs no
synchronisation
Unshared immutable
data needs no
synchronisation
Shared mutable
data needs
synchronisation
Shared immutable
data needs no
synchronisation

Mutable
Immutable
Unshared Shared
Unshared mutable
data needs no
synchronisation
Unshared immutable
data needs no
synchronisation
Shared mutable
data needs
synchronisation
Shared immutable
data needs no
synchronisation
The Synchronisation Quadrant

Systems have properties
— capabilities, features,
characteristics, etc. —
inside and out.

Functional
Operational
Developmental

This is the monstrosity in love,
lady, that the will is infinite,
and the execution confined;
that the desire is boundless,
and the act a slave to limit.
William Shakespeare
Troilus and Cressida

Multitasking is really just rapid
attention-switching.
And that'd be a useful skill, except it
takes us a second or two to engage
in a new situation we've graced with
our focus.
So, the sum total of attention is
actually decreased as we multitask.
Slicing your attention, in other
words, is less like slicing potatoes
than like slicing plums: you always
lose some juice.
David Weinberger

http://ithare.com/infographics-operation-costs-in-cpu-clock-cycles/

completion time
for single thread
𝑡 = 𝑡1

division of
labour
𝑡 =
𝑡1
𝑛

𝑡 = 𝑡1 1 − 𝑝
𝑛 − 1
𝑛
portion in
parallel
Amdahl's law

𝑡 = 𝑡1 1 − 𝑝
𝑛 − 1
𝑛
+ 𝑘
𝑛 𝑛 − 1
2
typical
communication
overhead
inter-thread
connections
(worst case)

𝑡 = 𝑡1 1 − 𝑝
𝑛 − 1
𝑛
+ 𝑘
𝑛 𝑛 − 1
2

template<typename TaskIterator, typename Reducer>
void map_reduce(
TaskIterator begin, TaskIterator end,
Reducer reduce)
{
std::vector<std::thread> threads;
for(auto task = begin; task != end; ++task)
threads.push_back(std::thread(*task));
for(auto & to_join : threads)
to_join.join();
reduce();
}

Command-line tools
can be 235x faster than
your Hadoop cluster
Adam Drake
http://aadrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html

A large fraction of the flaws in software development
are due to programmers not fully understanding all
the possible states their code may execute in.
In a multithreaded environment, the lack of
understanding and the resulting problems are greatly
amplified, almost to the point of panic if you are
paying attention.
John Carmack
http://www.gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php

Some people, when confronted with a
problem, think, "I know, I'll use threads,"
and then two they hav erpoblesms.
Ned Batchelder
https://twitter.com/#!/nedbat/status/194873829825327104

Shared memory is like a canvas where threads
collaborate in painting images, except that
they stand on the opposite sides of the canvas
and use guns rather than brushes.
The only way they can avoid killing each other
is if they shout "duck!" before opening fire.
Bartosz Milewski
"Functional Data Structures and Concurrency in C++"
http://bartoszmilewski.com/2013/12/10/functional-data-structures-and-concurrency-in-c/

There are several ways to
address the problem of
deadlock...
http://www.cs.rpi.edu/academics/courses/fall04/os/c10/index.html

Just ignore it and hope it
doesn't happen.
Ostrich Algorithm

Detection and recovery —
if it happens, take action.

Dynamic avoidance by careful
resource allocation — check to
see if a resource can be
granted, and if granting it will
cause deadlock, don't grant it.

Prevention — change the rules.

Habitability is the characteristic
of source code that enables
programmers, coders, bug-fixers,
and people coming to the code
later in its life to understand its
construction and intentions and
to change it comfortably and
confidently.

Habitability makes a place
livable, like home. And this is
what we want in software — that
developers feel at home, can
place their hands on any item
without having to think deeply
about where it is.

Simple Testing Can Prevent
Most Critical Failures
An Analysis of Production Failures in
Distributed Data-Intensive Systems
https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf

We want our code
to be unit testable.
What is a unit test?

A test is not a unit test if:
▪ It talks to the database
▪ It communicates across the network
▪ It touches the file system
▪ It can't run at the same time as any of your other
unit tests
▪ You have to do special things to your environment
(such as editing config files) to run it.
Michael Feathers
http://www.artima.com/weblogs/viewpost.jsp?thread=126923

A unit test is a test of behaviour
whose success or failure is wholly
determined by the correctness of
the test and the correctness of the
unit under test.
Kevlin Henney
http://www.theregister.co.uk/2007/07/28/what_are_your_units/

What do we want
from unit tests?

When a unit test
passes, it shows
the code is correct.

When a unit test
fails, it shows the
code is incorrect.

Future
Immediately return a ‘virtual’ data object—
called a future—to the client when it invokes a
service. This future [...] only provides a value
to clients when the computation is complete.

std::future<ResultType>
iou = std::async(function);
...
ResultType result = iou.get();

joiner<ResultType>
iou = thread(function);
...
ResultType result = iou();
"C++ Threading", ACCU Conference, April 2003
"More C++ Threading", ACCU Conference, April 2004
"N1883: Preliminary Threading Proposal for TR2", JTC1/SC22/WG21, August 2005

In functional programming, programs are
executed by evaluating expressions, in
contrast with imperative programming where
programs are composed of statements which
change global state when executed.
Functional programming typically avoids
using mutable state.
https://wiki.haskell.org/Functional_programming

Many programming languages support
programming in both functional and
imperative style but the syntax and facilities
of a language are typically optimised for only
one of these styles, and social factors like
coding conventions and libraries often force
the programmer towards one of the styles.
https://wiki.haskell.org/Functional_programming

William Cook, "On Understanding Data Abstraction, Revisited"

https://twitter.com/mfeathers/status/29581296216

To keep our C++ API boundary simple, we [...] adopted
one-way data flow. The API consists of methods to
perform fire-and-forget mutations and methods to
compute view models required by specific views.
To keep the code understandable, we write functional
style code converting raw data objects into immutable
view models by default. As we identified performance
bottlenecks through profiling, we added caches to avoid
recomputing unchanged intermediate results.
The resulting functional code is easy to maintain,
without sacrificing performance.
https://code.facebook.com/posts/498597036962415/under-the-hood-building-moments/

Immutable Value
Define a value object type whose instances
are immutable. The internal state of a value
object is set at construction and no
subsequent modifications are allowed.

Copied Value
Define a value object type whose instances
are copyable. When a value is used in
communication with another thread, ensure
that the value is copied.

class date
{
public:
date(int year, int month, int day_in_month);
date(const date &);
date & operator=(const date &);
...
int get_year() const;
int get_month() const;
int get_day_in_month() const;
...
void set_year(int);
void set_month(int);
void set_day_in_month(int);
...
};

Just because you
have a getter,
doesn't mean you
should have a
matching setter.

class date
{
public:
date(const date &);
...
...
void set(int year, int month, int day_in_month);
...
};
today.set(2016, 11, 16);

class date
{
public:
date(const date &);
...
...
};
today = date(2016, 11, 16);

class date
{
public:
date(const date &);
...
...
};
today = date { 2016, 11, 16 };

class date
{
public:
date(const date &);
...
...
};
today = { 2016, 11, 16 };

"Get something"
is an imperative
with an expected
side effect.

class date
{
public:
date(const date &);
...
...
};

class date
{
public:
date(const date &);
...
int year() const;
int month() const;
int day_in_month() const;
...
};

class date
{
public:
date(const date &);
...
int year() const;
int month() const;
date with_year(int) const;
...
};

Builder
Introduce a builder that provides separate
methods for constructing and disposing of
each different part of a complex object, or
for combining cumulative changes in the
construction of whole objects.

class date
{
public:
...
int year() const;
int month() const;
...
};

class date
{
public:
...
int year() const;
int month() const;
date with_year(int new_year) const
{
return { new_year, month(), day_in_month() };
}
...
};

class date
{
public:
...
int year() const;
int month() const;
date with_year(int new_year) const
{
return
new_year == year
? *this
: date { new_year, month(), day_in_month() };
}
...
};

class date
{
public:
...
int year() const;
int month() const;
date with_month(int) const;
date with_day_in_month(int) const
...
};

Asking a question
should not change
the answer.
Bertrand Meyer

Asking a question
should not change
the answer, and
nor should asking
it twice!

Referential transparency is a very
desirable property: it implies that
functions consistently yield the same
results given the same input,
irrespective of where and when they are
invoked. That is, function evaluation
depends less—ideally, not at all—on the
side effects of mutable state.
Edward Garson
"Apply Functional Programming Principles"

// "FTL" (Functional Template Library :->)
// container style
template<typename ValueType>
class container
{
public:
typedef const ValueType value_type;
typedef ... iterator;
...
bool empty() const;
std::size_t size() const;
iterator begin() const;
iterator end() const;
...
container & operator=(const container &);
...
};

class set
{
public:
typedef const ValueType * iterator;
...
set(std::initializer_list<ValueType> values);
...
bool empty() const;
iterator find(const ValueType &) const;
std::size_t count(const ValueType &) const;
iterator lower_bound(const ValueType &) const;
iterator upper_bound(const ValueType &) const;
pair<iterator, iterator> equal_range(const ValueType &) const;
...
private:
ValueType * members;
std::size_t cardinality;
};
set<int> c { 2, 9, 9, 7, 9, 2, 4, 5, 8 };

class array
{
public:
...
array(std::initializer_list<ValueType> values);
...
bool empty() const;
const ValueType & operator[](std::size_t) const;
const ValueType & front() const;
const ValueType & back() const;
const ValueType * data() const;
...
private:
ValueType * elements;
std::size_t length;
};
array<int> c { 2, 9, 9, 7, 9, 2, 4, 5, 8 };

In computing, a persistent data structure is a data structure
that always preserves the previous version of itself when it is
modified. Such data structures are effectively immutable, as
their operations do not (visibly) update the structure in-place,
but instead always yield a new updated structure.
http://en.wikipedia.org/wiki/Persistent_data_structure
(A persistent data structure is not a data structure committed
to persistent storage, such as a disk; this is a different and
unrelated sense of the word "persistent.")

class vector
{
public:
...
bool empty() const;
vector pop_front() const;
vector pop_back() const;
...
private:
ValueType * anchor;
iterator from, until;
};

class vector
{
public:
...
bool empty() const;
vector popped_front() const;
vector popped_back() const;
...
private:
ValueType * anchor;
};

I still have a deep fondness for the
Lisp model. It is simple, elegant, and
something with which all developers
should have an infatuation at least
once in their programming life.
Kevlin Henney
"A Fair Share (Part I)", CUJ C++ Experts Forum, October 2002

class list
{
public:
class iterator;
...
list popped_front() const;
list pushed_front() const;
...
private:
struct link
{
link(const ValueType & value, link * next);
ValueType value;
link * next;
};
link * head;
std::size_t length;
};

Hamlet: Yea, from the table of
my memory I'll wipe away all
trivial fond records.
William Shakespeare
The Tragedy of Hamlet
[Act I, Scene 5]

Garbage collection [...] is optional
in C++; that is, a garbage collector
is not a compulsory part of an
implementation.
Bjarne Stroustrup
http://stroustrup.com/C++11FAQ.html

assert(
std::get_pointer_safety() ==
std::pointer_safety::strict);

Ophelia: 'Tis in my memory
locked, and you yourself shall
keep the key of it.
William Shakespeare
The Tragedy of Hamlet
[Act I, Scene 3]

A use-counted class is more
complicated than a non-use-
counted equivalent, and all of
this horsing around with use
counts takes a significant
amount of processing time.
Robert Murray
C++ Strategies and Tactics

class vector
{
public:
...
bool empty() const;
vector popped_front() const;
vector popped_back() const;
...
private:
std::shared_ptr<ValueType> anchor;
};
Uses std::default_delete<ValueType[]>, but
cannot be initialised from std::make_shared

class list
{
public:
class iterator;
...
list popped_front() const;
list pushed_front() const;
...
private:
struct link
{
link(const ValueType & value, std::shared_ptr<link> next);
ValueType value;
std::shared_ptr<link> next;
};
std::shared_ptr<link> head;
std::size_t length;
};

{
list<Anything> chain;
std::fill_n(
std::front_inserter(chain),
how_many,
something);
}
On destruction, deletion of links is recursive through each link, causing
the stack to blow up for surprisingly small values of how_many.

Instead of using threads and shared
memory as our programming model, we
can use processes and message passing.
Process here just means a protected
independent state with executing code,
not necessarily an operating system
process.
Russel Winder
"Message Passing Leads to Better Scalability in Parallel Systems"

Languages such as Erlang (and occam
before it) have shown that processes are a
very successful mechanism for
programming concurrent and parallel
systems. Such systems do not have all
the synchronization stresses that shared-
memory, multithreaded systems have.
Russel Winder
"Message Passing Leads to Better Scalability in Parallel Systems"

class channel
{
public:
void send(const ValueType &);
bool try_receive(ValueType &);
private:
...
};

class channel
{
public:
private:
std::deque<ValueType> fifo;
};

class channel
{
public:
void send(const ValueType & to_send)
{
fifo.push_back(to_send);
}
...
};

class channel
{
public:
...
bool try_receive(ValueType & to_receive)
{
bool received = false;
if (!fifo.empty())
{
to_receive = fifo.front();
fifo.pop_front();
received = true;
}
return received;
}
...
};

class channel
{
public:
private:
std::mutex key;
};

{
std::lock_guard<std::mutex> guard(key);
}

bool try_receive(ValueType & to_receive)
{
bool received = false;
if (key.try_lock())
{
std::lock_guard<std::mutex> guard(key, std::adopt_lock);
if (!fifo.empty())
{
fifo.pop_front();
received = true;
}
}
return received;
}

class channel
{
public:
void receive(ValueType &);
private:
std::mutex key;
std::condition_variable_any non_empty;
};

{
non_empty.notify_all();
}

void receive(ValueType & to_receive)
{
non_empty.wait(
key,
[this]
{
return !fifo.empty();
});
fifo.pop_front();
}

https://twitter.com/richardadalton/status/591534529086693376

std::string fizzbuzz(int n)
{
return
n % 15 == 0 ? "FizzBuzz" :
n % 3 == 0 ? "Fizz" :
n % 5 == 0 ? "Buzz" :
std::to_string(n);
}

void fizzbuzzer(channel<int> & in, channel<std::string> & out)
{
for (;;)
{
int n;
in.receive(n);
out.send(fizzbuzz(n));
}
}

int main()
{
channel<int> out;
channel<std::string> back;
std::thread fizzbuzzing(fizzbuzzer, out, back)
for (int n = 1; n <= 100; ++n)
{
out.send(n);
std::string result;
back.receive(result);
std::cout << result << "n";
}
...
}

int main()
{
channel<int> out;
channel<std::string> back;
for (int n = 1; n <= 100; ++n)
{
out << n;
std::string result;
back >> result;
std::cout << result << "n";
}
...
}

void fizzbuzzer(channel<int> & in, channel<std::string> & out)
{
for (;;)
{
int n;
in >> n;
out << fizzbuzz(n);
}
}

class channel
{
public:
void operator<<(const ValueType &);
void operator>>(ValueType &);
private:
std::mutex key;
};

class channel
{
public:
receiving operator>>(ValueType &);
private:
std::mutex key;
};

class channel
{
public:
void operator<<(const ValueType & to_send)
{
send(to_send);
}
receiving operator>>(ValueType & to_receive);
{
return receiving(this, to_receive);
}
...
};

class receiving
{
public:
receiving(channel * that, ValueType & to_receive)
: that(that), to_receive(to_receive)
{
}
receiving(receiving && other)
: that(other.that), to_receive(other.to_receive)
{
other.that = nullptr;
}
operator bool()
{
auto from = that;
that = nullptr;
return from && from->try_receive(to_receive);
}
~receiving()
{
if (that)
that->receive(to_receive);
}
private:
channel * that;
ValueType & to_receive;
};

std::string fizzbuzz(int n)
{
if (n < 1 || n > 100)
throw std::domain_error(
"fizzbuzz(n) is defined for n in [1..100]")
return
n % 15 == 0 ? "FizzBuzz" :
n % 3 == 0 ? "Fizz" :
n % 5 == 0 ? "Buzz" :
std::to_string(n);
}

void fizzbuzzer(channel<int> & in, channel<std::any> & out)
{
for (;;)
{
try
{
int n;
in >> n;
out << fizzbuzz(n);
}
catch (...)
{
out << std::current_exception();
}
}
}

int main()
{
channel<int> out;
channel<std::any> back;
for (int n = 1; n <= 100; ++n)
{
out << n;
std::any result;
back >> result;
if (result.type() == typeid(std::string))
std::cout << std::any_cast<std::string>(result) << "n";
}
...
}

class channel
{
public:
channel();
channel(std::function<void(const ValueType &)>);
receiving operator>>(ValueType &);
private:
std::mutex key;
};

int main()
{
channel<int> out;
channel<std::any> back(
[](const any & received)
{
if (received.type() == typeid(std::exception_ptr))
std::rethrow_exception(any_cast<std::exception_ptr>(received));
});
...
}

int main()
{
...
for (int n = 1; n <= 100; ++n)
{
try
{
out << n;
std::any result;
back >> result;
std::cout << std::any_cast<std::string>(result) << "n";
}
catch (std::domain_error & caught)
{
std::cout << caught.what() << "n";
}
}
...
}

Pipes and Filters
Divide the application's task into
several self-contained data
processing steps and connect these
steps to a data processing pipeline
via intermediate data buffers.

Simple filters that can be arbitrarily
chained are more easily re-used, and
more robust, than almost any other
kind of code.
Brandon Rhodes
http://rhodesmill.org/brandon/slides/2012-11-pyconca/

Multithreading is just one
damn thing after, before, or
simultaneous with another.
Andrei Alexandrescu

Actor-based concurrency is
just one damn message after
another.

No matter what language you work in,
programming in a functional style provides
benefits.
You should do it whenever it is convenient,
and you should think hard about the
decision when it isn't convenient.
John Carmack

Programming in a functional style makes
the state presented to your code explicit,
which makes it much easier to reason
about, and, in a completely pure system,
makes thread race conditions impossible.
John Carmack

Think outside the
synchronisation
quadrant...

All computers
wait at the
same speed.

Thinking Outside the Synchronisation Quadrant

More Related Content

Similar to Thinking Outside the Synchronisation Quadrant (20)

More from Kevlin Henney (20)

Recently uploaded (20)

Thinking Outside the Synchronisation Quadrant