Dive into SObjectizer 5.5. Eighth Part: Dispatchers

Dive into SObjectizer-5.5
SObjectizer Team, Jan 2016
Eighth Part: Dispatchers
(at v.5.5.15)

This is the next part of the series of presentations with deep
introduction into features of SObjectizer-5.5.
This part is dedicated to dispatchers which are one of the
cornerstones of SObjectizer.
Dispatchers play a significant role in the behaviour of any
SObjectizer-based application.
Because of that understanding of dispatcher concept is
necessary for usage of SObjectizer framework.

What Is A Dispatcher?

Long story short:
Dispatchers manage event queues and provide
working threads on which agents handle their events.
If you don't understand what does it mean, don't worry.
There is a slightly longer explanation...

The main form of agents interaction in SObjectizer is
asynchronous message passing.
An agent sends a message to some mbox. If there is a
subscriber for that message type on that mbox then an event
is created. An event is an info about a specific message
which must be delivered to the specific agent.
Event are stored in events queue.

An event must be extracted from event queue and the
message from that event must be passed to agent-receiver
for processing.
Agent-receiver processes message by event handler which
is subscribed to this message type. Event handler is
represented by agent's method or just lambda-function.
Event handler is called on the context of some thread of
execution. This thread is provided by dispatcher.

Every working thread on the context of which an event
handler is invoked belongs to some dispatcher.
So a dispatcher can be seen as manager of working threads
to be used for invocation of event handlers.
Creation and deletion of working threads is an area of
responsibility of dispatcher.

There could be many dispatchers in SObjectizer
Environment. Each of them will have its own set of working
threads.
Every agent should be bound to some dispatcher. Binding is
done during registration of agent's coop. A programmer can
select a dispatcher for a specific agent.
Once agent is bound to a dispatcher all agent's events will
be handled on working threads of that dispatcher.

A dispatcher is not only a manager for working threads.
It also manages event queues for agents which are bound to
that dispatcher.
It is a significant difference of SObjectizer from other actor
frameworks: an event queue doesn't belong to an agent. A
queue is created and managed by dispatcher.

Another difference of SObjectizer from other actor
framework is a quantity of event queues.
In the "classical" actor-based approach every actor has its
own queue with messages to be processed.
In SObjectizer a dispatcher makes decision how many event
queues are necessary for serving agents bound to that
dispatcher.

The simplest dispatcher type in SObjectizer, one_thread
dispatcher, uses just one event queue for all agents bound
to that dispatcher.
Much more complex dispatchers like thread_pool and
adv_thread_pool can create as many event queues as it
required by a programmer.
Because of that every dispatcher has its own logic of
handling event queues and dispatching events for
processing by agents.

So now we can say that dispatcher is responsible for:
● creation and deletion of working threads;
● creation and deletion of event queues;
● extraction of events from event queues and invocation of
event-handlers on context of dispatcher's working
threads.
Different dispatchers do these action differently. So a user
can choose the most appropriate dispatcher for a particular
task.

Why Do We Need Dispatchers?

For what purpose does SObjectizer support different kinds of
dispatchers and allow to create several dispatchers in an
application?
Why a simple approach with one thread pool for all agents is
not used?
For what reasons a user should take care about dispatcher
and bind agents to some dispatcher?

It is a direct consequence of SObjectizer's primary target:
simplification of development of multithreading application.
SObjectizer doesn't hide threads from user.
But simplifies their usage and provides a way for safe
interaction between threads via asynchronous message
passing.

A user may require dedicated threads for performing long
and CPU-intensive calculations in background.
Or dedicated threads might be necessary for interaction with
some hardware attached to your computer.
Or dedicated threads might be needed to do blocking calls to
some 3rd party library.
Or thread pool can be necessary for efficient processing of
some event stream.

So there can be different reasons to use threads for solving
some real-world problems.
What we can do with that?
We can create and manage working threads manually. We
can pass information between threads via some ad-hoc
implementations of event queues or synchronize access to
shared data via low-level mechanisms like semaphores,
mutexes, condition variables and so on...

Or we can get a ready-to-use tools for management of
working threads and inter-thread message passing.
SObjectizer provides such tools:
● working thread management and events scheduling is
done via dispatchers;
● inter-thread message passing is implemented via general
message delivery mechanism (mboxes, event
subscriptions and so on).

And because different users require different thread
management policies there are different dispatchers in
SObjectizer.
And because of that there is a possibility to create any
number of dispatchers in an application.
All of that should allow to user to solve its domain-specific
task by most appropriate way.

But it doesn't mean that a user must take care about a
dispatcher for every agent.
A user can take care if he wants. But if not...
...there are such things as default dispatcher and coop's
main dispatcher binder which significantly simplify working
with SObjectizer.

An SObjectizer Environment has the default dispatcher. It is
created and started automatically.
The default dispatcher is an instance of one_thread
dispatcher. It means that all agents bound to that dispatcher
will work on the same working thread.
If a user doesn't specify a dispatcher for agent or agent's
coop then agent will be bound to the default dispatcher.

Dispatcher Binder

A binding of an agent to a dispatcher is done via separate
object called dispatcher binder.
Dispatcher binder knows how to bind agent to dispatcher
instance during agent registration. And how unbind the agent
during deregistration.
Every dispatcher implements its own dispatcher binder.
It means that if a user wants to bind agent to active_obj
dispatcher instance the user must create an active_obj
dispatcher binder.

Binder can be specified for an agent during agent creation:
auto coop = env.create_coop( so_5::autoname );
// An agent will be bound to one_thread dispatcher with name "Monitor".
coop->make_agent_with_binder< some_agent_type >(
// Creation of dispatcher binder.
so_5::disp::one_thread::create_disp_binder( "Monitor" ),
... /* Args for some_agent_type constructor */ );
// An agent will be bound to active_obj dispatcher witn name "Handler".
so_5::disp::active_obj::create_disp_binder( "Handler" ),
... );

Binder can also be specified for the whole coop. Such binder
will be main binder for that coop:
auto coop = env.create_coop( so_5::autoname,
// A main binder for coop will be a binder to active_obj dispatcher with the name "Handler".
so_5::disp::active_obj::create_disp_binder( "Handler" ) );
// This agent will be bound via coop's main binder to active_obj dispatcher with the name "Handler".
coop->make_agent< some_agent_type >( ... );
// This agent will also be bound to active_obj dispatcher with the name "Handler".
coop->make_agent< another_agent_type >( ... );
// But this agent will be bound to one_thread dispatcher with the name "Monitor". It is because
// a separate dispatcher binder is set for that agent.
so_5::disp::one_thread::create_disp_binder( "Monitor" ), ... );

If a dispatcher binder is not set for a coop then a binder for
the default dispatcher will be used as main binder for that
coop.

Public And Private Dispatchers

There are two kinds of dispatchers in SObjectizer: public and
private.
There are important differences in how to work with them.

Public dispatchers must have unique names. They are
accessed by means of their names.
Once created public dispatchers stay inside SObjectizer
Environment until Environment finished its work.
It means that an instance of public dispatcher will work even
when no one is using it.

Private dispatchers are accessible only via direct references
obtained during dispatchers creation.
Private dispatchers are destroyed automatically when no
one uses them.

A public dispatcher instance is passed to SObjectizer
Environment via environment_params_t object which could
be tuned in the launch() function:
so_5::launch( []( so_5::environment_t & ) { ... },
// Environment's parameters tuning lambda.
[]( so_5::environment_params_t & params ) {
// Addition of a public active_obj dispatcher with the name "Handler".
params.named_dispatcher( "Handler", so_5::disp::active_obj::create_disp() );
// Addition of a public one_thread dispatcher with the name "Monitor".
params.named_dispatcher( "Monitor", so_5::disp::one_thread::create_disp() );
} );

Another way of creation of a public dispatcher instance is the
usage of add_dispatcher_if_not_exists method:
void my_agent::evt_long_request( const request & evt )
{
// New child cooperation is necessary for handling the request.
// An instance of dispatcher is required for the cooperation.
// Create a dispatcher if it doesn't exist.
so_environment().add_dispatcher_if_not_exists(
"request_handler", [] { return so_5::disp::active_group::create_disp(); } );
so_5::introduce_child_coop( *this,
so_5::disp::active_group::create_disp_binder( "request_handler" ),
[&]( so_5::coop_t & coop ) {
...
} );
}

A binder for public dispatcher is created via appropriate
create_disp_binder() function which receives the name of
the dispatcher:
// A main binder for coop will be a binder to active_obj dispatcher with name "Handler".
so_5::disp::active_obj::create_disp_binder( "Handler" ) );
...
// This agent will be bound to one_thread dispatcher with name "Monitor". It is because
// a personal dispatcher binder is set for that agent.
so_5::disp::one_thread::create_disp_binder( "Monitor" ), ... );

An instance of private dispatcher is created by appropriate
create_private_disp() which is implemented for every
standard dispatcher type in SObjectizer.
A kind of smart pointer is returned by create_private_disp()
function. This handle can be used for binding agents for that
private dispatcher instance.
The instance will be destroyed automatically when no one
uses this handle anymore.

An example of creating private dispatchers:
// A main binder for coop will be a binder to a private active_obj dispatcher.
so_5::disp::active_obj::create_private_disp( env )->binder() );
// This agent will be bound via coop's main binder to active_obj dispatcher.
coop->make_agent< some_agent_type >( ... );
// This agent will also be bound to active_obj dispatcher.
coop->make_agent< another_agent_type >( ... );
// But this agent will be bound to private one_thread dispatcher.
so_5::disp::one_thread::create_private_disp( env )->binder(), ... );

Public dispatchers were created in the very first versions of
SObjectizer and they have some limitations. For example
there is no way to stop a public dispatcher if no one uses it.
Private dispatchers were added later and they solve some
problems related to public dispatchers.
Because of that usage of private dispatchers is preferable.

Event Handler Thread Safety

All event handlers are treated as not thread safe by default.
It means that if there are events E1 and E2 in event queue
for some agent A then these events will be delivered to A
one after another.
It is impossible that event handler for E1 is running at the
same time as event handler for E2 but on different thread.
SObjectizer guarantees that one event handler completed its
execution and only then the next event handler will be called.

Thread safety has sense on dispatchers which use pools of
threads.
Such dispatchers can migrate an agent from one working
thread to another.
Because such migration can take place the SObjectizer's
thread safety guarantee simplifies development of agents:
there is no need for synchronization of access to agent's
internals by using locks or something like that.

User can mark an event handler as thread safe.
For example if event handler doesn't modify state of the
agent. Or if such modification is implemented via usage of
appropriate mechanism like mutex or spinlock.
Most of standard SObjectizer's dispatchers will ignore that
mark and will treat every event handler as thread unsafe.
But there is adv_thread_pool dispatcher which can use that
mark appropriately.

When agent is bound to adv_thread_pool dispatcher and
event handler H is marked as thread safe then the
dispatcher can schedule calls of H for different events on
several of working threads at the same time.
Moreover if agent has thread safe event handler H and not
thread safe event handler G then adv_thread_pool will
guarantee that all event handlers H will finish its work before
a single event handler G will be invoked.

At v.5.5.15 there is only one dispatcher which can
distinguish between not thread safe and thread safe event
handlers.
But more dispatchers with such abilities can be introduced in
the future versions of SObjectizer.

A Brief Overview Of Existing Dispatchers

There are eight standard dispatchers in SObjectizer-5.5
Five of them do not support agent's priorities: one_thread,
active_obj, active_group, thread_pool and adv_thread_pool.
These dispatcher think that all agents have the same priority.
Three dispatchers support agent's priorities:
prio_one_thread::strictly_ordered, prio_one_thread::
quoted_round_robin, prio_dedicated_threads::one_per_prio.
They schedule agents with respect to agent's priority.

All stuff related to some dispatcher is defined in a specific
namespace inside so_5::disp.
For example:
so_5::disp::active_obj
so_5::disp::thread_pool
so_5::disp::prio_one_thread::strictly_ordered

Every dispatcher-specific namespace has the similar set of
functions and classes like:
create_disp, create_private_disp, create_disp_binder
queue_params_t, disp_params_t, bind_params_t
This makes usage of different dispatchers very similar: if you
know how to create one_thread dispatcher you easily create
create active_obj or thread_pool dispatcher.

one_thread Dispatcher

one_thread dispatcher is the oldest, simplest and,
sometimes, most useful dispatcher.
It creates just one working thread and just one event queue.
Events for all agents bound to that dispatcher are stored in
single event queue.
The single working thread gets the next event from the
queue and runs event handler form agent-receiver. Then
gets the next event and so on.

It is possible to say that one_thread implements cooperative
multitasking: if some agent hangs then all other agents
bound to that dispatcher will hang too.

active_obj Dispatcher

active_obj is another very old and simple dispatcher.
But now, after addition of private dispatchers to SObjectizer,
it is less useful than before.
active_obj dispatcher creates a separate working thread and
separate event queue for every agent bound to that
dispatcher.
It means that every agent will have its own working thread
and its own event queue.

active_obj dispatcher creates a new event queue and
launches a new working thread for every agent bound to it.
When agent is deregistered the dispatcher stops the working
thread of this agent and deletes the corresponding event
queue.
It means that if there are no agents bound to active_obj
dispatcher there are no working threads nor event queues.

active_obj dispatcher is useful if it is necessary to provide a
separate working thread for some agent.
For example if agent needs a dedicated context for long-
running activities like interaction with hardware devices,
performing blocking calls to 3rd party libraries or external
systems and so on.
The similar effect can be achieved by using a private
one_thread dispatcher. But sometimes usage of active_obj
dispatcher requires less efforts...

An example of usage of active_obj dispatcher:
// A main binder for coop will be binder to private active_obj dispatcher.
// It means that all agents of the coop without explicitly specified
// binder will be bound to that active_obj dispatcher and become
// active objects with own working thread.
so_5::disp::active_obj::create_private_disp( env )->binder() );
for( const auto & d : devices )
// Create a separate agent for every device.
coop->make_agent< device_manager >( d );
env.register_coop( std::move(coop) );

active_group Dispatcher

active_group is another very old and simple dispatcher.
But now, after addition of private dispatchers to SObjectizer,
it is less useful than before and can be replaced by private
one_thread dispatcher.
active_group dispatcher creates a separate working thread
and separate event queue for named group of agents. It
means that all agents from the same group will work on a
common working thread.

active_group dispatcher creates a new working thread and a
new event queue when the first agent of new group is
registered.
active_group dispatcher stops group's working thread and
destroy group's event queue when the last agent from the
group is deregistered. It means that all resources for a group
are deallocated when group lost the last member.

If a new agent is added to group which disappear earlier a
new group with the same name will be created.
For example:
● agents A and B is being registered as part of group G1;
● new group G1 is created, new working thread is started for G1;
● agents A and B are deregistered;
● working thread for G1 is stopped, group G1 is deleted;
● agents C and D is being registered as part of group G1;
● new group G1 is created, new working thread is started for G1;

Every instance of active_group dispatcher has its own set of
active groups.
A group name must be unique only inside one dispatcher
instance. It means that dispatcher instance ag1 can have
group G1 and dispatcher instance ag2 can have group G1
too.
These groups will be different. Lifetime of ag1.G1 and ag2.
G1 will be independent from each other.

active_group dispatcher can be useful if it is necessary to
bind a group of agents to the same working thread. For
example if these agents performs different stages of
complex requests processing.
Let's imagine a processing pipeline of three stages: requests
params validation, request processing and generation of
results. Every of these states can take a long time so it is
better to provide a separate context to every request.
It can be done by using active_group dispatcher...

An example of binding agents to the same active group:
std::string request_id = generate_request_id( request );
// Create coop with request ID used as coop name.
auto coop = env.create_coop( request_id,
// All agents of this coop will be part of active group on
// active_group dispatcher with name "Handlers".
// request_id will be used as group name.
so_5::disp::active_group::create_disp_binder( "Handlers", request_id ) );
// Filling coop with agents.
coop->make_agent< request_checker >( request );
coop->make_agent< request_handler >( request );
coop->make_agent< response_generator >( request );
env.register_coop( std::move(coop) );

After addition of private dispatcher to SObjectizer
active_group dispatcher can often be replaced by private
one_thread dispatcher.
But sometimes active_group dispatchers can still be useful.
For example if agents for request processing cannot be
created at the same time and agent for the next stage can
be registered only after completion of the previous state...

An example of addition of agents from different coops to the
same active group:
void request_acceptor::new_request( const request_type & request ) {
// A cooperation with the first stage must be created.
env.introduce_coop( request_id + "-checker",
so_5::disp::active_group::create_disp_binder( "Handlers", request_id ),
[&]( so_5::coop_t & coop ) { coop.make_agent< request_checker >( request ); } );
}
...
void request_checker::check_completed() {
// A cooperation with the second stage can be created.
so_environment().introduce_coop( request_id + "-handler",
so_5::disp::active_group::create_disp_binder( "Handlers", request_id ),
[&]( so_5::coop_t & coop ) { coop.make_agent< request_handler >( request ); } );
}

thread_pool Dispatcher

thread_pool dispatcher creates a pool with several working
threads.
An agent bound to thread_pool dispatcher can handle its
events on any of working threads from that pool.
thread_pool dispatcher guarantees that at any given moment
only single event handler of the agent might run thus using
only one of the working threads. It is impossible to run event
handlers e1 and e2 of agent A on different thread at the
same time.

The most complicated moment in thread_pool dispatcher is
event queues.
There are two important things which need to be mentioned
for understanding of thread_pool dispatcher logic:
● type of FIFO for agent/coop;
● max_demands_at_once parameter.

Type of FIFO determines the relationship between event
handlers of agents from one coop.
Suppose there is a coop with agents A, B and C. Some
agent D sends the following messages in exactly that order:
● M1 to A
● M2 to B
● M3 to A
● M4 to C
● M5 to B

If agents A, B and C use cooperation FIFO then all these
agents will use one common event queue. Sequence of
event handler calls will look like:
1. A's event handler for M1
2. B's event handler for M2
3. A's event handler for M3
4. C's event handler for M4
5. B's event handler for M5
Exactly in that order. B's event handler for M2 will be called only
when A's event handler for M1 finished its work.

Cooperation FIFO also means that agents with that FIFO
type cannot work on different threads at the same time.
For example it is possible that agent A will handle M1 on
thread T1. Then agent B will handle M2 on thread T2. Then
agent A will handle M3 on T3.
But it is impossible that A will handle M1 on T1 and B will
handle M2 on T2 at the same time.

If agents A, B and C use individual FIFO then all these
agents will use different and independent event queues. It is
hard to predict as a sequence of event handlers will look like.
For example:
1. A's event handler for M1 on thread T1
2. B's event handler for M2 on thread T2
3. C's event handler for M4 on thread T3
4. B's event handler for M5 on thread T2
5. A's event handler for M3 on thread T1

Individual FIFO also means that agents with that FIFO type
can work on different threads in parallel.
For example it is possible that agent A will handle M1 on
thread T1. At the same time agent B will handle M2 on
thread T2. At the same time agent C will handle M4 on
thread T3.

There is another side of FIFO type: thread safety.
If agents from a coop use cooperative FIFO then they do not
need to synchronize access to some shared data.
Suppose several agents from one coop use common std::map
object. Because these agents cannot work on different threads
at the same time they can read and modify that object without
any mutexes or something like that.
But if agents use individual FIFO every shared data must be
protected or completely avoided.

Parameter max_demands_at_once tells how many events
from an event queue can be processed by working thread
before switching to processing of another event queue.
This parameter is important because every working thread in
thread_pool dispatcher works this way:
● gets the next non-empty event queue;
● handle at most max_demands_at_once events from it;
● gets the next non-empty event queue;
● ...

Suppose there are agents A, B and C from one coop with
cooperation FIFO and events for messages M1, M2, ..., M5
in their event queue....

If max_demands_at_once is 4 then the following scenario
might happen:
● some working thread T1 gets this event queue and calls
event handlers for messages M1, ..., M4;
● working thread T1 switches for handling different event
queue. Event queue for A, B, C agents holds M5;
● some working thread Tn gets this event queue and calls
event handler for M5;
● event queue for A, B, C agents becomes empty and Tn
switches to another event queue.

If max_demands_at_once is 1 then a working thread will
handle just one event from A, B, C agent queue.
Then this queue can be handled by another working thread
which will handle just one event.
Then this queue can be handled by another working thread...
And so on.

A more interesting situation can be if agents A, B and C use
individual FIFO.
In that case there will be independent queues for these
agents:
● a queue for A with M1 and M3;
● a queue for B with M2 and M5;
● a queue for C with M4.

If max_demands_at_once is greater than 1 then there could
be the following scenario:
● thread T1 handles M1 and M3 for A;
● thread T2 handles M4 for C;
● thread T3 handles M2 and M5 for B.
But if there are just two working threads in the pool:
● thread T1 handles M1 and M3 for A;
● thread T2 handles M4 for C;
● thread T1 handles M2 and M5 for B.

Value of max_demands_at_once determines how often a
working thread will switch from one event queue to another.
It can have a huge impact on application performance: small
values of max_demands_at_once will lead to frequent queue
switching and this will slow down event processing.
So large values of max_demands_at_once can speed up
event processing if there is a dense flow of events.

Implementation of thread_pool dispatcher makes things yet
more complex and interesting: every agent for a coop can
have its own parameters for thread_pool dispatcher.
It means that agent A can have individual FIFO and
max_demands_at_once=100, but agents B and C from the
same coop will have cooperation FIFO and
max_demands_at_once=10.
In this case two different event queues will be created: one
for A and another for B and C.

This complex logic of thread_pool allows precise
performance tuning for complex use cases which can be
found in real-life problems.
But in the simple cases the default parameters can be used
(cooperation FIFO and max_demands_at_once=4). This
significantly simplifies usage of thread_pool dispatcher.
Especially when several coops are bound to the same
thread_pool dispatcher instance.

An example for thread_pool dispatcher:
void init( so_5::environment_t & env )
{
using namespace so_5::disp::thread_pool;
env.introduce_coop(
// Create an instance of thread_pool dispatcher with 3 working threads.
create_private_disp( env, 3 )->binder(
// All agents will use individual FIFO.
// Parameter max_demands_at_once will have the default value.
bind_params_t{}.fifo( fifo_t::individual ) ),
[]( so_5::coop_t & c ) {
auto collector = c.make_agent< a_collector_t >();
auto performer = c.make_agent< a_performer_t >( collector->so_direct_mbox() );
collector->set_performer_mbox( performer->so_direct_mbox() );
c.make_agent< a_generator_t >( collector->so_direct_mbox() );
});
}

adv_thread_pool Dispatcher

adv_thread_pool dispatcher is similar to thread_pool
dispatcher but has two important differences:
1. It allows to invoke several thread safe event handlers for
an agent at the same time on different threads.
2. As a consequence of previous point there is no
max_demands_at_once parameter.

Every working thread in adv_thread_pool works this way:
● gets some non-empty event queue;
● gets the first event from that queue;
● checks thread safety for event handler for that event;
● checks a possibility of invocation of this event handler;
● if event handler can be invoked it is called;
● if event handler cannot be invoked then event is returned
to event queue and thread switches to another non-
empty event queue.

An event handler for an agent can be called if:
● it is not thread safe and there is no any other event
handlers of that agent which are running now on one of the
threads;
● or it is thread safe and there is no any not thread safe event
handler which is working now on some thread.
It means that only one not thread safe event handler can be
called. And next event handler can be called only after
completion of previous handler. Several thread safe handlers
can work at the same time.

There are two types of FIFO for adv_thread_pool:
cooperation and individual. Just like in thread_pool
dispatcher.
But in adv_thread_pool FIFO type has influence on checking
for thread safety of event handler.
Suppose there are agents A and B from one coop and they
use cooperation FIFO. Suppose also that event handlers A.
e1 and B.e2 are thread safe, but event handler A.e3 is not.

If there is events e1, e2, e3, e2, e1, e3 in the event queue
for A and B agents then event handlers will be called in the
following sequence: A.e1 on thread T1, B.e2 on thread T2.
Only after completion of event handlers A.e1 and B.e2 event
handler A.e3 will be called.
Then A.e1 on T1 and B.e2 on T2. Then waiting for
completion and the call to A.e3.

If agents A and B use individual FIFO and there are events
sequence {e1, e3, e1} and {e2, e2} in two event queues then
there could be the following scenario:
● A.e1 on T1, then A.e3 on T1, then A.e1 on T1;
● B.e2 on T2;
● B.e2 on T3.

adv_thread_pool can be useful for spreading thread safe
event processing on several working threads.
For example there could be cryptographer agent which performs
make_signature, check_signature, encrypt_block and
decrypt_block operations.
These operations are thread safe because they don't change the
state of cryptographer agent. Event handler for these operations
can be marked as thread safe. This allows to handle several
cryptographic operations at the same time in parallel.

Subscription of thread safe event handler:
class cryptographer : public so_5::agent_t
{
void on_make_signature( const make_signature & ) {...}
void on_check_signature( const check_signature & ) {...}
void on_encrypt_block( const encrypt_block & ) {...}
void on_decrypt_block( const decrypt_block & ) {...}
...
virtual void so_define_agent() override {
so_subscribe_self()
.event( &cryptographer::on_make_signature, so_5::thread_safe )
.event( &cryptographer::on_check_signature, so_5::thread_safe )
.event( &cryptographer::on_encrypt_block, so_5::thread_safe )
.event( &cryptographer::on_decrypt_block, so_5::thread_safe )
...
}
}

An agent can have thread safe and not thread safe event
handlers.
For example cryptographer agent can have not thread safe
event handler for message reconfigure.
In this case adv_thread_pool dispatcher guarantees that all
thread safe event handlers finish their work before an event
handler for reconfigure message will be started.

An example of binding agent to adv_thread_pool dispatcher:
using namespace so_5::disp::adv_thread_pool;
env.introduce_coop(
// All agents of new coop will work on adv_thread_pool dispatcher.
// Every agent will use individual FIFO.
create_private_disp( env, 4 )->binder( bind_params_t{}.fifo( fifo_t::individual ) ),
[]( so_5::coop_t & coop ) {
coop.make_agent< cryptographer >();
...
} );

prio_one_thread::strictly_ordered Dispatcher

prio_one_thread::strictly_ordered dispatcher allows for
events of high priority agents to block events of low priority
agents.
It means that events queue is always strictly ordered: events
for agents with high priority are placed before events for
agents with lower priority.

For example if event queue is {e1(a7), e2(a6), e3(a4), e4
(a4)}, where a7 means an agent with priority p7, then events
will be handled in exact that order.
After handling e1 the queue will be {e2(a6), e3(a4), e4(a4)}.
If e5 for a5 arrived then the queue will become {e2(a6), e5
(a5), e3(a4), e4(a4)}.
It means that chronological order of events will be preserved
only for events of agents with the same priority.

This dispatcher could be useful if there is a necessity of
handling some messages before other messages.
For example there could be a stream of tasks represented
by take_job messages. There also could be a special
message for task processor’s reconfiguration: new_config
message.
It could have a sense to handle new_config as soon as
possible.

This can be done by two agents which are bound to single
prio_one_thread::strictly_ordered dispatcher.
One agent will have priority p1 and will handle new_config
message. Second agent will have priority p0 and will handle
take_job.
Both agents will have common shared data (at least
configuration parameters, may be something else). Dispatcher
prio_one_thread::strictly_ordered guarantees that new_config
will be handled as soon as processing of previous message
finished.

An example of binding agents to prio_one_thread::
strictly_ordered dispatcher:
namespace prio_disp = so_5::disp::prio_one_thread::strictly_ordered;
env.introduce_coop(
// Dispatcher instance and binder for it.
prio_disp::create_private_disp( env )->binder(),
// An agent with higher priority.
coop.make_agent< config_manager >( so_5::prio::p1 );
// An agent with lower priority.
coop.make_agent< job_performer >( so_5::prio::p0 );
} );

prio_one_thread::quoted_round_robin Dispatcher

prio_one_thread::quoted_round_robin dispatcher works on
round-robin principle.
It allows to specify maximum count of events to be
processed consequently for the specified priority. After
processing that count of events dispatcher switches to
processing events of lower priority even if there are yet more
events of higher priority to be processed.

Dispatcher handles no more than Q7 events of priority p7,
then no more than Q6 events of priority p6, ..., then no more
than Q0 events of priority p0.
If an event of higher priority is arrived during handling a
quote for lower priority no switching is performed.
For example if dispatcher handles events of priority p5 and
event of priority p7 is arrived the dispatcher will continue to
handle events of p5, then events of p4 (if any), ..., then
events of p0 (if any). And only then events of p7.

This working scheme means that agent’s priorities treated as
agent’s weight.
A programmer can set bigger quotes for more prioritized
(more heavyweight) agents and that agents will receive more
resources than less prioritized (less weighted) agents.

A very important detail: events of the same priority are
handled in chronological order.

A dispatcher of that type can be useful, for example, if there
are agents which handles clients of different types.
Some clients are VIP clients and they should receive first-class
quality of service and there could be other clients with lower
demands for service quality.
A high priority to agents for handling VIP-client requests can be
used and a large quote for that priority can be set. All other agents
will have lower priority and smaller quote. As result more requests
from VIP-clients will be handled but there also will be processing
of request from other clients.

An example of binding agents to prio_one_thread::
quoted_round_robin dispatcher:
namespace prio_disp = so_5::disp::prio_one_thread::quoted_round_robin;
env.introduce_coop(
// Create dispatcher and define quotes for several priorities.
prio_disp::create_private_disp( env,
// By default every priority will have quote for 20 events.
prio_disp::quotes_t{ 20 }
// Priority p7 will have different quote.
.set( so_5::prio::p7, 45 )
// Priority p6 will have different quote too.
.set( so_5::prio::p6, 35 ) )->binder(),
coop.make_agent< vip_client_processor >( so_5::prio::p7 );
coop.make_agent< ordinal_client_processor >( so_5::prio::p6 );
coop.make_agent< free_client_processor >( so_5::prio::p0 );
} );

prio_dedicated_threads::one_per_prio Dispatcher

prio_dedicated_threads::one_per_prio dispatcher creates a
single dedicated thread for every priority.
It means that events for agents with priority p7 will be
handled on different thread than events for agents with, for
example, priority p6.
Events of the same priority are handled in chronological
order.

Because of the fact that priority is assigned to an agent at its
creation and cannot be changed later an agent inside
prio_dedicated_threads::one_per_prio is bound to a
particular working thread and do not moved from that thread
to any other thread.
This property of the dispatcher can be used for binding
processing of different message types to different working
threads: messages of type M1 can be processed by agent
A1 with priority p1, messages of type M2 -- by agent A2 with
priority p2 and so on...

Assigning different priorities for agents which handle
different message type may have sense if OS-specific API
for changing thread priority is used.
For example an agent with priority p7 can set higher priority
for its working thread in so_evt_start() method than agent
with priority p6.

An example of binding agents to prio_dedicated_threads::
one_per_prio dispatcher:
namespace prio_disp = so_5::disp::prio_dedicated_threads::one_per_prio;
env.introduce_coop(
// An instance of dispatcher and a binder for it.
prio_disp::create_private_disp( env )->binder(),
coop.make_agent< m1_handler >( so_5::prio::p1 );
coop.make_agent< m2_handler >( so_5::prio::p2 );
...
} );

Some Final Words

Dispatchers play significant role in application development
on top on SObjectizer-5.
It is because SObjectizer do not hide multithreading from a
user. But simplifies a work with multithreading in application
domains where threads are important.
Application domains are different. Even inside one domain
different event scheduling policies might be needed. This is
why SObjectizer provides several dispatcher types.

This presentation has no intention to be a comprehensive
guide for SObjectizer's dispatcher.
It is just an attempt to explain a role of dispatchers and a
short overview of standard SObjectizer's dispatcher and their
features.
For more detailed information please see corresponding
sections in the Project's Wiki and materials in the Project's
Blog.

Additional Information:
Project’s home: http://sourceforge.net/projects/sobjectizer
Documentation: http://sourceforge.net/p/sobjectizer/wiki/
Forum: http://sourceforge.net/p/sobjectizer/discussion/
Google-group: https://groups.google.com/forum/#!forum/sobjectizer
GitHub mirror: https://github.com/masterspline/SObjectizer

Dive into SObjectizer 5.5. Eighth Part: Dispatchers

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Dive into SObjectizer 5.5. Eighth Part: Dispatchers (13)

More from Yauheni Akhotnikau (14)

Recently uploaded (20)

Dive into SObjectizer 5.5. Eighth Part: Dispatchers