Embedded Software For The Iot 3rd Edition Klaus Elk

Embedded Software For The Iot 3rd Edition Klaus
Elk download
https://ebookbell.com/product/embedded-software-for-the-iot-3rd-
edition-klaus-elk-51027530
Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.
Embedded Software Development For The Internet Of Things The Basics
The Technologies And Best Practices Klaus Elk
https://ebookbell.com/product/embedded-software-development-for-the-
internet-of-things-the-basics-the-technologies-and-best-practices-
klaus-elk-5903222
The Art Of Software Thermal Management For Embedded Systems 1st
Edition Mark Benson Auth
https://ebookbell.com/product/the-art-of-software-thermal-management-
for-embedded-systems-1st-edition-mark-benson-auth-4634992
Software And Compilers For Embedded Systems 8th International Workshop
Scopes 2004 Amsterdam The Netherlands September 23 2004 Proceedings
1st Edition Michael Uhler Auth
https://ebookbell.com/product/software-and-compilers-for-embedded-
systems-8th-international-workshop-scopes-2004-amsterdam-the-
netherlands-september-23-2004-proceedings-1st-edition-michael-uhler-
auth-1227400
If I Only Changed The Software Why Is The Phone On Fire Embedded
Debugging Methods Revealed Technical Mysteries For Engineers Lisa
Simone
https://ebookbell.com/product/if-i-only-changed-the-software-why-is-
the-phone-on-fire-embedded-debugging-methods-revealed-technical-
mysteries-for-engineers-lisa-simone-36441642

Embedded Software For Soc 1st Edition Ahmed Amine Jerraya Sungjoo Yoo
https://ebookbell.com/product/embedded-software-for-soc-1st-edition-
ahmed-amine-jerraya-sungjoo-yoo-2440582
Embedded Software Development For Safetycritical Systems Chris Hobbs
https://ebookbell.com/product/embedded-software-development-for-
safetycritical-systems-chris-hobbs-5266576
Embedded Software Development For Safetycritical Systems 2nd Edition
Chris Hobbs
https://ebookbell.com/product/embedded-software-development-for-
safetycritical-systems-2nd-edition-chris-hobbs-11091226
Embedded Systems Architecture Design And Write Software For Embedded
Devices 2nd Edition 2nd Daniele Lacamera
https://ebookbell.com/product/embedded-systems-architecture-design-
and-write-software-for-embedded-devices-2nd-edition-2nd-daniele-
lacamera-47545404
Design Patterns For Embedded Systems In C An Embedded Software
Engineering Toolkit 1st Edition Bruce Powel Douglass
https://ebookbell.com/product/design-patterns-for-embedded-systems-in-
c-an-embedded-software-engineering-toolkit-1st-edition-bruce-powel-
douglass-4167368

Klaus Elk
Embedded Software for the IoT

Klaus Elk
Embedded
Software
for the IoT
|
3rd edition

ISBN 978-1-5474-1715-5
e-ISBN (PDF) 978-1-5474-0102-4
e-ISBN (EPUB) 978-1-5474-0104-8
Library of Congress Control Number: 2018959434
Bibliographic information published by the Deutsche Nationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie;
detailed bibliographic data are available on the Internet at http://dnb.dnb.de.
© 2019 Klaus Elk
Information provided in this book is derived from many sources, standards, and analyses—including
subjective personal views. Any errors or omissions shall not imply any liability for direct or indirect
consequences arising from the use of this information. The author makes no warranty for the
correctness or for the use of this information, and assumes no liability for direct or indirect damages
of any kind arising from technical interpretation or technical explanations in this book, for
typographical or printing errors, or for any subsequent changes. All rights reserved.
See also https://klauselk.com
Published by Walter de Gruyter Inc., Boston/Berlin
Cover image: Tramino / iStock / Getty Images Plus
Typesetting: VTeX UAB, Lithuania
Printing and binding: CPI books GmbH, Leck
www.degruyter.com

About De|G PRESS
Five Stars as a Rule
De|G PRESS, the startup born out of one of the world’s most venerable publishers,
De Gruyter, promises to bring you an unbiased, valuable, and meticulously edited
work on important topics in the fields of business, information technology, comput-
ing, engineering, and mathematics. By selecting the finest authors to present, without
bias, information necessary for their chosen topic for professionals, in the depth you
would hope for, we wish to satisfy your needs and earn our five-star ranking.
In keeping with these principles, the books you read from De|G PRESS will be
practical, efficient and, if we have done our job right, yield many returns on their price.
We invite businesses to order our books in bulk in print or electronic form as a
best solution to meeting the learning needs of your organization, or parts of your or-
ganization, in a most cost-effective manner.
There is no better way to learn about a subject in depth than from a book that is
efficient, clear, well organized, and information rich. A great book can provide life-
changing knowledge. We hope that with De|G PRESS books you will find that to be the
case.
https://doi.org/10.1515/9781547401024-201

Preface
The internet of things is here, and soon 50 billion devices will be “connected.” So we
are told. This raises the question: “Who is going to program all of these devices?”
In a major survey at “StackOverflow” in 2018, 5.2 % of the 100k responding de-
velopers claimed to work with embedded applications or devices. This is twice the
percentage from the same survey in 2016. Still there is a large potential in attracting
developers from the remaining 94.8 %.
Common to all these developers is the overwhelming amount of new domains they
need to get into, on top of their basic programming skills.
The “2018 IoT Developer/Engineer Census and Analysis” from VDC Research
states that “growth and demand has slowed for traditional engineers as engineering
companies seek out ‘jack-of-all-trades’ IoT developers with domain-specific skills and
the cloud/IT skills to build connected solutions and applications.”
It is exactly this multitude of skills that this book aims to bring to the reader. The
author presents solid basic knowledge in the relevant domains in a structured way.
This creates a strong foundation, onto which all the scattered details from the web
may be attached.
Throughout the book, the author draws on 30+ years of good and bad real-life ex-
periences from the private industry as well as from university teaching, in an informal
and relevant way.
What is new in this edition?
Compared to the first edition, this book has two new chapters in the “IoT Technolo-
gies” part. It may not be a surprise that one of these is about internet security. This area
is growing in importance as fast as the internet of things is growing in size. The other
new chapter is about statistical process control (SPC). This is a less obvious choice.
However, as the introduction to this book shows, SPC is an important part of “Indus-
try 4.0,” another term closely related to IoT.
On top of these two new chapters, all the existing chapters are updated. The previ-
ous “Processes” chapter has been changed to “Code Maintenance” and now includes
sections on Yocto and especially git, while similar changes have been made in other
chapters. In terms of pages, this edition is more than 50% longer than the first.
The WireShark screenshots in the central network part are easier to read, while
the many new figures and tables improve the reading experience.
This third edition is published by De Gruyter. This has meant countless improve-
ments to the content as well as the print and design. Many details are updated to 2018
status using new information, and Python is now central in simulations.

XIV | Preface
About the author
Klaus Elk graduated as Master of Science in electronics from the Danish Technical Uni-
versity in Copenhagen in 1984, with a thesis in digital signal processing. Since then, he
has worked in the private industry within the domains of telecommunication, medical
electronics, and sound and vibration. He also holds a Bachelor’s degree in Marketing.
In a period of 10 years, Klaus—besides his R&D job—taught at the Danish Technical
University. The subjects were initially object oriented programming (C++ and Java),
and later the Internet Protocol Stack. Today he is R&D Manager in Instrumentation at
Brüel & Kjær Sound and Vibration.
Acknowledgments
I am very grateful to Stuart Douglas for his scouting—bringing this book into the
De Gruyter family. Thanks to my editor, Jeffrey Pepper, for his patient scrutinizing
work—my overuse of “initial caps” and hyphens alone did not make this easy. Jeffrey
has provoked many improvements to the text and figures that enhances the reading
experience. Likewise, I want to thank Paul Cohen for his good in-depth technical
review. Finally, thanks to my family for listening patiently and for their acceptance of
my many hours by the PC.
Klaus Elk

1 Introduction
1.1 The tale of the internet
Local Area Networks (LANs) became accessible during the 1970s. This technology al-
lowed computers within a building to connect to each other. The typical applications
were printing and basic file sharing, leaving the rest to the user. With the appearance
of the IBM PC in the mid-1980s, the market grew, and we saw a regular war between
network vendors with each of their technology, for example, Token-Ring, Token-Bus,
Ethernet, Novell, AppleTalk, and DECnet. Especially, the first three fought for the gen-
eral market, whereas AppleTalk and DECnet, respectively, were specific to Apple and
Digital. Novell quickly adopted Ethernet in its solutions—successfully focusing on net-
work software for years. Even though Ethernet was not the technically most advanced
solution, and only covered what we now know as the two lowest layers of the OSI stack,
it won the war. This is a great demonstration of how a solution which is available and
“good enough” can gain traction, and become popular, leading to lower prices, more
sales, etc.
Before Ethernet became the clear winner, engineers and scientists were struggling
with how to scale the LANs to become more than local. Different physical layers, ad-
dressing schemes, and protocols, almost made this impossible. Vinton Cerf and Robert
Khan from DARPA presented, in a paper in 1974, a concept of common virtual ad-
dresses on top of the various physical-rooted addresses. They presented a new com-
mon protocol, Transmission Control Protocol, or TCP, on top of these. TCP connected
the existing networks, and they called the concept the internet. The virtual addresses
became known as Internet Protocol Addresses, or IP addresses. Later on, TCP and IP
became separate layers, making it possible to run, for example, the simpler UDP on
top of the IP network protocol. TCP and IP are treated in depth in Chapter 7.
Looking back, it is amazing how Ethernet has scaled from 10 Mbit/s to 10 Gbit/s,
in affordable and still more robust solutions, and how TCP has been able to adapt to
this. IP version 4 is growing out of addresses and IP version 6 is gradually taking over,
as discussed in Chapter 7. The virtual addresses are hierarchical. This means that they
are routable, and can change as we move. This is exactly like post addresses and has
proven extremely useful.
In 1989, Tim Berners-Lee of CERN presented the world wide web. Many people
think of WWW as being the same as the internet, but WWW is some 20+ years younger
than the internet, while being simply one of many applications on it. The “hourglass”
architecture shown in Figure 1.1 with the single IP in the center, millions of applica-
tions on top, and many physical network solutions at the bottom, has also proven itself
to be extremely powerful. Software architecture is discussed in Chapter 4.
In the same period that the internet was developed, Unix was developed at AT&T
using the new machine-independent language C. Unix and C were both created by Ken

2 | 1 Introduction
Figure 1.1: Internet stack – an hourglass.
Thompson and Denis Ritchie. Unix was also one of the first adopters of TCP/IP. In other
words, many of the strengths Linux has today originates from the 1970s. Operating
Systems are discussed in Chapter 2.
1.2 The cloud
While the early internet was mostly of academic interest,1
the world wide web brought
it to the people with its easy clickable hyperlinks. Web servers moved into companies
and institutions, where they began to serve fixed content, text and figures, from file
servers to all of us.
With the help of script languages such as PHP and CGI, the content became more
dynamic, now often fetched from databases. This lasted for some years until we saw
what was coined Web 2.0. Before Web 2.0, it was possible for a web server to update
the content sent to the web browser, at the cost of a complete redraw of the page. Now
web applications began to look like Windows or Mac applications, updating fields,
graphs, etc. This was also the breakthrough of open source with the popular LAMP,
which stands for Linux (the OS), Apache (the web server), MySQL (the database), and
Python or PHP or Perl (all scripting languages).
As the complexity of web servers grew and the internet became faster and more
robust, it became economically wise for many companies and institutions to use
web hotels. The web hotels based their income on the scale: the constantly changing
know-how required to run a single server is much better utilized running many. Now,
1 E-mail came before the WWW and was driving a slow growth of the internet.

1.3 Internet of things | 3
Figure 1.2: The cloud abstraction.
not even the employees of a given company knew where their company website was
hosted, and they did not care. It was somewhere in the cloud. The cloud term became
even more popular as we started to use smartphones, clearly connecting over the
air—through the clouds. See Figure 1.2.
1.3 Internet of things
The first webcam was used at Cambridge University to show a coffee-machine, so that
no one needed to go to an empty machine—a fun idea, but not the most practical. In-
stead you might consider building a web server into the coffee-machine, telling the
user how many cups are left. The problem with this solution is that it is not very scal-
able. This is an important term that we will meet later. A web server in the small elec-
tronics of a cheap coffee-machine would typically be too limited to serve many web
browsers. At this point in history, we actually did see a lot of web-enabled devices.
It is practical that service technicians can connect to hardware using a standard web
browser, and we still use this concept. At the time, however, the devices typically were
not permanently connected to the internet, they just used the technology on rare oc-
casions in a local connection, directly to the technician’s PC.
The scalable solution to the coffee-machine problem would be a PC-based web
server, reading many coffee machines, so that each machine only has a single client
reading it, for example, twice a minute, and then many users could connect to the web
server.
Let the coffee machine be any small internet-attached device, place the web server
in the cloud, and let the users connect via a smartphone, PC, or tablet, and you have
the Internet of Things, in short IoT. This is another application of the hourglass ar-
chitecture, shown in Figure 1.3, only now the hourglass is not the implementation in

4 | 1 Introduction
Figure 1.3: An unusual—but architectural relevant—view.
a single PC, but the entire internet. Rather unusual, the clients are placed above the
cloud to demonstrate the architectural point. Another term sometimes used for all the
devices “on the ground” is “the fog.”
The inventor of Linux, Linus Thorvalds, once stated in an interview that he created
Linux for the desktop, and that it has overtaken the computerized world, except on
the desktop. This refers to open source initially being extremely popular in the servers
due to price, robustness, and ease of programming, while now it is moving into the
small devices for the same reasons, plus a few more. Common to the (web)servers and
devices is that they are only used directly by computer professionals. These people
are not relying on crisp graphical interfaces, but prefer programs and scripts featuring
automation.
Devices such as the BeagleBone, Arduino, and Raspbery Pi have opened the Linux-
device world to new generations, but real-world devices typically need to be more ro-
bust. These are discussed in Chapter 3.
1.4 IoT related terms
– Industry 4.0
Germany has been the most advanced European country in the automotive market
for a long time. The main player, VW, was shaken in 2015 by DieselGate. Even
before this, the Germans worried about disruption—the way new technologies may

1.4 IoT related terms | 5
turn a thriving business into bankruptcy in no time. They knew they could not
afford to rest on their laurels. As one of ten Future Projects in a 2020 plan, the
German government publicized Industrie 4.0 in 2013.
The fourth industrial revolution has been claimed several times in history. This
time it was an elaborate plan, envisioning the connectivity of IoT with its ability
to bring real-time factory-floor data from, for example, Brazil, to production con-
trol in Germany. These data are typically in the form of statistics relating to spe-
cific assembly lines, product type, production gear, day/night shift, etc. Equally
important, this plan focused on decentralization. This means more intelligent em-
bedded systems at the assembly line, taking more autonomous and advanced de-
cisions. These systems could possibly be calibrated remotely along the way, based
on the statistics collected. Chapter 12 deals with Statistical Process Control (SPC)
for embedded systems, and in Chapter 4 we are looking at CANOpen. This is one
of several standards in factory automation.
Today, this advanced embedded software will often communicate with the prod-
uct itself. It is no surprise that a car may be queried for its type, serial number,
etc., and that calibrated production data may be written back to the car. However,
via the car, the plant software can talk to its internal CAN buses, and via these to,
for example, brakes, airbags, and other subsystems.
– Industrial Internet of Things—IIoT
The IIoT term sounds very much like the Industry 4.0 concept, but it has a less
clear birth certificate. The Industrial Internet Consortium has taken the term to
heart. IIC was formed in 2014 by AT&T, Cisco, General Electric, IBM, and Intel,
and since then companies from the entire world has joined. The vision is to as-
sure the true interconnection of devices. The industries include manufacturing,
energy, health care, transportation, and smart cities.
– Internet of everything—IoE
Cisco is one of the members of the IIC. Cisco defines the Internet of Everything (IoE)
as the “networked connection of people, process, data, and things.” In this way,
it includes the internet of things, without forgetting the already existing internet
of people. This makes sense, as Cisco’s market is the infrastructure, it does not
really matter whether a person or a device is generating or subscribing to data. By
including the word “process,” Cisco opens a door for expanding the current cloud
with its web servers and databases with actual applications. This is something
that also interests major players like Google, Microsoft, Facebook, and Amazon.
– Big data
Big data is clearly not the same as IoT. However, all of these billions of internet-
attached devices will create unprecedented amounts of data. Chapter 11 shows
how some of these data may be filtered already at their birth in the devices. This
will be essential in many scenarios. Processing power has grown at an impressive
pace during the computer era, but the amounts of storage have grown even more.
Big data may be fantastic, but if we want fast algorithms, they need to work on

6 | 1 Introduction
less, but more relevant, data. We need to reduce data at the source and in the
various process steps, to a certain limit. To quote Einstein: “Make things as simple
as possible, but not simpler.” The subject of treating big data in the cloud is not
in the scope of this book. We will however see various ways to reduce data.
Should you want to browse through the amazing birth of the web and all the innova-
tions that helped it along, take a look at 100 Ideas that Changed the Web by Jim Boul-
ton. This book spends only two pages per innovation, and one of these is typically a
color image. This makes it entertaining, but still very informative.

2 How to select an OS
In “the old days,” you would pick a CPU first and then discuss the operating system
(OS)—after discussing whether such was really necessary at all. Today, it is more com-
mon to choose the OS first and then a CPU, or a CPU-family. In reality, it is often an it-
erative process. You may, for example, decide to go for Linux, but this requires a MMU
(memory management unit), and this may drive you toward too big and expensive
CPUs. In that case, you may have to redo you original choice. Table 2.1 is a “kick-start”
to this chapter. It shows an overall trend from simple to advanced.
Table 2.1: Task management—from simple to advanced.
OS/Kernel/Language Type
Simple main Strictly polling
Ruby Co-routines
Modula-2 Co-routines
Windows 3 Nonpreemptive scheduler
ARM mbed simple Interrupts + main with FSM
OS-9 Preemptive real-time kernel
Enea OSE Preemptive real-time kernel
Windows CE Preemptive real-time kernel
QNX Neutrino Preemptive real-time kernel
SMX Preemptive real-time kernel
Windows NT Preemptive OS
ARM mbed advanced Preemptive scheduler
Linux Preemptive OS
RT-Linux Preemptive real-time OS
VxWorks Preemptive real-time OS
We will dive into the various types and degrees of operating systems and their pros and
cons. Along the way, important parameters are introduced. The solutions are ordered
in the most reader-friendly way toward a full preemptive real time operating system
(RTOS).
2.1 No OS and strictly polling
The simplest embedded system has no operating system, leaving some low-level de-
tails to the programmer. If you are using “C” there is a main() function from which your
“official” program starts at power-up. Since there is no OS,this must be assured by con-

10 | 2 How to select an OS
figuring the compiler, linker, and locater.1
It is necessary to initially call a small assem-
bly program that copies the program to RAM, disables interrupts, clears the data-area,
and prepares the stack and stack-pointer.
I once used a compiler package that did all the above. Unfortunately, the vendor had
forgotten the call that executes the code with all the global C-variable initializations,
something that you normally take for granted. So after realizing this, I had the choice
between completing the tool myself or remembering to initialize all global variables
explicitly in a special “init” function in main(). This is a typical example of the differ-
ence between programming in the embedded world and on a PC, where the tools are
more “polished” than those for the smaller systems.
In an OS-less system, main() has an infinite loop that could look like this:
Listing 2.1: Round-robin scheduling
1 int main(int argc , char *argv [])
2 {
3 for (;;)
4 {
5 JobA ();
6 JobB ();
7 JobA ();
8 JobC ();
9 }
10 }
This is a “round-robin” scheme with the slight enhancement that JobA has re-
ceived more “attention” (not really priority) by giving it access to the CPU with shorter
intervals than the other processes. Within each job, we read the relevant inputs from
our code when we have the time. This is known as “polling.” We might even make a
loop where we test an input again and again until it goes from one state to another.
This is called “busy-waiting” as the CPU does nothing else than loop. Introducing such
a loop, in say JobB, is a disaster for JobA and JobC—they will not be executed until this
state-change occurs. And what if the state-change we are waiting for in this loop is
actually depending on JobA or JobC doing something? In such a scenario, we have a
deadlock. Another problem with a busy-wait loop is that you waste a lot of energy, as
the CPU is not allowed to go into any form of power-saving. So busy-waiting in a loop
may at times be okay, but not in a system as simple as this.
Another concept, still without any operating system whatsoever, but a lot more
clever, is to introduce finite state machines (FSMs) where you read all inputs, decide
what has changed, and take action as shown in Listing 2.2.
1 The locater is typically integrated with the linker so do not worry if you have not heard about it
before.

2.1 No OS and strictly polling | 11
Listing 2.2: Main with finite state machine
1 int main(int argc , char *argv [])
2 {
3 for (;;)
4 {
5 ReadSensors (); // Read all inputs
6 ExtractEvents (); // Is temp above limit?
7 StateEventHandling (); // Take action
8 }
9 }
Listing 2.3 is one of three finite state machines that together control a TOE—TCP of-
fload engine. The TOE implements the actual transmissions of TCP in hardware while
the rest is handled in the embedded software, via the FSMs. Later, we will look into
sockets and TCP, and it can be seen that the listing very directly represents a good part
of Figure 7.8, which is a graphic representation of the TCP connection states. For now,
it is more relevant to look into the concept of a FSM.
Each column is a state of a TCP socket, that at a given time is the “current state.”
Each row represents an event that occurs while in this state, for example, an ACK (ac-
knowledge) has been received. Each element in the table contains the action to take,
as well as the next state. In order to fit the table into this book, it has been split in
two. In the real C-code, the part after “table continuing here” is placed to the right of
the lines above, so that we have a table with 7 rows and 7 columns (it is coincidental
that the number of states and events is the same). FSMs are not just practical in a sim-
ple OS-less system but can be used anywhere. The FSM shown in Listing 2.3 was used
in a Linux system. FSMs are very popular among hardware designers, but not used
by many software designers, which is a pity. Nevertheless, many modern frameworks
contain FSMs inside, offering “event-driven” models.
Listing 2.3: One of three finite state machines for a TOE
1 struct action connected_map[EV_MINOR(EV_ENDING_COUNT )]
2 [ST_MINOR(ST_ENDING_COUNT )] =
3 {
4 // st_normal st_close_wait st_last_ack st_fin_wait_1 //EVENT
5 //NORM CL_W LACK FW_1 // ev_end_
6 {{error , NORM},{error , CL_W},{error , LACK},{error , FW_1},// <error >
7 {{send_fin ,FW_1},{send_fin ,LACK},{no_act , LACK},{no_act , FW_1},// close
8 {{error , NORM},{error , CL_W},{req_own ,OWN },{fw1_2 , FW_2},// ACK
9 {{ack_fin , CL_W},{error , CL_W},{error , LACK},{ack_fin ,CL_G},// FIN
10 {{error , NORM},{error , CL_W},{error , LACK},{ack_fin ,TM_W},// FIN_ACK
11 {{error , NORM},{error , CL_W},{fin_to , CL },{fin_to , CL },// TimeOut
12 {{abort , GHO },{abort , GHO },{abort , GHO },{abort , GHO },// Exc_RST
13 };
14 // Table continuing here
15 // st_fin_wait_2 st_closing st_time_wait //EVENT
16 //FW_2 CL_G TM_W // ev_end_
17 {error , FW_2},{error , CL_G},{error , TM_W}}, // <error >
18 {no_act , FW_2},{no_act , CL_G},{no_act , TM_W}}, // close
19 {error , FW_2},{cl_ack , TM_W},{error , TM_W}}, // ACK

20 {ack_fin , TM_W},{error , CL_G},{error , TM_W}}, // FIN
21 {error , FW_2},{error , CL_G},{error , TM_W}}, // FIN_ACK
22 {req_own , OWN },{req_own ,OWN },{req_own ,OWN }}, // TimeOut
23 {abort , GHO },{abort , GHO },{abort , GHO }}, // Exc_RST
One nice thing about FSMs is that they are both implementation and documenta-
tion, and they typically can fit into a single page on the screen. Compared to a number
of if-else or switch clauses, the use of FSMs is much “cleaner” and, therefore, much
easier to create and keep error-free. With the good overview, it is easy to spot a miss-
ing combination of a state and an event. It is also a compact solution. In the example,
the same FSM-code works for all sockets; we only need to keep the current state per
socket, and the statemachine is called with two parameters only: the socket-handle
and the incoming event. Incidentally, when not coding in C++ but in C, it is a com-
mon pattern that the first parameter is the “object you do not have.” Thus, if the C++
version is socket->open(a, b), this becomes open(socket, a, b) in C.
Figure 2.1: Finite state machine in main.
Figure 2.1 shows an OS-less system. The main advantage is simplicity. There is no third-
party OS that you need to understand and get updates of. This can be very relevant if
the system is to last many years. A part of this simplicity is that the application may
read inputs and write to outputs directly. There is no “driver” concept. This can be
practical in a small system with a single developer, but it is also the downside, as a
simple error can lead to disasters. Figure 2.1 introduces a small setup that we will see
a couple of more times:
– Input 1 – leading to some processing and eventually to a change in output 1.
– Input 2 – leading to some processing and eventually to a change in output 2.
– Input 3 – leading to some processing and eventually to a change in outputs 3
and 4.
2.2 Co-routines
Co-routines are not like the tasks (which we will get back to) of an OS, but they do have
similar traits:

2.3 Interrupts | 13
1. There can be many instances of the same co-routine, typically one per resource,
for example, an actor in a game or a cell in a bio-simulation.
2. Each instance can pause at some point while the CPU executes something else. It
keeps its state and can continue on from the given point.
3. This pause must be invoked by the co-routine itself by “yielding” to another co-
routine. There is, however, no caller and callee. This is supported by some lan-
guages, not “C” but, for example, by Ruby and Modula-2.
Co-routines are mostly of academic interest in today’s embedded world. They
might come in fashion again—you never know.
2.3 Interrupts
Instead of polling the various inputs, interrupts are generated when inputs change.
One or more interrupt routines read the inputs and take action. An interrupt is what
happens when an external event in hardware, asynchronously triggers a change in the
execution flow. Typically, a given pin on the CPU is mapped to an interrupt-number. In
a fixed place in the memory-layout, you find the interrupt-vector, which is an array with
a fixed number of bytes per interrupt—containing mainly the address that the CPU
must jump to. This is the address of the interrupt service routine (ISR). When entering
the ISR, most CPUs have all interrupts disabled.2
In such a “purely interrupt-controlled system,” the interrupt service routine may
in principle do everything that is related to a given event. See Figure 2.2.
Figure 2.2: Purely interrupt controlled system.
There are many variants of such a system:
1. Each input has its own interrupt service routine (ISR), and its own interrupt pri-
ority. Thus one interrupt may interrupt the main program (stacking the registers
it plans to use), and then this may become interrupted by the next, higher level
2 except for the NMI—nonmaskable interrupt—if such exists.

interrupt, etc. This is known as nested interrupts and typically only can happen
if the original ISR has reenabled interrupts. This can be done by the OS, if such
exists, before giving control to the programmer, or by the programmer himself in
our current OS-less case. Nested interrupts is very normal in the bigger systems,
but if all actions to inputs are done inside the interrupt routines, it becomes very
important that the nested interrupt “understands” the exact state of the system.
This again depends on how far we got in the interrupted interrupt, which is al-
most impossible to know. This is one of the many reasons why you should defer
as much of the action as possible, to something that runs later, at a lower prior-
ity. But then it’s no longer a purely interrupt-based system. Furthermore, many
systems do not have interrupt levels enough to match all the inputs.
2. As above, each input has its own interrupt service routine (ISR), and its own in-
terrupt priority. In this system, however, nested interrupts are not allowed. This
means that all other interrupts must wait until the first is done. This is really bad
for the interrupt latency—the worst-case reaction time—on the other interrupts.
Thus it normally becomes even more important to defer most work until later. This
again is bad news for a purely interrupt-based system.
3. Both of the above scenarios may have the option for many inputs to trigger the
same interrupt. The first thing the ISR must do then is to find out which input
actually changed state. This is daisy-chaining interrupts. The order in which you
test for the various events becomes a “subpriority” so to speak.
From the above, it is clear that a purely interrupt-controlled system, with nothing de-
ferred to low-priority handlers, has some huge challenges.
A particularly nasty problem with interrupts I once experienced, is related to the dif-
ference between “edge triggered” and “level triggered” interrupts. If an interrupt is
level triggered, you will keep getting the interrupt until the level is changed, either by
the hardware itself or from code, typically in your ISR. An edge triggered interrupt, on
the other hand, only happens at the up- or down-going edge of the pulse. If your in-
terrupts are disabled in that short instant, you never get an interrupt, unless the edge
is latched in CPU-hardware, which is not done in all CPUs.
The general rule in any good system with interrupts is like guerrilla warfare:
“move fast in, do the job, and move fast out.” This is to achieve the best interrupt
latency for the other interrupts. This means that the actual ISR only does the minimal
stuff needed. This could, for example, be to set a flag or read a sample from an A/D
converter before it is overwritten by the next sample. In the latter case, the ISR will
save the sample in a buffer, which later will be read from a standard process or task.
In such a system, the interrupt latency must be less than 1/fs—where fs is the sample
frequency. A system like this can thus detect external events very fast, but it does not
offer any help to the developer in terms of multitasking (a concept we will see shortly).

Figure 2.3: Interrupt system with finite state machine in main.
If, however, the main loop is broken up into small pieces of code, organized with the
help of finite state machines, it is possible to react to the flags set in the ISR, as soon
as one of the small pieces of code is done, and then decide (via the FSM) what the next
piece of code is; see Figure 2.3.
This is exactly what ARM has done in their basic version of the free ‘‘mbed’’ OS.
Here, the flags from the ISRs are called events. ARM mbed prioritizes the interrupts
in the usual way and also offers priority on the “pseudo threads”—the small pieces
of code. This simply means that if “pseudo threads” A and B both are waiting for an
event from the same interrupt, the one with the highest priority is started first. Since
all of these “pseudo threads” are started on a specific point and run to end, there is no
preemption. One task in the application code never takes over the CPU from another;
it is only interrupted by ISRs and these can use the single CPU stack for the specific
registers they use. This saves a lot of RAM space, and is very practical in a small system.
Thus mbed is tailored, for example, for the small 32-bit Cortex M0 CPUs that have
scarce resources (including no MMU). What makes mbed interesting is that it has a lot
of the extras, normally seen on larger OS’es: TCP/IP stack, Bluetooth LE stack, etc. It
also boasts a HAL (hardware abstraction layer), making the code the same for other
CPUs in the ARM family. In this way, mbed is well positioned and does look very inter-
esting.
Note that ARM mbed alternatively can be configured to use a preemptive scheduler
as described in the next section. This takes up more space, but also makes mbed a
member of a more serious club.
2.4 A small real-time kernel
Typically, the aforementioned concepts are only used in very small and simple sys-
tems. It is really practical to separate the various tasks. A real-time kernel offers exactly
that—a task concept. You can also say that a kernel is all about managing resources.
The basic theory is that you set aside a task for each independent resource in the sys-
tem. This could be a printer, a keyboard, a hard-drive, or a production-line “station”

(or parts of this). It is not uncommon though, to have more tasks if this makes your
code more maintainable. It is, nevertheless, not a good idea simply to assign a task to
each developer, as this will require more coordination among tasks. The less coordi-
nation needed between tasks, the better. In fact, almost all the quirks that can make
the use of a kernel complex are related to interaction between tasks. Figure 2.4 shows
a system with a preemptive scheduler (we will get back to this shortly).
Figure 2.4: OS with preemptive scheduling.
The states used in Figure 2.4 are:
– Dormant
The task is not yet brought to life. This must be done explicitly by the application.
– Ready
The task can run, only it’s waiting for the current “running” task to “get off the
CPU.”
– Running
Actually executing code. There can only be one running task per CPU—or rather
per CPU core. Many modern CPU chips contain several cores. This is really like
several CPUs in one house. Intel’s hyperthreaded virtual cores also count here.
– Blocked
The task is waiting for something to happen. This could, for example, be a socket
in a recv() call, waiting for input data. When data arrives, the task becomes
“Ready.” A socket will also block if you write to it with send() and the assigned
OS transmit buffer is full.
Most kernels today support preemption. This means that application code in tasks is
not only interrupted by ISRs. When a task has run for an allowed time—the so-called
timeslice—the scheduler may stop it “in mid-air” and instead start another task. As
no one knows which registers the current or the next task needs, all registers must be
saved on a stack per task. This is the context switch. This is different from an interrupt

routine where you only need to save the registers used by the routine itself.3
A context
switch can even occur before the time-slice is used. If somehow a higher prioritized
task has become ready, the scheduler may move the currently running, low priority
task to ready (thus not executing anymore but still willing to do so) to make room for
running the high-priority task.
More advanced kernels support priority inversion. This is when a high priority task
is currently blocked, waiting for a low priority task to do something that will unblock
it. In this case, the low priority tasks “inherits” the priority of the waiting task until
this is unblocked.
Figure 2.5 shows our little system again—now full-blown with interrupts and
tasks.
Figure 2.5: Tasks and interrupts.
In some ways, the tasks are now simpler, as each task is blocked until awakened by
the OS, due to something that happened in the ISR. The figure shows how the three
ISRs respectively use an x, y, or z data-structure, and that the three tasks each wait
on one of these. There is not necessarily a 1:1 correspondence—all three tasks might,
for example, have waited on “y.” The nature of x, y, and z is different from OS to OS.
In Linux, the awaiting tasks calls wait_event_interruptible() while the ISR calls
wake_up_interruptible(). Linux uses wait queues so that several tasks may be awo-
ken by the same event. The term “interruptible” used in the call does not refer to the ex-
ternal interrupt, but to the fact that the call may also unblock on a “signal” such as, for
example, CTRL-C from the keyboard. If the call returns nonzero, this is what has hap-
pened. In a normal embedded system, you are not so interested in the use of signals.
Tasks can communicate to each other with these low-level mechanisms, as well
as semaphores, but often a better way is to use messages. C-structs or similar can be
mapped to such messages that are sent to a queue. The queue is sometimes referred
to as a mailbox.
3 A classic error is that the ISR programmer saves only the registers used. Then later adds code and
forgets to save the extra register(s) used.

A highly recommended design is one where all tasks are waiting at their own spe-
cific queue, and are awakened when there is “mail.” A task may have other queues
it waits on in specific cases, or it may block waiting for data in or out, but it usually
returns to the main queue for the next “job.” This is not much different from a man-
ager, mainly driven by mails in his outlook in-box. One specific advantage in using
messages is that some kernels seamlessly extend this to work between cores—a local
network so to speak. Another advantage is that it allows you to debug on a higher level
which we will see later. “Zero message queue” is a platform independent implemen-
tation supporting this.
A word of caution: do not try to repair deadlocks by changing priorities on tasks.
The simple rule on mutexes, semaphores, etc. is that they must always be acquired in
the same order by all tasks, and released in the opposite order. If one task “takes A
then B” and another “takes B then A,” then one day the two tasks will have taken re-
spectively A and B; both will block and never unblock. Even though the rule is simple,
it is not easy to abide to. Thus it is preferable to keep resource-sharing at a minimum.
This is especially true in systems with multiple cores that actually do execute in par-
allel. Coordinating resources in this scenario is inefficient, and copying data may be
preferred in many cases.
2.5 A nonpreemptive operating system
We typically talk about an operating system when there is a user interface as well as
a scheduler. Even though the OS in this way is “bigger” than a kernel, it may have
a less advanced scheduler. A very good example here is Windows 3, released in May
1990. This was a huge and very complex operating system with a lot of new and ex-
citing GUI stuff and tools. With Windows 3.1 (1992), we got “true-types” which was a
breakthrough in printing for most people.
However, from an RTOS (real time operating system) point-of-view, Windows 3
was not so advanced. Windows 3 did support interrupts, and it had a task concept,
but contrary to a small RTOS kernel, it did not support preemption. Thus Windows 3
was like the system in Figure 2.4 without the preemption action. Another thing missing
in Windows 3—which all good OSs have today—was the support for the MMU. This was
supported in the Intel 80386 CPU, which was the standard at the time, but the software
hadn’t caught up with the hardware.
Anyway, an input could generate an interrupt as seen before. Even though this
meant that a long-awaited resource was now ready, the scheduler could not move a
low-prioritized task away from the CPU to make way for the now ready high-priority
task. A task could not get to the CPU until the currently running task yielded. This is
not the same kind of “yield” as with co-routines. The Windows version is easy to im-
plement in C. The way Windows 3 yields is that it performs specific OS calls, typically

2.5 A nonpreemptive operating system | 19
sleep(). Sleep takes as input the minimum number of seconds (or microseconds) the
process wants to be taken of the CPU—thus making time for other tasks.
In the Windows 3 code, you would often see sleep(0). This meant that the task
could continue, but on the other hand was also prepared to leave the CPU at this point.
Moreover, Windows 3 introduced a variant of Berkeley Sockets called WinSock. As we
will see later, if you try to read data that hasn’t arrived yet from a socket, your task will
be blocked in a preemptive system.
In the days of Windows 3, this was standard in Unix, but Windows couldn’t han-
dle it—yet. Instead, Microsoft invented WinSock where a socket could tell you that
it “WOULDBLOCK” if only it could, so could you please write a kind of loop around it
with a sleep(), so that you do not continue before there is data or the socket is closed.
It would not be surprising, if this was the kind of behavior that made Linus Thor-
valds start writing Linux. The lack of preemption support was also one of the main
reasons why Microsoft developed Windows NT—which in newer versions is known as
Windows XP, Vista, 7, 8, or 10—“Windows” to all modern people. This is not just an
anecdote. You may still see kernels for very small systems that are nonpreemptive, like
ARM mbed in its simple version.
It is important that not only the OS or kernel supports preemption, but also that
the C-library code is supporting this well. There are two overlapping terms we need to
consider here:
– Reentrant
A reentrant function can be used recursively; in other words, by the same thread.
In order for this to happen, it must not use static data, instead it uses the stack.
A classic example of a non-reentrant C-function is strtok(), which tokenizes a
string very fast and efficiently, but keeps and modifies the original string while
parsing it into tokens. It is called again and again until the original string is com-
pletely parsed. Should you want to start parsing another string, before the first is
completely done, you will overwrite what’s left of the first original string.
– Thread-safe
A thread-safe function may be used from different threads of execution in parallel.
This is accomplished by the use of OS locks or critical sections. If for instance, the
function increments a 16-bit number in external memory, things can go wrong.
To increment the number, we need to read it into the CPU, add one and write it
back. If the CPU has a word size of 8-bits, it reads the low byte first, adds one to it,
and writes it back. If this operation meant a carry, the next byte is incremented in
the same way. Unfortunately, another thread, or interrupt service routine, might
read the two bytes before the high-byte is incremented. Even with a 16-bit word
size this can happen if data is not word aligned. In this case, the CPU needs to
read two 16-bit words. You want to make the whole operation “atomic.” This can
be done by disabling interrupts for the duration of the complete read-modify-write
operation, using relevant OS-macros. Many modern CPUs have specific assembler
instructions for doing such atomic functions. C11 and C11++ compliant compil-

ers can use these functions—either explicitly like std::atomic<>::fetch_add(),
or by declaring variables like std::atomic<unsigned> counter, and then simply
write counter++ in the code.
More complex scenarios, like, for example, engine control, may require several
operations to be performed without something coming in between them. In this
case, the OS-macros or manual interrupt disable/enable is needed.
The two terms are sometimes mixed up. What you need as an embedded programmer
is typically “the full Monty”: you need library functions to be thread-safe as well as
reentrant. Many kernels and operating systems supply two versions of their libraries,
one for multitasking (thread-safe), and one not. The reason for having the latter at all,
is that it is smaller and executes faster. It makes sense in nonpreemptive systems, or
systems with no OS at all, as we saw at the beginning of this chapter.
It should be noted that there are now modern nonblocking sockets. In order to cre-
ate servers with tens of thousands of connections, there are various solutions known
as asynchronous I/O, IO-port-completion, and similar terms, creating an FSM within
the OS—thus featuring an event-driven programming model. The basic IoT device will
however not serve many clients directly. Typically, there will only be one or two clients
in the cloud, with these servicing the many real clients. On top of this, we often see a
local Bluetooth or Wi-Fi control. For this reason, and because classic sockets are uni-
versally implemented, we are focusing on the classic socket paradigm. In any case,
the underlying TCP is the same.
2.6 Full OS
The small preemptive kernel gives you everything you need in order to multitask and
handle interrupts. Gradually, kernels have added file systems and TCP/IP stacks to
their repertoire. When a kernel comes with drivers for many different types of hard-
ware, as well as tools that the user may run on the commandline prompt, and typically
a Graphical User Interface (GUI)—then we have a full OS.
Today Linux is the best known and used full OS in the embedded world. Win-
dows also comes in versions targeting the embedded world, but somehow Microsoft
never really had its heart in it. Windows CE is dying out. Very few hardware vendors
are supporting it, thus there is not the wealth of drivers that we are accustomed to
with desktop Windows. The development environment, “Platform Builder,” can be
very disappointing if you are used to Visual Studio. Microsoft marketed first Windows
XP, later Windows 8 in a “fragmented” version for the embedded world, and is now
marketing Windows 10. However, the embedded world typically demands operating
systems that are maintained longer than it takes for Microsoft to declare something a
“legacy,” and Windows is difficult to shrink down to small systems. If you can use a
standard industrial PC with standard Windows in an application, then by all means

2.6 Full OS | 21
do it. You can take advantage of Visual Studio with C# and all its bells and whistles.
This is a fantastic and very productive environment.
Neither Linux, nor Windows are what can be called real-time systems (except Win-
dows CE). There are many definitions of the term “real-time,” but the most commonly
used is that there must be a deterministic, known, interrupt latency. In other words,
you need to know the worst-case time it takes from something in the hardware changes
state, until the relevant ISR is executing its first instruction. Both Linux and Windows
are designed for high throughput, not for deterministic interrupt latency. An example
of a true real-time OS (RTOS) is VxWorks from WindRiver.
If it’s not real-time, how come Linux is so popular in the embedded world? First
and foremost, it is popular for its availability of drivers and libraries for almost any-
thing. Your productivity as an embedded developer is much higher when you can draw
on this massive availability. Secondly, there’s the community. Should you get stuck,
there are many places to ask for help. In most cases, you only have to browse a little to
find a similar question with a good answer. Finally, we have the open source, which
we will discuss separately.
The fact remains that generally high throughput is actually very nice, and in real-
ity there are typically not many hard real-time demands in a system. Reading the A/D
converter sample before the next sample overwrites it is one such example. There are
several solutions to this problem:
– Apply a real-time patch to Linux. In this way, Linux becomes a real-time system,
but there is no such thing as a free lunch. In this case, the cost is that some stan-
dard drivers are not working any more. As this is one of the main reasons for choos-
ing Linux, it can be a high price.
– Add external hardware to handle the few hard real-time cases. This could, for ex-
ample, be an FPGA collecting 100 samples from the A/D. Theoretically, Linux still
might not make it, but in reality it’s not a problem with the right CPU.
– Add internal hardware. Today, we see ARM SoCs containing two cores: one with a
lot of horsepower, perfect for Linux, and another one that is small and well suited
for handling interrupts. As the latter does nothing else, it can work without an
OS, or with a very simple kernel. This CPU shares some memory space with the
bigger CPU, and can thus place data in buffers, ready for the bigger brother. The
DS-MDK environment from ARM/Keil actually supports such a concept, for devel-
opment hosted on Windows as well as Linux. In the simple example with an A/D
converter, many CPUs are however capable of buffering data directly from an I2
S
bus or similar.
Another problem with Linux is that it demands a memory management unit (MMU).
This is, in fact, a very nice component in the larger CPUs that cooperate with the OS.
It guarantees that one task cannot in any way mess up another task, or even read its
data. Actually, tasks in such a system are often called processes. A process is protected
from other processes by the MMU, but this also means that there is no simple sharing

of memory. When this is relevant, a process may spawn threads. Threads in the same
process-space can share memory, and thus are very much like tasks in a smaller sys-
tem without MMU. Processes are very relevant on a PC, and practical to have in an
embedded system, but if you want a really small system it won’t have an MMU. It is
possible to compile Linux to work without the MMU. Again, this may hurt software
compatibility.
There is a lesson to learn from Ethernet. Ethernet is not perfect and it cannot guar-
antee a deterministic delay like Firewire can. Still Firewire has lost the battle, while
Ethernet has survived since 1983 (i. e., at various speeds and topologies). The cheap
“good-enough” solution—in this case the nonreal-time Linux—wins over the expen-
sive perfect solution, if it can solve problems outside a small community.
2.7 Open source, GNU licensing, and Linux
It is a well-known fact that Linux brought the ways of Unix to the PC-world, mainly
servers, and now is taking over the embedded world. Interestingly, Unix clones are
old news to the embedded world. Many years before Linux was born, RTOSs such as
OS-9, SMX, QNX, and many others, used the Unix style. It was even standardized as
“POSIX.” The idea was to make it feasible to switch from one to another. So why is
Linux so successful? One explanation is the massive inertia it got via PC-hardware in
embedded systems. Another explanation is that it is open source.
Kernels and OSs come in two major flavors: open or closed source. If you come
from the Windows world you might wonder why so many embedded developers want
open source. Surely a lot of people believe in the cause, the concept of not monopo-
lizing knowledge. However, to many developers “open” literally means that you can
open the lid and look inside.
Here are some reasons why this is so important:
1. If you are debugging a problem, you may eventually end in the kernel/OS. If you
have the source, you may be able to find out what is wrong. Often times you may
be able to make a work-around in your own code. This would be pure guesswork
using closed source.
2. As above—but there is no possible work-around, you need to change the OS. With
open source you can actually do this. You should definitely try to get your change
into the official code base, so that the next update contains your fix. There is noth-
ing more frustrating than finding a bug, and then realize that you have found it
before. Also, the GNU Public License, GPL, requires you to make the improvement
public, so getting it back into the official kernel makes life easier, which is the
whole point.
3. As stated earlier, a lot of embedded codes lives for many years. It’s a pain if the OS
doesn’t, and if it is open source you can actually maintain it, even if no one else
does.

2.7 Open source, GNU licensing, and Linux | 23
If you come from the “small kernel” embedded world, you are probably used to com-
piling and linking one big “bin” or “exe” or similar. This keeps you in total control of
what the user has on his or her device. You may have noticed that embedded Linux
systems look much like a PC in the way that there are tons of files in a similar-looking
file-system. This is a consequence of the open source licensing concept. If you are a
commercial vendor, you charge for your system which includes a lot of open source
code besides your application. This is okay, as long as the parts originating from open
source are redistributed “as is.” This makes configuration control much more diffi-
cult, and you may want to create your own distribution, or “distro.” See Section 6.9 on
Yocto.
GNU means “GNU is Not Unix.” It was created in the US university environment
as a reaction to some lawsuits on the use of Unix. The purpose is to spread the code
without prohibiting commercial use. GNU is very focused on not being “fenced in”
by commercial interests. The basic GNU license allows you to use all the open source
programs. However, you cannot merge them into your source code, and not even link
to them without being affected by the “copy-left” rule which means that your program
source must then also be public.
Many sites claim that you are allowed to dynamically link without being affected
by the copy-left clause. However, a FAQ on gnu.org raises and answers the question:
Does the GPL have different requirements for statically versus dynamically linked mod-
ules with a covered work? No. Linking a GPL covered work statically or dynamically with
other modules is making a combined work based on the GPL covered work. Thus the
terms and conditions of the GNU general public license cover the whole combination.
This means that your code must call all this GPL code as executables. This is not
a “work-around” but the intended way. This fact is probably responsible for keeping
one of the really nice features from Unix: you can call programs on the command-line
and you can call them from your program or script—it works the same way. The Linux
philosophy states that programs should be “lean and mean” or in other words: do only
one thing, but do it good. This, together with the fact that most programs use files, or
rather stdin and stdout, allows you to really benefit from the GPL programs this way.
This is very different from Windows where commandline programs are rarely used
from within applications; see Section 4.4.
But if you are not allowed to link to anything, then what about libraries? It will
be impossible to create anything proprietary working with open source. This is where
the “Lesser GNU Public License” (LGPL) comes into the picture. The founders of GNU
realized it would inhibit the spread of the open source concept if this was not possible.
All the system libraries are under this license, allowing linking in any form. However, if
you statically link, you must distribute your object file (not the source), thus enabling
other people to update when newer libraries become available. This makes dynamic
linking the preferred choice.
The GNU org are very keen on not having too much code slip into the LGPL. There
is even a concept called “sanitized headers.” This is typically headers for LGPL li-

braries that are shaved down and pre-approved by GNU for use in proprietary code.
In order to use a library, you need the header files, and the fact that someone even
thinks that sanitizing these is needed, shows how serious the GPL is. The main rule is
to keep things completely separate—never start a proprietary code module based on
open source. There are alternatives to the GPL, such as FreeBSD and MIT licenses, aim-
ing to make it easier to make a living on products based on their code. Such libraries
may also be used from the proprietary code.
Still, Linux adheres to GNU. There is a much debated case on LKMs (loadable ker-
nel modules). As stated by the name, these are program parts, dynamically loaded
into the kernel. One vendor has made a proprietary LKM. I am not a lawyer, but I fail
to see how this cannot violate the GPL. The way I understand it, this has been ignored,
not accepted by the GNU community.
2.8 OS constructs
Table 2.2 presents a short list and explanation of some OS constructs.
Table 2.2: OS primitives for scheduling.
Concept Basic Usage
atomic A Linux macro assuring atomicity on variables, not normally atomic, for example,
a variable in external memory.
critical section Generally a block of code that must only be accessed by one thread at a time.
Typically protected by a mutex. Specifically, on Windows a critical section is a
special, effective mutex for threads in the same process.
event Overloaded term. Kernel-wise Windows uses events which other threads/processes
may wait on—blocking or not—in, for example, WaitForMultipleObjects().
semaphore Can handle access to n instances of a resource at the same time. The semaphore is
initialized to “n.” When a process or thread wishes to access the protected data,
the semaphore is decremented. If it becomes 0, the next requesting process/thread
is blocked. When releasing the resource, the semaphore is incremented.
lock A mutex can be said to implement a lock.
mutex Like a semaphore initialized to 1. However, only the owner of the “lock” can
“unlock.” The ownership facilitates priority inversion.
signal A Unix/Linux asynch event like CTRL-C or kill -9. A process can block until it is
ready, but it may also be “interrupted” in the current flow to run a signal handler.
Like interrupts, signals can be masked.
spinlock A low-level mutex in Linux that does not sleep, and thus can be used inside the
kernel. It is a busy-wait, effective for short waits. Used in multiprocessor systems to
avoid concurrent access.
queue High-level construct for message passing.

2.9 Further reading
– Andrew S. Tanenbaum: Modern Operating Systems
This is a kind of classic and very general in its description of operating systems.
The latest edition is the 4th.
– Jonathan Corbet and Alessandro Rubini: Linux Device Drivers
This is a central book on Linux drivers, and if you understand this, you understand
everything about critical sections, mutexes, etc. The latest edition is the 3rd.
– lxr.linux.no
This is a good place to start browsing the Linux source code. There are also git
archives at www.kernel.org, but lxr.linux.no is very easy to simply jump around
in. It is good when you just want to learn the workings of Linux, but is also good
to have in a separate window when you are debugging.
– Mark Russinovich et al.: Windows Internals Parts 1 and 2
To avoid it all being Linux, these books by the fabulous developers of “Sysinter-
nals” are included. This was originally a website with some fantastic tools that
were, and still are, extremely helpful for a Windows developer. These guys knew
more about Windows than Microsoft did, until they “merged” with Microsoft.
– Simon: An Embedded Software Primer
This book includes a small kernel called uC, and is using this for samples on how
to setup and call tasks, ISRs, etc. It includes a description of some specific low-
level HW circuits. The book uses Hungarian notation which can be practical in
samples, but not recommended in daily use.
– C. Hallinan: Embedded Linux Primer
This is a very thorough book on the Linux OS.
– Michael Kerrisk: The Linux Programming Interface
This is a huge reference book—not something you read from cover to cover. Never-
theless, once you start reading, you may end up having read more than planned—
simply because it is well written and filled with good examples.

3 Which CPU to use?
3.1 Overview
As stated in Chapter 2, the choice of CPU is very related to the choice of operating
system. As an embedded developer, you may not always get to choose which CPU
you will be working with. This may have been decided long ago in a previous project,
or by someone else in the organization. Still, understanding the various parameters
will help you get the best out of what you have, as well as make it easier for you to
drive the choice of the next CPU. It will also enhance your dialogue with the digital
designers.
Historically, the CPU was the basic chip executing code. Everything else was out-
side this in other chips. Then came the “microcontroller.” This was specifically made
for smaller embedded systems, containing a few built-in peripherals such as timers, a
little on-board RAM, interrupt-controller, and sometimes EPROM to store a program.
As integration increased, we ended with the modern SoC (system-on-chip). In such
chips, what used to be called the CPU is now known as the “core” (but is typically
much more powerful in itself than older CPUs) and the term “CPU” can mean anything
from the core to the full SoC chip.
An example of such a SoC chip is the Texas Instruments AM335x, where the “x”
refers to variations in clock cycles and on-board peripherals. This is also nicknamed
“Sitara” and is shown in Figure 3.1.
AM335x is particularly interesting as it is used in the BeagleBone Black Board.
This is a hobbyist/prototyping board resembling the Raspberry Pi. The BeagleBone is
referred to, in this book, in several contexts.1
Inside the AM335x is an ARM Cortex-A8,
and a lot of integrated peripherals. The core runs the actual program, dictating the
instruction set, while the other components decide the overall functionality and in-
terfaces. This chapter is dedicated to the full integrated chip concept which is referred
to as the CPU. The term “core” is used for the part executing the code.
Modern computerized devices are thus created like “Russian dolls”2
or “Chinese
boxes”; see Figure 3.2.
A companycreatesa productthat maybe based on a computer-boardfrom another
vendor. The vendor of this board has used an SoC like the AM335x from TI, and TI
has bought the IP (intellectual property) for the core from ARM. ARM delivers cores to
many modern designs from many vendors, but there are alternatives to ARM. Table 3.1
gives an overview of the major concepts discussed in this chapter.
In the following, we will look a little closer into each of the subjects in Table 3.1.
1 The BeagleBone is also a reference hardware solution for the Yocto project; see Section 6.9.
2 The correct name is Matryoshka dolls.

28 | 3 Which CPU to use?
Figure 3.1: AM335x—courtesy of Texas Instruments.

3.2 CPU core | 29
Figure 3.2: System design is like Russian dolls.
Table 3.1: Overview of concepts related to CPU-choice.
Concept Purpose (short)
CPU core Basic programmable unit
MMU support Needed for high-end OSs
DSP support Signal analysis
Power consumption Battery powered, heat generation
Peripheral units A/D, UART, MAC, USB, Wi-Fi…
Basic architecture For specific solutions
Built-in RAM For speed and simplicity
Built-in cache For speed
Built-in EEPROM or flash Field-upgradeable
Temperature range Environmental requirements
RoHS Compliance (typically) needed
JTAG debugger support HW debug without ICE
OSs supporting See Chapter 2
Upgrade path If you grow out of current solution
Second sources If the vendor stops or raises price
Price in quantity Basic but also complex
Evaluation boards Benchmark and/or prototype
Tool-chain Compilers, debuggers, etc.
3.2 CPU core
Though not everything is about ARM, it is very popular in modern embedded designs.
ARM has created three different “Cortex” (for “Core Technology”) “profiles”:

– A – for application
These are the biggest, highest performing, and most integrated designs with
“bells and whistles.” These CPUs can do hardware control, but are better suited
for number-crunching, including DSP and NEON media processing engines; see
Figure 3.1. They integrate a lot of the peripherals as we shall see in this chapter.
They are very well suited for running Linux or other high-end OSs. The Cortex-50
series even supports 64-bit and is used in some high-end mobile phones.
– R – for realtime
These favor engine control, robotics, etc. where it is important that latency is low
and security is high. They are a good choice for network routers, media players,
and similar devices that do not require the performance of the Cortex-As, but still
need data here and now.
– M – for microcontroller
These are classic microcontrollers for handling external hardware. They are of-
fered as soft cores for FPGA, but are also sold in numerous versions integrated
with memory and peripherals. They cannot run standard Linux due to the miss-
ing MMU.
It’s probably not a coincidence that the three types above spell “ARM.” Still, they do re-
flect some major segments within the embedded world, which other microcontrollers
and SoCs fit into as well. It is, however, not given which is most relevant in an IoT de-
sign, as IoT is a very broad term covering anything from intelligent light-bulbs (“M”
profile), over ATMs (“R” profile) to advanced image processing (“A” profile).
As discussed before: Linux is not really well suited for real-time, but is a great help
to get the most out of an advanced “application profile” CPU. In many ways, it is even
a waste to use such a device to control simple I/O. For these reasons, some vendors are
integrating, for example, both A- and M-profile CPUs in the same chip. This may allow
us to get “the best of both worlds.” The A-CPU can do advanced number-crunching on
Linux, and the M-CPU can focus on HW-interaction, and maybe do completely without
an OS as discussed in Section 2.1 and Section 2.3. The integration thus continues, now
toward multiple cores and/or DSPs.
3.3 CPU architecture
The best known CPU-architecture is “von Neumann.” Here, program and data are ac-
cessed by the same databus and addressed via the same addressbus. The major ad-
vantage is that the CPU has few pins and is easy to “design in.” Another advantage
is that it is (or rather may be) possible to update the program in the field. In a very
security-aware project, it may be preferable NOT to be able to change the instructions
that the device executes.
The alternative to von Neumann is “Harvard.” Here, data and program have sepa-
rate buses. This is particularly popular with DSPs. A digital signal processor executes

3.3 CPU architecture | 31
a lot of “multiply-accumulate” instructions, and typically (as we will see in Chapter 11)
the multiply operands are one of each:
– A constant—from the program.
– A data value, for example, from an A/D converter or previous calculations.
Thus the Harvard architecture allows the DSP and/or core to fetch constants and data
at the same time. In the ARM series, the v6 family is Von Neumann, used in, for exam-
ple, Cortex M0. The v7 family is Harvard and is used in the “application profile” CPUs
that mostly also have DSP extensions. The ARM Cortex-A8 was the first to utilize the
v7 family (yes—the names and numbers are not that easy to follow).
Another parameter on microprocessor architecture is whether it is CISC or RISC
based. CISC (complex instruction set computing) is “the classic way.” Here, a single as-
sembly instruction may perform a complex operation, using many cycles. If you want
to program in assembly-language, CISC has many instructions that resemble their C-
equivalent. With RISC (reduced instruction set computing), each assembly instruction
is simpler, executing much faster. However, the same complex operation demands
more instructions, making assembly coding harder while also taking up more space
for code, and using a little more performance on instruction decoding. With the in-
structions being simpler, it is possible for a clever programmer or compiler to do only
what is needed in the current context.
Little code is written in assembly language today. C and higher-level languages
are totally dominating, while CPUs are getting faster, memory cheaper, and compil-
ers better. Thus the important thing is “how fast does my C program run?” Typically,
RISC wins today. The main reason we still have a lot of CISC processors around, is the
fact that Intel 80x86 CPUs (or AMD lookalikes) are used in almost all modern desktop
computers, no matter whether they run Windows, Mac OS-X, or Linux.
An important architectural parameter for a CPU is the degree of “pipelining.”3
The execution of an instruction has several stages, for example, fetch the instruction,
decode it, fetch the operands, execute the instruction, and store the result. To utilize
the buses efficiently, the CPU may have several pipeline stages, handling the instruc-
tions in an overlapping fashion. This has inherent “hazards,” as one instruction may
change an operand that the next instruction has already fetched, with the old value.
This is yet a reason for using a compiler.
CPUs come as either big-endian or little-endian; see Figure 3.3.
If the ASCII string “Hello” starts at address 0x100, this is where the “H” is (assum-
ing C-style) and at address 0x101 we have “e” then “l,” etc. Similarly, a byte array is
also laid out sequentially. However, the bytes in the 32-bit dataword 0x12345678 can
be laid out in two major ways (see Figure 3.3):
3 A similar term is used in relation to TCP sockets.

Figure 3.3: Endian-ness.
1. Big-Endian
0x12 is placed at address 0x100, then 0x34 at address 0x101, etc. An argument for
this is that when data is viewed in a debugger as bytes, they are placed in the same
order as when viewed as 32-bit words. Thus it is relatively easy for a human to get
an overview of memory content. Motorola was on this side in the “endian-wars.”
2. Little-Endian
0x78 is placed at address 0x100, then 0x56 at address 0x101, etc. An argument for
this is that the most significant byte is at the highest address. Intel was on this side
with most of their CPUs—including 80x86—although 8051 actually is big-endian.
Big-endian is also defined as “network byte order,” meaning that it should be used for
network data interchanged between platforms. We will see this when setting up port
numbers (16-bit) and IPv4 addresses (32-bit), but it should also be used for the ac-
tual application data, though this rule is not followed by all protocols. When sending
application data between two PCs, which are both little-endian, there is quite a perfor-
mance overhead in changing to network order and back again. At least the “endian-
ness” of a protocol must be documented. Many modern CPUs can be configured to one
or the other.
Similar to byte ordering, bit ordering could be a problem, but we are normally
spared. All Ethernet PHYs transmit the LSB (least significant bit) first. Other serial
buses, like CAN, send the MSB (most significant bit) first, but since the PHYs are stan-
dardized and bought for the purpose, we do not really have to care, unless we are
debugging at the wire with an oscilloscope.
3.4 Word size
The word size of a CPU is defined as the width of its internal data bus. The address
bus may be the same size, or wider. There are also examples of variants with different

3.5 MMU-memory managed unit | 33
internal and external bus sizes. The best known example is the 16-bit 8086 that had
an external 16-bit data bus, but later came in an 8-bit external data bus variant called
8088, immortalized by the IBM PC XT.
64-bit CPUs have not really taken off in the embedded world. The question is
whether to use 8, 16, or 32 bits. Some may be surprised that 8 bits are still relevant.
The site “embedded.com” had some interesting discussions on this. On January 6,
2014, Bernard Cole takes up the thread from an earlier article by Jack Ganssle and
argues that the Intel 8051 in the world of IoT can be a big threat to modern ARM CPUs.
The basic advantage of 8-bit controllers is that they are very cheap. The underly-
ing patents have run out, and vendors can now make derivatives of the 8051 without
paying license fees to Intel, and there are definitely still many 8051-based designs on
the market. Universities and others have ported minimal TCP/IP stacks, even with IPv6
support, to 8051. Hardware vendors are still upgrading these designs with more built-
in peripherals while constantly speeding them up. This happens by executing instruc-
tions on fewer cycles as well as running at higher clock speeds. Finally, there is a large
base of developers that are very confident in, and efficient with, this architecture.
Consequently, if a design is extremely price sensitive, the 8-bit CPUs may be inter-
esting. If, on the other hand, a highly integrated and high-performing modern System-
on-Chip (SoC) solution is targeted, the 32-bit CPUs are definitely the place to look. It
is a fact, however, that only recently have the 32-bit CPUs overtaken the 8-bit CPUs as
the ones most sold.
That leaves the 16-bit CPUs in a kind of limbo and looking at Internet discussions,
it certainly does seem like 16-bit designs are not really worth discussing.
I am not sure that 16-bit CPUs can be written off. Some Vendors such as Texas Instru-
ments, Frescale/NXP and Microchip Technology supply to many designs, whereas we
don’t hear much about designs from Renesas (the result of a merge between Hitachi
and Mitsubishi). Nevertheless, this is one of the major vendors—offering many 16-bit
based CPUs. The explanation may be that Renesas is targeting the automotive in-
dustry where the number of units is high, but the number of integrators is relatively
low—causing less hype on the Internet.
3.5 MMU-memory managed unit
As described in Chapter 2, the MMU is a hardware entity in the CPU, isolating the OS
processes completely from each other. With this, it is impossible for one process to
overwrite data in another process. In fact, it cannot even read it. This also has some
overhead. If, for example, a process asks for a socket buffer to put data into, this buffer
must be cleared by the OS before it is handed to the process, as it might have been used
by another process earlier on, and the new process may not see these data.

Apart from these IT-security concerns, a MMU is our friend in catching bugs.
A common, and sometimes very nasty problem, is memory overwrite. This comes in
many flavors; some simply can write anywhere in memory, typically due to a “wild” or
noninitialized C pointer. If this happens in a system with a MMU, it typically triggers
an “exception.” If this is caught, it may be easy to find the culprit.
3.6 RAM
No real program can run without RAM. Naturally, if one can fit the RAM inside the CPU,
nothing is needed outside it. Also, accessing internal RAM is faster than accessing
external RAM.
A desktop computer may have Gigabytes of Dynamic RAM (DRAM). This cannot
possibly fit into the CPU. The advantage of Dynamic RAM is that it takes less space than
Static RAM (SRAM). It does however need a “DRAM controller,” constantly “visiting”
RAM cells to keep contents fresh. This is often built into the CPU. SRAM is typically
faster and more expensive than DRAM and does not need refreshing. Modern RAM is
known as SDRAM which is very confusing as we now have both the “S” and the “D,”
so which is it? It turns out that the “S” here is for “synchronous.” It is still dynamic and
the latest ones are called DDR with a generation number—DDR, DDR2, and DDR3, etc.
DDR means “Double Data Rate,” specifying that data is fetched on both the up- and
down-going flank of the clock.
Microcontrollers like Intel 8051 and the newer Atmel Atmega128A, have internal
SRAM. In some scenarios, this may be all that is needed. In other scenarios, we may,
for example, in the Atmel case, utilize the fact that various parts of the chip may be put
into sleep mode while the internal SRAM still works. The system stack and interrupt
vectors are kept here. It may even be possible to turn external RAM completely off in
sleep mode, but this requires the design to be able to restart almost from scratch.
3.7 Cache
Over the last many years, CPU performance has improved more than memory perfor-
mance. As described earlier, it is therefore common to have the hardware insert “wait-
states” when fetching data from external memory. To handle this growing gap, cache
was invented. This is intermediate memory, inside the CPU, which is a “shadow” of
the latest blocks of data fetched from the external memory. In a Von Neumann sys-
tem (see Section 3.3), a single cache is needed. In a Harvard system, a “split cache” is
needed if both program and data is cached.
If, in a system without cache, code is executing a fast “inner loop,” it is spending
many clock cycles waiting for the same parts of the program to be fetched from exter-
nal memory—with wait-states—again and again. If instead we have a cache big enough

3.8 EEPROM and flash | 35
to hold all the code in the inner loop, the code executes much faster. Cache may come
in several layers, and can be complex to set up. A common problem is that most mod-
ern CPUs are using “memory-mapped I/O.” When we read from this, we may read new
data at every read, unless it is cached, in which case the cache will keep supplying the
same first value. This is the reason why memory-mapped I/O must be declared non-
cacheable on a low level. It would seem that the C keyword “volatile” will help us, but
it won’t.
Many compilers offer the option to prioritize either speed or size, in the sense that
the compiler either maximizes the speed of the program or minimizes the number of
instructions. In a system with cache, it is recommended to prioritize size. This is due to
the fact that the fewer bytesa program takesup, the bigger is the chance that important
loops fit into the cache. This boosts speed much more than what any speed-optimizing
compiler can do. Not all modern CPUs have cache. This is a feature typically only in-
cluded in the larger CPUs, for example, the ARM Cortex M0, M1, M2, M3, and M4 do
not have internal cache.
3.8 EEPROM and flash
If the CPU has built-in flash memory, it can store its own program here. Unlike the RAM
case, we normally don’t see CPUs that have both internal and external flash. It’s either
one or the other. EEPROM (Electrically Erasable Programmable Memory) could also
be used for storing the program, but typically there is not room for a program here.
Instead EEPROM is mostly used for setup data. This could be user-changeable data as
an IP address, production data like a MAC address, or even something like “Number
of hours in service.” The latter is a challenge as the EEPROM only allows a limited
amount of writes.
3.9 FPU-floating point unit
There are many degrees of math assistance. For example, many newer derivatives of
the old Intel 8051 have added special multipliers to assist the small 8-bit core with
16-bit integer multiplications. Some have even added true IEEE 754 Floating-Point
units with single precision (8-bit exponent and 24-bit mantissa), or double precision
(11-bit exponent and 53-bit mantissa), respectively known as float and double in C.
When comparing FPUs, it is important to note how many cycles they need, in order
to do the different kinds of multiplications and divisions. An accurate comparison is
quite difficult as the various CPU structures don’t fetch their variables in the same
way or at the same speed. This is where the number of MFLOPS (million floating point
operations per second) becomes interesting.

3.10 DSP
When doing digital signal processing, the number of possible MACs (multiply and ac-
cumulate) per second is a vital parameter. This must be measured in the relevant accu-
racy (integer, single, or double). As we shall see in Chapter 11, a multiply-accumulate
with an output shifter is very important for filters implemented with integer arith-
metic. If an FFT is implemented, so-called bit-reversed addressing can save a lot of
cycles. Some of the high-performance CPUs include SIMD extensions. SIMD is single
instruction on multiple data. This means that exactly the same instruction can be per-
formed on many elements of an array at the same time. The ARM NEON has 32, 64-bit
wide registers that can be seen as an array. This is particularly relevant in image pro-
cessing. Note, however, that compilers do not support this directly.
3.11 Crypto engine
In Chapter 10, we dive into the huge subject of security. In order to, for example, par-
ticipate in https, the system must support SSL (secure sockets layer) or TLS (transport
layer security). It requires a lot of math to support encryption and decryption, etc. All
of this can be done in software, but it is faster and, interestingly, also safer to do it in
hardware.
As an example, the crypto engine in the TI AM335x boosts SSL performance a fac-
tor of two (blocksize 1 kB) to five (blocksize 8 kB). This engine can also work together
with Linux DM-Crypt to encrypt a hard disk. In this case, it boosts performance a fac-
tor two to three. The crypto-engine in TI AM335x only supports a subset of the TPM
specification,4
and, for example, asymmetric key algorithms5
are run in software.
3.12 Upgrade path
Most CPU cores are part of a family with many levels of performance, on-board pe-
ripherals and temperature variants. ARM is a special case, as the core CPU blueprint
is leased to many chip vendors, building on top of this. In this way, the number of ARM
derivatives is huge compared to any other embedded CPU. On the other hand, there
may be large differences between the various vendors versions. An upgrade path is
very important as systems tend to grow. If the bigger and better device is even plug-in
compatible with the old, then your day is made. Different manufacturers tend to grow
in different directions. Whereas one keeps pushing performance upwards, another is
pushing down the energy used per instruction, while introducing better sleep modes.
4 TPM is discussed in Section 10.17.
5 Asymmetric key encryption is explained in Section 10.7.

Random documents with unrelated
content Scribd suggests to you:

"I doubt it. But only Lovel can prove that, and he denies that he met
her on that night."
"Do you believe him?"
"No!" said Herne savagely. "I received a note in London which
advised me that they were going to meet."
"Who wrote the note?"
"I can't tell you yet. The person who wrote it wishes to remain
unknown for the present. But I believe that Lovel met Milly and killed
her because she would not marry him. Mind you," continued Herne,
energetically, "I have no proof of this; but I mean to obtain proof in
order to hang Lovel and save you."
"I'm afraid I'm past saving," sighed Lester. "Even Drek believes me
to be guilty, and, as I cannot recall the events of the night, I dare
not swear that I am innocent. Oh, God! that I should be in such a
position! ignorant of my own acts; and all on account of that
accursed drink! I am rightly punished for my vice."
Herne said nothing, for the present was no time for reproaches, but,
taking Lester by the arm, he led him into the room where the jury
were seated. Already the proceedings had begun, and the witnesses
summoned by Inspector Drek were giving evidence. Mr. Chaskin was
called first, and deposed that after evening service on Sunday he
had been summoned to a house on the other side of the common to
pray with a dying man. He returned to Barnstead by the short way
of the Winding Lane, and on entering the wood he had stumbled
over a body which was lying in the roadway near the stile. Thinking
that she had fainted--for by the touch of the garments and the faint
glimmer of the moonlight he perceived that the deceased was a
woman--he lighted a match to see who she was, and what was the
matter with her. Then he recognised the face of Millicent Lester, and
that she was dead. There was a wound in the back of the head. The
body was lying face downward, and he had to turn it over in order to

perceive the features. At once he went on to The Herne Arms and
roused up four or five men. These returned with him to the stile and
carried the body to the house of Dr. Lester, whence it was removed
subsequently to the inn for the inquest. Mr. Chaskin said he heard no
shot, and that he had seen no one about either on the common or in
the wood. It was about eleven, or a little after, when he discovered
the body. He had no idea as to who could have killed the deceased.
The next witness was Dr. Rollin, the rival to Lester in Barnstead, and
the medical man who had examined the body. He deposed that he
had made the examination on Monday morning. The deceased had
been shot from behind, and the bullet had passed right through the
brain. It had entered a little above the nape of the neck, and had
come out on one side of the nose. Death must have been
instantaneous. He examined the body at nine o'clock on Monday
morning; and from its condition he could state that death must have
taken place between eight and nine of the previous night; twelve
hours, more or less, elapsed, as he believed, between the death and
the examination.
Inspector Drek stated that he had been called to Barnstead from
Marborough by the information that Millicent Lester had been
murdered. He came at once to the house of the deceased. She had
died from the effects of a pistol shot, as Dr. Rollin had stated. He
had examined the spot where the body had been found, but could
discover no evidence there likely to lead to the identification of the
criminal. The pistol could not be found; and as the bullet had passed
right through the head of the deceased it could not be found either.
The spot where the body was discovered was of a deep-red clay,
somewhat softened by recent rain. There were many footmarks
about, but these were probably those of the bearers who had
brought home the body.
Iris Link, on being sworn, declared that the deceased had said
nothing to her about going to the Winding Lane on that night. She
(deceased) had left St. Dunstan's Church during the service and had

not been seen alive since leaving. Witness did not know why
deceased had left. She knew that the dead girl was in the habit of
meeting Mr. Lucas Lovel, but did not know for certain if she had met
him on that night. Still, she suspected, as deceased had not come
home that such a meeting might have taken place. The body of
deceased was brought home shortly after midnight on Sunday night.
She had no idea who had killed deceased, nor had any knowledge of
the motive for the crime.
Mr. Mexton watched the face and listened to the voice of Iris as she
made this last statement, for he recalled how she had asked him not
to seek for the assassin. For this reason he believed that she knew
who had killed Milly, and for some reason--of which he was naturally
ignorant--she desired to screen the guilty person. It struck him that
she might betray herself while under examination, but in this he was
wrong. Without a change of expression, in a firm voice she denied
all knowledge of the possible murderer. After this final assertion she
stepped down and gave place to Lucas Lovel.
This young man, who was pale but composed, stated that he had
not met Milly Lester on the fatal night. He had intended to do so, but
meeting with Gran Jimboy he had gone with her to her tent on the
other side of the common, and had not returned to The Herne Arms,
where he resided, till ten o'clock. He had walked over by the road,
and had not taken the short cut through the woods. He swore that
he had not been in the Winding Lane on Sunday night.
Gran Jimboy was summoned by Lovel to corroborate this evidence.
The old gipsy stated that she had met Lucas at eight o'clock,
immediately after service in St. Dunstan's Church, and had induced
him to come to her tent to hear some information which nearly
concerned him. The information was private, and had nothing to do
with the murder. Lovel, said the woman, had stayed with her till
nearly ten o'clock, and then had walked back to the village by the
high road. She knew this, as she had gone part of the way with him.

Thus, by the evidence of Gran Jimboy, an alibi in favour of Lovel was
clearly proved; and he was exonerated from any complicity in the
crime. Still, Herne did not believe the evidence, as Mexton could see
by the mocking smile on his lips. However, he made no attempt to
speak, and the proceedings continued.
Eliza, the servant of Dr. Lester, was the next witness, and she told
her story with shrill volubility. For the moment she was the most
important person in the room, as on her evidence was based the
charge which was known to be made against Dr. Lester. Eliza knew
that her master would be arrested on the statements she could
make against him, and relished the situation exceedingly. She had
no idea of the cruelty of her feelings towards the man whose bread
she had eaten.
Eliza stated, with many airs and graces, that she was the domestic
servant of Dr. Lester, and had been in this situation for some years.
Her master was in the habit of getting drunk two or three times in
the week; when in this condition, he always went about with a
loaded revolver, so that the inmates of the house were in peril of
their lives. Dr. Lester had been delighted by the engagement of the
deceased to Mr. Herne; and he was angry at the meeting of Miss
Lester with Mr. Lovel. Eliza knew that they met, as it was common
gossip. On the night of the murder Miss Lester and Miss Link went to
church, while the doctor remained at home drinking.
Miss Milly did not return; but Miss Iris did, in the company of Mrs.
Drass. When Mrs. Drass departed, Eliza heard high words between
the doctor and Miss Link relative to the meetings of the deceased
with Mr. Lovel. Afterwards Miss Iris went out to seek Miss Milly,
whom she thought was with Mr. Lovel; but Eliza did not know if this
were so. Dr. Lester continued drinking, and, fearing lest he should
cause trouble, witness watched the door of the consulting-room.
Shortly after half-past eight Dr. Lester came out, holding a pistol in
his hand; he was mad with drink, and cried out about his daughter
and Lovel. Then he rushed out, and witness thought he intended

murder. He did not come home till seven in the morning, and then
he had no pistol, but his clothes were daubed with red mud, such as
is found in the Winding Lane. He began to drink again; but before
doing so he changed his clothes. Witness swore that he went out
with the intention of killing his daughter, but she did not think he did
it deliberately, as he was mad with drink.
Dr. Lester was then called to refute this evidence if he was able. He
stated that he had gone out as Eliza described, and with a pistol. He
wished to kill Lovel, but he did not know what he said. He did not
remember what he did or where he went after leaving the house;
but he had an indistinct recollection of meeting someone--man or
woman he could not say. His pistol was gone when he returned
home at seven in the morning, but he did not know where he lost it.
Also his clothes were covered with red mud, so it was possible he
might have wandered into the Winding Lane, and have fallen in the
moist clay. But he recollected nothing. He had no intention of
harming his daughter, as he loved her too much.
Iris Link, recalled, said that Dr. Lester was in the habit of carrying
about a loaded revolver when drunk. She did not know if he took it
out on the Sunday night, but it was not in its case the next day. She
stated also that she heard a shot fired at nine o'clock, when she was
standing in the shadow of St. Dunstan's Church. She had never
heard Dr. Lester threaten his daughter; but he was certainly very
irate at the behaviour of Mr. Lovel.
This was all the evidence which had been collected by Drek; and it
certainly was against Lester. His own testimony rather inculpated
than exonerated him; and from the faces of the jury Paul saw that
they inclined to believe Lester guilty. Mexton himself could not make
up his mind; appearances pointed to the perpetration of the crime
by Dr. Lester when in a state of intoxication; but it was possible that
he might be innocent after all. Still, how his innocence was to be
proved it was difficult to say.

"Herne might do it," thought Paul, as he took down the evidence,
"for he seems to believe Lovel guilty, although Gran Jimboy's
statement goes to clear him. Also Herne knows something in
connection with that feather which he picked up. I wonder what that
odd-coloured feather has to do with the matter, and whether it could
prove the guilt of some person of whom at present we know
nothing?"
There was no answer to this question, and Herne made no sign of
making any statement about the feather in favour of Lester. He
stood quite still, and listened to the summing-up of the coroner--a
summing-up which was dead against Lester. The coroner declared
that Lester must have been in the Winding Lane on that night, else
he could not have got the red clay on his clothes. The question was
whether he was there after or before nine o'clock, the hour when,
according to Miss Link's evidence, the fatal shot was fired. The
coroner was inclined to think that Lester went straight from his own
house to the Winding Lane, knowing that was the spot in which his
daughter usually met Lovel; and there, finding his daughter waiting
for Lovel--who was then in the tent of Gran Jimboy--had fired and
killed her. Perhaps he would have killed Lovel had he been present,
but in his absence he vented his rage on the deceased. The crime,
however, was committed while Lester was drunk, and therefore he
was not responsible for his actions.
The result of this speech was that the jury--already prejudiced--
found Lester guilty; and immediately the wretched father was
arrested by Inspector Drek.

CHAPTER X.
THE PROPHECY AGAIN.
When the inquest was over, and Dr. Lester had departed for
Marborough gaol under the escort of Inspector Drek, the young
journalist remained standing thoughtfully in the square before the
inn. Nobody was surprised at the verdict, and everyone--as Paul
could hear asserted on all sides--believed that Dr. Lester had
murdered his own daughter while in a state of frenzy induced by
intoxication. But Mexton had his doubts about the matter, principally
on account of the words spoken by Iris when she wished him to
cease from searching for the assassin. He wished to question her as
to what she meant; and implore her, if she knew the truth, to reveal
it and save her unfortunate stepfather. While he was considering the
advisability of following Iris to Poverty Villa, he felt a touch on his
arm. It was Eliza, and her face was grave.
"I want to speak t' you, sir, if y' don't mind," she said quietly, with an
entire absence of her former self-importance; "but not here; I want
t' speak you--alone."
"Why? Is anything wrong?"
"I think so, Mr. Mexton--and with Miss Iris."
"Miss Iris?" repeated Paul, glancing round. "Where is she?"
"She's gone home. You follow her, sir, and ask her a question."

"What kind of question?" demanded Paul, startled by this hint.
Eliza drew Mexton to one side, until they were both out of earshot of
the scattered groups, and bent forward to whisper in his ear, "Ask
her why she went out after they brought home the corpse of Miss
Milly?" she said; and before Paul could make any comment on this
remark, she laid her finger on her mouth, and walked away.
At first Paul intended to follow her, and demand an explanation; but
on consideration he deemed it best to take her advice, and ask the
question directly of Iris herself. More would be learnt by thus going
to the fountain-head. Eliza evidently suspected something; and,
afraid to question Iris directly, had hinted her suspicions to Paul that
he might do so. With his usual promptitude Mexton sent over his
notes on the trial by special messenger to the editor of the "Tory
Times" at Marborough; and set forth at a brisk walk to Poverty Villa.
He believed firmly at the moment that the saving of Dr. Lester from
suffering unjustly lay in the hands of his step-daughter.
As he passed along the street towards the desolate house in which
the poor girl was waiting, he was surprised to meet with Herne, and
still more surprised when Herne stopped to speak; for the man was
not over-friendly towards him.
"What do you think of the verdict?" asked the squire abruptly.
"It seems just enough, going by the evidence," replied Mexton
cautiously.
"No doubt. This is one of those cases in which circumstantial
evidence accumulates to hang an innocent man."
"You believe Dr. Lester to be innocent?"
"I do--as surely as I believe Lovel to be guilty."

"My dear sir!" protested the journalist. "Lovel proved his innocence
by an alibi."
"No doubt; on the evidence of that old witch Mother Jimboy. Bah! a
made-up plot!"
"I don't think so, Herne. Why should Mother Jimboy assist Lovel?"
"Why?" repeated the squire--"because blood is thicker than water;
and, I told you the other day, Lovel has got gipsy blood in his veins."
"Who told you so?"
"The lady at whose name you blushed when I mentioned it in the
Winding Lane."
"Catinka?" said Paul, blushing again.
"Yes; Catinka, the violinist. Lovel knows her, and told her that his
mother was Romany, perhaps the daughter of Gran Jimboy--who
knows? That is why the old woman lied."
"Because Lovel is her grandson?"
"No, no; I am not sure of that; but because Lovel is a half-gipsy. But
in spite of the alibi I believe he is guilty. I'll prove his guilt and hang
him!"
"Why do you hate him so, Herne?"
"Because he led that poor girl to her death. I wished to save the soul
of Milly; but it is lost, and Lovel is the cause. Besides, I believe it is
my duty to succor the afflicted, and of the afflicted Dr. Lester is one.
An innocent man shall not die on the scaffold if I can help it. God
forbid! I'll save Lester, and hang Lovel. The end of this tragedy has
not yet come, Mexton."
"But if you----"

Herne waved his hand and interrupted Mexton.
"I can't waste any more time discussing the matter," he said,
retreating. "I'll see you again when I have proofs to hang Lovel."
After which speech he walked rapidly away, without the courtesy of
an adieu.
"Mad!" said Paul to himself, and resumed his interrupted journey
towards Poverty Villa. In his own heart the young man believed that
Herne was insane; his fanaticism in religion was a proof of an ill-
balanced mind; and now this furious hatred of Lovel--just enough, in
the face of Lovel's attentions to Milly in wilful disregard of the
engagement with Herne--threatened to rob him of all his self-
control. Failing to fasten the crime on Lovel, and it seemed
impossible to do so, Herne was quite capable of shooting the man in
a fit of rage. Knowing that Chaskin had most influence over Darcy,
the journalist determined to put him on his guard relative to the
squire's hatred of Lovel. But this warning word need not be spoken
immediately; and in the meantime Paul was anxious to see Iris.
The door of Poverty Villa was wide open; and the untidy house in its
neglected garden looked more desolate than ever. Lester was on his
way to Marborough gaol; Milly was lying in her coffin at The Herne
Arms; and Eliza had not yet returned. Therefore Paul knew that Iris
was alone in the house with a heavy burden of grief to bear. Slipping
lightly into the passage, he glanced through the open door of the
dining-room, but she was not there. The drawing-room was also
empty; so as a last resource he softly opened the door of the
consulting-room, and beheld the poor girl seated at the desk with
her head bowed on her folded arms. Sobs were shaking her frame,
and she looked as though the sorrows of the past week were
crushing her to the earth.
"Iris," he said softly, "my poor girl."

With an exclamation she lifted her head, and on seeing Paul rose to
her feet hastily, brushing away the tears from her face. Then, with a
little gasp, she moved forward with outstretched hands, to greet the
only friend who remained to her in the desolation of her life.
"Paul," she said with relief, "oh, my dear, I am glad to see you!"
He led her to a seat, and, taking a chair beside her, pressed her
hand warmly. "My dear Iris," said he, "at such a time you need the
services of your best friend. Let me be that friend."
"Thank you, Paul," she said faintly. "Oh, this horrible tragedy! Shall I
ever get it out of my head?"
"Time will bring comfort, Iris. In the meantime, let me ask what you
intend to do now? You cannot remain here."
"No; you are right there. Milly is dead; her father is in gaol on the
charge of having killed her, and I am alone in the world."
"Have you any money?"
"Not one penny. The last money I got from my step-father went to
pay last week's bills."
"Then you cannot remain here, as I said before."
"Where am I to go?" asked Iris helplessly.
"To Marborough--to my mother. She told me to ask you."
"How good and kind of her, Paul! I should like--but, oh!" she burst
out, "how can I go to Marborough to be pointed out as the relative
of a murderer?"
"Wait one moment before you call Dr. Lester by that name, Iris. Are
you sure that he is the murderer of Milly?"

"I don't know. I can't say. The verdict at the inquest----"
"Never mind the verdict at the inquest," interposed Paul quickly. "I
want to know what you think."
"Why do you want to know what I think?"
"Because I believe you can save an innocent man from being
hanged."
"I? No, no! I can do nothing!"
"Iris," said Mexton, taking her hand, "you asked me never to look for
the assassin of Milly. Did you do so to save Dr. Lester?"
"No. At that time I did not think that he would be accused."
"Then you suspect someone?"
"I--I have my suspicions," she said, in hesitating tones.
"What are they? To whom do they point?"
"I can't tell you. I am not certain. I may be deceived. Paul!" cried Iris
in desperation, "don't ask me. My answer may condemn an innocent
person!"
"Your silence acts in the same way, Iris. Dr. Lester is in danger of
death, and you know he is innocent."
"He is--he is! I don't believe that he killed Milly. But how should I
know the name of the real assassin?"
"Because you saw him on that night."
"I? I was not out on that night--at least, after the body was brought
home."

"Iris, why will you lie to me? Eliza saw you leave the house after
midnight."
"Eliza! Ah, that wretched girl has brought ruin on us all!"
"Not so--if I can save you. Tell me--did you go out?"
"Wait--wait! I'll answer in a moment. Give me time."
She rose to her feet, and, with clasped hands, walked twice or thrice
up and down the room. Evidently she was considering what to say,
and after some thought she faced round on Paul.
"I shall tell you," she said slowly, "but you will use the knowledge to
hunt down the assassin of Milly?"
"Assuredly! I wish to save Dr. Lester from suffering an unjust death."
"So do I, so do I! But, oh!"--she struck her hand together--"was ever
a woman placed in such a position? If I could only speak!"
"You must," said Paul determinedly, "or else have your step-father's
death at your door. Come, Iris, do you know the name of the
assassin?"
"No, but I suspect----"
"Suspect whom?"
"Lucas Lovel."
Mexton rose from his seat in astonishment. "Do you believe him
guilty, as Herne does?"
"Does Mr. Herne believe in his guilt?" asked Iris quickly.
"So thoroughly that he intends to bring Lovel to the scaffold."

"He will never succeed in doing so," cried the girl involuntarily.
"Why not?"
"He will not be able to obtain any evidence."
"I'm not so sure of that," said Mexton drily. "Herne is a fanatic; he is
clever; he is extremely pertinacious; and he hates Lovel like poison."
"For all that, I do not believe he will be able to accumulate sufficient
evidence to get Mr. Lovel arrested. Besides, he has a clever foe, who
will defend Mr. Lovel."
"A foe?" said Paul, puzzled--"and the name of the foe?"
"Mother Jimboy."
"What! that old fool! How can she defend Lovel?"
"She did so to-day by that alibi. She will do so again, you may be
sure."
"What reason have you to believe that Mrs. Jimboy is implicated in
the case!"
Iris thought for a moment. "On the day before Milly was killed," she
said, slowly, "she and Mr. Lovel met with Gran Jimboy, who
prophesied by palmistry that Milly would die a violent death."
"You don't say so! Go on."
"Well, Milly did perish by violence the next night. I truly believe that
she met Lovel in the Winding Lane, and that he killed her."
"Why should he kill her? He loved her."
"He did--so much that he killed her rather than that Mr. Herne
should marry her. I tell you, Paul, that Mr. Lucas is a man of violent

passions, and I believe he was egged on by Mother Jimboy to the
murder."
"Why should Mother Jimboy desire Milly's death?"
"I don't know; no more than I can guess why she provided that lying
alibi. I am sure that Lovel shot Milly, and then went across the
common to Mother Jimboy's tent so as to appear innocent."
"But why do you believe all this?"
"Because of the prophecy which was fulfilled; because of the
unexplained association of Mother Jimboy and Lovel, and because I
saw Lovel when I went out after midnight."
"You saw Lovel?" said Paul, incredulously.
"Yes; I fancied that Dr. Lester might have killed Milly; and to save
him I went to look for him. I could not find him on the fatal spot, but
there was a man there who ran away when he heard my approach. I
saw his face in the moonlight. He was Mr. Lovel."
CHAPTER XI.
BRENT SPEAKS OUT.
"It was Mr. Lovel," repeated Iris; "and if he was not concerned in the
murder, what was he doing at midnight on the very spot where it
occurred?"

"He may have been there after twelve o'clock," said Paul; "but to
inculpate him you must prove that he met Milly between eight and
nine."
"I can't prove it; no one can prove it."
"I am not sure of that," replied Paul, with sudden recollection; "there
is a man called Brent who was in the Winding Lane on that night,
and about that time. I'll see him."
Iris shook her head. "If Brent had known anything he would have
come forward at the inquest."
"No doubt--if he had not been bribed."
"What makes you think that Brent has been bribed?" asked Iris, in
surprise.
"I do not think so; but Herne insists upon it."
"Mr. Herne!" said Iris, in a low voice, and with a flush--"he believes
Lovel guilty also?"
"Yes--and without your grounds for belief. Also, he declares that
Lovel bribed Brent to hold his tongue."
"Does Mr. Herne think that Brent saw the murder committed?"
"Oh, no! but he thinks that Brent saw Lovel with Milly."
"I am certain Milly, poor girl, was with Lovel on that night, and I
believe he killed her."
There was a few minutes' silence, and then Paul turned quickly
towards Iris. "I want to ask you a rather rude question," said he,
awkwardly.
"What is it?"

"You won't be angry?"
"I am long past feeling anger after what I have gone through," said
Iris, sadly. "What is it you wish to know, friend?"
"You asked me not to search for the assassin of Milly; and now I find
that you believe the assassin to be Lovel. Are you in love with the
man, that you sought to screen him?"
"In love with Mr. Lovel!" cried Iris, indignantly. "Not I! I despise him
too much! A man who would act as he has done with Milly, knowing
that she was engaged to Herne, is not worthy of a woman's love!
No; I do not love, or even respect, Mr. Lovel."
"Then why do you seek to screen him?"
Iris rose to her feet with a cold look. "I cannot answer that question
now. I had my reasons for acting as I did."
"What do you mean?" asked Mexton, rising in his turn. "I don't
understand you."
"If I told you my reasons, you would understand still less," said Iris
bitterly. "I do not understand myself. But don't ask me any more
questions, Paul. I have told you all I know."
"All!" said Mexton, with emphasis, his eyes searching her face.
"All I can tell you now, at all events," she replied, obstinately.
After this last remark Mexton was satisfied that Iris, for reasons of
her own which he could not guess, had not confessed all she knew.
Yet as he was unaware of her motives for this reserve, he did not
think it wise to press his questions. Better, he thought, to accept her
refusal for the moment, and question her on some future occasion,
when she might be more inclined to take him into her confidence.
Moreover, by examining Brent, and forcing him into confession, he

might get at her knowledge without the necessity of procuring it
through herself. The matter thus settled in his own mind, Paul
discarded the subject of the murder, and addressed himself to the
question of Miss Link's position.
"You will accept my mother's offer, I suppose?" said he, quietly. "At
all events you will stay with her until after the trial of your
stepfather?"
Iris winced. "I do not care about facing Marborough gossip," she
said; "but I think it best to stay with Mrs. Mexton, as I am afraid to
remain here alone. I shall go over to Marborough by the six o'clock
coach. Eliza can stay here in charge of the house."
"Very good, Iris. I shall meet you at six o'clock at The Herne Arms
and take you over."
"And in the meantime--?"
"I intend to find out Brent, and force him to confess the truth."
This arrangement having been come to, Paul left Poverty Villa, and
went off in the direction of the village. On his way towards the
market-place, where he expected to find Brent--for it was market-
day in Barnstead, and the town was full of farmers and labourers--
Mexton remembered that the ploughman had confessed to being
with one Jane Bilway in the Winding Lane. If this were so, the
woman must have seen as much as the man; and if she had not
been bribed also, it was more likely that he would be able to extract
the truth from her. Mexton knew most people, high and low, in
Barnstead, amongst these Jane Bilway, who was a servant at The
Chequers, a little public-house on the outskirts of the village. Thither
he turned his steps to see what he could learn from the woman.
Jane was a broad, squat wench with a healthy red face and dull
eyes. She had about as much intelligence as a cow, and was only
useful in doing rough work and common drudgery. She was, at the

moment of Paul's arrival, cleaning the front windows of The
Chequers, and recognised him with a friendly grin. At once Mexton
began to ask her questions on the subject which was uppermost in
his mind.
"Jane," he said, quietly, "you are to marry Giles Brent, they say?"
"Yes, Mr. Mexton. We've bin keepin' company since Christmas."
"You see him occasionally?"
"Most ivery day. He comes here a lot; he's inside now, havin' a wet,"
said Jane, pointing to the window of the tap-room.
This was better news than Paul expected, for it gave him the chance
of an immediate conversation with Brent. But before entering the
public-house, he pursued his plan of gaining information from Jane.
"Were you walking with him on the night Miss Lester was killed?"
"I were," replied Miss Bilway, frankly. "We went to the Methody
Chapel together."
"Where did you meet him?"
"Just by the church, sir. We heard the shot fired when the bell was
ringing."
"But you were with him in the Winding Lane?"
Jane shook her head emphatically. "No, I wasn't, sir," she denied. "I
couldn't git away in time to go there. I wasn't in the lane on that
night."
"Oh!" Paul noted that Brent had been telling a lie. "You met Brent by
St. Dunstan's Church at nine o'clock, and went to the Methodist
Chapel?"

"Yes, I did. And I 'eard the shot fired, but I thought it was nothin',
though Giles he wanted to go back."
"You didn't see Miss Lester on that night?"
"No, sir; but I see Miss Iris, her sister, by the church at nine. She
must 'ave heard the shot, too."
"I daresay," replied Paul, with assumed carelessness. "Well, Jane,
here's a sovereign to buy yourself a wedding-present."
"Thank you, sir," said Jane, slipping the coin into her pocket. "I
wants all I can git, though to be sure Giles ain't badly off for money."
"Oh, he has money, has he?" said Mexton, recollecting Herne's idea
of the bribery; "a few shillings, no doubt?"
"A good few shillings, sir! Five pounds of 'em! We're goin' to spend
'em on the weddin'. Giles saved up the money from his wages. He's
a good fellow, is Giles, sir."
"I'm sure he is; I hope he'll make you a good husband."
"I'll see to that!" replied Miss Bilway, grimly, and she went on
cleaning the windows.
Paul laughed as he entered the tap-room, and thought of the
ingenious Mr. Brent's device for accounting for his possession of the
money. He was well known to be a thriftless wastrel, who spent
most of his earnings in strong ale; and was as likely to save five
pounds as he was to do an honest day's work. No one but simple
Jane Bilway, blinded by love, would have believed so improbable a
story. There was now no doubt in Paul's mind that the theory of
Herne was correct. Lovel and Milly had met in the Winding Lane
between eight and nine o'clock on the night of the murder, and had
been seen by Brent as he was on his way to meet Jane near the

church. Lest he should tell Herne of the meeting Lovel had bribed
him with the five pounds.
"Though it is a large sum for a man like Lovel to give," thought Paul;
"he is not well off, and would not part with so much money unless
he was forced to. I hope the five pounds was not given to conceal a
worse affair than a simple meeting. However, I'll play a game of bluff
with Brent, and wring the truth, whatever it may be, out of him."
Brent, who was a huge, bull-headed fellow with a sulky face, sat
alone in the tap-room with a mug of ale before him. He touched his
hat to Paul, whom he recognised, and looked puzzled for the
moment at the sight of a gentleman in a low-class public-house,
which was usually patronised by himself and those of his class.
"Well, Brent," said Paul, in a cheerful voice, "how are you? All right--
eh? I have just come to have a few moments of conversation with
you."
Brent took his pipe from his lips, and gave a sulky growl. "What
about, sir?"
"I'll tell you in good time," replied Paul, taking a chair, and selecting
a cigarette from his case. "In the meantime, I am thirsty, and wish
to drink. You'll have some ale with me?"
"I'd 'ave ale wi' anyone," said Brent, suspiciously; "but I don't know,
sir, what the likes o' you wants with the likes o' me."
"We'll come to that soon," said Mexton, and hammered on the table.
"Two tankards of bitter," he added to the slip-slop landlady, who
entered with a deferential smile.
The liquor was soon brought, and after a deep draught Paul lighted
his cigarette, and looked closely at the ploughman. Brent took a
drink also, and tried to appear at ease, although he was visibly
disturbed by the scrutiny of his visitor. Having reduced him to a

doubtful frame of mind, Mexton addressed himself to the matter in
hand.
He knew the manner of the man he had to do with, and that it
would not be an easy matter to extract information from such a
sulky brute. Threats also would avail little, as Brent was one of these
pig-headed men, who begin by denying, and go on doing so in the
face of the clearest evidence with incredible obstinacy. The sole
chance of getting at the truth was to assume that Lovel had
confessed the bribery to him--that is, to Paul Mexton--and had sent
him on an errand connected therewith to Brent. This attitude
necessitated the telling of a few lies; but Mexton was quite prepared
to tell them. He was cool-headed and pertinacious, and not the man
to stick at a trifle for the gaining of his own ends.
"I have come to you from Mr. Lovel," said Paul, slowly.
Brent's jaw dropped. "What's the likes of him want with the likes of
me?" he said.
"A little decency, in the first place," replied Mexton. "You promised to
hold your tongue about the meeting of Mr. Lovel and Miss Lester on
the night of the murder."
"How d'ye know they met?" asked Brent, with dogged suspicion.
"Mr. Lovel told me. Do you think I would know if he had not?--or
that I would be aware that he paid you five pounds to hold your
tongue?"
Brent, whose brain worked slowly, fell into the trap at once. Unless
Lovel had spoken, as Mexton declared, he did not think Paul could
have come by such exact information; the more particularly as the
precise amount of the bribe was mentioned. It never occurred to
Brent at the moment that Jane had innocently betrayed him.
"Well, I've earned the money all right, ain't I?" he growled.

Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.
More than just a book-buying platform, we strive to be a bridge
connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.
Join us on a journey of knowledge exploration, passion nurturing, and
personal growth every day!
ebookbell.com

Embedded Software For The Iot 3rd Edition Klaus Elk

More Related Content

Similar to Embedded Software For The Iot 3rd Edition Klaus Elk (20)

Recently uploaded (20)

Embedded Software For The Iot 3rd Edition Klaus Elk