![Finns—further, the “Gautigoth,” generally taken for the West Göter,
who were a specially “brave and warlike people,” the “Ostrogothæ”
[East Göter] and many more.
Then he crosses the Norwegian frontier and mentions
“The ‘Raumarici’[of Romerike] and ‘Ragnaricii’ [of Ranrike or
Bohuslen], the very mild [peaceful] ‘Finns’ (‘Finni mitissimi’), who are
milder than all the other inhabitants of Scandza;[136] further their
equals the ‘Vinoviloth’; the ‘Suetidi’ are known among this people
[‘hac gente’ must doubtless mean the Scandinavians] as towering
above the rest in bodily height, and yet the ‘Danes,’ who are
descended from this very race [i.e., the Scandinavians ?] drove out
the ‘Heruli’ from their own home, who claimed the greatest fame
[i.e., of being the foremost] among the peoples [‘nationes’] of
Scandia for very great bodily size. Yet of the same height as these
are also the ‘Granii’ [of Grenland, the coast-land of Bratsberg and
Nedenes], the ‘Augandzi’ [people of Agder],[137] ‘Eunix’ [islanders,
Holmryger in the islands ?], ‘Ætelrugi’ [Ryger on the mainland in
Ryfylke], ‘Arochi’ [== ‘arothi,’ i.e., Harudes, Horder of Hordaland],
‘Ranii’ [in other MSS. ‘Rannii’ or ‘Rami,’ Sophus Bugge (1907) and A.
Bugge see in this a corruption of ‘*Raumi,’ that is, people of
Romsdal], over whom not many years ago Roduulf was king, who,
despising his own kingdom, hastened to the arms of Theodoric king
of the Goths, and found what he had hankered after. These people
fight with the savageness of beasts, more mighty than the Germans
in body and soul.”
The small (?), “very mild” Finns must, from the order in which they
are named, have lived in the forest districts—Solör, Eidskogen, and
perhaps farther south—on the Swedish border. P. A. Munch [1852, p.
83] saw in their kinsmen the “Vinoviloth” the inhabitants of
“Vingulmark” (properly “vingel-skog,” thick, impenetrable forest),
which was the forest country on Christiania fjord from Glommen to
Lier. Müllenhoff agrees with this [ii., 1887, pp. 65 f.], but thinks that
“-oth,” the last part of the word, belongs to the next name, Suetidi,
and that “Vinovil” may be a corruption of Vingvili or Vinguli (cf.](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-56-320.jpg)
![Paulus Warnefridi’s “Vinili” ?). But however this may be, we must
regard this people and the foregoing as “Finnish” and as inhabiting
forest districts, as hunters, as well as a third Finnish people,
“Finnaithæ” in Småland. We shall return later to these “Finns” in
Scandinavia. It has been thought that “Suetidi” may be from the
same word as “Sviþjoð”; but as Jordanes has already mentioned the
Svear (“Suehans”), and as the name occurs among the Norwegian
tribes, and there is evidently a certain order in their enumeration,
Müllenhoff may be right in seeing in it a corruption of a Norwegian
tribal name. He thinks that “Othsuetidi” may be a corruption of
“Æthsævii,” i.e., “Eiðsivar” (cf. Eidsivathing), “Heiðsævir” or “Heiðnir”
in Hedemarken, who were certainly a very tall people. The mention
of the Norwegian warriors has a certain interest in that it is due to
the Roman statesman Cassiodorus (or his authority), who glorified
the Goths and had no special reason for praising the Northmen.[138]
It shows that even at that time our northern ancestors were famed
for courage and bodily size, and that too above all other Germanic
peoples, who were highly esteemed by the Romans. It is not clear
whether Rodulf was King of the “Ranii” (Raumer ?) alone, or of all
the Norwegian tribes from Grenland to Romsdal. It may be supposed
that he was a Norwegian chief who migrated south through Europe
at the head of a band of warriors, composed of men from the tribes
mentioned, and that finally on the Danube, hard pressed by other
warlike people, he sought alliance and support from the mighty king
of the Goths, Theodoric or Tjodrik (Dietrich of Berne). This may have
been just before 489, when the latter made his expedition to Italy.
Many circumstances combine to make such a hypothesis probable.
[139]
We know that about 489 the Eruli were just north of the Danube,
and were the Goths’ nearest neighbours. Now, as we shall see later,
Eruli was perhaps at first a common name for bands of northern
warriors, and these Eruli on the Danube may therefore certainly
have consisted to a greater or less extent of Norwegians. We know,
further, that at this time there was a king of the Eruli to whom
Theodoric sent as a gift a horse, sword and shield, thereby making](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-57-320.jpg)
![him his foster-son [cf. Cassiodorus, Varia iii. 3, iv. 2]. Finally, we
know from Procopius that the Eruli just at this time had a king,
Rodulf, who fell in battle against the Langobards (about 493). When
we compare this with what Jordanes says about the Norwegian king
Rodulf, who hastened to Theodoric’s arms and found there what he
sought, it will be easy to conclude that this Norwegian chief is the
same as the chief of Eruli here spoken of. Rodulf, or “Hrodulfr,” is a
known Norwegian name. “Rod-,” or “Hrod,” is the same as the
modern Norwegian “ros” (i.e., praise), and means probably here
renowned.
One is further inclined to believe that it was from this Rodulf or his
men, of whom some may have come from And in Hålogaland, that
Cassiodorus or his authority obtained the information about
Scandinavia and northern Norway, which is also partly repeated in
Procopius.
Sophus Bugge [cf. 1910, pp. 87 ff.; see also A. Bugge, 1906, pp. 35
f.] has suggested that the “Ráðulfr,” who is mentioned in the runic
inscription on the celebrated Rök-stone in Östergötland (of about the
year 900), in which Theodoric (“Þiaurikr”) is also mentioned, may be
the same Norwegian chief Rodulf who came to Theodoric and who
fell in battle with the Langobards. He even regards it as possible that
it is an echo of this battle which is found in the inscription, where it
is said that “twenty kings lie slain on the field”; in that case the
battle has been moved north from the Danube to “Siulunt” (i.e.,
Sealand). There are other circumstances which agree with this: it is
said of the Eruli that they had peace for three years before the battle
[cf. Procopius]; on the Rök-stone it is stated that the twenty kings
stayed in Siulunt four winters; the latter must have been Norwegian
warriors of different tribes: Ryger, Horder, and Heiner (from
Hedemarken), perhaps under a paramount king Ráðulfr, who settled
in Sealand—while the Eruli were bands of northern warriors, who
under a king Rodulf had established themselves on the north bank of
the Danube. Bugge’s supposition may be uncertain, but if it be
correct it greatly strengthens the view (see p. 145) that the Eruli](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-58-320.jpg)
![were largely Norwegian warriors, since in that case the king of the
Eruli, Rodulf (== Ráðulfr), would have been in command of tribes
for the most part Norwegian: Ryger, Horder, and Heiner.
Procopius,
circa 552
A.D.
The Byzantine historian Procopius, of Cæsarea (ob. after 562),
became in 527 legal assistant, “assessor,” to the general Belisarius,
and accompanied him on his campaigns until 549, amongst others
that against the Goths in Italy. In his work (in Greek) on the war
against the Goths (“De bello Gothico,” t. ii. c. 14 and 15), written
about 552, he gives information about the North which is of great
interest. He tells us of the warlike Germanic people, the Eruli, who
from old time[140] were said to have lived on the north bank of the
Danube, and who, with no better reason than that they had lived in
peace for three whole years and were tired of it, attacked their
neighbours the Langobards, but suffered a decisive defeat, and their
king, Rodulf, fell in the battle (about 493).[141]
“They then hastily left their dwelling-places, and set out with their
women and children to wander through the whole country
[Hungary] which lies north of the Danube. When they came to the
district where the Rogians had formerly dwelt, who had joined the
army of the Goths and gone into Italy, they settled there; but as
they were oppressed by famine in that district, which had been laid
waste, they soon afterwards departed from it, and came near to the
country of the Gepidæ [Siebenbürgen]. The Gepidæ allowed them to
establish themselves and to become their neighbours, but began
thereupon, without the slightest cause, to commit the most revolting
acts against them, ravishing their women, robbing them of cattle
and other goods, and omitting no kind of injustice, and finally began
an unjust war against them.” The Eruli then crossed the Danube to
Illyria and settled somewhere about what is now Servia under the
eastern emperor Anastasius (491-518). Some of the Eruli would not
“cross the Danube, but decided to establish themselves in the
uttermost ends of the inhabited world. Many chieftains of royal blood](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-59-320.jpg)
![now undertaking their leadership, they passed through all the tribes
of the Slavs one after another, went thence through a wide,
uninhabited country, and came to the so-called Varn. Beyond them
they passed by the tribes of the Danes [in Jutland], without the
barbarians there using violence towards them. When they thence
came to the ocean [about the year 512] they took ship, and landed
on the island of Thule [i.e., Scandinavia] and remained there. But
Thule is beyond comparison the largest of all islands; for it is more
than ten times as large as Britain. But it lies very far therefrom
northwards. On this island the land is for the most part uninhabited.
But in the inhabited regions there are thirteen populous tribes, each
with a king. Every year an extraordinary thing takes place; for the
sun, about the time of the summer solstice, does not set at all for
forty days, but for the whole of this time remains uninterruptedly
visible above the earth. No less than six months later, about the
winter solstice, for forty days the sun is nowhere to be seen on this
island; but continual night is spread over it, and therefore for the
whole of that time the people are very depressed, since they can
hold no intercourse. It is true that I have not succeeded, much as I
should have wished it, in reaching this island and witnessing what is
here spoken of; but from those who have come thence to us I have
collected information of how they are able [to count the days] when
the sun neither rises nor sets at the times referred to,” etc. When,
during the forty days that it is above the horizon, the sun in its daily
course returns “to that place where the inhabitants first saw it rise,
then according to their reckoning a day and a night have passed. But
when the period of night commences, they find a measure by
observation of the moon’s path, according to which they reckon the
number of days. But when thirty-five days of the long night are
passed, certain people are sent up to the tops of mountains, as is
the custom with them, and when from thence they can see some
appearance of the sun, they send word to the inhabitants below that
in five days the sun will shine upon them. And the latter assemble
and celebrate, in the dark it is true, the feast of the glad tidings.
Among the people of Thule this is the greatest of all their festivals. I
believe that these islanders, although the same thing happens every](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-60-320.jpg)
![year with them, nevertheless are in a state of fear lest some time
the sun should be wholly lost to them.
“Among the barbarians inhabiting Thule, one people, who are called
Skridfinns [Scrithifini], live after the manner of beasts. They do not
wear clothes [i.e., of cloth] nor, when they walk, do they fasten
anything under their feet, [i.e., they do not wear shoes], they
neither drink wine nor eat anything from the land, because they
neither cultivate the land themselves nor do the women provide
them with anything from tilling it, but the men as well as the women
occupy themselves solely and continually in hunting; for the
extraordinarily great forests and mountains which rise in their
country give them vast quantities of game and other beasts. They
always eat the flesh of the animals they hunt and wear their skins,
and they have no linen or anything else that they can sew with. But
they fasten the skins together with the sinews of beasts, and thus
cover their whole bodies. The children even are not brought up
among them as with other peoples; for the Skridfinns’ children do
not take women’s milk, nor do they touch their mothers’ breasts, but
they are nourished solely with the marrow of slain beasts. As soon
therefore as a woman has given birth, she winds the child in a skin,
hangs it up in a tree, puts marrow into its mouth, and goes off
hunting; for they follow this occupation in common with the men.
Thus is the mode of life of these barbarians arranged.
“Nearly all of the remaining inhabitants of Thule do not, however,
differ much from other peoples. They worship a number of gods and
higher powers in the heavens, the air, the earth and the sea, also
certain other higher beings which are thought to dwell in the waters
of springs and rivers. But they always slay all kinds of sacrifice and
offer dead sacrifices. And to them the best of all sacrifices is the
man they have taken prisoner by their arms. Him they sacrifice to
the god of war, because they consider him to be the greatest. But
they do not sacrifice him merely by using fire at the sacrifice; they
also hang him up in a tree, or throw him among thorns, and slay
him by other cruel modes of death. Such is the life of the inhabitants](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-61-320.jpg)
![of Thule, among whom the most numerous people are the Gauti
(Göter), with whom the immigrant Eruli settled.”
Erulian
sources of
Procopius
Common
source of
Procopius
and
Jordanes
This description by Procopius of Thule (Scandinavia) and its people
bears the stamp of a certain trustworthiness. If we ask whence he
has derived his information, our thoughts are led at once to the
Eruli, referred to by him in such detail, who in part were still the
allies of the Eastern Empire, and of whom the emperor at Byzantium
had a bodyguard in the sixth century. There were many of them in
the army of the Eastern Empire both in Persia and in Italy; thus
Procopius says that there were two thousand of them in the army
under the eunuch Narses, which came to Italy to join Belisarius.
Procopius thus had ample opportunity for obtaining first-hand
information from these northern warriors, and his account of them
shows that the Eruli south of the Danube kept up communication
with their kinsmen in Scandinavia, for when they had killed their king
“Ochon” without cause, since they wished to try being without a
king, and had repented the experiment, they sent some of their
foremost men to Thule to find a new king of the royal blood. They
chose one and returned with him; but he died on the way when they
had almost reached home, and they therefore turned again and
went once more to Thule. This time they found another, “by name
‘Datios’ [or ‘Todasios’ == Tjodrik ?]. He was accompanied by his
brother ‘Aordos’ [== Vard ?] and two hundred young men of the
Eruli in Thule.” Meanwhile, as they were so long absent, the Eruli of
Singidunum (the modern Belgrade) had sent an embassy to the
emperor Justinianus at Byzantium asking him to give them a chief.
He sent, therefore, the Erulian “Svartuas” (== Svartugle, i.e., black
owl ?), who had been living with him for a long time. But when](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-62-320.jpg)
![to the west coast of Norway; there had therefore been an ancient
connection, and perhaps, indeed, Horder from Norway and Harudes
from Jutland may have been among Rodulf’s men, and there may
also have been Harudes among the Eruli whom the Danes, according
to Jordanes, drove out of their home (in Jutland ?). There was also,
from the very beginning of Norwegian history, much connection
between Norway and Jutland.
Another disagreement between the descriptions of Procopius and
Jordanes is that according to the former there were thirteen tribes,
each with a king, in Thule, while Jordanes enumerates twice as
many tribal names in Scandza, but of these perhaps several may
have belonged to the same kingdom.[142]
A remarkable similarity between the two authors is the summer day
forty days long and the equally long winter night among the people
of Thule as with the Adogit, and the fact that in immediate
connection therewith the Scrithifini and Screrefennæ, which must
originally be the same name, are mentioned. The description in
Procopius of festivals on the reappearance of the sun, etc., points
certainly to information from the North; but, as already pointed out,
the statement in this form, that the summer day was of the same
length as the winter night, cannot be due to the Norsemen
themselves; it is a literary invention, which points to a common
literary origin; for it would be more than remarkable if it had arisen
independently both with the authority of Procopius and with that of
Jordanes. An even more striking indication in the same direction is
the resemblance which we find in the order of the two descriptions
of Thule and of Scandza. First comes the geographical description of
the island, which in both is of very great size and lies far out in the
northern ocean; then occurs the statement that in this great island
are many tribes.[143] Next we have in both the curious fact that the
summer day and the winter night both last for forty days. Then
follows in both a more detailed statement of how the long summer
day and winter night come about, and of how the sun behaves
during its course, etc. Immediately after this comes the description](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-64-320.jpg)
![in the eighth century B.C. made an inroad into Asia Minor. Six
hundred years later, in the second century B.C., bands of Cimbri and
Teutones came down from northern Europe and were pressing
towards Rome, till they were defeated by Marius and gradually
disappeared. Five hundred years later still, in the third to the fifth
centuries A.D., the Eruli come on the scene, and after they have
disappeared come the Saxons and Danes, and then the Normans.
We may perhaps suppose, to a certain extent at all events, that the
races which formed these restless and adventurous bands were in
part the same, and that it is the names that have changed. The Eruli
are also mentioned by Jordanes and by many other authorities
besides Procopius. Together with the Goths they played a part in the
“Scythian” war in the third century, but afterwards disappear to the
north of the Black Sea. They must have been the most migratory
people of their time; we find them roaming over the whole of
Europe, from Scandinavia on the north to Byzantium on the south,
from the Black Sea on the east to Spain on the west; from the third
to the fifth century we find Eruli from Scandinavia as pirates on the
coasts of western Europe, and even in the Mediterranean itself,
where in 455 they reached Lucca in Italy [cf. Zeuss, 1837, p. 477 f.;
Müllenhoff, 1889, p. 19]. When we read in Procopius that some of
the Eruli would not “cross the Danube, but determined to establish
themselves in the uttermost ends of the world,” this means, of
course, that they had come from thence, and that rather than be
subject to the Eastern Empire they would return home to
Scandinavia. The name also frequently appears in its primitive Norse
form, “erilar,“ in Northern runic inscriptions.[144] Since ”erilar” (in
Norwegian “jarl,” in English “earl”) means leader in war, and is not
known in Scandinavia as the original name of a tribe which has given
its name to any district in the North, we must suppose that it was
more probably an appellative in use in the more southern parts of
Europe for bands of northern warriors of one or more Scandinavian
tribes [cf. P. A. Munch, 1852, p. 53]. They may have called
themselves so; it was, in fact, characteristic of the Scandinavian
warrior that he was not disposed to acknowledge any superior; they
were all free men and chiefs in contradistinction to thralls. Gradually](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-66-320.jpg)
![these bands in foreign countries may have coalesced into one nation
[cf. A. Bugge, 1906, p. 32]. But as expeditions of Eruli are spoken of
in such widely different parts of Europe, the name must, up to the
end of the fifth century, have often been used for Norsemen in
general, to distinguish them from the nations of Germany, like the
designation Normans, and sometimes also Danes, in later times.
That the latter was used as an appellative as early as the time of
Procopius seems to result from his mentioning the tribes (“ethne”) of
the Danes in just the same way as he speaks of those of the Slavs.
What is said about the Eruli suits the Scandinavians: they were very
tall (cf. Jordanes, above, p. 136) and fair, were specially famed for
their activity, and were lightly armed; they went into battle without
helmet or coat of mail, protected only by a shield and a thick tunic,
which they tucked up into a belt. Their thralls, indeed, had to fight
without shields; but when they had shown their courage they were
allowed to carry a shield [Procopius, De bello Pers., ii. 25]. “At that
time,” says Jordanes, “there was no nation that had not chosen the
light-armed men of its army from among them. But if their activity
had often helped them in other wars, they were vanquished by the
slow steadiness of the Goths,” and they had to submit to
Hermanaric, King of the Goths by the Black Sea, the same who is
called Jörmunrek in the Völsunga Saga. The people here described
can scarcely have been typical dwellers in plains, who are usually
slow and heavy; we should rather think of them as tough and active
Scandinavian mountaineers, who by their hard life in the hills had
become light of foot and practised in the use of their limbs; but who,
on the other hand, had been ill-supplied with heavier weapons and
had had scant opportunities of exercise as heavy-armed men, for
which indeed they had no taste. This also explains their remarkable
mobility. We are thus led once more to think of Norway as the
possible home of some of the Eruli. To sum up, we find then that
they had a king with the Norse name Rodulf, and there are many
indications that he was the same as the Norwegian king Rodulf
(from Romsdal ?) who came to Theodoric. They returned through
Jutland and sailed thence to Thule, where they settled by the side of
the Gauti, i.e., to the west of them in Norway, which from old time](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-67-320.jpg)
![had had frequent communication with Jutland, from whence the
Horder (and probably also the Ryger ?) had immigrated. They are
described as having characteristics which are typical of
mountaineers, but not of lowlanders. An Erulian name, “Aruth”
(Ἀρουθ), mentioned by Procopius [De bello Goth., iv. 26], also points
to Norway, since it appears to be the same as the Norwegian tribal
name “Horder” (“*Haruðr,” gen. “Haruþs,” on the Rök-stone [cf. S.
Bugge, 1910, p. 98], or “Arothi” in Jordanes).
Other Erulian names in Procopius may be common to the northern
Germanic languages. In the opinion of Professor Alf Torp it is
probable that “Visandos” is bison, “Aluith” is Alvid or Alvith (all-
knowing); in “Fanitheos” the first syllable may be “fan” or “fen”
(English, fen) and the second part “-theos” may be the Scandinavian
termination “-ther”; “Aordos” may be Vard. The King’s name “Ochon”
seems to resemble the Norwegian Håkon; but the latter name
cannot have had such a form at that time, it must have been longer.
What Procopius tells us [De bello Goth., ii. 14] about the manners
and customs of the Eruli agrees with what we know of the
Norsemen generally. They worshipped many gods, whom they
considered it their sacred duty to propitiate with human sacrifices.
Aged and sick persons were obliged to ask their relatives to help
them to get rid of life;[145] they were killed with a dagger by one
who did not belong to the family, and were burnt on a great pile,
after which the bones were collected and buried, as was the custom
in western Norway amongst other places. “When an Erulian died, his
wife, if she wished to show her virtue and leave a good name behind
her, had to hang herself not long after with a rope by her husband’s
grave and thus make an end of herself. If she did not do this, she
lost respect for the future, and was an offence to her husband’s
family. This custom was observed by the Eruli from old time.” Their
many gods and human sacrifices agree, as we see, with Procopius’s
description of the inhabitants of Thule, and with what we know of
the Scandinavians from other quarters. As human sacrifices with
most peoples were connected with banquets, at which slain enemies](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-68-320.jpg)
![were eaten,[146] the assertion that our Germanic ancestors did not
practise cannibalism rests upon uncertain ground. When, therefore,
in finds of the Stone Age in Denmark, Sweden and Norway broken or
scraped human bones occur, which point to cannibalism, it cannot be
argued from this, as is done by Dr. A. M. Hansen [1907], that the
finds belong to a non-Germanic people.
For the rest, Procopius paints the Eruli in crude colours; they are
covetous, domineering and violent towards their fellow men, without
being ashamed of it. They are addicted to the grossest debauchery,
are the most wicked of men, and utterly depraved.
Skridfinns
The “Scrithifini” of Procopius (and Jordanes’ corrupted form,
“Screrefennæ” or “Scretefennæ”) are undoubtedly a people of the
same kind as Tacitus’s “Fenni” (Ptolemy-Marinus’s “Finni”); but they
have here acquired the descriptive prefix “scrithi-,” which is generally
understood as the Norse “skriða” (== to slide, e.g., on the ice, to
glide; cf. Swedish “skridsko,” skate). The Norsemen must have
characterised their Finnish (i.e., Lappish) neighbours on the north as
sliding (walking) on ski (“skriða á skiðum”), to distinguish them from
other peoples in the outlying districts whom they also called Finns. If
this is so, it is the first time that a reference to ski-running is found
in literature. There is, moreover, considerable similarity between
Procopius’s description of these hunters and Tacitus’s account of the
“Fenni,” who must certainly also have lived in Scandinavia (see
above, p. 113), and who may have been the same people. They
have many peculiar characteristics in common, e.g., that both men
and women go hunting; and the statement that while the mothers
go hunting, the children, in Tacitus, are hidden in a shelter of
boughs (i.e., a tent), and in Procopius are hung up in a tree
(perhaps the Lapps’ “komse,” i.e., a cradle made of wood to hang up
in the tent). Procopius himself probably did not know Tacitus’s
“Germania,” but it is possible that his unknown authority did so,
although this work was generally forgotten at that time. But even if
the description of Procopius may thus be partly derived from Tacitus,](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-69-320.jpg)
![and weak. He was also to have an influence on cartography. With his
fondness for bad etymological interpretations he derived the word
“rotunditas,” for the roundness of the earth, from “rota,” wheel, and
he taught that “the word ‘orbis’ is used on account of the roundness
of the circumference, since it is like a wheel. For in every part the
circumfluent ocean surrounds its borders in a circle.” Hence the
conception of the earth’s disc as a wheel came to be general in the
early Middle Ages, and hence the designation of wheel-maps. Isidore
divided the earth’s disc into three parts, Asia (including Paradise) at
the top of the wheel-map, and Europe and Africa, also called Lybia,
at the bottom; and the boundaries between these continents formed
a T with the rivers Tanais and Nile horizontally at the top, and the
Mediterranean (“Mare Magnum”) below. Therefore maps of this type,
which was maintained for a long time, are also called T-maps.[147]
Otherwise Isidore declared clearly enough in favour of the spherical
form of the earth.
Bede, 673-
735
The Anglo-Saxon monk and scholar, Beda Venerabilis (673-735), who
in his work “Liber de natura rerum” also mentions the countries of
the earth, but without making any fresh statement about the North,
was strongly influenced by Isidore. He asserts, however, the
spherical form of the earth in an intelligent way, giving, amongst
other reasons, that of the ancient Greeks, that earth and water are
attracted towards a central point. The form of a sphere was also the
only one that would explain why certain stars were visible in the
north, but not in the south.](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-72-320.jpg)
![priest. He bases his work on older authors; the Bible, some Latin,
some Greek, and some later writers; but he certainly had a Roman
itinerary map like the Tabula Peutingeriana. His statements about
the North are in part taken from Jordanes, but he also quotes three
other “Gothic scholars,” who are otherwise entirely unknown. One of
them, Aithanarit (or Athanaric ?), is mentioned particularly in
connection with the Skridfinns. The other two, Eldevaldus (or
Eldebald ?) and Marcomirus (or Marcomeres ?), have also described
western Europe; the latter is specially used in the description of the
countries of the Danes, Saxons and Frisians.
The Ravenna geographer regarded the earth’s disc as approximately
round, and surrounded by ocean, but the latter was not entirely
continuous, for it did not extend behind India. It was true that some
cosmographers had described it so, but no Christian ought to believe
this, for Paradise was in the extreme East, near to India; and as the
pollen is wafted by the breath of the wind from the male palm to the
female near it, so does a beneficent perfume from Paradise blow
upon the aromatic flowers of India. Some thought that the sun in its
course returned to the east under the depths of ocean; but the
Ravenna geographer agreed with those who said that the sun
moved all night along paths which cannot be traced, behind lofty
mountains, in the north beyond the ocean, and in the morning it
came forth again from behind them.
[iv. 12.] “In a line with Scythia and the coast of the ocean is the
country which is said to be that of the ‘Rerefeni’ and ‘Sirdifeni’
(‘Scirdifrini’). The people of this country, according to what the
Gothic scholar Aithanarit says, dwell among the rocks of the
mountains, and both men and women are said to live by hunting,
and to be entirely unacquainted both with meat and wine. This land
is said to be colder than all others. Farther on by the side of the
Serdifenni on the coast of the ocean is the land which is called
Dania; this land, as the above-mentioned Aithanaridus and
Eldevaldus and Marcomirus, the Gothic scholars, say, produces
people who are swifter than all others.” [These must be the Eruli.]](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-74-320.jpg)
![“This Dania is now called the land of the Nordomanni.” This is the
first time the name Norman is used, so far as is known.
[v. 30.] “In the northern ocean itself, after the land of the Roxolani,
is an island which is called Scanza, which is also called Old Scythia
by most cosmographers. But in what manner the island of Scanza
itself lies, we will with God’s help relate.”
He says, following Jordanes (see above, p. 130), that from this
island other nations, amongst them the Goths and the Danes,
besides the Gepidæ, migrated.
It will be seen that the Ravenna geographer’s statements about the
Skridfinns, whose name is varied and corrupted even more than in
Jordanes, bear a striking resemblance to those of Procopius,
although he says he derived them from the Goth Aithanarit; if this is
correct, then the latter must either have borrowed from Procopius,
which is very probable, or he is older and was the common authority
both of Procopius and the Ravenna geographer, and, if so, perhaps
also of Cassiodorus (?).](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-75-320.jpg)
![Cynocephali on a peninsula north-east of Norway
(from the Hereford map)
Æthicus
Istricus,
seventh
century (?)
An enigmatical work, probably dating from about the seventh
century, which was much read in the Middle Ages, professes to be a
Latin translation, by a certain Hieronymus, from a Christian book of
travel by a Greek commonly called Æthicus Istricus.[148] He is said to
have travelled before the fourth century. The translator asserts that
Æthicus had related many fabulous things, which he has not
repeated, as he wished to keep to the sure facts; but among them
we find many remarkable pieces of information, as that Æthicus had
seen with his own eyes on the north of the Caspian Sea the
Amazons give the breast to Centaurs and Minotaurs, and when he](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-76-320.jpg)
![was living in the town of Choolisma, built by Japhet’s son Magog, he
saw the sea of bitumen which forms the mouth of Hell and from
which the cement for Alexander’s wall of iron came. In Armenia he
looked in vain for Noah’s ark; but he saw dragons, ostriches, griffins,
and ants as large and ferocious as dogs. He also mentioned griffins
and treasures of gold in the north between the Tanais and the
northern ocean. “The Scythians, Griffins, Tracontians and Saxons
built ships of wattles smeared over with pitch” (perhaps it is meant
that they were also covered with hides). These ships were
extraordinarily swift. Among the Scythians there was said to be an
able craftsman and great teacher, Grifo, who built ships with prows
in the northern ocean. He was like the griffins or the flying fabulous
birds. Æthicus visited an island called Munitia north of Germania.
There he found “Cenocephali” (dog-headed men). They were a
hideous race. The Germanic peoples came to the island as
merchants and called the people “Cananei.” They go with bare
calves, smear their hair with oil or fat and smell foully. They lead a
dirty life and feed on unclean animals, mice, moles, etc. They live in
felt tents in the woods far away by fens and swampy places. They
have a number of cattle, fowls and eggs.[149] They know no god and
have no king. They use more tin than silver. One might be tempted
to think that this fable of dog-headed people in the north had arisen
from the word “Kvæn” (Finn), which to a Greek like Æthicus would
sound like “cyon” (dog). The name “Cenocephali” may have been
introduced in this way, while that of “Cananei” may have arisen by a
sort of corrupt similarity of sound between Kvæn and the Old
Testament people of Canaan. It might thus be Kvænland or Finland
that is here spoken of. Their going with bare calves and living in felt
tents may remind us of the Argippæi of Herodotus, who were bald
(while in Mela they went bare-headed) and had felt tents in winter.](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-77-320.jpg)
![The Seven Sleepers in the Cave by the North Sea (from Olaus
Magnus)
Paulus
Warnefridi,
720-790
The Langobard author Paulus Warnefridi, also called Diaconus (about
720-790), gives for the most part more or less confused extracts
from earlier authors, but he seems besides to have obtained some
new information about the North. Just as the Goth Jordanes (or
Cassiodorus, or Ablabius) makes the Goths emigrate from Ptolemy’s
Scandza, so Paulus, following earlier authors,[150] makes the
Langobards proceed from Pliny’s island Scatinavia, far in the north. It
looks as though at that time a northern origin was held in high
esteem. But Paulus describes the country, from the statements of
those who have seen it, as not “really lying in the sea, but the waves
wash the low shores.” This points to a confusion here with a district
called Scatenauge by the Elbe, which in a somewhat later MS.
(about 807) of the Langobardic Law is mentioned as the home of the
Langobards [cf. Lönborg, 1897, p. 27]. Paulus further relates that on
the coast “north-west towards the uttermost boundaries of](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-78-320.jpg)
![Germany” there lie seven men asleep in a cave, for how long is
uncertain. They resemble the Romans in appearance, and both they
and their clothes are unharmed, and they are regarded by the
inhabitants as holy. The legend of the Seven Sleepers is already
found in Gregory of Tours, who has it from Asia Minor, where it arose
in the third century and was located at Ephesus [cf. J. Koch, 1883].
The legend was very common in Germania, and we find it again later
in tales of shipwreck on the coast of Greenland.[151]
“Near to this place [i.e., the cave with the seven men] dwell the
‘Scritobini’;[152] thus is this people called; they have snow even in
summer time, and they eat nothing but the raw flesh of wild beasts,
as they do not differ from the beasts themselves in intelligence, and
they also make themselves clothes of their skins with the hair on.
Their name is explained from the word ‘to leap’ in the foreign tongue
[i.e., Germanic], for by leaping with a certain art they overtake the
wild beasts with a piece of wood bent like a bow. Among them is an
animal which is not much unlike a stag, and I have seen a dress
made of the hide of this animal, just as if it was bristling with hairs,
and it was made like a tunic and reached to the knees, as the
above-mentioned Scritobini wear it, as I have told. In these parts, at
the summer solstice, there is seen for several days, even at night,
the clearest light, and they have there much more daylight than
elsewhere, as on the other hand, about the winter solstice, even if
there is daylight, the sun itself is not seen there, and the day is
shorter than in any other place, the nights also are longer; for the
farther one goes away from the sun, the nearer the sun appears to
the earth [the horizon], and the shadows become longer.”...](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-79-320.jpg)
![The oldest known picture of a ski-runner
(from the Hereford map’s representation
of Norway, thirteenth century)
“And not far from the shore which we before spoke of [by the cave]
on the west, where the ocean extends without bounds, is that very
deep abyss of the waters which we commonly call the ocean’s navel.
It is said twice a day to suck the waves into itself, and to spew them
out again; as is proved to happen along all these coasts, where the
waves rush in and go back again with fearful rapidity. Such a gulf or
whirlpool is called by the poet Virgil Caribdis, and in his poem he
says it is in the strait by Sicily, as he says:
‘Scilla lies on the right hand
and the implacable Caribdis on the left.
And three times it sucks the vast billows
down into the abyss with the deep whirlpool](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-80-320.jpg)
![of the gulf, and it sends them up again into the air,
and the wave lashes the stars.’
“By the whirlpool of which we have spoken it is asserted that ships
are often drawn in with such rapidity that they seem to resemble the
flight of arrows through the air; and sometimes they are lost in this
gulf with a very frightful destruction. Often just as they are about to
go under, they are brought back again by a sudden shock of the
waves, and they are sent out again thence with the same rapidity
with which they were drawn in. It is asserted that there is also
another gulf of the same kind between Britain and the Gallician
province” [i.e., northern Spain], whereupon there follows a
description of the tides on the south coast of France and at the
mouths of the rivers, after which there is a highly coloured account
of the horrors of the Ebudes, where they can hear the noise of the
waters rushing towards a similar Caribdis.
Paulus Warnefridi evidently had a very erroneous idea of ski-running,
which he made into a leaping instead of a gliding motion. He may
have imagined that they jumped about on pieces of wood bent like
bows. That the abyss of waters or navel of the sea is thought to be
in the North may be due to reports either of the current in the
Pentland Firth or of the Mosken-ström or the Salt-ström, which thus
make their appearance here in literature, and which were afterwards
developed into the widespread ideas of the Middle Ages about
maelstroms and abysses in the sea, perhaps by being connected
with the ancient Greek conception of the uttermost abyss (Tartarus,
Anostus, Ginnungagap; see pp. 11, 12, 17), and as here with the
description of the current in the Straits of Messina.](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-81-320.jpg)
![The Maelstrom near the Lofoten Islands (from Olaus Magnus)
Viktor Rydberg [1886, pp. 318, 425, ff.] supposed Paulus’s
description of the whirlpool to be derived from the Norse legends of
the world’s well, “Hvergelmer”—which causes the tides by the water
flowing up and down through its subterranean channels—and of the
quern “Grotte” at the bottom of the sea, which forms whirlpools
when the waters run down into the hole in the mill-stone.[153] But it
is perhaps just as probable that it is the southern, originally classical
ideas which have been localised in the Norse legends. As we have
seen, we find in Virgil the same conception of a gulf in the sea which
sucks the water into itself and sends it up again. Isidore says of the
abyss (also repeated in Hrabanus Maurus):
“Abyssus is the impenetrable deep of the waters, or the caves of the
hidden waters, from whence springs and rivers issue forth, but also
those which run concealed beneath the ground. Therefore it is called
Abyssus, for all streams return by hidden veins to their mother
Abyssus.”
It is credible that ideas such as this may have originated, or at any
rate coloured, the myth of “Hvergelmer” (i.e., the noisy or bubbling](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-82-320.jpg)
![occurs in a MS. As the addition about Thule, so far as I can judge,
must show that this country is regarded as Norway, and as there are
many indications that it was made by an Irish monk, it is further
probable that it belongs to the period before the Irish discovery of
Iceland, which then, according to Dicuil’s book, became regarded as
Thule. I think, therefore, we can place the addition at the beginning
of the eighth century, and it will then be evidence of the knowledge
of Norway which prevailed in the British Isles at that time. After
having mentioned Britain and the neighbouring islands the account
proceeds [Solinus, c. 22]:
“From the Caledonian Promontory it is two days’ sail for those who
voyage to Tyle [Thule]. From thence begin the Ebudes islands
[Hebrides], five in number [the five principal islands]. Their
inhabitants live on fruits, fish and milk. Though there are many
islands, they are all separated by narrow arms of the sea. They all
together have but one king. The king owns nothing for himself
alone, all is common property. Justice is imposed upon him by fixed
laws, and lest he should be led away from the truth by
covetousness, he learns righteousness by poverty, since he has no
possessions; he is therefore supported by the people. No woman is
given him in marriage, but he takes in turn her who pleases him at
the moment. Thus he has neither the desire nor the hope of
children. The second station for the voyager [to Thule] is provided
by the Orcades. But the Orcades lie seven days’ and the same
number of nights’ sail from the Ebudes, they are three in number
[i.e., the three principal isles of the Shetlands]. They are uninhabited
(‘vacant homines’). They have no woods, but are rough with reeds
and grass, the rest is bare sandy beach and rocks. From the Orcades
direct to Thule is five days’ and nights’ sail. But Thule is fertile and
rich in late-ripening fruits. The inhabitants there live from the
beginning of spring with their cattle, and feed on herbs and milk; the
fruits of the trees they keep for winter. They have women in
common, regular marriage is not known among them.”](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-84-320.jpg)
![The earliest voyages northward to the Arctic Circle, of which there is
certain literary mention in the early Middle Ages, are the Irish
monks’ expeditions across the sea in their small boats, whereby they
discovered the Faroes and Iceland, and, at all events for a time, lived
there. Of these the Irish monk Dicuil gave an account, as early as
about the year 825, in his description of the earth, “De Mensura
Orbis Terræ” [cf. Letronne, 1814, pp. 38 f., 131 f.]. It is characteristic
of the spiritual tendency of that period of the Middle Ages that these
remarkable voyages were not, like other voyages of discovery,
undertaken from love of gain, thirst for adventure, or desire of
knowledge, but chiefly from the wish to find lonely places, where
these anchorites might dwell in peace, undisturbed by the turmoil
and temptations of the world.[154] In this way the unknown islands
near the Arctic Ocean must have seemed to satisfy all their
requirements; but their joy was short-lived; the disturbers of the
North, the Vikings from Norway, soon came there also and drove
them out or oppressed them.
What Dicuil tells us of the Scandinavian North is chiefly derived from
Pliny, and contains nothing new. But of the unknown islands in the
northern ocean he writes [7, 3]:
Discovery
of the
Faroes by
the Irish
“There are many more islands in the ocean north of Britain, which
can be reached from the northern British Isles in two days’ and two
nights’ direct sailing with full sail and a favourable wind. A
trustworthy priest (‘presbyter religiosus’) told me that he had sailed
for two summer-days and an intervening night in a little boat with
two thwarts [i.e., two pairs of oars],[155] and landed on one of these
islands. These islands are for the most part small; nearly all are
divided from one another by narrow sounds, and upon them
anchorites, who proceeded from our Scotia [i.e., Ireland], have lived
for about a hundred years (‘in centum ferme annis’). But as since the
beginning of the world they had always been deserted, so are they](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-86-320.jpg)
![now by reason of the Northman pirates emptied of anchorites, but
full of innumerable sheep and a great number of different kinds of
sea-birds. We have never found these islands spoken of in the books
of authors.”
The Faroes
This description best suits the Faroes,[156] where, therefore, Irish
monks had previously lived, and from whence they had been driven
out by Norwegian seafarers, probably at the close of the eighth
century. As, however, Dicuil is so well aware of the islands being full
of sheep, the Irish may have continued to visit them occasionally,
like the trustworthy priest referred to, who sailed there in a boat
with two thwarts. Dicuil’s statement that they were then “emptied of
anchorites” must doubtless be interpreted to mean that they were
uninhabited; but this does not sound very probable. Rather, there](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-87-320.jpg)
![are many indications that the islands had an original Celtic
population, which continued to live there after the settlement of the
Norsemen.
There are some Celtic place-names, such as “Dímon” (the islands
“Stora Dímon” and “Litla Dímon,” or “Dímun meiri” and “Dímun
minni”) from the Celtic “dimun” (== double neck, thus like
Norwegian “Tviberg”).[157] As such Celtic place-names cannot have
been introduced later, the Norwegians must have got them from the
Celts who were there before, and with whom they had intercourse.
The language of the Faroes has also many loan-words from Celtic,
mostly for agriculture and cattle-farming, and for the flora and fauna
of the islands. These might be explained by many of the Norwegian
settlers having previously lived in the Scottish islands or in Ireland,
or having had frequent communication with those countries [cf. A.
Bugge, 1905, p. 358]; but it seems more natural to suppose that the
loan-words are derived from a primitive Celtic population. To this
must be added that the people of the Southern Faroes are still dark,
with dark eyes and black hair, and differ from the more Germanic
type of the northern islands [cf. D. Bruun, 1902, p. 5]. The name
“Færöene” (sheep-islands) shows that there probably were sheep
before the Norsemen came, which so far agrees with Dicuil; these
sheep must then have been introduced by the earlier Celts.
According to this it seems possible that the Irish monks came to the
islands not merely as anchorites, but also to spread Christianity
among a Celtic population. The Norwegians arrived later, took
possession of the islands, and oppressed the Celts.
Irish
Discovery
of Iceland
But the bold Irish monks extended their voyages farther north. Dicuil
has also to tell us how they found Iceland, which he calls Thule, and
lived there. After having mentioned what Pliny, Solinus, Isidore
(Hispalensis) and Priscianus say about Thule (Thyle), he continues
[7, 2, 6]:](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-88-320.jpg)
![“It is now thirty years since certain priests, who had been on that
island from the 1st of February to the 1st of August, told that not
only at the time of the summer solstice, but also during the days
before and after, the setting sun at evening conceals itself as it were
behind a little mound, so that it does not grow dark even for the
shortest space of time, but whatsoever work a man will do, even
picking the lice out of his shirt (pediculos de camisia extrahere), he
may do it just as though the sun were there, and if they had been
upon the high mountains of the island perhaps the sun would never
be concealed by them [i.e., the mountains]. In the middle of this
very short time it is midnight in the middle of the earth, and on the
other hand I suppose in the same way that at the winter solstice and
for a few days on either side of it the dawn is seen for a very short
time in Thule, when it is midday in the middle of the earth.
Consequently I believe that they lie and are in error who wrote that
there was a stiffened (concretum) sea around it [i.e., Thyle], and
likewise those who said that there was continuous day without night
from the vernal equinox till the autumnal equinox, and conversely
continuous night from the autumnal equinox till the vernal, since
those who sailed thither reached it in the natural time for great cold,
and while they were there always had day and night alternately
except at the time of the summer solstice; but a day’s sail northward
from it they found the frozen (congelatum) sea.”
This description, written half a century before the Norwegians,
according to common belief, came to Iceland, shows that the
country was known to the Irish, at any rate before the close of the
eighth century (thirty years before Dicuil wrote in 825), and how
much earlier we cannot say. With the first-hand information he had
received from people who had been there, Dicuil may have blended
ideas which he had obtained from his literary studies. The sun hiding
at night behind a little mound reminds us of the older ideas that it
went behind a mountain in the north (cf. Cosmas Indicopleustes and
the Ravenna geographer); but of course it may also be due to local
observation. The idea that the frozen sea (“congelatum mare”) had
been found a day’s sail north of this island is precisely the same as](https://image.slidesharecdn.com/21185778-250519104953-f1914441/85/Understanding-Distributed-Systems-2nd-Edition-2nd-Edition-Roberto-Vitillo-89-320.jpg)
Understanding Distributed Systems 2nd Edition 2nd Edition Roberto Vitillo
- 1. Understanding Distributed Systems 2nd Edition 2nd Edition Roberto Vitillo download https://ebookbell.com/product/understanding-distributed- systems-2nd-edition-2nd-edition-roberto-vitillo-42371556 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Understanding Distributed Systems What Every Developer Should Know About Large Distributed Applications Roberto Vitillo https://ebookbell.com/product/understanding-distributed-systems-what- every-developer-should-know-about-large-distributed-applications- roberto-vitillo-49169040 Understanding Behaviour Of Distributed Systems Using Mcrl2 1st Muhammad Atif https://ebookbell.com/product/understanding-behaviour-of-distributed- systems-using-mcrl2-1st-muhammad-atif-47921432 Benchmarking Peertopeer Systems Understanding Quality Of Service In Largescale Distributed Systems 1st Edition Wolfgang Effelsberg https://ebookbell.com/product/benchmarking-peertopeer-systems- understanding-quality-of-service-in-largescale-distributed- systems-1st-edition-wolfgang-effelsberg-4293988 Handson Azure Repos Understanding Centralized And Distributed Version Control In Azure Devops Services 1st Edition Chaminda Chandrasekara https://ebookbell.com/product/handson-azure-repos-understanding- centralized-and-distributed-version-control-in-azure-devops- services-1st-edition-chaminda-chandrasekara-50195564
- 3. Handson Azure Repos Understanding Centralized And Distributed Version Control In Azure Devops Services 1st Edition Chaminda Chandrasekara https://ebookbell.com/product/handson-azure-repos-understanding- centralized-and-distributed-version-control-in-azure-devops- services-1st-edition-chaminda-chandrasekara-53001822 Collaborative And Distributed Chemical Engineering From Understanding To Substantial Design Process Support Results Of The Improve Project 1st Edition W Marquardt https://ebookbell.com/product/collaborative-and-distributed-chemical- engineering-from-understanding-to-substantial-design-process-support- results-of-the-improve-project-1st-edition-w-marquardt-4240570 Distributed Ambient And Pervasive Interactions Understanding Humans 1st Ed Norbert Streitz https://ebookbell.com/product/distributed-ambient-and-pervasive- interactions-understanding-humans-1st-ed-norbert-streitz-7151184 Understanding Enterprise Soa 1 Illustrated Edition Eric Pulier https://ebookbell.com/product/understanding-enterprise- soa-1-illustrated-edition-eric-pulier-981346 Black Swan Startups Understanding The Rise Of Successful Technology Business In Unlikely Places Mahroum https://ebookbell.com/product/black-swan-startups-understanding-the- rise-of-successful-technology-business-in-unlikely-places- mahroum-22006572
- 5. What every developer should know about large distributed applications S e c o n d E d i t i o n ROBERTO VITILLO UNDERSTANDING DISTRIBUTED SYSTEMS
- 7. Understanding Distributed Systems Version 2.0.0 Roberto Vitillo March 2022
- 9. Contents Copyright ix About the author xi Acknowledgements xiii Preface xv 1 Introduction 1 1.1 Communication . . . . . . . . . . . . . . . . . . . . 2 1.2 Coordination . . . . . . . . . . . . . . . . . . . . . 3 1.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Resiliency . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Maintainability . . . . . . . . . . . . . . . . . . . . 6 1.6 Anatomy of a distributed system . . . . . . . . . . 7 I Communication 11 2 Reliable links 17 2.1 Reliability . . . . . . . . . . . . . . . . . . . . . . . 18 2.2 Connection lifecycle . . . . . . . . . . . . . . . . . 18 2.3 Flow control . . . . . . . . . . . . . . . . . . . . . . 20 2.4 Congestion control . . . . . . . . . . . . . . . . . . 21 2.5 Custom protocols . . . . . . . . . . . . . . . . . . . 23 3 Secure links 25 3.1 Encryption . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Authentication . . . . . . . . . . . . . . . . . . . . 26 3.3 Integrity . . . . . . . . . . . . . . . . . . . . . . . . 28 3.4 Handshake . . . . . . . . . . . . . . . . . . . . . . 29
- 10. CONTENTS iv 4 Discovery 31 5 APIs 35 5.1 HTTP . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2 Resources . . . . . . . . . . . . . . . . . . . . . . . 39 5.3 Request methods . . . . . . . . . . . . . . . . . . . 41 5.4 Response status codes . . . . . . . . . . . . . . . . 42 5.5 OpenAPI . . . . . . . . . . . . . . . . . . . . . . . 43 5.6 Evolution . . . . . . . . . . . . . . . . . . . . . . . 45 5.7 Idempotency . . . . . . . . . . . . . . . . . . . . . 46 II Coordination 53 6 System models 57 7 Failure detection 61 8 Time 63 8.1 Physical clocks . . . . . . . . . . . . . . . . . . . . 63 8.2 Logical clocks . . . . . . . . . . . . . . . . . . . . . 65 8.3 Vector clocks . . . . . . . . . . . . . . . . . . . . . 67 9 Leader election 71 9.1 Raft leader election . . . . . . . . . . . . . . . . . . 72 9.2 Practical considerations . . . . . . . . . . . . . . . . 73 10 Replication 77 10.1 State machine replication . . . . . . . . . . . . . . . 78 10.2 Consensus . . . . . . . . . . . . . . . . . . . . . . . 81 10.3 Consistency models . . . . . . . . . . . . . . . . . . 83 10.4 Chain replication . . . . . . . . . . . . . . . . . . . 90 11 Coordination avoidance 95 11.1 Broadcast protocols . . . . . . . . . . . . . . . . . . 96 11.2 Conflict-free replicated data types . . . . . . . . . . 98 11.3 Dynamo-style data stores . . . . . . . . . . . . . . . 103 11.4 The CALM theorem . . . . . . . . . . . . . . . . . . 105 11.5 Causal consistency . . . . . . . . . . . . . . . . . . 106 11.6 Practical considerations . . . . . . . . . . . . . . . . 110 12 Transactions 111 12.1 ACID . . . . . . . . . . . . . . . . . . . . . . . . . 112 12.2 Isolation . . . . . . . . . . . . . . . . . . . . . . . . 113
- 11. CONTENTS v 12.3 Atomicity . . . . . . . . . . . . . . . . . . . . . . . 119 12.4 NewSQL . . . . . . . . . . . . . . . . . . . . . . . . 122 13 Asynchronous transactions 127 13.1 Outbox pattern . . . . . . . . . . . . . . . . . . . . 128 13.2 Sagas . . . . . . . . . . . . . . . . . . . . . . . . . . 130 13.3 Isolation . . . . . . . . . . . . . . . . . . . . . . . . 133 III Scalability 137 14 HTTP caching 145 14.1 Reverse proxies . . . . . . . . . . . . . . . . . . . . 148 15 Content delivery networks 151 15.1 Overlay network . . . . . . . . . . . . . . . . . . . 151 15.2 Caching . . . . . . . . . . . . . . . . . . . . . . . . 153 16 Partitioning 155 16.1 Range partitioning . . . . . . . . . . . . . . . . . . 157 16.2 Hash partitioning . . . . . . . . . . . . . . . . . . . 158 17 File storage 163 17.1 Blob storage architecture . . . . . . . . . . . . . . . 163 18 Network load balancing 169 18.1 DNS load balancing . . . . . . . . . . . . . . . . . . 174 18.2 Transport layer load balancing . . . . . . . . . . . . 175 18.3 Application layer load balancing . . . . . . . . . . . 178 19 Data storage 181 19.1 Replication . . . . . . . . . . . . . . . . . . . . . . 181 19.2 Partitioning . . . . . . . . . . . . . . . . . . . . . . 184 19.3 NoSQL . . . . . . . . . . . . . . . . . . . . . . . . . 185 20 Caching 191 20.1 Policies . . . . . . . . . . . . . . . . . . . . . . . . 192 20.2 Local cache . . . . . . . . . . . . . . . . . . . . . . 193 20.3 External cache . . . . . . . . . . . . . . . . . . . . . 194 21 Microservices 197 21.1 Caveats . . . . . . . . . . . . . . . . . . . . . . . . 199 21.2 API gateway . . . . . . . . . . . . . . . . . . . . . . 202
- 12. CONTENTS vi 22 Control planes and data planes 209 22.1 Scale imbalance . . . . . . . . . . . . . . . . . . . . 211 22.2 Control theory . . . . . . . . . . . . . . . . . . . . . 214 23 Messaging 217 23.1 Guarantees . . . . . . . . . . . . . . . . . . . . . . 221 23.2 Exactly-once processing . . . . . . . . . . . . . . . 223 23.3 Failures . . . . . . . . . . . . . . . . . . . . . . . . 224 23.4 Backlogs . . . . . . . . . . . . . . . . . . . . . . . . 224 23.5 Fault isolation . . . . . . . . . . . . . . . . . . . . . 225 IV Resiliency 229 24 Common failure causes 233 24.1 Hardware faults . . . . . . . . . . . . . . . . . . . . 233 24.2 Incorrect error handling . . . . . . . . . . . . . . . 234 24.3 Configuration changes . . . . . . . . . . . . . . . . 234 24.4 Single points of failure . . . . . . . . . . . . . . . . 235 24.5 Network faults . . . . . . . . . . . . . . . . . . . . 236 24.6 Resource leaks . . . . . . . . . . . . . . . . . . . . . 237 24.7 Load pressure . . . . . . . . . . . . . . . . . . . . . 238 24.8 Cascading failures . . . . . . . . . . . . . . . . . . 238 24.9 Managing risk . . . . . . . . . . . . . . . . . . . . . 240 25 Redundancy 243 25.1 Correlation . . . . . . . . . . . . . . . . . . . . . . 244 26 Fault isolation 247 26.1 Shuffle sharding . . . . . . . . . . . . . . . . . . . . 248 26.2 Cellular architecture . . . . . . . . . . . . . . . . . 250 27 Downstream resiliency 253 27.1 Timeout . . . . . . . . . . . . . . . . . . . . . . . . 253 27.2 Retry . . . . . . . . . . . . . . . . . . . . . . . . . . 255 27.3 Circuit breaker . . . . . . . . . . . . . . . . . . . . 258 28 Upstream resiliency 261 28.1 Load shedding . . . . . . . . . . . . . . . . . . . . 261 28.2 Load leveling . . . . . . . . . . . . . . . . . . . . . 262 28.3 Rate-limiting . . . . . . . . . . . . . . . . . . . . . 263 28.4 Constant work . . . . . . . . . . . . . . . . . . . . 269
- 13. CONTENTS vii V Maintainability 275 29 Testing 279 29.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . 280 29.2 Size . . . . . . . . . . . . . . . . . . . . . . . . . . 281 29.3 Practical considerations . . . . . . . . . . . . . . . . 283 29.4 Formal verification . . . . . . . . . . . . . . . . . . 285 30 Continuous delivery and deployment 289 30.1 Review and build . . . . . . . . . . . . . . . . . . . 290 30.2 Pre-production . . . . . . . . . . . . . . . . . . . . 292 30.3 Production . . . . . . . . . . . . . . . . . . . . . . . 293 30.4 Rollbacks . . . . . . . . . . . . . . . . . . . . . . . 293 31 Monitoring 297 31.1 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 298 31.2 Service-level indicators . . . . . . . . . . . . . . . . 301 31.3 Service-level objectives . . . . . . . . . . . . . . . . 304 31.4 Alerts . . . . . . . . . . . . . . . . . . . . . . . . . 306 31.5 Dashboards . . . . . . . . . . . . . . . . . . . . . . 308 31.6 Being on call . . . . . . . . . . . . . . . . . . . . . . 312 32 Observability 315 32.1 Logs . . . . . . . . . . . . . . . . . . . . . . . . . . 316 32.2 Traces . . . . . . . . . . . . . . . . . . . . . . . . . 319 32.3 Putting it all together . . . . . . . . . . . . . . . . . 321 33 Manageability 323 34 Final words 327
- 15. Copyright Understanding Distributed Systems by Roberto Vitillo Copyright © Roberto Vitillo. All rights reserved. The book’s diagrams have been created with Excalidraw. While the author has used good faith efforts to ensure that the in- formation and instructions in this work are accurate, the author disclaims all responsibility for errors or omissions, including with- out limitation responsibility for damages resulting from the use of or reliance on this work. The use of the information and instruc- tions contained in this work is at your own risk. If any code sam- ples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
- 17. About the author Authors generally write this page in the third person as if some- one else is writing about them, but I like to do things a little bit differently. I have over 10 years of experience in the tech industry as a software engineer, technical lead, and manager. After getting my master’s degree in computer science from the Uni- versity of Pisa, I worked on scientific computing applications at the Berkeley Lab. The software I contributed is used to this day by the ATLAS experiment at the Large Hadron Collider. Next, I worked at Mozilla, where I set the direction of the data plat- form from its very early days and built a large part of it, including the team. In 2017, I joined Microsoft to work on an internal SaaS for telemetry. Since then, I have helped launch multiple public SaaS products, like Playfab and Customer Insights. The data ingestion platform I am responsible for is one of the largest in the world, ingesting millions of events per second from billions of devices worldwide.
- 19. Acknowledgements I am very thankful for the colleagues and mentors who inspired me and believed in me over the years. Thanks to Chiara Roda, Andrea Dotti, Paolo Calafiura, Vladan Djeric, Mark Reid, Pawel Chodarcewicz, and Nuno Cerqueira. Rachael Churchill, Doug Warren, Vamis Xhagjika, Alessio Placitelli, Stefania Vitillo, and Alberto Sottile provided invaluable feedback on early drafts. Without them, the book wouldn’t be what it is today. A very special thanks to the readers of the first edition who reached out with feedback and suggestions over the past year and helped me improve the book: Matthew Williams, Anath B Chat- terjee, Ryan Carney, Marco Vocialta, Curtis Washington, David Golden, Radosław Rusiniak, Sankaranarayanan Viswanathan, Praveen Barli, Abhijit Sarkar, Maki Pavlidis, Venkata Srinivas Namburi, Andrew Myers, Aaron Michaels, Jack Henschel, Ben Gummer, Luca Colantonio, Kofi Sarfo, Mirko Schiavone, and Gaurav Narula. I am also very thankful for all the readers who left reviews or reached out to me and let me know they found the book useful. They gave me the motivation and energy to write a second edition. Finally, and above all, thanks to my family: Rachell and Leonardo. Without your unwavering support, this book wouldn’t exist.
- 21. Preface Learning to build distributed systems is hard, especially if they are large scale. It’s not that there is a lack of information out there. You can find academic papers, engineering blogs, and even books on the subject. The problem is that the available information is spread out all over the place, and if you were to put it on a spectrum from theory to practice, you would find a lot of material at the two ends but not much in the middle. That is why I decided to write a book that brings together the core theoretical and practical concepts of distributed systems so that you don’t have to spend hours connecting the dots. This book will guide you through the fundamentals of large-scale distributed systems, with just enough details and external references to dive deeper. This is the guide I wished existed when I first started out. If you are a developer working on the backend of web or mobile applications (or would like to be!), this book is for you. When building distributed applications, you need to be familiar with the network stack, data consistency models, scalability and reliability patterns, observability best practices, and much more. Although you can build applications without knowing much of that, you will end up spending hours debugging and re-architecting them, learn- ing hard lessons that you could have acquired in a much faster and less painful way. However, if you have several years of experience designing and building highly available and fault-tolerant applications that scale to millions of users, this book might not be for you. As an expert,
- 22. CONTENTS xvi you are likely looking for depth rather than breadth, and this book focuses more on the latter since it would be impossible to cover the field otherwise. The second edition is a complete rewrite of the previous edition. Every page of the first edition has been reviewed and where ap- propriate reworked, with new topics covered for the first time. This book is available both in a physical and digital format. The digital version is updated occasionally, which is why the book has a version number. You can subscribe to receive updates from the book’s landing page1 . As no book is ever perfect, I’m always happy to receive feedback. So if you find an error, have an idea for improvement, or simply want to comment on something, always feel free to write me2 . I love connecting with readers! 1 https://understandingdistributed.systems/ 2 roberto@understandingdistributed.systems
- 23. Chapter 1 Introduction “A distributed system is one in which the failure of a com- puter you didn’t even know existed can render your own computer unusable.” – Leslie Lamport Loosely speaking, a distributed system is a group of nodes that cooperate by exchanging messages over communication links to achieve some task. A node can generically refer to a physical ma- chine, like a phone, or a software process, like a browser. Why do we bother building distributed systems in the first place? Some applications are inherently distributed. For example, the web is a distributed system you are very familiar with. You access it with a browser, which runs on your phone, tablet, desktop, or Xbox. Together with other billions of devices worldwide, it forms a distributed system. Another reason for building distributed systems is that some appli- cations require high availability and need to be resilient to single- node failures. For example, Dropbox replicates your data across multiple nodes so that the loss of a single one doesn’t cause your data to be lost.
- 24. CHAPTER 1. INTRODUCTION 2 Some applications need to tackle workloads that are just too big to fit on a single node, no matter how powerful. For example, Google receives tens of thousands of search requests per second from all over the globe. There is no way a single node could handle that. And finally, some applications have performance requirements that would be physically impossible to achieve with a single node. Netflix can seamlessly stream movies to your TV at high resolution because it has a data center close to you. This book tackles the fundamental challenges that need to be solved to design, build, and operate distributed systems. 1.1 Communication The first challenge derives from the need for nodes to communi- cate with each other over the network. For example, when your browser wants to load a website, it resolves the server’s IP address from the URL and sends an HTTP request to it. In turn, the server returns a response with the page’s content. How are the request and response messages represented on the wire? What happens when there is a temporary network outage, or some faulty network switch flips a few bits in the messages? How does the server guarantee that no intermediary can snoop on the communication? Although it would be convenient to assume that some networking library is going to abstract all communication concerns away, in practice, it’s not that simple because abstractions leak1 , and you need to understand how the network stack works when that hap- pens. 1 “The Law of Leaky Abstractions,” https://www.joelonsoftware.com/2002/ 11/11/the-law-of-leaky-abstractions/
- 25. CHAPTER 1. INTRODUCTION 3 1.2 Coordination Another hard challenge of building distributed systems is that some form of coordination is required to make individual nodes work in unison towards a shared objective. This is particularly challenging to do in the presence of failures. The “two generals” problem is a famous thought experiment that showcases this. Suppose two generals (nodes), each commanding their own army, need to agree on a time to jointly attack a city. There is some dis- tance between the armies (network), and the only way to commu- nicate is via messengers, who can be captured by the enemy (net- work failure). Under these assumptions, is there a way for the gen- erals to agree on a time? Well, general 1 could send a message with a proposed time to general 2. But since the messenger could be captured, general 1 wouldn’t know whether the message was actually delivered. You could argue that general 2 could send a messenger with a re- sponse to confirm it received the original message. However, just like before, general 2 wouldn’t know whether the response was actually delivered and another confirmation would be required. As it turns out, no matter how many rounds of confirmation are made, neither general can be certain that the other army will attack the city at the same time. As you can see, this problem is much harder to solve than it originally appeared. Because coordination is such a key topic, the second part of the book is dedicated to understanding the fundamental distributed algorithms used to implement it. 1.3 Scalability The performance of an application represents how efficiently it can handle load. Intuitively, load is anything that consumes the sys- tem’s resources such as CPU, memory, and network bandwidth. Since the nature of load depends on the application’s use cases and architecture, there are different ways to measure it. For example,
- 26. CHAPTER 1. INTRODUCTION 4 the number of concurrent users or the ratio of writes to reads are different forms of load. For the type of applications discussed in this book, performance is generally measured in terms of throughput and response time. Throughput is the number of requests processed per second by the application, while response time is the time elapsed in seconds between sending a request to the application and receiving a response. As load increases, the application will eventually reach its capac- ity, i.e., the maximum load it can withstand, when a resource is ex- hausted. The performance either plateaus or worsens at that point, as shown in Figure 1.1. If the load on the system continues to grow, it will eventually hit a point where most operations fail or time out. Figure 1.1: The system throughput on the y axis is the subset of client requests (x axis) that can be handled without errors and with low response times, also referred to as its goodput.
- 27. CHAPTER 1. INTRODUCTION 5 The capacity of a distributed system depends on its architecture, its implementation, and an intricate web of physical limitations like the nodes’ memory size and clock cycle and the bandwidth and latency of network links. For an application to be scalable, a load increase should not degrade the application’s performance. This requires increasing the capacity of the application at will. A quick and easy way is to buy more expensive hardware with better performance, which is also referred to as scaling up. Unfor- tunately, this approach is bound to hit a brick wall sooner or later when such hardware just doesn’t exist. The alternative is scaling out by adding more commodity machines to the system and hav- ing them work together. Although procuring additional machines at will may have been daunting a few decades ago, the rise of cloud providers has made that trivial. In 2006 Amazon launched Amazon Web Ser- vices (AWS), which included the ability to rent virtual machines with its Elastic Compute Cloud (EC22 ) service. Since then, the number of cloud providers and cloud services has only grown, democratizing the ability to create scalable applications. In Part III of this book, we will explore the core architectural pat- terns and building blocks of scalable cloud-native applications. 1.4 Resiliency A distributed system is resilient when it can continue to do its job even when failures happen. And at scale, anything that can go wrong will go wrong. Every component has a probability of failing — nodes can crash, network links can be severed, etc. No matter how small that probability is, the more components there are and the more operations the system performs, the higher the number of failures will be. And it gets worse because a failure of one com- ponent can increase the probability that another one will fail if the components are not well isolated. 2 “Amazon EC2,” https://aws.amazon.com/ec2/
- 28. CHAPTER 1. INTRODUCTION 6 Failures that are left unchecked can impact the system’s availabil- ity3 , i.e., the percentage of time the system is available for use. It’s a ratio defined as the amount of time the application can serve re- quests (uptime) divided by the total time measured (uptime plus downtime, i.e., the time the application can’t serve requests). Availability is often described with nines, a shorthand way of ex- pressing percentages of availability. Three nines are typically con- sidered acceptable by users, and anything above four is considered to be highly available. Availability % Downtime per day 90% (“one nine”) 2.40 hours 99% (“two nines”) 14.40 minutes 99.9% (“three nines”) 1.44 minutes 99.99% (“four nines”) 8.64 seconds 99.999% (“five nines”) 864 milliseconds If the system isn’t resilient to failures, its availability will inevitably drop. Because of that, a distributed system needs to embrace fail- ures and be prepared to withstand them using techniques such as redundancy, fault isolation, and self-healing mechanisms, which we will discuss in Part IV, Resiliency. 1.5 Maintainability It’s a well-known fact that the majority of the cost of software is spent after its initial development in maintenance activities, such as fixing bugs, adding new features, and operating it. Thus, we should aspire to make our systems easy to modify, extend and op- erate so that they are easy to maintain. Any change is a potential incident waiting to happen. Good test- ing — in the form of unit, integration, and end-to-end tests — is a 3 “AWS Well-Architected Framework, Availability,” https://docs.aws.amazon. com/wellarchitected/latest/reliability-pillar/availability.html
- 29. CHAPTER 1. INTRODUCTION 7 minimum requirement to modify or extend a system without wor- rying it will break. And once a change has been merged into the codebase, it needs to be released to production safely without af- fecting the system’s availability. Also, operators need to monitor the system’s health, investigate degradations and restore the service when it can’t self-heal. This requires altering the system’s behavior without code changes, e.g., toggling a feature flag or scaling out a service with a configuration change. Historically, developers, testers, and operators were part of differ- ent teams, but the rise of microservices and DevOps has changed that. Nowadays, the same team that designs and implements a sys- tem is also responsible for testing and operating it. That’s a good thing since there is no better way to discover where a system falls short than being on call for it. Part V will explore best practices for testing and operating distributed systems. 1.6 Anatomy of a distributed system Distributed systems come in all shapes and sizes. In this book, we are mainly concerned with backend applications that run on commodity machines and implement some kind of business service. So you could say a distributed system is a group of machines that communicate over network links. However, from a run-time point of view, a distributed system is a group of software processes that communicate via inter-process communication (IPC) mechanisms like HTTP. And from an implementation perspective, a distributed system is a group of loosely-coupled components (services) that communicate via APIs. All these are valid and useful architectural points of view. In the rest of the book, we will switch between them depending on which one is more appropriate to discuss a particular topic. A service implements one specific part of the overall system’s ca- pabilities. At the core of a service sits the business logic, which exposes interfaces to communicate with the outside world. Some
- 30. CHAPTER 1. INTRODUCTION 8 interfaces define the operations that the service offers to its users. In contrast, others define the operations that the service can invoke on other services, like data stores, message brokers, etc. Since processes can’t call each other’s interfaces directly, adapters are needed to connect IPC mechanisms to service interfaces. An inbound adapter is part of the service’s Application Programming Interface (API); it handles the requests received from an IPC mech- anism, like HTTP, by invoking operations defined in the service interfaces. In contrast, outbound adapters grant the business logic access to external services, like data stores. This architectural style is also referred to as the ports and adapters architecture4 . The idea is that the business logic doesn’t depend on technical details; in- stead, the technical details depend on the business logic (depen- dency inversion principle5 ). This concept is illustrated in Figure 1.2. Going forward, we will refer to a process running a service as a server, and a process sending requests to a server as a client. Some- times, a process will be both a client and a server. For simplic- ity, we will assume that an individual instance of a service runs entirely within a single server process. Similarly, we will also as- sume that a process has a single thread. These assumptions will allow us to neglect some implementation details that would only complicate the discussion without adding much value. 4 “Ports And Adapters Architecture,” http://wiki.c2.com/?PortsAndAdapter sArchitecture 5 “Dependency inversion principle,” https://en.wikipedia.org/wiki/Depend ency_inversion_principle
- 31. CHAPTER 1. INTRODUCTION 9 Figure 1.2: In this example, the business logic uses the reposi- tory interface, implemented by the PostgreSQL adapter, to access the database. In contrast, the HTTP adapter handles incoming re- quests by calling operations defined in the service interface.
- 35. Introduction “The network is reliable.” – Fallacies of distributed computing, L. Peter Deutsch Communication between processes over the network, or inter- process communication (IPC), is at the heart of distributed systems — it’s what makes distributed systems distributed. In order for processes to communicate, they need to agree on a set of rules that determine how data is processed and formatted. Network protocols specify such rules. The protocols are arranged in a stack6 , where each layer builds on the abstraction provided by the layer below, and lower layers are closer to the hardware. When a process sends data to another through the network stack, the data moves from the top layer to the bottom one and vice-versa at the other end, as shown in Figure 1.3: • The link layer consists of network protocols that operate on lo- cal network links, like Ethernet or Wi-Fi, and provides an in- terface to the underlying network hardware. Switches oper- ate at this layer and forward Ethernet packets based on their destination MAC address7 . • The internet layer routes packets from one machine to another across the network. The Internet Protocol (IP) is the core pro- tocol of this layer, which delivers packets on a best-effort ba- 6 “Internet protocol suite,” https://en.wikipedia.org/wiki/Internet_protocol_s uite 7 “MAC address,” https://en.wikipedia.org/wiki/MAC_address
- 36. 14 Figure 1.3: Internet protocol suite sis (i.e., packets can be dropped, duplicated, or corrupted). Routers operate at this layer and forward IP packets to the next router along the path to their final destination. • The transport layer transmits data between two processes. To enable multiple processes hosted on the same machine to communicate at the same time, port numbers are used to ad- dress the processes on either end. The most important proto- col in this layer is the Transmission Control Protocol (TCP), which creates a reliable communication channel on top of IP. • Finally, the application layer defines high-level communica- tion protocols, like HTTP or DNS. Typically your applica- tions will target this level of abstraction. Even though each protocol builds on top of another, sometimes the abstractions leak. If you don’t have a good grasp of how the lower layers work, you will have a hard time troubleshooting network- ing issues that will inevitably arise. More importantly, having an appreciation of the complexity of what happens when you make a network call will make you a better systems builder. Chapter 2 describes how to build a reliable communication chan- nel (TCP) on top of an unreliable one (IP), which can drop or dupli- cate data or deliver it out of order. Building reliable abstractions on top of unreliable ones is a common pattern we will encounter again in the rest of the book.
- 37. 15 Chapter 3 describes how to build a secure channel (TLS) on top of a reliable one (TCP). Security is a core concern of any system, and in this chapter, we will get a taste of what it takes to secure a network connection from prying eyes and malicious agents. Chapter 4 dives into how the phone book of the internet (DNS) works, which allows nodes to discover others using names. At its heart, DNS is a distributed, hierarchical, and eventually consistent key-value store. By studying it, we will get the first taste of even- tual consistency8 and the challenges it introduces. Chapter 5 concludes this part by discussing how loosely coupled services communicate with each other through APIs by describing the implementation of a RESTful HTTP API built upon the proto- cols introduced earlier. 8 We will learn more about consistency models in chapter 10.
- 39. Chapter 2 Reliable links At the internet layer, the communication between two nodes hap- pens by routing packets to their destination from one router to the next. Two ingredients are required for this: a way to address nodes and a mechanism to route packets across routers. Addressing is handled by the IP protocol. For example, IPv6 pro- vides a 128-bit address space, allowing 2128 addresses. To decide where to send a packet, a router needs to consult a local routing table. The table maps a destination address to the address of the next router along the path to that destination. The responsibility of building and communicating the routing tables across routers lies with the Border Gateway Protocol (BGP1 ). Now, IP doesn’t guarantee that data sent over the internet will arrive at its destination. For example, if a router becomes over- loaded, it might start dropping packets. This is where TCP2 comes in, a transport-layer protocol that exposes a reliable communica- tion channel between two processes on top of IP. TCP guarantees that a stream of bytes arrives in order without gaps, duplication, or corruption. TCP also implements a set of stability patterns to 1 “RFC 4271: A Border Gateway Protocol 4 (BGP-4),” https://datatracker.ietf.o rg/doc/html/rfc4271 2 “RFC 793: Transmission Control Protocol,” https://tools.ietf.org/html/rfc793
- 40. CHAPTER 2. RELIABLE LINKS 18 avoid overwhelming the network and the receiver. 2.1 Reliability To create the illusion of a reliable channel, TCP partitions a byte stream into discrete packets called segments. The segments are sequentially numbered, which allows the receiver to detect holes and duplicates. Every segment sent needs to be acknowledged by the receiver. When that doesn’t happen, a timer fires on the sending side and the segment is retransmitted. To ensure that the data hasn’t been corrupted in transit, the receiver uses a checksum to verify the integrity of a delivered segment. 2.2 Connection lifecycle A connection needs to be opened before any data can be transmit- ted on a TCP channel. The operating system manages the connec- tion state on both ends through a socket. The socket keeps track of the state changes of the connection during its lifetime. At a high level, there are three states the connection can be in: • The opening state in which the connection is being created. • The established state in which the connection is open and data is being transferred. • The closing state in which the connection is being closed. In reality, this is a simplification, as there are more states3 than the three above. A server must be listening for connection requests from clients be- fore a connection is established. TCP uses a three-way handshake to create a new connection, as shown in Figure 2.1: 1. The sender picks a random sequence number x and sends a SYN segment to the receiver. 3 “TCP State Diagram,” https://en.wikipedia.org/wiki/Transmission_Control _Protocol#/media/File:Tcp_state_diagram_fixed_new.svg
- 41. CHAPTER 2. RELIABLE LINKS 19 2. The receiver increments x, chooses a random sequence num- ber y, and sends back a SYN/ACK segment. 3. The sender increments both sequence numbers and replies with an ACK segment and the first bytes of application data. The sequence numbers are used by TCP to ensure the data is de- livered in order and without holes. Figure 2.1: Three-way handshake The handshake introduces a full round-trip in which no applica- tion data is sent. So until the connection has been opened, the bandwidth is essentially zero. The lower the round trip time is, the faster the connection can be established. Therefore, putting servers closer to the clients helps reduce this cold-start penalty. After the data transmission is complete, the connection needs to be closed to release all resources on both ends. This termi- nation phase involves multiple round-trips. If it’s likely that another transmission will occur soon, it makes sense to keep the connection open to avoid paying the cold-start tax again. Moreover, closing a socket doesn’t dispose of it immediately as it
- 42. CHAPTER 2. RELIABLE LINKS 20 transitions to a waiting state (TIME_WAIT) that lasts several min- utes and discards any segments received during the wait. The wait prevents delayed segments from a closed connection from be- ing considered part of a new connection. But if many connections open and close quickly, the number of sockets in the waiting state will continue to increase until it reaches the maximum number of sockets that can be open, causing new connection attempts to fail. This is another reason why processes typically maintain connec- tion pools to avoid recreating connections repeatedly. 2.3 Flow control Flow control is a backoff mechanism that TCP implements to prevent the sender from overwhelming the receiver. The receiver stores incoming TCP segments waiting to be processed by the application into a receive buffer, as shown in Figure 2.2. Figure 2.2: The receive buffer stores data that hasn’t yet been pro- cessed by the destination process. The receiver also communicates the size of the buffer to the sender whenever it acknowledges a segment, as shown in Figure 2.3.
- 43. CHAPTER 2. RELIABLE LINKS 21 Assuming it’s respecting the protocol, the sender avoids sending more data than can fit in the receiver’s buffer. Figure 2.3: The size of the receive buffer is communicated in the headers of acknowledgment segments. This mechanism is not too dissimilar to rate-limiting at the service level, a mechanism that rejects a request when a specific quota is exceeded (see section 28.3). But, rather than rate-limiting on an API key or IP address, TCP is rate-limiting on a connection level. 2.4 Congestion control TCP guards not only against overwhelming the receiver, but also against flooding the underlying network. The sender maintains a so-called congestion window, which represents the total number of outstanding segments that can be sent without an acknowledg- ment from the other side. The smaller the congestion window is, the fewer bytes can be in flight at any given time, and the less band- width is utilized.
- 44. CHAPTER 2. RELIABLE LINKS 22 When a new connection is established, the size of the congestion window is set to a system default. Then, for every segment ac- knowledged, the window increases its size exponentially until it reaches an upper limit. This means we can’t use the network’s full capacity right after a connection is established. The shorter the round-trip time (RTT), the quicker the sender can start utilizing the underlying network’s bandwidth, as shown in Figure 2.4. Figure 2.4: The shorter the RTT, the quicker the sender can start utilizing the underlying network’s bandwidth. What happens if a segment is lost? When the sender detects a missed acknowledgment through a timeout, a mechanism called congestion avoidance kicks in, and the congestion window size is reduced. From there onwards, the passing of time increases the window size4 by a certain amount, and timeouts decrease it by an- other. As mentioned earlier, the size of the congestion window defines the maximum number of bytes that can be sent without receiving 4 “CUBIC: A New TCP-Friendly High-Speed TCP Variant,” https://citeseerx.is t.psu.edu/viewdoc/download?doi=10.1.1.153.3152&rep=rep1&type=pdf
- 45. CHAPTER 2. RELIABLE LINKS 23 an acknowledgment. Because the sender needs to wait for a full round trip to get an acknowledgment, we can derive the maximum theoretical bandwidth by dividing the size of the congestion win- dow by the round trip time: Bandwidth = WinSize RTT The equation5 shows that bandwidth is a function of latency. TCP will try very hard to optimize the window size since it can’t do anything about the round-trip time. However, that doesn’t always yield the optimal configuration. Due to the way congestion control works, the shorter the round-trip time, the better the underlying network’s bandwidth is utilized. This is more reason to put servers geographically close to the clients. 2.5 Custom protocols TCP’s reliability and stability come at the price of lower band- width and higher latencies than the underlying network can deliver. If we drop the stability and reliability mechanisms that TCP provides, what we get is a simple protocol named User Datagram Protocol6 (UDP) — a connectionless transport layer protocol that can be used as an alternative to TCP. Unlike TCP, UDP does not expose the abstraction of a byte stream to its clients. As a result, clients can only send discrete packets with a limited size called datagrams. UDP doesn’t offer any relia- bility as datagrams don’t have sequence numbers and are not ac- knowledged. UDP doesn’t implement flow and congestion control either. Overall, UDP is a lean and bare-bones protocol. It’s used to bootstrap custom protocols, which provide some, but not all, of the stability and reliability guarantees that TCP does7 . 5 “Bandwidth-delay product,” https://en.wikipedia.org/wiki/Bandwidth- delay_product 6 “RFC 768: User Datagram Protocol,” https://datatracker.ietf.org/doc/html/ rfc768 7 As we will later see, HTTP 3 is based on UDP to avoid some of TCP’s short- comings.
- 46. CHAPTER 2. RELIABLE LINKS 24 For example, in multiplayer games, clients sample gamepad events several times per second and send them to a server that keeps track of the global game state. Similarly, the server samples the game state several times per second and sends these snapshots back to the clients. If a snapshot is lost in transmission, there is no value in retransmitting it as the game evolves in real-time; by the time the retransmitted snapshot would get to the destination, it would be obsolete. This is a use case where UDP shines; in contrast, TCP would attempt to redeliver the missing data and degrade the game’s experience.
- 47. Chapter 3 Secure links We now know how to reliably send bytes from one process to an- other over the network. The problem is that these bytes are sent in the clear, and a middleman could intercept the communication. To protect against that, we can use the Transport Layer Security1 (TLS) protocol. TLS runs on top of TCP and encrypts the communica- tion channel so that application layer protocols, like HTTP, can leverage it to communicate securely. In a nutshell, TLS provides encryption, authentication, and integrity. 3.1 Encryption Encryption guarantees that the data transmitted between a client and a server is obfuscated and can only be read by the communi- cating processes. When the TLS connection is first opened, the client and the server negotiate a shared encryption secret using asymmetric encryption. First, each party generates a key pair consisting of a private and public key. The processes can then create a shared secret by ex- changing their public keys. This is possible thanks to some math- 1 “RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3,” https: //datatracker.ietf.org/doc/html/rfc8446
- 48. CHAPTER 3. SECURE LINKS 26 ematical properties2 of the key pairs. The beauty of this approach is that the shared secret is never communicated over the wire. Although asymmetric encryption is slow and expensive, it’s only used to create the shared encryption key. After that, symmetric en- cryption is used, which is fast and cheap. The shared key is peri- odically renegotiated to minimize the amount of data that can be deciphered if the shared key is broken. Encrypting in-flight data has a CPU penalty, but it’s negligible since modern processors have dedicated cryptographic instruc- tions. Therefore, TLS should be used for all communications, even those not going through the public internet. 3.2 Authentication Although we have a way to obfuscate data transmitted across the wire, the client still needs to authenticate the server to verify it’s who it claims to be. Similarly, the server might want to authenti- cate the identity of the client. TLS implements authentication using digital signatures based on asymmetric cryptography. The server generates a key pair with a private and a public key and shares its public key with the client. When the server sends a message to the client, it signs it with its private key. The client uses the server’s public key to verify that the digital signature was actually signed with the private key. This is possible thanks to mathematical properties3 of the key pair. The problem with this approach is that the client has no idea whether the public key shared by the server is authentic. Hence, the protocol uses certificates to prove the ownership of a public key. A certificate includes information about the owning entity, expiration date, public key, and a digital signature of the third- party entity that issued the certificate. The certificate’s issuing 2 “A (Relatively Easy To Understand) Primer on Elliptic Curve Cryptography,” https://blog.cloudflare.com/a-relatively-easy-to-understand-primer-on-elliptic- curve-cryptography/ 3 “Digital signature,” https://en.wikipedia.org/wiki/Digital_signature
- 49. CHAPTER 3. SECURE LINKS 27 entity is called a certificate authority (CA), which is also represented with a certificate. This creates a chain of certificates that ends with a certificate issued by a root CA, as shown in Figure 3.1, which self-signs its certificate. For a TLS certificate to be trusted by a device, the certificate, or one of its ancestors, must be present in the trusted store of the client. Trusted root CAs, such as Let’s Encrypt4 , are typically included in the client’s trusted store by default by the operating system ven- dor. Figure 3.1: A certificate chain ends with a self-signed certificate issued by a root CA. When a TLS connection is opened, the server sends the full cer- tificate chain to the client, starting with the server’s certificate and 4 “Let’s Encrypt: A nonprofit Certificate Authority,” https://letsencrypt.org/
- 50. CHAPTER 3. SECURE LINKS 28 ending with the root CA. The client verifies the server’s certificate by scanning the certificate chain until it finds a certificate that it trusts. Then, the certificates are verified in reverse order from that point in the chain. The verification checks several things, like the certificate’s expiration date and whether the digital signature was actually signed by the issuing CA. If the verification reaches the last certificate in the path (the server’s own certificate) without er- rors, the path is verified, and the server is authenticated. One of the most common mistakes when using TLS is letting a certificate expire. When that happens, the client won’t be able to verify the server’s identity, and opening a connection to the re- mote process will fail. This can bring an entire application down as clients can no longer connect with it. For this reason, automa- tion to monitor and auto-renew certificates close to expiration is well worth the investment. 3.3 Integrity Even if the data is obfuscated, a middleman could still tamper with it; for example, random bits within the messages could be swapped. To protect against tampering, TLS verifies the integrity of the data by calculating a message digest. A secure hash function is used to create a message authentication code5 (HMAC). When a process receives a message, it recomputes the digest of the message and checks whether it matches the digest included in the message. If not, then the message has either been corrupted during trans- mission or has been tampered with. In this case, the message is dropped. The TLS HMAC protects against data corruption as well, not just tampering. You might be wondering how data can be corrupted if TCP is supposed to guarantee its integrity. While TCP does use a checksum to protect against data corruption, it’s not 100% reliable6 : 5 “RFC 2104: HMAC: Keyed-Hashing for Message Authentication,” https://da tatracker.ietf.org/doc/html/rfc2104 6 “When the CRC and TCP checksum disagree,” https://dl.acm.org/doi/10.11 45/347057.347561
- 51. CHAPTER 3. SECURE LINKS 29 it fails to detect errors for roughly 1 in 16 million to 10 billion pack- ets. With packets of 1 KB, this is expected to happen once per 16 GB to 10 TB transmitted. 3.4 Handshake When a new TLS connection is established, a handshake between the client and server occurs during which: 1. The parties agree on the cipher suite to use. A cipher suite specifies the different algorithms that the client and the server intend to use to create a secure channel, like the: • key exchange algorithm used to generate shared secrets; • signature algorithm used to sign certificates; • symmetric encryption algorithm used to encrypt the ap- plication data; • HMAC algorithm used to guarantee the integrity and authenticity of the application data. 2. The parties use the key exchange algorithm to create a shared secret. The symmetric encryption algorithm uses the shared secret to encrypt communication on the secure channel going forward. 3. The client verifies the certificate provided by the server. The verification process confirms that the server is who it says it is. If the verification is successful, the client can start send- ing encrypted application data to the server. The server can optionally also verify the client certificate if one is available. These operations don’t necessarily happen in this order, as modern implementations use several optimizations to reduce round trips. For example, the handshake typically requires 2 round trips with TLS 1.2 and just one with TLS 1.3. The bottom line is that creating a new connection is not free: yet another reason to put your servers geographically closer to the clients and reuse connections when possible.
- 53. Chapter 4 Discovery So far, we have explored how to create a reliable and secure chan- nel between two processes running on different machines. How- ever, to create a new connection with a remote process, we must first discover its IP address somehow. The most common way of doing that is via the phone book of the internet: the Domain Name System1 (DNS) — a distributed, hierarchical, and eventually con- sistent key-value store. In this chapter, we will look at how DNS resolution2 works in a browser, but the process is similar for other types of clients. When you enter a URL in your browser, the first step is to resolve the hostname’s IP address, which is then used to open a new TLS con- nection. For example, let’s take a look at how the DNS resolution works when you type www.example.com into your browser (see Fig- ure 4.1). 1. The browser checks its local cache to see whether it has re- solved the hostname before. If so, it returns the cached IP address; otherwise, it routes the request to a DNS resolver, 1 “RFC 1035: Domain Names - Implementation and Specification,” https://da tatracker.ietf.org/doc/html/rfc1035 2 “A deep dive into DNS,” https://www.youtube.com/watch?v=drWd9HIhJ dU
- 54. CHAPTER 4. DISCOVERY 32 a server typically hosted by your Internet Service Provider (ISP). 2. The resolver is responsible for iteratively resolving the host- name for its clients. The reason why it’s iterative will become obvious in a moment. The resolver first checks its local cache for a cached entry, and if one is found, it’s returned to the client. If not, the query is sent to a root name server (root NS). 3. The root name server maps the top-level domain (TLD) of the request, i.e., .com, to the address of the name server responsi- ble for it. 4. The resolver sends a resolution request for example.com to the TLD name server. 5. The TLD name server maps the example.com domain name to the address of the authoritative name server responsible for the domain. 6. Finally, the resolver queries the authoritative name server for www.example.com, which returns the IP address of the www hostname. If the query included a subdomain of example.com, like news.example.com, the authoritative name server would have returned the address of the name server responsible for the subdomain, and an additional request would be required. The original DNS protocol sent plain-text messages primarily over UDP for efficiency reasons. However, because this allows any- one monitoring the transmission to snoop, the industry has mostly moved to secure alternatives, such as DNS on top of TLS3 . The resolution process involves several round trips in the worst case, but its beauty is that the address of a root name server is all that’s needed to resolve a hostname. That said, the resolution would be slow if every request had to go through several name 3 “RFC 7858: Specification for DNS over Transport Layer Security (TLS),” https: //en.wikipedia.org/wiki/DNS_over_TLS
- 55. Another Random Document on Scribd Without Any Related Topics
- 56. Finns—further, the “Gautigoth,” generally taken for the West Göter, who were a specially “brave and warlike people,” the “Ostrogothæ” [East Göter] and many more. Then he crosses the Norwegian frontier and mentions “The ‘Raumarici’[of Romerike] and ‘Ragnaricii’ [of Ranrike or Bohuslen], the very mild [peaceful] ‘Finns’ (‘Finni mitissimi’), who are milder than all the other inhabitants of Scandza;[136] further their equals the ‘Vinoviloth’; the ‘Suetidi’ are known among this people [‘hac gente’ must doubtless mean the Scandinavians] as towering above the rest in bodily height, and yet the ‘Danes,’ who are descended from this very race [i.e., the Scandinavians ?] drove out the ‘Heruli’ from their own home, who claimed the greatest fame [i.e., of being the foremost] among the peoples [‘nationes’] of Scandia for very great bodily size. Yet of the same height as these are also the ‘Granii’ [of Grenland, the coast-land of Bratsberg and Nedenes], the ‘Augandzi’ [people of Agder],[137] ‘Eunix’ [islanders, Holmryger in the islands ?], ‘Ætelrugi’ [Ryger on the mainland in Ryfylke], ‘Arochi’ [== ‘arothi,’ i.e., Harudes, Horder of Hordaland], ‘Ranii’ [in other MSS. ‘Rannii’ or ‘Rami,’ Sophus Bugge (1907) and A. Bugge see in this a corruption of ‘*Raumi,’ that is, people of Romsdal], over whom not many years ago Roduulf was king, who, despising his own kingdom, hastened to the arms of Theodoric king of the Goths, and found what he had hankered after. These people fight with the savageness of beasts, more mighty than the Germans in body and soul.” The small (?), “very mild” Finns must, from the order in which they are named, have lived in the forest districts—Solör, Eidskogen, and perhaps farther south—on the Swedish border. P. A. Munch [1852, p. 83] saw in their kinsmen the “Vinoviloth” the inhabitants of “Vingulmark” (properly “vingel-skog,” thick, impenetrable forest), which was the forest country on Christiania fjord from Glommen to Lier. Müllenhoff agrees with this [ii., 1887, pp. 65 f.], but thinks that “-oth,” the last part of the word, belongs to the next name, Suetidi, and that “Vinovil” may be a corruption of Vingvili or Vinguli (cf.
- 57. Paulus Warnefridi’s “Vinili” ?). But however this may be, we must regard this people and the foregoing as “Finnish” and as inhabiting forest districts, as hunters, as well as a third Finnish people, “Finnaithæ” in Småland. We shall return later to these “Finns” in Scandinavia. It has been thought that “Suetidi” may be from the same word as “Sviþjoð”; but as Jordanes has already mentioned the Svear (“Suehans”), and as the name occurs among the Norwegian tribes, and there is evidently a certain order in their enumeration, Müllenhoff may be right in seeing in it a corruption of a Norwegian tribal name. He thinks that “Othsuetidi” may be a corruption of “Æthsævii,” i.e., “Eiðsivar” (cf. Eidsivathing), “Heiðsævir” or “Heiðnir” in Hedemarken, who were certainly a very tall people. The mention of the Norwegian warriors has a certain interest in that it is due to the Roman statesman Cassiodorus (or his authority), who glorified the Goths and had no special reason for praising the Northmen.[138] It shows that even at that time our northern ancestors were famed for courage and bodily size, and that too above all other Germanic peoples, who were highly esteemed by the Romans. It is not clear whether Rodulf was King of the “Ranii” (Raumer ?) alone, or of all the Norwegian tribes from Grenland to Romsdal. It may be supposed that he was a Norwegian chief who migrated south through Europe at the head of a band of warriors, composed of men from the tribes mentioned, and that finally on the Danube, hard pressed by other warlike people, he sought alliance and support from the mighty king of the Goths, Theodoric or Tjodrik (Dietrich of Berne). This may have been just before 489, when the latter made his expedition to Italy. Many circumstances combine to make such a hypothesis probable. [139] We know that about 489 the Eruli were just north of the Danube, and were the Goths’ nearest neighbours. Now, as we shall see later, Eruli was perhaps at first a common name for bands of northern warriors, and these Eruli on the Danube may therefore certainly have consisted to a greater or less extent of Norwegians. We know, further, that at this time there was a king of the Eruli to whom Theodoric sent as a gift a horse, sword and shield, thereby making
- 58. him his foster-son [cf. Cassiodorus, Varia iii. 3, iv. 2]. Finally, we know from Procopius that the Eruli just at this time had a king, Rodulf, who fell in battle against the Langobards (about 493). When we compare this with what Jordanes says about the Norwegian king Rodulf, who hastened to Theodoric’s arms and found there what he sought, it will be easy to conclude that this Norwegian chief is the same as the chief of Eruli here spoken of. Rodulf, or “Hrodulfr,” is a known Norwegian name. “Rod-,” or “Hrod,” is the same as the modern Norwegian “ros” (i.e., praise), and means probably here renowned. One is further inclined to believe that it was from this Rodulf or his men, of whom some may have come from And in Hålogaland, that Cassiodorus or his authority obtained the information about Scandinavia and northern Norway, which is also partly repeated in Procopius. Sophus Bugge [cf. 1910, pp. 87 ff.; see also A. Bugge, 1906, pp. 35 f.] has suggested that the “Ráðulfr,” who is mentioned in the runic inscription on the celebrated Rök-stone in Östergötland (of about the year 900), in which Theodoric (“Þiaurikr”) is also mentioned, may be the same Norwegian chief Rodulf who came to Theodoric and who fell in battle with the Langobards. He even regards it as possible that it is an echo of this battle which is found in the inscription, where it is said that “twenty kings lie slain on the field”; in that case the battle has been moved north from the Danube to “Siulunt” (i.e., Sealand). There are other circumstances which agree with this: it is said of the Eruli that they had peace for three years before the battle [cf. Procopius]; on the Rök-stone it is stated that the twenty kings stayed in Siulunt four winters; the latter must have been Norwegian warriors of different tribes: Ryger, Horder, and Heiner (from Hedemarken), perhaps under a paramount king Ráðulfr, who settled in Sealand—while the Eruli were bands of northern warriors, who under a king Rodulf had established themselves on the north bank of the Danube. Bugge’s supposition may be uncertain, but if it be correct it greatly strengthens the view (see p. 145) that the Eruli
- 59. were largely Norwegian warriors, since in that case the king of the Eruli, Rodulf (== Ráðulfr), would have been in command of tribes for the most part Norwegian: Ryger, Horder, and Heiner. Procopius, circa 552 A.D. The Byzantine historian Procopius, of Cæsarea (ob. after 562), became in 527 legal assistant, “assessor,” to the general Belisarius, and accompanied him on his campaigns until 549, amongst others that against the Goths in Italy. In his work (in Greek) on the war against the Goths (“De bello Gothico,” t. ii. c. 14 and 15), written about 552, he gives information about the North which is of great interest. He tells us of the warlike Germanic people, the Eruli, who from old time[140] were said to have lived on the north bank of the Danube, and who, with no better reason than that they had lived in peace for three whole years and were tired of it, attacked their neighbours the Langobards, but suffered a decisive defeat, and their king, Rodulf, fell in the battle (about 493).[141] “They then hastily left their dwelling-places, and set out with their women and children to wander through the whole country [Hungary] which lies north of the Danube. When they came to the district where the Rogians had formerly dwelt, who had joined the army of the Goths and gone into Italy, they settled there; but as they were oppressed by famine in that district, which had been laid waste, they soon afterwards departed from it, and came near to the country of the Gepidæ [Siebenbürgen]. The Gepidæ allowed them to establish themselves and to become their neighbours, but began thereupon, without the slightest cause, to commit the most revolting acts against them, ravishing their women, robbing them of cattle and other goods, and omitting no kind of injustice, and finally began an unjust war against them.” The Eruli then crossed the Danube to Illyria and settled somewhere about what is now Servia under the eastern emperor Anastasius (491-518). Some of the Eruli would not “cross the Danube, but decided to establish themselves in the uttermost ends of the inhabited world. Many chieftains of royal blood
- 60. now undertaking their leadership, they passed through all the tribes of the Slavs one after another, went thence through a wide, uninhabited country, and came to the so-called Varn. Beyond them they passed by the tribes of the Danes [in Jutland], without the barbarians there using violence towards them. When they thence came to the ocean [about the year 512] they took ship, and landed on the island of Thule [i.e., Scandinavia] and remained there. But Thule is beyond comparison the largest of all islands; for it is more than ten times as large as Britain. But it lies very far therefrom northwards. On this island the land is for the most part uninhabited. But in the inhabited regions there are thirteen populous tribes, each with a king. Every year an extraordinary thing takes place; for the sun, about the time of the summer solstice, does not set at all for forty days, but for the whole of this time remains uninterruptedly visible above the earth. No less than six months later, about the winter solstice, for forty days the sun is nowhere to be seen on this island; but continual night is spread over it, and therefore for the whole of that time the people are very depressed, since they can hold no intercourse. It is true that I have not succeeded, much as I should have wished it, in reaching this island and witnessing what is here spoken of; but from those who have come thence to us I have collected information of how they are able [to count the days] when the sun neither rises nor sets at the times referred to,” etc. When, during the forty days that it is above the horizon, the sun in its daily course returns “to that place where the inhabitants first saw it rise, then according to their reckoning a day and a night have passed. But when the period of night commences, they find a measure by observation of the moon’s path, according to which they reckon the number of days. But when thirty-five days of the long night are passed, certain people are sent up to the tops of mountains, as is the custom with them, and when from thence they can see some appearance of the sun, they send word to the inhabitants below that in five days the sun will shine upon them. And the latter assemble and celebrate, in the dark it is true, the feast of the glad tidings. Among the people of Thule this is the greatest of all their festivals. I believe that these islanders, although the same thing happens every
- 61. year with them, nevertheless are in a state of fear lest some time the sun should be wholly lost to them. “Among the barbarians inhabiting Thule, one people, who are called Skridfinns [Scrithifini], live after the manner of beasts. They do not wear clothes [i.e., of cloth] nor, when they walk, do they fasten anything under their feet, [i.e., they do not wear shoes], they neither drink wine nor eat anything from the land, because they neither cultivate the land themselves nor do the women provide them with anything from tilling it, but the men as well as the women occupy themselves solely and continually in hunting; for the extraordinarily great forests and mountains which rise in their country give them vast quantities of game and other beasts. They always eat the flesh of the animals they hunt and wear their skins, and they have no linen or anything else that they can sew with. But they fasten the skins together with the sinews of beasts, and thus cover their whole bodies. The children even are not brought up among them as with other peoples; for the Skridfinns’ children do not take women’s milk, nor do they touch their mothers’ breasts, but they are nourished solely with the marrow of slain beasts. As soon therefore as a woman has given birth, she winds the child in a skin, hangs it up in a tree, puts marrow into its mouth, and goes off hunting; for they follow this occupation in common with the men. Thus is the mode of life of these barbarians arranged. “Nearly all of the remaining inhabitants of Thule do not, however, differ much from other peoples. They worship a number of gods and higher powers in the heavens, the air, the earth and the sea, also certain other higher beings which are thought to dwell in the waters of springs and rivers. But they always slay all kinds of sacrifice and offer dead sacrifices. And to them the best of all sacrifices is the man they have taken prisoner by their arms. Him they sacrifice to the god of war, because they consider him to be the greatest. But they do not sacrifice him merely by using fire at the sacrifice; they also hang him up in a tree, or throw him among thorns, and slay him by other cruel modes of death. Such is the life of the inhabitants
- 62. of Thule, among whom the most numerous people are the Gauti (Göter), with whom the immigrant Eruli settled.” Erulian sources of Procopius Common source of Procopius and Jordanes This description by Procopius of Thule (Scandinavia) and its people bears the stamp of a certain trustworthiness. If we ask whence he has derived his information, our thoughts are led at once to the Eruli, referred to by him in such detail, who in part were still the allies of the Eastern Empire, and of whom the emperor at Byzantium had a bodyguard in the sixth century. There were many of them in the army of the Eastern Empire both in Persia and in Italy; thus Procopius says that there were two thousand of them in the army under the eunuch Narses, which came to Italy to join Belisarius. Procopius thus had ample opportunity for obtaining first-hand information from these northern warriors, and his account of them shows that the Eruli south of the Danube kept up communication with their kinsmen in Scandinavia, for when they had killed their king “Ochon” without cause, since they wished to try being without a king, and had repented the experiment, they sent some of their foremost men to Thule to find a new king of the royal blood. They chose one and returned with him; but he died on the way when they had almost reached home, and they therefore turned again and went once more to Thule. This time they found another, “by name ‘Datios’ [or ‘Todasios’ == Tjodrik ?]. He was accompanied by his brother ‘Aordos’ [== Vard ?] and two hundred young men of the Eruli in Thule.” Meanwhile, as they were so long absent, the Eruli of Singidunum (the modern Belgrade) had sent an embassy to the emperor Justinianus at Byzantium asking him to give them a chief. He sent, therefore, the Erulian “Svartuas” (== Svartugle, i.e., black owl ?), who had been living with him for a long time. But when
- 63. Datios from Thule approached, all the Eruli went over to him by night, and Svartuas had to flee quite alone, and returned to Byzantium. The emperor now exerted all his power to reinstate him; “but the Eruli, who feared the power of the Romans, decided to migrate to the Gepidæ.” This happened in Procopius’s own time, and may therefore be regarded as trustworthy; it shows how easy communication must have been at that time between Scandinavia and the south, and also with Byzantium, so that Procopius may well have had his information by that channel. But he may also have received information from another quarter. His description of Thule shows such decided similarities with Jordanes’ account of Scandza and its people that they point to some common source of knowledge, even though there are also dissimilarities. Among the latter it may be pointed out that Jordanes makes a distinction between Thule (north of Britain) and Scandza, while Procopius calls Scandinavia Thule, which, however, like Jordanes, he places to the north of Britain, and he does not mention Scandia. It may seem surprising that Jordanes’ authority, Cassiodorus (or Ablabius ?), should have known Ptolemy better than the Greek Procopius. The explanation may be that when Procopius heard from the statements of the Eruli themselves that some of them had crossed the ocean from the land of the Danes (Jutland) to a great island in the north, he could not have supposed that this was Scandia, which on Ptolemy’s map lay east of the Cimbrian peninsula and farther south than its northern point; it would seem much more probable that it was Thule, which, however, as he saw, must lie farther from Britain and be larger than it was shown on Ptolemy’s map; for which reason Procopius expressly asserts that Thule was much larger than Britain and lay far to the north of it. As it was not Procopius’s habit to make a show of unnecessary names, he keeps the well-known name of Thule and does not even mention Scandia. It may even be supposed that it was to west Norway itself, or the ancient Thule, that the Eruli sailed. If their king Rodulf was a Norwegian, as suggested above, this would be probable, as in that case many of themselves would have come from there too; besides which, we know of a people, the Harudes or Horder, who had formerly migrated by sea from Jutland
- 64. to the west coast of Norway; there had therefore been an ancient connection, and perhaps, indeed, Horder from Norway and Harudes from Jutland may have been among Rodulf’s men, and there may also have been Harudes among the Eruli whom the Danes, according to Jordanes, drove out of their home (in Jutland ?). There was also, from the very beginning of Norwegian history, much connection between Norway and Jutland. Another disagreement between the descriptions of Procopius and Jordanes is that according to the former there were thirteen tribes, each with a king, in Thule, while Jordanes enumerates twice as many tribal names in Scandza, but of these perhaps several may have belonged to the same kingdom.[142] A remarkable similarity between the two authors is the summer day forty days long and the equally long winter night among the people of Thule as with the Adogit, and the fact that in immediate connection therewith the Scrithifini and Screrefennæ, which must originally be the same name, are mentioned. The description in Procopius of festivals on the reappearance of the sun, etc., points certainly to information from the North; but, as already pointed out, the statement in this form, that the summer day was of the same length as the winter night, cannot be due to the Norsemen themselves; it is a literary invention, which points to a common literary origin; for it would be more than remarkable if it had arisen independently both with the authority of Procopius and with that of Jordanes. An even more striking indication in the same direction is the resemblance which we find in the order of the two descriptions of Thule and of Scandza. First comes the geographical description of the island, which in both is of very great size and lies far out in the northern ocean; then occurs the statement that in this great island are many tribes.[143] Next we have in both the curious fact that the summer day and the winter night both last for forty days. Then follows in both a more detailed statement of how the long summer day and winter night come about, and of how the sun behaves during its course, etc. Immediately after this comes the description
- 65. of the Skridfinns, who have a bestial way of life, and do not live on corn, but on the flesh of wild beasts, etc., with an addition in Jordanes about fen-fowl’s eggs (perhaps taken from Mela), while Procopius has a more detailed description of their mode of life, which reminds one somewhat of Tacitus. Finally, there is a reference to the Germanic people of Thule or Scandza; but while Procopius mentions their religious beliefs and human sacrifices, and only gives the name of the most numerous tribe, the Gauti, Jordanes has for the most part a rigmarole of names. Even if the method of treating the material is thus very different in the two works, the order in which the material is arranged, and to some extent also the material itself, are in such complete agreement that there must be a historical connection, and undoubtedly a common literary source, through a greater or less number of intermediaries, is the basis of both descriptions. One might think of the unknown Ablabius, or perhaps of the unknown Gothic scholar Aithanarit, whom the Ravenna geographer mentions in connection with his reference to the Skridfinns, if indeed he did not live later than Procopius. It is striking also that the passage about Thule in Procopius gives rather the impression of having been inserted in the middle of his narrative about the Eruli, without any very intimate connection therewith, and it may therefore be for the most part taken from an earlier author, perhaps with alterations and additions by Procopius himself; but it is not his habit to inform us of his authorities. The Eruli are Norsemen Procopius’s description of the Eruli is of great interest. It is a remarkable feature in the history of the world that at certain intervals, even from the earliest times, roving warrior peoples appear in Europe, coming from the unknown North, who for a time fill the world with dread, and then disappear again. One of these northern peoples was perhaps, as already mentioned, the “Cimmerians,” who
- 66. in the eighth century B.C. made an inroad into Asia Minor. Six hundred years later, in the second century B.C., bands of Cimbri and Teutones came down from northern Europe and were pressing towards Rome, till they were defeated by Marius and gradually disappeared. Five hundred years later still, in the third to the fifth centuries A.D., the Eruli come on the scene, and after they have disappeared come the Saxons and Danes, and then the Normans. We may perhaps suppose, to a certain extent at all events, that the races which formed these restless and adventurous bands were in part the same, and that it is the names that have changed. The Eruli are also mentioned by Jordanes and by many other authorities besides Procopius. Together with the Goths they played a part in the “Scythian” war in the third century, but afterwards disappear to the north of the Black Sea. They must have been the most migratory people of their time; we find them roaming over the whole of Europe, from Scandinavia on the north to Byzantium on the south, from the Black Sea on the east to Spain on the west; from the third to the fifth century we find Eruli from Scandinavia as pirates on the coasts of western Europe, and even in the Mediterranean itself, where in 455 they reached Lucca in Italy [cf. Zeuss, 1837, p. 477 f.; Müllenhoff, 1889, p. 19]. When we read in Procopius that some of the Eruli would not “cross the Danube, but determined to establish themselves in the uttermost ends of the world,” this means, of course, that they had come from thence, and that rather than be subject to the Eastern Empire they would return home to Scandinavia. The name also frequently appears in its primitive Norse form, “erilar,“ in Northern runic inscriptions.[144] Since ”erilar” (in Norwegian “jarl,” in English “earl”) means leader in war, and is not known in Scandinavia as the original name of a tribe which has given its name to any district in the North, we must suppose that it was more probably an appellative in use in the more southern parts of Europe for bands of northern warriors of one or more Scandinavian tribes [cf. P. A. Munch, 1852, p. 53]. They may have called themselves so; it was, in fact, characteristic of the Scandinavian warrior that he was not disposed to acknowledge any superior; they were all free men and chiefs in contradistinction to thralls. Gradually
- 67. these bands in foreign countries may have coalesced into one nation [cf. A. Bugge, 1906, p. 32]. But as expeditions of Eruli are spoken of in such widely different parts of Europe, the name must, up to the end of the fifth century, have often been used for Norsemen in general, to distinguish them from the nations of Germany, like the designation Normans, and sometimes also Danes, in later times. That the latter was used as an appellative as early as the time of Procopius seems to result from his mentioning the tribes (“ethne”) of the Danes in just the same way as he speaks of those of the Slavs. What is said about the Eruli suits the Scandinavians: they were very tall (cf. Jordanes, above, p. 136) and fair, were specially famed for their activity, and were lightly armed; they went into battle without helmet or coat of mail, protected only by a shield and a thick tunic, which they tucked up into a belt. Their thralls, indeed, had to fight without shields; but when they had shown their courage they were allowed to carry a shield [Procopius, De bello Pers., ii. 25]. “At that time,” says Jordanes, “there was no nation that had not chosen the light-armed men of its army from among them. But if their activity had often helped them in other wars, they were vanquished by the slow steadiness of the Goths,” and they had to submit to Hermanaric, King of the Goths by the Black Sea, the same who is called Jörmunrek in the Völsunga Saga. The people here described can scarcely have been typical dwellers in plains, who are usually slow and heavy; we should rather think of them as tough and active Scandinavian mountaineers, who by their hard life in the hills had become light of foot and practised in the use of their limbs; but who, on the other hand, had been ill-supplied with heavier weapons and had had scant opportunities of exercise as heavy-armed men, for which indeed they had no taste. This also explains their remarkable mobility. We are thus led once more to think of Norway as the possible home of some of the Eruli. To sum up, we find then that they had a king with the Norse name Rodulf, and there are many indications that he was the same as the Norwegian king Rodulf (from Romsdal ?) who came to Theodoric. They returned through Jutland and sailed thence to Thule, where they settled by the side of the Gauti, i.e., to the west of them in Norway, which from old time
- 68. had had frequent communication with Jutland, from whence the Horder (and probably also the Ryger ?) had immigrated. They are described as having characteristics which are typical of mountaineers, but not of lowlanders. An Erulian name, “Aruth” (Ἀρουθ), mentioned by Procopius [De bello Goth., iv. 26], also points to Norway, since it appears to be the same as the Norwegian tribal name “Horder” (“*Haruðr,” gen. “Haruþs,” on the Rök-stone [cf. S. Bugge, 1910, p. 98], or “Arothi” in Jordanes). Other Erulian names in Procopius may be common to the northern Germanic languages. In the opinion of Professor Alf Torp it is probable that “Visandos” is bison, “Aluith” is Alvid or Alvith (all- knowing); in “Fanitheos” the first syllable may be “fan” or “fen” (English, fen) and the second part “-theos” may be the Scandinavian termination “-ther”; “Aordos” may be Vard. The King’s name “Ochon” seems to resemble the Norwegian Håkon; but the latter name cannot have had such a form at that time, it must have been longer. What Procopius tells us [De bello Goth., ii. 14] about the manners and customs of the Eruli agrees with what we know of the Norsemen generally. They worshipped many gods, whom they considered it their sacred duty to propitiate with human sacrifices. Aged and sick persons were obliged to ask their relatives to help them to get rid of life;[145] they were killed with a dagger by one who did not belong to the family, and were burnt on a great pile, after which the bones were collected and buried, as was the custom in western Norway amongst other places. “When an Erulian died, his wife, if she wished to show her virtue and leave a good name behind her, had to hang herself not long after with a rope by her husband’s grave and thus make an end of herself. If she did not do this, she lost respect for the future, and was an offence to her husband’s family. This custom was observed by the Eruli from old time.” Their many gods and human sacrifices agree, as we see, with Procopius’s description of the inhabitants of Thule, and with what we know of the Scandinavians from other quarters. As human sacrifices with most peoples were connected with banquets, at which slain enemies
- 69. were eaten,[146] the assertion that our Germanic ancestors did not practise cannibalism rests upon uncertain ground. When, therefore, in finds of the Stone Age in Denmark, Sweden and Norway broken or scraped human bones occur, which point to cannibalism, it cannot be argued from this, as is done by Dr. A. M. Hansen [1907], that the finds belong to a non-Germanic people. For the rest, Procopius paints the Eruli in crude colours; they are covetous, domineering and violent towards their fellow men, without being ashamed of it. They are addicted to the grossest debauchery, are the most wicked of men, and utterly depraved. Skridfinns The “Scrithifini” of Procopius (and Jordanes’ corrupted form, “Screrefennæ” or “Scretefennæ”) are undoubtedly a people of the same kind as Tacitus’s “Fenni” (Ptolemy-Marinus’s “Finni”); but they have here acquired the descriptive prefix “scrithi-,” which is generally understood as the Norse “skriða” (== to slide, e.g., on the ice, to glide; cf. Swedish “skridsko,” skate). The Norsemen must have characterised their Finnish (i.e., Lappish) neighbours on the north as sliding (walking) on ski (“skriða á skiðum”), to distinguish them from other peoples in the outlying districts whom they also called Finns. If this is so, it is the first time that a reference to ski-running is found in literature. There is, moreover, considerable similarity between Procopius’s description of these hunters and Tacitus’s account of the “Fenni,” who must certainly also have lived in Scandinavia (see above, p. 113), and who may have been the same people. They have many peculiar characteristics in common, e.g., that both men and women go hunting; and the statement that while the mothers go hunting, the children, in Tacitus, are hidden in a shelter of boughs (i.e., a tent), and in Procopius are hung up in a tree (perhaps the Lapps’ “komse,” i.e., a cradle made of wood to hang up in the tent). Procopius himself probably did not know Tacitus’s “Germania,” but it is possible that his unknown authority did so, although this work was generally forgotten at that time. But even if the description of Procopius may thus be partly derived from Tacitus,
- 70. in any case fresh information has been added, the name Skridfinns itself to begin with, and certain correct details, such as their fastening the skins together with the sinews of beasts. The fable that the children did not touch their mothers’ breasts may (like the masculine occupation of the women) be due to legends about the Amazons, who were not brought up on their mothers’ milk. That the children were given marrow instead may be due to the fact that this people of hunters, like the Lapps of the present day, ate much animal fat and marrow. The Eskimo often give their children raw blubber to chew. Map of the world in the MS. of Isidore, tenth century, St. Gallen (K. Miller)
- 71. The oldest known map of the world, from the MS. of Isidore of the end of the seventh century, St. Gallen (K. Miller) Isidorus Hispalensis , before 636 A.D. Thus while valuable information about the North is to be found in the early mediæval authors we have mentioned, this is not the case with the well-known Isidorus Hispalensis of Seville (ob. 636, as bishop of that city), who, however, exercised the greatest influence on the geographical ideas of the Middle Ages. His geographical knowledge was derived from late Latin authors, especially Orosius, Hieronymus and Solinus, and contributed nothing new of value. But as he was one of the most widely read authors of the early Middle Ages, he is of importance for having in that dark time continued the thread of the learning of antiquity, even though that thread was thin
- 72. and weak. He was also to have an influence on cartography. With his fondness for bad etymological interpretations he derived the word “rotunditas,” for the roundness of the earth, from “rota,” wheel, and he taught that “the word ‘orbis’ is used on account of the roundness of the circumference, since it is like a wheel. For in every part the circumfluent ocean surrounds its borders in a circle.” Hence the conception of the earth’s disc as a wheel came to be general in the early Middle Ages, and hence the designation of wheel-maps. Isidore divided the earth’s disc into three parts, Asia (including Paradise) at the top of the wheel-map, and Europe and Africa, also called Lybia, at the bottom; and the boundaries between these continents formed a T with the rivers Tanais and Nile horizontally at the top, and the Mediterranean (“Mare Magnum”) below. Therefore maps of this type, which was maintained for a long time, are also called T-maps.[147] Otherwise Isidore declared clearly enough in favour of the spherical form of the earth. Bede, 673- 735 The Anglo-Saxon monk and scholar, Beda Venerabilis (673-735), who in his work “Liber de natura rerum” also mentions the countries of the earth, but without making any fresh statement about the North, was strongly influenced by Isidore. He asserts, however, the spherical form of the earth in an intelligent way, giving, amongst other reasons, that of the ancient Greeks, that earth and water are attracted towards a central point. The form of a sphere was also the only one that would explain why certain stars were visible in the north, but not in the south.
- 73. Europe on the reconstructed map of the world of the Ravenna geographer (after K. Miller) The Ravenna geographe r, seventh century A few new facts about the North are to be found in the anonymous author who wrote a cosmography at the close of the seventh century. As, according to his own statement, he was born at Ravenna, he is usually known as the Ravenna geographer, but otherwise nothing is known of him, except that he was probably a
- 74. priest. He bases his work on older authors; the Bible, some Latin, some Greek, and some later writers; but he certainly had a Roman itinerary map like the Tabula Peutingeriana. His statements about the North are in part taken from Jordanes, but he also quotes three other “Gothic scholars,” who are otherwise entirely unknown. One of them, Aithanarit (or Athanaric ?), is mentioned particularly in connection with the Skridfinns. The other two, Eldevaldus (or Eldebald ?) and Marcomirus (or Marcomeres ?), have also described western Europe; the latter is specially used in the description of the countries of the Danes, Saxons and Frisians. The Ravenna geographer regarded the earth’s disc as approximately round, and surrounded by ocean, but the latter was not entirely continuous, for it did not extend behind India. It was true that some cosmographers had described it so, but no Christian ought to believe this, for Paradise was in the extreme East, near to India; and as the pollen is wafted by the breath of the wind from the male palm to the female near it, so does a beneficent perfume from Paradise blow upon the aromatic flowers of India. Some thought that the sun in its course returned to the east under the depths of ocean; but the Ravenna geographer agreed with those who said that the sun moved all night along paths which cannot be traced, behind lofty mountains, in the north beyond the ocean, and in the morning it came forth again from behind them. [iv. 12.] “In a line with Scythia and the coast of the ocean is the country which is said to be that of the ‘Rerefeni’ and ‘Sirdifeni’ (‘Scirdifrini’). The people of this country, according to what the Gothic scholar Aithanarit says, dwell among the rocks of the mountains, and both men and women are said to live by hunting, and to be entirely unacquainted both with meat and wine. This land is said to be colder than all others. Farther on by the side of the Serdifenni on the coast of the ocean is the land which is called Dania; this land, as the above-mentioned Aithanaridus and Eldevaldus and Marcomirus, the Gothic scholars, say, produces people who are swifter than all others.” [These must be the Eruli.]
- 75. “This Dania is now called the land of the Nordomanni.” This is the first time the name Norman is used, so far as is known. [v. 30.] “In the northern ocean itself, after the land of the Roxolani, is an island which is called Scanza, which is also called Old Scythia by most cosmographers. But in what manner the island of Scanza itself lies, we will with God’s help relate.” He says, following Jordanes (see above, p. 130), that from this island other nations, amongst them the Goths and the Danes, besides the Gepidæ, migrated. It will be seen that the Ravenna geographer’s statements about the Skridfinns, whose name is varied and corrupted even more than in Jordanes, bear a striking resemblance to those of Procopius, although he says he derived them from the Goth Aithanarit; if this is correct, then the latter must either have borrowed from Procopius, which is very probable, or he is older and was the common authority both of Procopius and the Ravenna geographer, and, if so, perhaps also of Cassiodorus (?).
- 76. Cynocephali on a peninsula north-east of Norway (from the Hereford map) Æthicus Istricus, seventh century (?) An enigmatical work, probably dating from about the seventh century, which was much read in the Middle Ages, professes to be a Latin translation, by a certain Hieronymus, from a Christian book of travel by a Greek commonly called Æthicus Istricus.[148] He is said to have travelled before the fourth century. The translator asserts that Æthicus had related many fabulous things, which he has not repeated, as he wished to keep to the sure facts; but among them we find many remarkable pieces of information, as that Æthicus had seen with his own eyes on the north of the Caspian Sea the Amazons give the breast to Centaurs and Minotaurs, and when he
- 77. was living in the town of Choolisma, built by Japhet’s son Magog, he saw the sea of bitumen which forms the mouth of Hell and from which the cement for Alexander’s wall of iron came. In Armenia he looked in vain for Noah’s ark; but he saw dragons, ostriches, griffins, and ants as large and ferocious as dogs. He also mentioned griffins and treasures of gold in the north between the Tanais and the northern ocean. “The Scythians, Griffins, Tracontians and Saxons built ships of wattles smeared over with pitch” (perhaps it is meant that they were also covered with hides). These ships were extraordinarily swift. Among the Scythians there was said to be an able craftsman and great teacher, Grifo, who built ships with prows in the northern ocean. He was like the griffins or the flying fabulous birds. Æthicus visited an island called Munitia north of Germania. There he found “Cenocephali” (dog-headed men). They were a hideous race. The Germanic peoples came to the island as merchants and called the people “Cananei.” They go with bare calves, smear their hair with oil or fat and smell foully. They lead a dirty life and feed on unclean animals, mice, moles, etc. They live in felt tents in the woods far away by fens and swampy places. They have a number of cattle, fowls and eggs.[149] They know no god and have no king. They use more tin than silver. One might be tempted to think that this fable of dog-headed people in the north had arisen from the word “Kvæn” (Finn), which to a Greek like Æthicus would sound like “cyon” (dog). The name “Cenocephali” may have been introduced in this way, while that of “Cananei” may have arisen by a sort of corrupt similarity of sound between Kvæn and the Old Testament people of Canaan. It might thus be Kvænland or Finland that is here spoken of. Their going with bare calves and living in felt tents may remind us of the Argippæi of Herodotus, who were bald (while in Mela they went bare-headed) and had felt tents in winter.
- 78. The Seven Sleepers in the Cave by the North Sea (from Olaus Magnus) Paulus Warnefridi, 720-790 The Langobard author Paulus Warnefridi, also called Diaconus (about 720-790), gives for the most part more or less confused extracts from earlier authors, but he seems besides to have obtained some new information about the North. Just as the Goth Jordanes (or Cassiodorus, or Ablabius) makes the Goths emigrate from Ptolemy’s Scandza, so Paulus, following earlier authors,[150] makes the Langobards proceed from Pliny’s island Scatinavia, far in the north. It looks as though at that time a northern origin was held in high esteem. But Paulus describes the country, from the statements of those who have seen it, as not “really lying in the sea, but the waves wash the low shores.” This points to a confusion here with a district called Scatenauge by the Elbe, which in a somewhat later MS. (about 807) of the Langobardic Law is mentioned as the home of the Langobards [cf. Lönborg, 1897, p. 27]. Paulus further relates that on the coast “north-west towards the uttermost boundaries of
- 79. Germany” there lie seven men asleep in a cave, for how long is uncertain. They resemble the Romans in appearance, and both they and their clothes are unharmed, and they are regarded by the inhabitants as holy. The legend of the Seven Sleepers is already found in Gregory of Tours, who has it from Asia Minor, where it arose in the third century and was located at Ephesus [cf. J. Koch, 1883]. The legend was very common in Germania, and we find it again later in tales of shipwreck on the coast of Greenland.[151] “Near to this place [i.e., the cave with the seven men] dwell the ‘Scritobini’;[152] thus is this people called; they have snow even in summer time, and they eat nothing but the raw flesh of wild beasts, as they do not differ from the beasts themselves in intelligence, and they also make themselves clothes of their skins with the hair on. Their name is explained from the word ‘to leap’ in the foreign tongue [i.e., Germanic], for by leaping with a certain art they overtake the wild beasts with a piece of wood bent like a bow. Among them is an animal which is not much unlike a stag, and I have seen a dress made of the hide of this animal, just as if it was bristling with hairs, and it was made like a tunic and reached to the knees, as the above-mentioned Scritobini wear it, as I have told. In these parts, at the summer solstice, there is seen for several days, even at night, the clearest light, and they have there much more daylight than elsewhere, as on the other hand, about the winter solstice, even if there is daylight, the sun itself is not seen there, and the day is shorter than in any other place, the nights also are longer; for the farther one goes away from the sun, the nearer the sun appears to the earth [the horizon], and the shadows become longer.”...
- 80. The oldest known picture of a ski-runner (from the Hereford map’s representation of Norway, thirteenth century) “And not far from the shore which we before spoke of [by the cave] on the west, where the ocean extends without bounds, is that very deep abyss of the waters which we commonly call the ocean’s navel. It is said twice a day to suck the waves into itself, and to spew them out again; as is proved to happen along all these coasts, where the waves rush in and go back again with fearful rapidity. Such a gulf or whirlpool is called by the poet Virgil Caribdis, and in his poem he says it is in the strait by Sicily, as he says: ‘Scilla lies on the right hand and the implacable Caribdis on the left. And three times it sucks the vast billows down into the abyss with the deep whirlpool
- 81. of the gulf, and it sends them up again into the air, and the wave lashes the stars.’ “By the whirlpool of which we have spoken it is asserted that ships are often drawn in with such rapidity that they seem to resemble the flight of arrows through the air; and sometimes they are lost in this gulf with a very frightful destruction. Often just as they are about to go under, they are brought back again by a sudden shock of the waves, and they are sent out again thence with the same rapidity with which they were drawn in. It is asserted that there is also another gulf of the same kind between Britain and the Gallician province” [i.e., northern Spain], whereupon there follows a description of the tides on the south coast of France and at the mouths of the rivers, after which there is a highly coloured account of the horrors of the Ebudes, where they can hear the noise of the waters rushing towards a similar Caribdis. Paulus Warnefridi evidently had a very erroneous idea of ski-running, which he made into a leaping instead of a gliding motion. He may have imagined that they jumped about on pieces of wood bent like bows. That the abyss of waters or navel of the sea is thought to be in the North may be due to reports either of the current in the Pentland Firth or of the Mosken-ström or the Salt-ström, which thus make their appearance here in literature, and which were afterwards developed into the widespread ideas of the Middle Ages about maelstroms and abysses in the sea, perhaps by being connected with the ancient Greek conception of the uttermost abyss (Tartarus, Anostus, Ginnungagap; see pp. 11, 12, 17), and as here with the description of the current in the Straits of Messina.
- 82. The Maelstrom near the Lofoten Islands (from Olaus Magnus) Viktor Rydberg [1886, pp. 318, 425, ff.] supposed Paulus’s description of the whirlpool to be derived from the Norse legends of the world’s well, “Hvergelmer”—which causes the tides by the water flowing up and down through its subterranean channels—and of the quern “Grotte” at the bottom of the sea, which forms whirlpools when the waters run down into the hole in the mill-stone.[153] But it is perhaps just as probable that it is the southern, originally classical ideas which have been localised in the Norse legends. As we have seen, we find in Virgil the same conception of a gulf in the sea which sucks the water into itself and sends it up again. Isidore says of the abyss (also repeated in Hrabanus Maurus): “Abyssus is the impenetrable deep of the waters, or the caves of the hidden waters, from whence springs and rivers issue forth, but also those which run concealed beneath the ground. Therefore it is called Abyssus, for all streams return by hidden veins to their mother Abyssus.” It is credible that ideas such as this may have originated, or at any rate coloured, the myth of “Hvergelmer” (i.e., the noisy or bubbling
- 83. kettle). Isidore was early known in England, Ireland and Scandinavia. The whirlpool is also found among Orientals; thus Sindbad is drawn into it. Paulus’s mention of whirlpools not only in the North, and off the Hebrides, but also between Britain and Spain and in the Straits of Messina, does not show that he derived the legend solely from the North. Later, on the other hand, in Adam of Bremen, the whirlpool becomes more exclusively northern, and later still we shall get it even at the North Pole itself. Paulus Warnefridi also mentions Greek fabulous people such as the Dog-heads (Cynocephali) and the Amazons in North Germania. He says that the Langobards fought with a people called “Assipitti,” who lived in “Mauringa,” and that they frightened them by saying that they had Cynocephali in their army, who drank human blood, their own if they could not get that of others. The Langobards were said to have been stopped by the Amazons at a river in Germany. The Langobard king, Lamissio, fought with the bravest of them, while he was swimming in the river, and slew her; and according to a prearranged agreement he thereby obtained for his people the right of crossing unhindered. Paulus regards the story as untrue, as the Amazons were supposed to have been destroyed long before; but he had nevertheless heard that there was a tribe of such women in the interior of Germany. The same idea of a female nation in Germany occurs again later in literature (cf. King Alfred’s “Mægða-land”). Interpolati on in Solinus, circa eighth century It has already been mentioned (p. 123) that in the MSS. of Solinus of the ninth century and later there is found a mention of the Ebudes, the Orcades and Thule which in the opinion of Mommsen is a later addition; and as it is not found in Isidore Hispalensis, who made extensive use of Solinus, it must have been introduced after his time (seventh century), but before the ninth century, when it
- 84. occurs in a MS. As the addition about Thule, so far as I can judge, must show that this country is regarded as Norway, and as there are many indications that it was made by an Irish monk, it is further probable that it belongs to the period before the Irish discovery of Iceland, which then, according to Dicuil’s book, became regarded as Thule. I think, therefore, we can place the addition at the beginning of the eighth century, and it will then be evidence of the knowledge of Norway which prevailed in the British Isles at that time. After having mentioned Britain and the neighbouring islands the account proceeds [Solinus, c. 22]: “From the Caledonian Promontory it is two days’ sail for those who voyage to Tyle [Thule]. From thence begin the Ebudes islands [Hebrides], five in number [the five principal islands]. Their inhabitants live on fruits, fish and milk. Though there are many islands, they are all separated by narrow arms of the sea. They all together have but one king. The king owns nothing for himself alone, all is common property. Justice is imposed upon him by fixed laws, and lest he should be led away from the truth by covetousness, he learns righteousness by poverty, since he has no possessions; he is therefore supported by the people. No woman is given him in marriage, but he takes in turn her who pleases him at the moment. Thus he has neither the desire nor the hope of children. The second station for the voyager [to Thule] is provided by the Orcades. But the Orcades lie seven days’ and the same number of nights’ sail from the Ebudes, they are three in number [i.e., the three principal isles of the Shetlands]. They are uninhabited (‘vacant homines’). They have no woods, but are rough with reeds and grass, the rest is bare sandy beach and rocks. From the Orcades direct to Thule is five days’ and nights’ sail. But Thule is fertile and rich in late-ripening fruits. The inhabitants there live from the beginning of spring with their cattle, and feed on herbs and milk; the fruits of the trees they keep for winter. They have women in common, regular marriage is not known among them.”
- 85. This description cannot well be pure invention, and unless it may be thought to be transferred from another place, we must believe it to be derived from a distant knowledge of Norway. Their living with the cattle in spring is in accordance with this, but not their subsistence on the fruits of the trees. Here one would rather be led to think of the Hesperides and their golden apples, unless we are to suppose that they collected nuts and berries. That the inhabitants of Thule had women in common might be connected with the predilection of the Scandinavians for polygamy, of which we also hear from other sources; but this is uncertain. Even the Greeks and Romans saw in the absence of regular marriage a sign of barbarism, which brought man near to the beasts, and which they therefore attributed to people at the extreme limits of the earth; cf. Herodotus, and Strabo’s description of the Irish (p. 81). If the Caledonian Promontory means Scotland, it is surprising that it should be two days’ sail to the Hebrides, and that these were the first and the Orcades the second station on the way to Thule. We must then suppose that there has been a jumbling together of several authorities, which is not very probable if this is a later interpolation, since we must doubtless believe the interpolating copyist to have thought himself possessed of knowledge of these matters. If, however, we suppose him to have been an Irishman, and to have looked upon the voyage to Thule with Ireland as a starting-point, then it becomes more consistent. It is then two days’ sail from Ireland to the Hebrides, seven days thence to the Shetlands, and then five to Thule; that is, the whole voyage will last fourteen days; and this may be about right. It is undeniably somewhat surprising that there should be no inhabitants on the Orcades, or Shetland, at that time. THE DISCOVERY OF THE FAROES AND ICELAND BY THE IRISH IN THE EIGHTH CENTURY Dicuil, circa 825
- 86. The earliest voyages northward to the Arctic Circle, of which there is certain literary mention in the early Middle Ages, are the Irish monks’ expeditions across the sea in their small boats, whereby they discovered the Faroes and Iceland, and, at all events for a time, lived there. Of these the Irish monk Dicuil gave an account, as early as about the year 825, in his description of the earth, “De Mensura Orbis Terræ” [cf. Letronne, 1814, pp. 38 f., 131 f.]. It is characteristic of the spiritual tendency of that period of the Middle Ages that these remarkable voyages were not, like other voyages of discovery, undertaken from love of gain, thirst for adventure, or desire of knowledge, but chiefly from the wish to find lonely places, where these anchorites might dwell in peace, undisturbed by the turmoil and temptations of the world.[154] In this way the unknown islands near the Arctic Ocean must have seemed to satisfy all their requirements; but their joy was short-lived; the disturbers of the North, the Vikings from Norway, soon came there also and drove them out or oppressed them. What Dicuil tells us of the Scandinavian North is chiefly derived from Pliny, and contains nothing new. But of the unknown islands in the northern ocean he writes [7, 3]: Discovery of the Faroes by the Irish “There are many more islands in the ocean north of Britain, which can be reached from the northern British Isles in two days’ and two nights’ direct sailing with full sail and a favourable wind. A trustworthy priest (‘presbyter religiosus’) told me that he had sailed for two summer-days and an intervening night in a little boat with two thwarts [i.e., two pairs of oars],[155] and landed on one of these islands. These islands are for the most part small; nearly all are divided from one another by narrow sounds, and upon them anchorites, who proceeded from our Scotia [i.e., Ireland], have lived for about a hundred years (‘in centum ferme annis’). But as since the beginning of the world they had always been deserted, so are they
- 87. now by reason of the Northman pirates emptied of anchorites, but full of innumerable sheep and a great number of different kinds of sea-birds. We have never found these islands spoken of in the books of authors.” The Faroes This description best suits the Faroes,[156] where, therefore, Irish monks had previously lived, and from whence they had been driven out by Norwegian seafarers, probably at the close of the eighth century. As, however, Dicuil is so well aware of the islands being full of sheep, the Irish may have continued to visit them occasionally, like the trustworthy priest referred to, who sailed there in a boat with two thwarts. Dicuil’s statement that they were then “emptied of anchorites” must doubtless be interpreted to mean that they were uninhabited; but this does not sound very probable. Rather, there
- 88. are many indications that the islands had an original Celtic population, which continued to live there after the settlement of the Norsemen. There are some Celtic place-names, such as “Dímon” (the islands “Stora Dímon” and “Litla Dímon,” or “Dímun meiri” and “Dímun minni”) from the Celtic “dimun” (== double neck, thus like Norwegian “Tviberg”).[157] As such Celtic place-names cannot have been introduced later, the Norwegians must have got them from the Celts who were there before, and with whom they had intercourse. The language of the Faroes has also many loan-words from Celtic, mostly for agriculture and cattle-farming, and for the flora and fauna of the islands. These might be explained by many of the Norwegian settlers having previously lived in the Scottish islands or in Ireland, or having had frequent communication with those countries [cf. A. Bugge, 1905, p. 358]; but it seems more natural to suppose that the loan-words are derived from a primitive Celtic population. To this must be added that the people of the Southern Faroes are still dark, with dark eyes and black hair, and differ from the more Germanic type of the northern islands [cf. D. Bruun, 1902, p. 5]. The name “Færöene” (sheep-islands) shows that there probably were sheep before the Norsemen came, which so far agrees with Dicuil; these sheep must then have been introduced by the earlier Celts. According to this it seems possible that the Irish monks came to the islands not merely as anchorites, but also to spread Christianity among a Celtic population. The Norwegians arrived later, took possession of the islands, and oppressed the Celts. Irish Discovery of Iceland But the bold Irish monks extended their voyages farther north. Dicuil has also to tell us how they found Iceland, which he calls Thule, and lived there. After having mentioned what Pliny, Solinus, Isidore (Hispalensis) and Priscianus say about Thule (Thyle), he continues [7, 2, 6]:
- 89. “It is now thirty years since certain priests, who had been on that island from the 1st of February to the 1st of August, told that not only at the time of the summer solstice, but also during the days before and after, the setting sun at evening conceals itself as it were behind a little mound, so that it does not grow dark even for the shortest space of time, but whatsoever work a man will do, even picking the lice out of his shirt (pediculos de camisia extrahere), he may do it just as though the sun were there, and if they had been upon the high mountains of the island perhaps the sun would never be concealed by them [i.e., the mountains]. In the middle of this very short time it is midnight in the middle of the earth, and on the other hand I suppose in the same way that at the winter solstice and for a few days on either side of it the dawn is seen for a very short time in Thule, when it is midday in the middle of the earth. Consequently I believe that they lie and are in error who wrote that there was a stiffened (concretum) sea around it [i.e., Thyle], and likewise those who said that there was continuous day without night from the vernal equinox till the autumnal equinox, and conversely continuous night from the autumnal equinox till the vernal, since those who sailed thither reached it in the natural time for great cold, and while they were there always had day and night alternately except at the time of the summer solstice; but a day’s sail northward from it they found the frozen (congelatum) sea.” This description, written half a century before the Norwegians, according to common belief, came to Iceland, shows that the country was known to the Irish, at any rate before the close of the eighth century (thirty years before Dicuil wrote in 825), and how much earlier we cannot say. With the first-hand information he had received from people who had been there, Dicuil may have blended ideas which he had obtained from his literary studies. The sun hiding at night behind a little mound reminds us of the older ideas that it went behind a mountain in the north (cf. Cosmas Indicopleustes and the Ravenna geographer); but of course it may also be due to local observation. The idea that the frozen sea (“congelatum mare”) had been found a day’s sail north of this island is precisely the same as
- 90. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com