![Annotation
The wireless network coding [26,30,39,54] (wireless extension of the network coding [1,67]) was shown to
have a potential to significantly improve the performance in the wireless networks against the conventional
networks based on independent (or interfering) point to point channels directed by higher layers. The
whole network must be however designed jointly to take advantages from the wireless network coding. The
receiver processing then should take into account this global design. This thesis aims to (1) the global
design of the channel encoders and corresponding decoders among all nodes within the wireless network
and (2) the receiver processing of a particular node in the wireless communication network.
A joint solution of the channel coders design across the network is called the block structure layered
design [46]. This solution allows to incorporate arbitrary particular channel code rates. The design is
based on a linear description of the wireless network containing hierarchical maps (network coding) and
channel coding. The design combines already known approach suited just for symmetric channel code rates
(equal channel encoders) [30] properly utilizing advances of the network coding paradigm [67] in wireless
extension. This principle is mixed with traditional joint decoding such that the desired code rates and
relay information flows are achieved in average over whole codeword. This mixture results in the block
diagonal structure using the proposed linear system description. We numerically evaluate the performance
for a particular wireless network that is designed by means of the wireless network coding to prove the
design properties.
The fundamental topic underlaying the whole thesis is the sum product algorithm on factor graphs
[27,33]. This generic tool enables a construction of the receiver processing for a general class of scenarios
including the wireless networks with properly respected global design. The thesis aims to a direct im-
plementation of the sum product algorithm that is applied in the receiver within the wireless networks.
Our endeavour results in a generic implementation framework of the sum-product algorithm [44,47] that
is applicable in a wide class of scenarios including the receiver processing in the wireless networks. The
framework is based on the proposed KLT message representation. The KLT message representation is
capable to methodically describe an arbitrary continuous message in FG-SPA, whenever the stochastic
description (even numerical) is available and the KLT of the random process formed by random message
realizations can be evaluated. The description is moreover the best one among all linear message repre-
sentations for a given message dimension as a consequence of the KLT. We stress that the uniqueness of
the proposed message representation consists in its methodical nature and applicability on the continuous
messages (the FG-SPA definition straightforwardly enables only representation of discrete messages). We
further derive the update rules upon linear message representations with orthogonal kernels. The KLT
message representation satisfies both the linearity and the orthogonality of the kernels and therefore the
proposed update rules design is applicable also for the KLT message representation. The KLT message
representation jointly with the update rules design therefore form the generic FG-SPA implementation
framework. The proposed implementation framework is verified in a particular scenario (a receiver aiming
with joint phase estimation data detection) and compared with a conventional solution.
i](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-2-320.jpg)
![1 Introduction
1.1 Motivation
The wireless network coding attained a large popularity recently, because it stands for one of the possible
flows improving the current state of art of the digital communications. The gaps between the fundamental
limits and the current state of art implementations shrank to negligible values for almost all utility metrics
in conventional point to point channels. Nevertheless, if we consider a wireless network as a collection
of independent point to point channels that are directed by higher layers (conventional OSI model) the
situation is no longer optimal from the global point of view. The situation in wireless networks is therefore
fundamentally different.
Considering the whole wireless network as one unit to be optimized opens a whitespace for new fun-
damental limits related to the wireless network communication as well as for novel procedures improving
the network performance. The main part of the research community in communication aims to fill this
whitespace, which can be proven by majority of such topics in the communication journals and conferences
nowadays. A lot of work has been done, especially in simple network models such as two-way relay channel
containing two nodes that want to exchange the data using an auxiliary relay. A performance very close
(half bit below) to the first order cut-set bound was proven in [38] for the two way relay channel upon
AWGN channels. It means that the relay receives a superimposed signal from two sources simultaneously
and the performance of the multiple access channel is limited almost only by the Gaussian noise, the
interference limitation is therefore almost avoided.
The proof of the achievability is, however, based on the lattice code approach that does not allow
a straightforward implementation. There is therefore need for a directly implementable channel coding
strategy allowing to approach the performance close to this limit. Many papers suggest to adopt the
conventional channel coding strategies to properly respect the wireless network coding, but only limited
results are available so far. Particularly a natural benchmark, i.e. the joint decoding followed by the pure
network coding, can be obviously implemented for arbitrary channel code rates in sources, nevertheless the
advantage of the wireless network coding is not fully exploited. On the other hand, for a special case, one
can compose a compound decoder processing jointly the channel decoding and the network operation. This
approach is however limited for a particular class of scenarios, where the equal source rates are assumed.
A solution utilizing the advantage of the wireless network coding and allowing arbitrary channel code rates
is an obvious target.
Suppose now that we have already described the receiver in a mathematical way. The point is how to
properly implement the processing in the receiver. This processing should be efficient to avoid waisting
the resources and also it should be as exact as possible. One can feel that this is another crucial step for
the system implementation. As an example from the history, we can mention the conventional capacity
achieving channel codes that were known (LDPC codes developed by Gallager) for a long time, but their
implementation was trackable since the iterative processing was employed. Nevertheless, this took over
twenty years.
The sum product algorithm on factor graph [27, 33] was shown as a tool that successfully solves a
wide family of problems. The generic nature of the sum product algorithm allows its use also to describe
the receiver processing in a wireless network properly taking into account the global design. The sum
product algorithm is, however, exactly described in systems leading to a cycle-free factor graph. The
capacity achieving channel codes that are supposed to be utilized in the receiver typically lead to the
factor graphs containing loops. One should take attention about the system convergence in such a case,
which is rather analytical than the implementation problem. Another problem is that the sum product
algorithm is described as an iterative algorithm that is directed by some update rules. These update rules
provide basically marginalization of a soft information (message) describing a particular local variable in
the system. The problem arising from the implementation of the sum product algorithm is therefore given
1](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-14-320.jpg)
![1.2. GOALS AND SUMMARY OF MAIN RESULTS
by a description of a general continuously valued message for the processing purposes and the numerical
intensity of the continuous message marginalization (integration) required by the update rules.
1.2 Goals and Summary of Main Results
The goal of this thesis is twofold. The first goal aims to the overall channel codes construction within the
network that utilizes the wireless network coding. The second goal is dedicated to the implementation
issue of generic sum product algorithm that can be used for the receiver processing design. The generic
nature of the sum product algorithm enables to incorporate a various number of flavours resulting from
the overall network design.
The answer to the first goal is contained in the block structured layered design describing the network
design for an arbitrary channel code rates and simultaneously profiting from the wireless network coding
advantage. This contribution is situated in a self-contained form in Chapter 7 as well as in [46]. The
block structured layered design is build upon the system linearity assumption, that is linear channel codes
and hierarchical functions. The channel codes can be then described by matrices (LDPC codes) that
are properly composed within a global matrix in each node. Such a (block) construction indeed allows to
incorporate arbitrary source rates as well as an arbitrary information flow division between the hierarchical
streams (paths of the mixed information) in the network. To best of our knowledge, no such a design is
available in the current state of art literature.
The second goal is an implementation framework for the generic sum product algorithm applicable
among others also in the wireless networks. This framework should efficiently and methodically solve the
message representation as well as an efficient implementation of the update rules upon the given message
parameterization. We therefore formally describe the problem of the message representation in Chapter
4. The solution is then situated into the sequel Chapter 5, where we first serve a generic KLT message
representation that is based on the stochastic description of the message. The message representation
(approximation) obtaining is both methodical and efficient, where the optimized criterion is the MSE of
the approximated message (Sec. 5.2.1 or [44]). The update rules upon the message representation for
a special type of linear canonical message representation with orthogonal kernels are then proposed in
Section 5.2.3 or [47]. An example demonstrating the power of the proposed implementation framework can
be found in Section 5.3.
1.3 Structure of the Thesis
The thesis is divided into three main parts, where the core of the work is situated into the second and
third part. The main contribution is contained in Chapters 5 (also in [44,47]) and 7 (also in [46] and partly
in [58]). These Chapters are written such that they are supposed to be self-contained. It means that the
rest of the thesis is some kind of supplementary material for these two main Chapters.
The first part situated immediately after introduction contains a collection of prerequisites that will
be further used in the sequel parts. A common characteristic of the first part is that the prerequisites are
supposed to be presented just for the completeness. The first part is therefore not written in the tutorial
form at all. Instead, some basic ideas and definitions are discussed with reference to the current state of
art. We often point out the relation of a particular topic with the contribution that is situated into the
sequel parts.
The second part contains topic related to the implementation of the sum product algorithm. We
formally state the problem of the implementation framework and then we introduce the proposed imple-
mentation framework containing the KLT message representation and the generic update rules design.
The third part is dedicated to the channel code design across the wireless network properly respecting
the global design requests (wireless aware network coding). This is presented after a short introduction to
2](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-15-320.jpg)
![CHAPTER 1. INTRODUCTION
the wireless network coding.
The contents of the individual chapters can be summarized as:
• Part I contains the prerequisites for the sequel part.
– Chapter 2 provides a collection of some basic terms that form some fundamental limits (chan-
nel capacity) and starting points for the receiver algorithm derivation (maximum a posteriory
estimation criterion).
– Chapter 3 briefly lists conventional channel coding and decoding techniques that approach the
channel capacity. Namely the turbo and LDPC codes. The decoding algorithms are also dis-
cussed including the tools dedicated for the investigation of the decoder convergence properties
(EXIT charts).
• Part II presents a solution to the first goal, that is the generic implementation framework.
– Chapter 4 is supposed to formally define the sum product algorithm on factor graph as well as
what does it exactly mean the sum product algorithm implementation framework. Although the
implementation framework can be explained by two sentences, we found a formalism underlaying
the message representation (approximation) and update rules upon the message representation
forming the implementation framework together. Some examples of implementation frameworks
are considered for an illustration. For the sake of completeness, we then also discuss the issues
of the sum product algorithm resulting from its use on the looped factor graphs.
– Chapter 5 presents one of the main contribution of this work, that is the implementation
framework based on the KLT message representation. This message representation is intro-
duced as a message representation solution methodically minimizing the MSE of the message
approximation for a given message dimensionality. The generic KLT message representation
is supplemented by the generic update rules design that is applicable for a wide class of mes-
sages (among others the KLT message representation). The proposed framework is numerically
verified and compared with a conventional solution.
• Part III covers the topic of the error protection in the wireless network operating with the wireless
network coding.
– Chapter 6 offers a basic overview of the wireless network coding and its basic classification.
Some basic assumptions and constraints are discussed. The fundamental principles charac-
terizing the wireless network coding are demonstrated when the basic network topologies are
presented.
– Chapter 7 serves the second main contribution of this work. This chapter basically corresponds
to [46]. We describe both hierarchical operation and channel codes by a linear operation. The
block structured layered design is then introduced upon this description. The block structured
layered design enables to design jointly the hierarchical functions and the channel codes such
that arbitrary desired rates can be incorporated. The principle is finally verified in a concrete
wireless network example.
1.4 Thesis Organization
The aim of this Section is to guide through some common rules that are used within this thesis (e.g. some
global notation). We try to avoid overloading of all symbols (at least locally).
3](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-16-320.jpg)
![1.4. THESIS ORGANIZATION
Vector Notation and Operations
A vector is denoted by a bold-face lower-case letter and it is implicitly assumed to be in a column form. It
means that a vector [b1, . . . , bk]T
is denoted b. A zero vector with a single non-zero unit value on the ith
position is denoted by ei, for example e3 is a vector given by [0, 0, 1, 0, . . . ]T
. If the order of a particular
symbol in the vector is not important or clear from the context, the subscript is dropped, i.e. b instead of
bi.
The vector can be also written in the form bA, where A is an ordered set or vector of indexes, e.g.
A = {1, 2, 5} gives bA = b1,2,5 = [b1, b2, b5]T
. We further allow to remove a particular dimension by the
tilde operation1
∼ . As an example, we consider a vector b = [b1, b2, b3]T
. The vector b ∼ b2 is therefore
given by b ∼ b2 = [b1, b3]T
.
Matrix Notation and Operations
A matrix is conventionally denoted by a bold upper-case letter. For example Xk×n = [x1 . . . xn] is k × n
matrix composed from n vectors x1, . . . , xn, where each of them is k symbols long. If the matrix is square,
its dimension is denoted by a single index, that is Xk×k = Xk. The matrix dimension in the subscript is
omitted whenever the matrix dimension is clear from the context. A zero matrix 0 is a matrix containing
all zero elements and a matrix E denotes the unitary matrix that can be written as E = [e1, e2, . . . ]. A
matrix Xk×n is said full-rank if rank(X) = min(k, n).
We denote R to be a set of real numbers. An n-dimensional set An
is denoted by A∗
, whenever the
number of dimension is not important for that time. We further implicitly assume a symbol-wise operations
upon given sets excepting conventional vector and matrix multiplication.
Random Variables and Density Functions
The random variables and random processes [42] are widely used within this thesis. We assume a random
variable X described by its probability function (either pdf or pmf) pX(x) = ∂Pr(X < x)/∂x. This is
denoted just by p(x) for simplicity if it is clear which probability function is meant. The conditional
probability px|y(x, y) = ∂Pr(X < x ∩ Y < y)/∂xPr(Y < y) will be simply denoted by p(x|y). We denote a
random realization of the random process in subscript, i.e. µx(t) as a random process of a variable t. We
will consider basic moment operations upon the random variables and process. Particularly mean value
given by Ex[·] = ·p(x)dx for a real domain of x. Note that the mean value is sometimes denoted by
an over-bar symbol or by m letter, that is Ex[·] = m· = ¯·. Similarly the variance is given by the second
moment, i.e. var[x] = Ex[(x − Ex[x])2
]. The variance is sometimes denoted by σ letter, i.e. var[x] = σx.
The most notable distribution used within this thesis is the Gaussian distribution given by
ρσ(y − x) =
1
σ2π
exp
−|y − x|2
σ2
, (1.1)
for complex random variable and
ρσ(y − x) =
1
√
2σ2π
exp
−|y − x|2
2σ2
, (1.2)
for real random variable.
1The tilde operation is also used for the proportionality relation. The concrete meaning of a given tilde operation can be
recognized from the context within this thesis.
4](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-17-320.jpg)
![2 Theoretical Background
This chapter aims with the theories essentially affecting the decoding procedure in wireless communication.
We will consider the information and estimation theory. Although some results from other fields can be
also applied, we focus just to these. This Chapter is dedicated as a reference and therefore it is not written
in the tutorial form. We point out some important definitions and theorems that will be used in later part
of the work. We refer excellent tutorials [11] for the information theory and [24] for the estimation theory.
2.1 Information Theory
The information theory answers some principal questions from the communication design, mainly regarding
the fundamental limits that can be used as a benchmark for a real application, i.e. the gap between the
fundamental limit and the real application refers to the space for an improvement for a given realization.
The fundamental limits are ordinarily connected with some utility metric. These limits state the best
possible performance of the system from a given metric point of view for given constraints. The main
utility metric that is considered within this work is the channel capacity in a power constraint wireless
channel, because it answers an essential natural question, how many information can be reliably transmitted
through a noisy channel for a given transmit power. We also touch the latency issue, because the capacity
optimization leads to processing of long sequences unavoidably causing the latency.
2.1.1 Capacity of Scalar Point to Point AWGN Channel
The channel capacity is given by the stochastic description of the channel. The most important channel for
our consideration is the Additive White Gaussian Channel (AWGN) described by the Gaussian conditional
pdf, that is
p(y|x) =
1
√
2πσ2
exp
−|y − x|2
2σ2
= ρσ(y − x), x, y ∈ R. (2.1)
The power constraint channel capacity for such a channel is given by
CAW GN = 1/2 log(1 + γ), (2.2)
where γ = P/N is the signal to noise ratio and the unit is bit per transmission in case of binary base of
the logarithm, which is implicitly assumed. The capacity of complex channel, i.e. x, y ∈ R becomes to
CAW GN = log(1 + γ).
The channel capacity for AWGN is shown to be achievable in [11, Theorem 9.1.1]. The proof of the
capacity theorem also reveals how the codeword should look like to attain the capacity. Particularly,
the distribution p(x) maximizing the mutual information I(X, Y ) is the Gaussian one. The channel code
approaching the channel capacity moreover relies on the asymptotic arguments and therefore the channel
codeword is expected to be infinitely long to achieve the channel capacity.
Practically, the codeword cannot be infinitely long, so one must choose some trade-off between the
codeword sequence length and the gap from the channel capacity. The processing of a long sequence
brings two issues: (1) a time required for the long sequence processing unavoidably brings latency and
(2) one need to design a channel code which is implementable with a reasonable complexity (e.g. linear
with respect to the codeword length) and that approaches the channel capacity. The random coding used
in [11, Theorem 9.1.1] cannot be directly implemented due to its exponential complexity with respect to
the codeword length. We dedicate Chapter 3 as an overview of the conventional coding strategies including
their implementation in P2P channels.
6](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-19-320.jpg)
![CHAPTER 2. THEORETICAL BACKGROUND
2.1.2 Capacity of Multiple Access AWGN Channel
If more nodes transmit simultaneously to one receiver, the situation is referred as a Multiple Access Channel
(MAC). The receiver hears a superposition of the signals from more nodes corrupted by a noise. This can be
described by a conditional pdf p(y|x1, . . . , xn). We assume that no other1
information about the particular
streams is available in the receiver (the information is passed only through the channel) for this moment.
We also assume that the receiver wishes to fully decode all of the received signals.
The channel capacity (2.2) does not refer only to a single rate, but now it is given by the rates of
all transmitting nodes. We therefore speak about the capacity region, which is given by a union of all
achievable rates for a given MAC channel [11].
In special case of two sources (operating at rates R1 and R2), the capacity region (complex channels
assumed) is given by R1 ≤ log(1 + P1/N) and R2 ≤ log(1 + P2/N), where P1 and P2 stand for the power
of individual signals. The achievable single rate cannot be larger than in case, where the second signal is
not present . This is referred as the first order cut-set bound of the classical MAC channel. The second
order cut-set bound is given by R1 + R2 ≤ log(1 + (P1 + P2)/N) and if refers to the joint decoding, i.e. full
decoding of data from both sources.
Note 2.1 (Moving Beyond the Second Order Cut Set Bound in More Complex Networks). We will discuss
the wireless network coding in sequel part of this work. This strategy is able to operate beyond the second
order cut-set bound thanks to avoiding of the full decoding in auxiliary relays. Instead a function of the
pair is decoded and one relies that a complement of this function is delivered by an alternative path into
the final destination. The final destination then solves a system of equations.
This capacity region can be achieved e.g. by a Successive Decoding-Interference Cancellation (SD-IC)
method, where the receiver decodes one of the signal threating the latter as noise and after decoding, this
signal is re-encoded and subtracted from the superimposed signal. In case of error-less decoding of the first
stream, the subtracted signal is equal to the ordinary P2P channel as one can find e.g. in [11].
2.2 Estimation Theory
Estimation and detection theory stands for one of the fundamental cornerstones for the data communica-
tion. This theory provides some criteria of a communication receiver design optimality. A nice tutorial for
the estimation theory can be found e.g. in [24].
In this section, we briefly point out the most important criteria (ML, MAP, MMSE estimation) for the
receiver design within the wireless communication.
2.2.1 Problem Statement
Assume a random parameter ξ = [. . . , ξi, . . . ] to be estimated (e.g. data vector, phase shift, etc.) that is
observed through a (noisy) observation θ = [. . . , θi, . . . ]. A system description defines a joint pdf p(ξi, θ),
∀i.
We can now consider several approaches of the desired vector ξ estimation. We will distinguish symbol
estimation ˆξi and the whole vector estimation ˆξ.
1Breaking this assumption opens a way to the wireless network coding significantly favouring this work. The wireless
network coding will be discussed in Chapters 6 and 7
7](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-20-320.jpg)
![2.2. ESTIMATION THEORY
2.2.2 Maximum Likelihood Estimator
The likelihood function (LF) is defined2
by
Λ(ξ) = p(θ|ξ). (2.3)
The maximization of (2.3), i.e.
ˆξ = arg max
ˇξ
p(θ|ˇξ) (2.4)
is called maximum likelihood (ML) estimator. From the monotonicity of the logarithm operation, the
maximization of log-likelihood function (LLF) Λ (ξ) = ln Λ(ξ) in (2.4) instead of the LF leads to the same
result.
Note 2.2 (Optimality of ML Estimator). One can show that [24, Theorem 7.3] the estimation is asymptot-
ically minimal variance unbiased (MVU), that is the estimated vector ˆξ asymptotically obeys a Gaussian
distribution ρ(ξ, I−1
(ξ))), where the variance matrix is given by the inverse of the Fisher n×n information
matrix given by
−E
∂2
ln Λ(ξ)
∂ξi∂ξj
, ∀i, j. (2.5)
The ML estimator therefore asymptotically achieves the Crammer-Rao bound [24].
2.2.3 Bayesian Estimators
An another branch of estimators is based on a Bayesian approach. According to nature of the estimated
parameter ξ, a proper type of the estimator is chosen. The general Bayesian estimator is defined by a cost
function C( ), where = ξi − ˆξi denotes an error in the estimation. For our purposes, we consider
• hit or miss cost function defined by
C( ) =
0 | | ≤ δ
1 | | > δ
, (2.6)
where δ defines a threshold,
• vector hit or miss function defined by
C( ) =
0 ≤ δ
1 > δ
, (2.7)
where = ξ − ˆξ and 2
= i
2
i and
• mean square error cost function defined by
C( ) = 2
. (2.8)
A minimization of the Bayes risk given by
R = E[C( )] = C(ξi − ˆξi)p(ξi, θ)dξidθ
for a given observation θ provides [24]:
2One can consider a symbol-LF Λ(ξi) = p(θ|ξi).
8](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-21-320.jpg)
![CHAPTER 2. THEORETICAL BACKGROUND
• Minimal mean square error (MMSE) estimation for the cost function (2.8) is given by
ˆξi = ξip(ξi|θ)dξi = E[ξi|θ], (2.9)
where the vector θ may contain additionally the whole vector ξ excepting ξi. Note that the vector
form of the MMSE estimator, that is
ˆξ = E[ξ|θ]
minimizes the MSE of each component ˆξi. The MMSE criterion is commonly used e.g. in the phase
estimation.
• Maximal a posterior probability (MAP) estimation for the cost function (2.6) is given by
ˆξi = arg max
ξi
p(ξi|θ). (2.10)
The MAP criterion is suitable to use for the data detection, because it is equivalent to the minimal
average bit error rate (BER) criterion standing for a natural goal of the reliable communication. A
considerable part of this work aims with the evaluation of p(ξi, θ) serving for the MAP estimation
(2.10).
• Vector maximal a posterior probability estimation for the cost function (2.7) is given by
ˆξ = arg max
ξ
p(ξ|θ). (2.11)
Note that the vector MAP estimation does not generally give the same result as the classical MAP
estimator (2.10).
If a priory distribution of ξi is uniform (as data in the digital communication are conventionally as-
sumed), the MAP estimator is equal to the ML estimator. As a consequence, the likelihood metric is a
sufficient statistics for the minimal average BER criterion for a given observation.
9](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-22-320.jpg)
![3 Error Protection Techniques
This Chapter shortly reviews the channel coding techniques with accent to the decoding processing and the
investigation of the decoding behaviour. Since the core of this work is basically the decoding processing,
the basic coding theory forms one of the prerequisites for the sequel part serving the main contribution of
this work.
The main purpose of the channel code is a protection of the data for a reliable transmission through
an unreliable channel (wireless medium). The fundamental limits are given by means of the information
theory, mainly the channel capacity. This channel code design (Sec. 3.1) aims to approach the channel
capacity. The receiver then needs to properly incorporate the received signal taking into account the con-
sidered channel encoder in the source. A direct application of the optimal solution leads to the intrackable
implementation. We therefore focus on some advanced techniques (iterative detection) incorporating a
slightly suboptimal, but trackable, approach to the code design and the corresponding receiver processing
in Sec. 3.2. The suboptimality (convergence) investigation techniques are the topic of Sec. 3.3.
Throughout this Chapter, we will consider a memoryless only channel, that is the channel described
by a conditional pdf p(x|c) = i p(xi|ci). All memory is therefore situated in the channel code.
3.1 Encoding Techniques
We briefly list several encoding techniques attempting to approach the channel capacity (e.g. [49]). We
will consider exclusively linear codes. As it was shown in the lattice code case [17], the linearity of the
code still enables to achieve the channel capacity.
3.1.1 FSM/Block Codes
The block and FSM based codes are the oldest ways of the channel coding techniques. The block code is
described by a matrix G such that the codeword is given by c = Gd, where d is a data vector and c is a
codeword vector. The rate of this code is given by ratio of dimension of the matrix G in case of full-rank
generator matrix.
The trellis (FSM) based codes [51] are described by a FSM. We will use the conventional octal notation
to identify the code. The analysis of the FSM-based codes from the performance point of view can be
found e.g. in [51].
Since the instant (ith) codeword symbol is given by the data vector up to the ith symbol, the trellis
codes are causal in contrast to the block codes, where one needs a whole data vector for obtaining the
instant codeword symbol in general1
.
The block code can be also described by a FSM, but both output and state functions are generally time
variant in the corresponding FSM.
3.1.2 Turbo Codes
In contrast with the FSM/block codes described above having a significant gap from the channel capacity,
the turbo codes [6,61] attain the performance very close to the capacity. The main idea is in concatenation
of FSM encoders separated by an interleaver to reduce the probability of terminating sequences for the
recursive FSM code and therefore to almost meet the assumption of the infinite codeword length. The
concatenation can be considered either parallel or serial.
We will again not investigate the performance of the turbo coding here, since it is not directly related
with the core of this work. Instead we refer some literature containing the details on the turbo codes
including the performance analysis, e.g. [51, 61]. We note here, that the "turbo" principle is quite more
1Using a systematic form of the code, the whole data vector is needed only for the parity part of the codeword.
10](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-23-320.jpg)
![CHAPTER 3. ERROR PROTECTION TECHNIQUES
general. The "turbo principle" basically means two correlated entities (e.g. FSM), that are able to some-
how exchange a randomized (interleaved) soft information such that the information input improve the
information output for each entity. One can find some other systems, where the turbo principle is applied
(e.g. MIMO equalization [60] or decoding of correlative sources [3]).
The decoding of the turbo code is based on an iterative algorithm that will be described later. Note
that the use of the interleaver corrupts the causality of the overall encoding process.
3.1.3 LDPC Codes
The LDPC codes [14] are also popular easy to implement encoding technique that attains a close-capacity
performance. The LDPC code is basically a block code described by a large dense parity check matrix H
such that HG = 0. Due to the matrix description, the LDPC code is linear and it will form the basic
building block in Chapter 7.
The LDPC decoding is a nice example of the FG-SPA implementation that will be discussed in Example
4.13. Although the FG-SPA can be directly applied for the LDPC decoding, it is much more efficient to
employ an implementation framework, where the message is represented by soft bit and the corresponding
update rules are shown e.g. in [23].
Note 3.1 (Enconding Procedure). The encoding of the LDPC codes is not straightforward, because they
are defined using the parity check matrix. One can obtain the corresponding generator matrix directly
using the Gaussian elimination from the parity check matrix. The problem is in the numerical intensity
for large matrices. We refer [49, Appendix A] for an efficient way of the LDPC encoding.
Note 3.2 (Parity Check Matrix Obtaining). Design of good parity check matrix is a nutshell, because the
matrix should be very large to respect the very long codeword assumption for the capacity achieving code.
Particularly the properties of the LDPC code are given by degree distribution as well as by the girth, that
is the length of the shortest loop in the corresponding graph [49]. The size of the matrix makes numerically
intensive to provide some additional loops removing algorithms from a given code as suggested e.g. in [34].
3.1.4 Other Encoding Techniques
One can find much more channel codes with various utility targets (channel capacity, latency). This is
however not directly related with this thesis. We therefore point out mainly the lattice codes [17] from the
other encoding techniques. This family of codes is very nice tool used mainly in the information theory.
The problem of the lattice codes is that their direct implementation is not straightforward and therefore
they serve rather as a tool proving some performance than a manual for a direct implementation.
3.2 Receiver Processing
The receiver is assumed to obtain a noisy observation of the encoded vector. It aims to recover the original
data [25,43]. First a (symbol-wise) metric is evaluated, since we assume a memoryless channel. This metric
is served to the decoder. We will shortly discuss some fundamental algorithms capable to provide the MAP
estimation (or some approximation) for the aforementioned encoding procedures [9].
3.2.1 FSM Decoding - Viterbi Algorithm
The Viterbi Algorithm (VA) [9] is a well known algorithm providing the vector-MAP estimation of the
FSM-coded data vector in with a linear complexity.
The algorithm optimizes a metric across an underlaying FSM trellis in the forward part keeping sur-
viving paths active, that is a hard decision about a path in the trellis according to the metric increments
11](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-24-320.jpg)
![3.2. RECEIVER PROCESSING
is performed. In the backward part, one resulting path is selected according to the overall accumulated
optimized metric.
We start with the vector MAP description to derive a metric for the VA. The causality of the FSM
coding and ability to be described by the trellis [18] gives
p(d|x) ∼ p(x|d)p(d) (3.1)
=
i
p(xi|x0,...,i−1, d)p(di) (3.2)
=
i
p(xi|x0,...,i−1, d0,...,i)p(di) (3.3)
=
i
p(xi|x0,...,i−1, ςi−1, di)p(di), (3.4)
where the causality was used in (3.3) and the FSM description in (3.4). Using the memoryless property of
the channel, we finally obtain
p(d|x) ∼
i
p(xi|ςi−1, di)p(di) (3.5)
The trellis transition is unambiguously given by a pair Ti = (ςi−1, di) and the probability of the
transition with respect to a given received vector x refers to (3.5), namely
γ(Ti) = p(xi|ςi−1, di)p(di) (3.6)
for memoryless channel. The term γ(Ti) can be viewed as an increment of the metric for a corresponding
transition Ti since p(d|x) ∼ i γ(Ti).
Thanks to the monotonicity of the logarithm operation, the equation (3.4) can be also written as
log(p(d|x)) ∼
i
log(γ(Ti)) (3.7)
and the corresponding vector MAP estimation can be written in logarithm form as ˆd = arg maxd log(p(d|x)).
The VA now presents an efficient way of this maximization. The VA relies on a fact that each possible
data vector corresponds unambiguously with a path in the whole trellis. The main simplification is that
one is interested only is the path with the maximal probability leading to a given state in the ith step. All
of the other pats can be trashed, because they surely does not refer to the part of data vector maximizing
the vector MAP estimation.
We define a metric ρi ∈ R as an accumulated metric in the ith state and ∆ρ as the metric incre-
ment referred to a transition Ti, e.g. ∆ρ(Ti) → γ(Ti) and ρi =
i
j=1 γ(Tj). The accumulated metric ρi
corresponds to a certain path 2
in the trellis, i.e. {Tj}i
j=0. The Viterbi Algorithm is then given by:
1. initialization: set the worst possible metric to all states excepting the initial one ς0
2. the forward part of the VA also know as add-compare select unit: In each possible state ςi provide
the following:
(a) assuming that the metric is already evaluated in the i − 1th step, evaluate the metric in the
next state by ρi = ∆ρ(Ti)(add)ρi−1 for all possible transitions leading to a given state Ti :
S(di, ςi−1) = ςi, where the add operation stands for plus in the log-domain and product in the
probability domain.
2The certain means is the path optimizing ρi among all possible paths in VA.
12](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-25-320.jpg)
![CHAPTER 3. ERROR PROTECTION TECHNIQUES
(b) compare all evaluated metrics evaluated in the previous step,
(c) select the transition with the best ρi as the surviving path, save that and discard all other
transitions.
3. backward part: select the final state with optimal metric and by the surviving paths select the overall
path in the trellis. The final selected path refers to the vector MAP estimation of the data vector.
3.2.2 FSM decoding - FBA Algorithm
The Forward Backward Algorithm (FBA) is suited for the turbo codes decoding. The FBA aims with the
symbol-MAP estimation for a given FSM. The algorithm consists from a recursive forward and backward
part.
Using a similar argumentation as in the VA derivation, we can derive [18] the symbol MAP estimation
with the FSM description:
p(di|x) =
ςi−1
p(ςi−1, di|x) (3.8)
∼
ςi−1
p(x|ςi−1, di)p(ςi−1, di) (3.9)
=
ςi−1
p(x0,...,i−1|ςi−1, di)p(xi|ςi−1, di, x0,...,i−1)
p(xi+1,...,N−1|ςi−1, di, x0,...,i)p(ςi−1, di) (3.10)
=
ςi−1
p(x0,...,i−1|ςi−1)p(ςi−1)
α(ςi−1)
p(xi|ςi−1, di, x0,...,i)p(di)
γ(Ti)
p(xi+1,...,N−1|ςi)
β(ςi)
, (3.11)
where (3.10) follows from the chain rule and (3.11) is given by causality of the FSM. Since ςi = S(ςi−1, d),
we can rewrite (3.11) to
p(di|x) ∼
ςi−1
α(ςi−1)γ(T(di, ςi−1))β(S(di, ςi−1)), (3.12)
where T(di, ςi−1) = Ti.
One can derive (e.g. [18]) a recursive formula
α(ςi−1) =
Ti−1:ςi−1
α(ςi−2)γ(Ti−1) (3.13)
β(ςi−1) =
Ti:ςi
β(ςi)γ(Ti), (3.14)
for α and β metrics evaluation serving jointly with (3.12) for an efficient evaluation of the symbol-wise
MAP estimation which is called the Forward Backward Algorithm.
Note 3.3 (Turbo Decoding). The main purpose of the FBA lies in the turbo decoding. The FBA is
applied to the SISO blocks referring to the concatenated FSM based encoders. The algorithm is iterative
and the convergence can be described by EXIT-charts. The decoder consists of two either parallel or serial
concatenation of Soft In Soft Out (SISO) blocks connected by (de)interleaver.
The FBA is iteratively performed in the SISO blocks, where the output is evaluated according to an
input from the complementary SISO block and the output serves as the input for the second SISO block
and conversely.
13](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-26-320.jpg)
![3.3. CONVERGENCE CRITERIA
3.2.3 Graph Based Solution
A common point of the aforementioned decoding algorithms is that the system can be described using a
graph, that is
• FSM trellis in the VA case,
• compound FSM trellis with interleaver in the FBA-based turbo decoding.
One can consider a generalization of these algorithms. The factor graph describing a structure of an
arbitrary system can be also used for the aforementioned decoding techniques. Some algorithms can be
then defined upon the factor graph. Since the algorithms operate upon the decomposed system, they are
much more efficient than the brute-force solution optimizing the global goal at the same time.
One of such algorithm (sum product algorithm) will be described later in greater detail. The LDPC
codes can be decoded using this algorithm as it will be shown later.
3.3 Convergence Criteria
The turbo decoding as well as the graph based decoding have an iterative nature. The essential question
is a convergence of such algorithms. The solution is achieved in a suboptimal way, that is the prize for the
system trackability. The main tool that we will discuss are the EXIT charts [19,56] enabling to describe the
convergence properties by means of the mutual information transfer investigation between some entities.
Some other convergence criteria are discussed in Sec. 4.3.2.
3.3.1 Log-likelihood Arithmetics
The log-likelihood algebra [21] is used with advantage for a description of a random variable. This de-
scription is best suited in the binary case, that is random variable with two possible states. This refers to
binary data in the digital communications. This algebra simplifies the description of the random variable.
Assume a random variable S with pmf given by P(S = −1) = p−1 and P(S = 1) = p1. The domain of
S is a binary GF with zero element −1. The Log-Likelihood Ratio (LLR) is defined as λ = log(p1/p−1).
One can see that the binary random variable is stochastically described just by one number. Using the
natural identity p−1 + p1 = 1, we can express the pmf as
p−1 =
1
exp(λ) + 1
=
exp(−λ/2)
exp(λ/2) + exp(−λ/2)
p1 =
exp(λ)
exp(λ) + 1
=
exp(λ/2)
exp(λ/2) + exp(−λ/2)
. (3.15)
The a posteriori probability after receiving an observation x is p(s|x) ∼ p(x|s)p(s), which can be written
using LLR as
λs|x = log
p(s = 1|x)
p(s = −1|x)
= log
p(x|s = 1)
p(x|s = −1
xλC
+ log
p(s = 1)
p(s = −1
λs
. (3.16)
If we consider Gaussian channel p(x|s), one can show that λC = 4γ. We further note that
λs|x1,x2
= λC1
x1 + λC2
x2 + λs. (3.17)
for conditionally independent x1 and x2.
14](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-27-320.jpg)
![CHAPTER 3. ERROR PROTECTION TECHNIQUES
The GF(2) addition denoted by ⊕ is a cornerstone operation for the coding. We now assume a random
variable given by x = x1 ⊕ x2, where x1 and x2 are described by their LLR λx1
and λx2
. It can be shown
that
λx = 2 tanh−1
(tanh(λx1
/2) tanh(λx2
/2)). (3.18)
3.3.2 Binary EXIT Charts
The EXIT charts developed in [56] bring a semi-analytical tool well describing the convergence of the paral-
lel turbo codes. The principle consists in measuring of the extrinsic information exchange of concatenated
soft in soft out blocks (SISO). Nevertheless this principle is more general and it can be extended also for
LDPC codes, where the SISO blocks are given by check nodes and variable nodes.
We assume a SISO block C. A mutual information between systematic data d and a prior input of the
SISO block A is called an intrinsic information I(d, A) = IA. The mutual information between the data
and the SISO output I(d, E) = IE is called an extrinsic information. A function T given by IE = T(IA) is
called extrinsic transfer function. If the prior information is in the Gaussian form, the mutual information
is given only by variance of A denoted σA.
In case of a concatenation of more SISO blocks, where the extrinsic information of the first one becomes
to the intrinsic information of an another block and conversely, the convergence of the system as whole
can be described using the extrinsic transfer functions of individual blocks.
We demonstrate the principle on the parallel turbo code example [56]. We assume that the received
signal is given by x = s + w, where w is an AWGN and s ∈ {−1, 1} is a systematic bit. The likelihood is
then given by
Λ(s) = p(x|s) =
1
2πσ2
w
exp −
(x − s)2
2σ2
w
. (3.19)
We use the log-likelihood form
X = log
p(x|s = 1)
p(x|s = −1)
=
(x + 1)2
− (x − 1)2
2σ2
w
=
2x
σ2
w
=
2
σ2
w
(s + w), (3.20)
which can be rewritten in the form X = µXs + nX with mean
µX =
2
σ2
w
(3.21)
and a zero-mean Gaussian random variable nX with variance
σX =
4
σ2
w
. (3.22)
The variance and mean of X are connected through σ2
w as
µX =
σ2
X
2
. (3.23)
The log-likelihood form of A in case of SISO block in AWGN channel was shown to have the Gaussian
distribution. Therefore it can be written in the form A = µAs + nA, where s is the systematic bit. The
mean is given by
µA =
σ2
A
2
. (3.24)
15](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-28-320.jpg)
![3.3. CONVERGENCE CRITERIA
Using this mean, we can write the likelihood (3.19) as
pA(a|s) =
1
2πσ2
A
exp −
(a − µAs)2
2σ2
A
(3.25)
=
1
2πσ2
A
exp −
(a −
σ2
A
2 s)2
2σ2
A
. (3.26)
A joint pdf p(a, s) used to evaluate the intrinsic mutual information is given by
pA(a, s) = pA(a|s = −1) + pA(a|s = 1) (3.27)
and the intrinsic information is therefore given by
IA =
s
pA(a, s)
pA(a)p(s)
pA(a, s)
da (3.28)
=
1
2 s=±1
∞
−∞
pA(a|S = s)
log
2pA(a|S = s)
pA(a|s = 1) + pA(a|s = −1)
da. (3.29)
The extrinsic information can be evaluated in the same way, where the extrinsic distribution is used,
that is
IE =
s
pE(a, s)
pE(a)p(s)
pE(a, s)
da (3.30)
=
1
2 s=±1
∞
−∞
pE(a|S = s)
log
2pE(a|S = s)
pE(a|s = 1) + pE(a|s = −1)
da. (3.31)
Note 3.4 (Investigation of turbo decoding). The EXIT function is evaluated for the SISO blocks given
by the FSMs. During the evaluation of the EXIT function, one must pay attention that the probability
serving for the evaluation (3.29) and (3.31) is in a proper form.
Note 3.5 (Investigation of LDPC decoding). The first SISO block is formed by variable nodes with
connection to observation and the second SISO block is given by the check node. One can find the EXIT
chart application on LDPC codes e.g. [57].
16](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-29-320.jpg)
![4 Factor Graphs and Related Algorithms
The aim of this Chapter is to introduce the sum product algorithm on factor graphs and to formally describe
the issues that should be taken into account during the implementation of this algorithm. We formally
provide motivation for the sequel Chapter 5, where a concrete implementation framework methodically
solving the issues is introduced as one of the main contribution of this work. Finally, we briefly touch the
extension to looped FG-SPA and corresponding problems within this Chapter.
The factor graph (FG) is a tool graphically describing an arbitrary function (system) called the global
function using its inner structure. The sum-product algorithm on factor graph (FG-SPA) is a highly-
efficient algorithm capable to solve a wide family of problems. The most important one for this work is
the MAP-detection.
The FG-SPA evolved as a generalization of trellis algorithms, where a general graph is considered
instead of a trellis [62]. The framework of the FG-SPA is introduced in [27, 33], where the authors also
show a number of applications such as the FSM/decoding, Kalman filtering, etc.
The problem of the inference (marginalization of the global function serving for e.g. the MAP-detection)
is complex when it is applied on the global function. The point of the FG-SPA is that it marginalizes
local functions in the decomposed system instead of the global function with typically significantly less
complexity. A crucial assumption is a system decomposability. If some variables represented in the FG are
random and independent, their joint probability (global function) can be composed as a product of some
parts referring to the independent variables. The independence implies a strict limitation of the FG-SPA,
that is the FG representing the global function must be cycle-free1
.
It was shown that the FG-SPA often works well also in the systems leading to a looped-graph, where
the FG-SPA was not originally designed for. We stress that the inference is only approximation in the
FG-SPA operating on the looped graphs (called also the loopy belief propagation [36,66]). The algorithms
very similar to the looped FG-SPA were re-discovered independently in more fields.
Many practical problems across many fields lead to the looped-graphs. Most notably for this work,
we again mention the MAP-detection of the capacity achieving codes such as turbo or LDPC codes. We
stress that the FG-SPA (or some of its variant) is capable to cope among others all decoding algorithms
discussed in Chapter 3.
Furthermore, the global function need not to be restricted only to the decoder itself. We can consider
e.g. a joint decoder and phase estimator as one global function. Therefore all decoding and signal processing
can be covered by one global function. The advantage is that the individual tasks are processed jointly
and therefore with a potentially better performance than solving the tasks independently.
The major problems that are presented in the looped FG-SPA are:
• convergence of the FG-SPA issue,
• FG-SPA message representation.
The convergence of the FG-SPA issue stands for a principle-nature issue. The cycled graph implies
unknown fixed points existence and their uniqueness. The convergence of the FG-SPA to a proper fixed
point is not also guaranteed in general. We refer [36] and references from there (e.g. [66]) for the reader
interested in the investigation of the looped FG-SPA convergence topic.
While the issue of the FG-SPA convergence is a principle-nature problem, the message representation
stands for the implementation nature problem. The problem is, roughly said, how to represent a general
continuously valued function by means of several parameters (the less the better), because the messages
are passed through the corresponding factor graph and therefore their description is unavoidable for the
implementation. A methodical solution for this problem is one of the main contribution of this work. We
refer Section 5.1.2 or e.g. [10, 13, 28] and references from there as an overview of the current state of art
for the FG-SPA message representation problem.
1According to the original definition presented in [27,33].
18](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-31-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
4.1 Cycle-Free Factor Graph and Sum Product Algorithm
We will first consider a cycle-free FG-SPA and present a fundamental theorem 4.1 stating the possibility
of an exact inference evaluation in the cycle-free FG-SPA. A discussion of the implementation issues will
follow.
4.1.1 Formal Definition of the Factor Graph
We start with some formal description of the system. A motivation of the following notation is to describe
a collection of functions {fI}I forming some local parts of the global function. The factor graph for such
a system is then defined.
Let x ∈ Ax be a multi-dimensional variable with dimensions [x1, . . . , xN ] ∈ Ax1
× · · · × AxN
= Ax,
where a set2
given by the individual dimensions is denoted by X, i.e. X = {x1, . . . xN }. Further let
F = {f1, . . . fM }, fI : Aψ(fI ) → CI, ∀I ∈ {1, . . . , M} be a set of functions, where Aψ(fI ) is a domain of
fI and CI is its codomain. We defined a mapping ψ(fI) : F → XnI
assigning a vector of arguments to
a given function fI and nI denoting the number of arguments of fI. Similarly, we also define a mapping
υ(i) giving a vector of functions, where xi as the argument.
We consider two functions f1(x1, x3, x4) and f2(x2, x4). The domain of f2(x2, x4) = f2(x2,4) =
f2(ψ(f2)) is Aψ(f2) = Ax2,x4
= Ax2
× Ax4
, the codomain is C2, the mapping ψ(f2) = x2,4 =
[x2, x4] and n2 = 2. Similarly domain of f1(x1, x3, x4) = f1(x1,3,4) = f1(ψ(f1)) is Aψ(f1) =
Ax1,x3,x4 = Ax1 × Ax3 × Ax4 , the codomain is C1, the mapping ψ(f1) = [x1, x3, x4] and n1 = 3.
Finally υ(x1) = f1, υ(x2) = f2, υ(x3) = f1 and υ(x4) = [f1, f2].
Example 4.1 (Simple Demonstration of Notation).
Using the introduced notation, we can formally define the factor graph as a visualization of a global
function, where the global function (whole factor graph) is composed from local functions as in Example
4.1. The global function g(x) can be composed from two local functions f1(x1, x3, x4) and f2(x2, x4) as
shown in Fig. 4.1. One can see that the factor graph respects inner structure of the global function (see
Example 4.2), where the functions are visualized by factor (check) nodes and the variables by variable
nodes. The arguments of the function are represented by edge between the corresponding factor and
variable node.
If C = I CI forms a commutative semi-ring, the factor graph is defined as a bipartite graph
G = (N, V, E), where N = {F1, . . . FM } is a set of FN referring one to one to the set of functions
F, V = {ν1, . . . , νN } is a set of VN referring one to one to the set X and E = {e1, . . . , eL} is a set
of edges connecting FN and VN if and only if the variable represented by the VN is argument
of the local function referring to the FN. It means that an edge connects a FN FI ∈ N referring
to fI and a VN νi ∈ V if the variable xi represented by the VN νi is an argument of fI. This
can be written using the mapping ψ as νi is connected with FI if and only if xi is in the vector
given by ψ(fI).
Definition 4.1 (Factor Graph).
2We mean an ordered set by default.
19](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-32-320.jpg)
![4.1. CYCLE-FREE FG-SPA
For each edge given by a FN XI and a VN xi, we assign two messages. The first one is from the
VN to the FN (µxi→XI
(xi)) and the second one with a converse direction (µXI →xi
(xi)). The
message is a function of the variable xi. A domain of the message is a set µ ∈ Aµ = M.
Definition 4.2 (Message in FG-SPA).
Note 4.3 (Notation comments to messages). The domain inherits the identification of the message, i.e.
µxi→XI
(xi) ∈ Mxi→XI
. Each message can be parameterized by some further parameters which are situated
into the subscript. We drop (simplify) the message parameters or its identification, whenever the this
information is clear or not important.
Motivated by the marginal evaluation using the messages, we define the update rules defining a relation
of the messages. Note that the definition corresponds to a commonly used update rules [27,33].
The FN update rule of a given messagea
µFI →xi
is a mapping UF : Mψ(FI )∼xi→FI
→ MFI →xi
,
where µψ(FI )∼xi→FI
∈ Mψ(FI )∼xi→FI
are the messages inputing the node FI excepting µxi→FI
and similarly for the variable node, i.e. UV : Mυ(xi)∼FI →xi
→ Mxi→FI
. The particular form
of the update rules is given by
µFI →xi
k+1 (xi) =
ψ(fI )∼xi
fI(ψ(fI))
xj :ψ(fI )∼xi
µ
xj →FI
k (xj) (4.6)
µxi→FI
k+1 (xi) =
FJ :υ(xi)∼fI
µFJ →xi
k+1 (xi), (4.7)
where (4.6) defines the update rule in FN and (4.7) defines the VN-update. Note that the
sum operator becomes to integral whenever the domain Axi is a continuous interval. The
subscript k denotes the iteration.
aThe unary tilde operation means the whole vector expecting the argument. See Section 1.4 for details.
Definition 4.3 (Update Rules).
The marginal of a variable xi given by
I∈υ(i)
µXI →xi
(xi) (4.8)
and it represents a marginalized global function serving for further purposes.
We state a fundamental theorem of the FG-SPA serving for the marginal evaluation using the message
passing directed by the update rules.
Let x be a multi-dimensional random variable described by a joint pf g(x). If the FG G referring
to g is cycle-free, the message passing directed by the update rules defined by (4.6) and (4.7)
provides messages enabling an exact inference evaluation (4.8) with an arbitrary scheduling
algorithm containing all messages.
Theorem 4.1 (Fundamental FG-SPA Theorem).
22](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-35-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
x1 F1
x3
x4 F2 x2
A2A3Oθ
µx1→F1
(x1)
µx3→F1
(x3)
µF1→x4
(x4) µF2→x2
(x2)
µx4→F2
(x4)
µA2→x2
(x2)
µA3→x3
(x3)
µO→x1
(x1)
Figure 4.2: Simple demonstration of FG-SPA corresponding to Example 4.5. The messages required for
the marginalization of p(x2, θ) are denoted.
The theorem is so fundamental that the proof can be found in many tutorials to FG-SPA e.g. in [27].
4.1.3 Sum Product Algorithm
Although the FG can be established for any commutative semi-ring codomain, we consider Ax to be a
support of a multi-dimensional random variable x. Providing further specification, we can formally define
the sum product algorithm.
We consider a factor graph G. The sum-product algorithm on the factor graph is the message
passing algorithm directed by the update rules (Def. 4.3), where
• the local functions fI are conditional pf.
• the marginal (4.8) is a marginalized pf p(xi), i.e. p(xi) = I∈υ(xi) µXI →xi
(xi).
We recognize forward messages having a prior form and backward messages in likelihood form.
Definition 4.4 (Sum Product Algorithm).
The local functions fI are some probability functions (pf - pdf, pmf or their mixture) up to a non-negative
scale factor describing a multi-dimensional random variable ψ(fI). Consequently we state the codomain
CI ∈ 0, ∞), ∀fI.
Note 4.4 (Other Algorithms on FG). The composition and marginalization operations are more general.
One can consider e.g. max-sum algorithm, where the marginalization is given by maximization and the
composition by sum operation. The Viterbi Algorithm in the log-domain can be mentioned as an instance
of the max-sum algorithm.
23](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-36-320.jpg)
![4.1. CYCLE-FREE FG-SPA
We return to Example 4.2. We specify that x1 is connected to an observation θ. Further, we
assume an a prior information x2 and x3. This is expressed by adding further factor node to
the graph as one can see in Fig. 4.2 The application of the update rules gives
µF1→x4
(x4) =
x1,x3
p(x1|x3, x4)
f1(x1,x3,x4)
p(x3)
µx3→F1
p(θ|x1)
µx1→F1
=
x1,x3
p(θ, x1, x3|x4) = p(θ|x4). (4.9)
Using the VN update, we can see that µF1→x4
(x4) = µx4→F2
(x4) and the second FN update is
given by
µF2→x2
(x2) =
x4
p(x4|x2)
f2(x2,x4)
p(θ|x4)
µx4→F2 (x4)
= p(θ|x2). (4.10)
Using a prior information about x2 (message µA→x2
(x2) = p(x2)), we evaluate the marginal as
µF2→x2
(x2)µA→x2
(x2) = p(x2)p(θ|x2) = p(θ, x2). Note that we can recognize forward form of
the messages (a prior form) and backward messages (likelihood).
Example 4.5 (Message passing in a simple example).
One can recognize the message µ(x4) evaluated in Example 4.2 in (4.9).
4.1.4 MAP Decoding and MMSE Estimation Using FG-SPA
The MAP decoding is a fundamental capability of the FG-SPA for our purposes. The maximum a posterior
probability of a parameter ξi given by an observation θ is given by (2.10) ˆξi = arg maxξi
p(ξi|θ). This can
be rewritten as
ˆξi = arg max
ξi
p(ξi|θ) (4.11)
= arg max
ξi
p(ξi, θ) (4.12)
= arg max
ξi
x∼ξi
g(x) (4.13)
= arg max
ξi
x∼ξi I
fI(ψ(fI)). (4.14)
Assuming a cycle-free FG G referring to the global function g, we can use fundamental SPA Theorem 4.1
to evaluate exactly the marginal pf p(ξi, θ). An evaluation of the MAP estimation is then straightforward
with a reasonable complexity. The MMSE estimation can be done in a similar manner, i.e. using (2.9), we
have
ˆξi = E[p(ξi|θ)] (4.15)
= E
x∼ξi I
fI(ψ(fI))
(4.16)
for a given observation θ.
24](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-37-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
4.2 FG-SPA Implementation
One of the main advantage of the FG-SPA is that it defines powerful algorithms. These algorithms are
described precisely using the FG-SPA. Nevertheless if one would like to implement these algorithms, some
difficulties arise. We browse these difficulties including their formal description. According to this descrip-
tion, we form a formal requirement for the implementation framework serving for a direct implementation
of the FG-SPA.
4.2.1 Message Types in FG-SPA and their Representation
We start with description of the message representation. We have to somehow cope the representation
of the message for the processing purposes. We are able to store and process a finite number of discrete
values. This gives as a basic constraint, which is not important in case of discrete messages. The FG-SPA
is, however, defined also for messages describing continuously valued variables (pdf) requiring uncountable
number of parameters for their direct exact representation in general. This brinks a question about a
parameterization of these messages.
Since we are able to store a finite number of parameters, the message is therefore parameterized by an
n-dimensional vector q ∈ Aq ⊆ Rn
. The message µ ∈ M is then given by µ = Φ(q), where Φ : Aq → M is
a function defining the parameterization. Obviously, the less number n the more efficient parameterization.
On the other hand some messages may need an infinite number of parameters to be exactly represented.
We will also use an inverse mapping Φ−1
: M → Aq to determine the parameter vector for a given message,
i.e. q = Φ−1
(µ).
The parameterization of the message is therefore given by the mappings Φ : Aq → M and Φ−1
:
M → Aq determining the relation between the message µ ∈ M and its parameterization q ∈ Aq ⊆ Rn
.
We further define a dimensionality of the message as number of parameters D required for the message
representation, that is the length of the vector q.
Within this work, we assume all variables xi to be scalar variables in complex domain or some of its
subset, i.e. xi ∈ Axi
⊆ C, ∀xi ∈ X. According to the domain Axi
of a variable xi, we recognize the following
types of the messages connected to the VN xi:
Discrete Messages
The discretely valued message is a message connected to xi, if Axi
has a finite number of elements. A
straightforward representation of such a message is q = [. . . , qj, . . . ], where the components qj refers to
the elements of Axi
. This refers to the pmf of the corresponding variable xi. The minimal dimensionality
over all parameterizations of this message is obviously upper-bounded by the number of parameters, i.e.
min(D) ≤ |Aq| = Mq.
25](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-38-320.jpg)
![4.2. FG-SPA IMPLEMENTATION
A typical example is a binary message, where xi ∈ Axi = {−1, 1} and µ(xi) = [q0, q1] = [p−1, p1].
An another form of the binary message representation is log-likelihood ratio (LLR). Since xi is
a binary random value, it can be described by a log likelihood ratio given by λ = log(p1/p−1)
standing for mapping Φ−1
. Since p−1 + p1 = 1, we can express the message by (3.15), i.e.
µ(xi) = [p−1, p1] = Φ(λ) =
1
1 + exp(λ)
,
exp(λ)
1 + exp(λ)
. (4.17)
Note that the dimension of the message is D = 1, because the message can be fully described by
a single real number (λ). We therefore found a parameterization of a pmf with lower dimension
then the description by the pmf itself, where D = 2 for the binary variable.
Example 4.6 (Log-likelihood message representation).
Parameterizable Continuous Messages
The continuous parameterizable message is a message connected to xi, if Axi
is an infinite set, e.g. a
continuous interval I ⊆ R. This message is continuous, but directly parameterizable. The direct parame-
terization is a concrete form of the function Φ. The number of parameters again forms the upper-bound
on the message dimensionality.
As an example, we mention the real Gaussian message fully described by its mean and variance
q = [¯xi, σx] and the message is given by
µ(xi) = Φ(¯xi, σx) =
1
2πσ2
x
exp
|xi − ¯xi|2
2σ2
x
. (4.18)
The inverse mapping Φ−1
is given by q = Φ−1
(µ(x)) = [Ex[µ(x)], var[µ(x)]]. Note that it can
be shown that the mapping Φ is bijection in this case.
Example 4.7 (Gaussian message representation).
General Continuous Message
A general continuous message is a message connected to xi, if Axi
is an infinite set and there is not any
straightforward parameterization of the message. This means that the dimension of the message is either
infinite, or we are not able to find a parameterization with a reasonable dimension.
The message appears whenever both discretely and continuously valued messages are presented
in one FG-SPA. We mention the phase model of the joint decoder phase estimator as an instance.
The problem is increasing number of parameters required for the message parametrization with
increasing iteration. See a demonstration in Fig. 4.3.
Example 4.8 (Mixed-Type Message).
26](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-39-320.jpg)
![4.2. FG-SPA IMPLEMENTATION
M
ˆM
AqE
Aq
Φ−1 (µ)
Φ(q)
Figure 4.4: Principle of the message approximation given by the parameterization ˆµ = Φ(q). The true
message µ ∈ M is mapped through a mapping Φ−1
(dashed arrows) to a parameter vector q ∈ Aq. The
vector q ∈ Aq is unambiguously related with an approximated message ˆµ ∈ ˆM by a mapping Φ (dotted
arrows). We denoted a hypothetical vector qE ∈ AqE
exactly representing the true message µ (dash-dotted
set).
If a domain of the variable xi that is represented by a message µ(xi) is some real interval, i.e.
xi ∈ Axi = I ⊆ R, the message can be surely parameterized by samples. It means that an
original true continuously valued message µ(xi) ∈ I = a, b) in the pdf form is represented a
vector q = [q1, . . . , qD]
T
= [µ(a), . . . , µ(a + (D − 1)∆)]]
T
, where ∆ = (b − a)/D. This formally
defines the mapping Φ−1
(µ). And the approximated continuous message is then composed
(mapping Φ) as a piecewise function from the samples
ˆµ(t) =
D−1
i=0
qi+1ν(t − i∆), (4.19)
where ∆ = (b − a)/D, ν(t) = 1 for t ∈ 0; ∆) and ν(t) = 0 otherwise. Using a finite number
of samples (see Fig.4.8a), we are able to represent an arbitrary message in a finite interval.
Intuitively, the more samples the more exact representation can be expected but at expanse of
higher computational complexity.
Example 4.9 (Representation of a Continuous Message by Samples).
The number of samples in Example 4.9 suggests a crucial natural question about quality of the approxi-
mation. One can feel a trade-off between the fidelity of the message representation and the dimension of the
parameterization. The fidelity can be measured by means of the overall FG-SPA performance. Neverthe-
less, it would be useful to define a metric M(µ, ˆµ) that assigns some quality to the message representation
computable just from the message approximation and the true message. As an example of such a metric,
we mention the mean square error of the approximated message. This metric will be optimized by the
proposed KLT message representation in Sec. 5.2.1.
28](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-41-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
χ1(x) χ2(x) χ3(x)
µ(x) ≈ c1χ1(x) + c2χ2(x) + c3χ3(x)
Figure 4.5: Demonstration of the canonical message representation.
Canonical Message Representation
A general concept of the message representation is proposed in [63], where the authors propose the canonical
message representation consisting in a set of kernel function representing the message as a superposition
of some kernel functions. The message is then defined by a parameter vector qj.
The message is written in the form
ˆµ(xi) =
j
χj(qj, xi), (4.20)
where χj are the kernel canonical functions and qj is a set of parameters (see Fig 4.5 for an illustration).
This is, however, rather a generic concept how to describe a certain class of message representation,
because no way how to obtain methodically the kernel functions is not given. Furthermore, the canonical
representation solves only the parameterization ˆµ = Φ(q). No way of the evaluation of the parameter q is
given in general (function Φ−1
).
Note 4.10 (Linear Canonical Message Representation). If the mapping Φ can be written as a superposition
a some kernel set, we refer such a representation given by Φ as a Linear Canonical Message Representation
(LCMR). It means that a message can be written in form
µ(x) ≈ ˆµ(x) =
D
i=1
qiχi(x), (4.21)
where q is the vector parameterizing the message and {χi(x)}D
i=1 is a collection of (canonical) kernel
functions. Note that in this case, we still have available only the mapping Φ. In the other words, we no
way of the parameterization obtaining from the true message is given in general. An example with three
canonical kernels is shown in Fig. 4.5.
Note 4.11 (Linear Canonical Message Representation with Orthogonal Kernel). If we assume a LCMR,
where the kernel set {χi(x)}D
i=1 is orthogonal, we are able to construct also the mapping Φ−1
by means of
the inner product, since χi, χj = 0, ∀i = j and therefore the i-th component of the parameter vector is
given by qi = µ, χi .
4.2.3 Update Rules Upon the Message Representation
We introduced the update rules upon the messages in (4.6) and (4.7) as mapping6
U : Mn
→ M, for some n.
That is the update rules defined upon the true messages. Nevertheless during the processing, the message
6We assume an equal domain for all messages corresponding to the update for the notation simplicity. Nevertheless there
is no reason for this assumption in general.
29](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-42-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
provided by a chain of mappings given by Φ ◦ U ◦ Φ−1
: An
q → ˆMn
→ ˆM → Aq for some n, where we
assume the parameterization of the message as both input and output to the update rules. See Fig. 4.7
for demonstration.
This is some middle-step between the direct definition of the update rules and the desired implemen-
tation framework.
Update Rules Directly Upon the Parameterization
This case refers to the target of the derivation of the update rules for a general message representation.
It provides a direct relation between the message parameterizations. It means that the chain of mappings
from the previous case is reduced to Aqn → Aq and the integration required in Def. 4.3 is avoided for
finite length of vector q (dimension D). We will closely focus on this topic in Section 5.2.1, where we
introduce the update rules for a generic LCMR with orthogonal kernels introduced in Note. 4.11.
As an example, we mention the update rules upon sampling of the continuously valued message
showed in Example 4.9. One can show [13] that the update rules (Def. 4.3) applied on the
samples refer to the numerical integration with rectangular rule and therefore a natural solution
for the update rules. The samples (parameterization) of pdf (true message) are taken as pmf
inputting into the update rules.
Example 4.12 (Update Rules Upon Sampling).
Another example is the update rules design for the LLR (or rather soft bit) message (4.17) suited
for the decoding of LDPC codes. The factor nodes refer to the check nodes proving GF-sum,
i.e. ⊕ operation. The solution can be found e.g. in [23].
Example 4.13 (Update LLR for LDPC).
4.2.4 FG-SPA Implementation Framework
The FG-SPA can be easily (and exactly) implemented for the discrete (especially binary) messages. The
messages are exactly represented by their pmf and the update rules are performed directly according to
their definition (4.6) and (4.7). This is a straightforward way, which is not necessarily the most efficient.
As an example, we can mention the soft bit implementation of LDPC. This implementation attains less
complexity, because both the message representation and the update rules process a single-valued messages
for the binary case as we discussed in Example 4.13.
To unify the ways of the FG-SPA, we define the implementation framework as a message representation
and the corresponding update rules upon the given message representation.
The FG-SPA implementation framework is a tuple (Φ, U ), where Φ defines a message parame-
terization (even approximately) and the corresponding update rules U are defined directly upon
the given message representation.
Definition 4.5 (Implementation FG-SPA Framework).
31](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-44-320.jpg)
![4.2. FG-SPA IMPLEMENTATION
(a) Discretization of the continuously valued
message.
(b) Representation using the coefficients of the
Fourier decomposition.
(c) Gaussian representation given by mean
and variance of the original message.
(d) Representation by Dirac-delta function
posed into the argument of maximum of the
original message.
Figure 4.8: Different message representations of general continuously valued messages.
Note 4.14 (Scheduling Algorithm). We said that the implementation framework is given by the message
representation and the corresponding update rules upon the message representation. We should remind
that in case of looped FG-SPA, we have to also consider some scheduling algorithm. The scheduling
algorithms will be discussed in Sec. 4.3.1.
Criteria of the FG-SPA Implementation Framework
We have already defined the implementation framework, but we did not consider any way, how to compare
the frameworks from some quality point of view. A natural requests of the FG-SPA implementation are:
• efficiency,
• exactness.
Moreover, we are often by a limited memory available to processing. These requests are naturally in
contradiction and therefore some trade-off is often needed. The efficiency request basically means a sim-
ple message representation (represented by a few parameters) that leads to efficient update rules design
(relation between the few parameters).
4.2.5 Instances of the FG-SPA Implementation Frameworks
We show some possible FG-SPA message representation frameworks that can be found in literature in
greater detail (see Fig 4.8). Some of these representations serve later as reference implementation frame-
works for the numerical verification of the proposed design.
Sample Representation with Numerical Integration
The discretization of the continuous message is the most straightforward method of the practically feasible
representation as it is discussed in [12,13]. An arbitrary precision might be achieved using this represen-
tation by increasing the number of samples (complexity of the approximated message). This framework is
given by merging Examples 4.9 and 4.12.
A straightforward evaluation of the update rules makes this representation case very important, because
it is principally usable for every case, where we do not care about the numerical intensity of the FG-SPA.
A direct application of the marginalization upon the message representation (the discrete vector p taking
32](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-45-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
as pmf) leads to an approximation of the marginalization upon pdf, where the integration is performed by
the numerical integration with rectangular rule (see e.g. [13]).
Fourier Representation
The well known Fourier decomposition enables to parameterize the message by the Fourier’s series as
ˆµ(t) = Φ(α, β) =
α0
2
√
π
+
1
√
π
D
i=1
αi cos(it) + βi sin(it), (4.22)
where αi = 1/
√
π µ(t) cos(it)dt and βi = 1/
√
π µ(t) sin(it)dt gives the inverse mapping Φ−1
. Dimen-
sionality is given by D = 2N + 1. In [10], the authors have derived update rules for the Fourier coefficients
in a special case of the random-walk phase model.
This message representation is the LCMR with orthogonal kernels as we stated in Note 4.11. The
Fourier representation also exemplifies the case when ˆM is not a subset of M as one may wonder, when
we discussed the message approximation (see Fig. 4.4). Consider that the set ˆM is given by all pdfs, that
is all non-negative functions that satisfies µ(t)dt = 1. The Fourier approximation, however, does not
guarantee a non-negative message as one can see in Fig. 4.8b.
Dirac-Delta Representation
The message is represented by a single number situated in the maximum of the message (see Fig. 4.8d).
The approximated message is given by
ˆµ(t) = δ(t − arg max
ˇt
µ(ˆt)). (4.23)
The update rules upon the Dirac-Delta message approximation can be found e.g. in [12] for the phase
estimation problem. The update rules are based on the gradient methods
Gaussian Representation
Gaussian representation and its modifications are widely used message representations (e.g. [52]). This
is given by simplicity of the message representation given just by mean and variance (m, σ) with the
interpretation
ˆµ(t) =
1
√
2πσ2
exp −
|t − m|2
2σ2
, (4.24)
where m = E[µ(t)] and σ2
= var[µ(t)] (see Fig. 4.8c) for a real message. We have already discussed this
case in Example 4.7.
The update rules for linear only factor nodes can be found in [32]. The Gaussian representation is
moreover often used to approximate such a message representation that is difficult to be parametrized
directly (e.g. [52,53,55]).
4.3 SPA on Looped Factor Graphs
This section is written as an overview what happens if the factor graph contains loop. Since the FG-SPA
implementation framework as we discussed so far does not explicitly need a cycle-free corresponding FG,
we can freely extend the terms defined in Sec. 4.2 to the looped FG-SPA. Nevertheless, we must pay
attention to the conventional problems of the looped FG-SPA.
33](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-46-320.jpg)
![4.3. SPA ON LOOPED FACTOR GRAPHS
The FG-SPA as defined for the cycle-free FG can be used also in the looped case. The main differences
are summarized bellow:
• The FG-SPA becomes to be an iterative algorithm and a convergence uncertainty occurs.
• Even, if the looped FG-SPA converges, the inference is evaluated only approximately and a further
result investigation is required. Moreover, the stationary point must not be unique.
• The performance of the FG-SPA depends on a message scheduling.
• The message representation issue often becomes to be hardly trackable for mixed-type messages
(Example 4.8).
Although the mentioned issues, the looped FG-SPA presents a tool to solve much more problems then
the cycle-free FG-SPA. We have already mentioned some examples, but the most important application is
capability of the algorithms described in Chapter 3.
4.3.1 Message Scheduling in FG-SPA
We have several choices of the scheduling. There are three fundamentally different approaches, concretely:
1. Random scheduling selects randomly the messages to be updated.
2. Predefined fixed scheduling strictly defines the scheduling prior the SPA starts. It can be parallel,
i.e. all messages are updated once per iteration, e.g. flood schedule algorithm (FSA), or sequential,
i.e. there is a given order and the updated messages are used directly after evaluation. A notable
sequential scheduling algorithm is the FBA used in the turbo decoding.
3. Adaptive scheduling selects message to be updated according to a comparison between a potentially
updated message with the non-updated message. It means that a metric M(µk, µk+1) is selected and
then the message with the metric satisfying some properties e.g. [16] is updated.
We can mix the mentioned principles. For example, one part of messages are passed randomly and the
second one by a predefined scheduling. This approach is reasonable, when the system is composed from a
smaller sub-systems and each subsystem has its own well defined scheduling order.
4.3.2 Convergence of the Looped FG-SPA
The convergence of the looped FG-SPA becomes a huge complication. We cannot use the fundamental
theorem 4.1 and thus we have no guarantee of an approximated inference evaluation. We can mention
several approaches of the looped FG-SPA convergence that can be found in literature.
• Analytical approach [36,37], where the authors describe the update function by an analytical function
of the all messages in one iteration and then they investigate the properties of the function by the
Banach theorem. Their analysis gives a very practical method for binary messages and relatively
small graphs. The FG-SPA with more complex (even non-binary) messages suffers from very difficult
analytical formulas and the convergence test is dropped absolutely for FG-SPA containing continuous
messages in [37].
• Statistical physics approach presented in [66] is completely different to [36, 37]. The authors have
used methods of statistical physics. They found a correspondence between the fixed points of the
FG-SPA and the stationary points of Bethe approximation of the free energy.
34](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-47-320.jpg)
![CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS
• EXIT chart convergence description 3.3.2 [56] was designed as a description of the turbo decoding.
However, the principle is more general, it describes the convergence according to the mutual infor-
mation transfer of some entities. As it was stated in [2], the FG automatically handle correctly the
extrinsic and intrinsic information. Nevertheless the difficulty occurs with the numerical evaluation
of the mutual information, since we cannot generally assume that it can be fully determined by a
single parameter as in case of binary EXIT charts.
We can finally mention that it is known that the correlation decays exponentially with the loop length in
the FG.. Therefore (if possible) it is convenient to maximize the FG-loops length (girth) for the FG-SPA
performance optimization, because the long loops with reduced correlation are closer to the crucial FG-SPA
assumption, that is the decomposability of the system (i.e. independence of the parameters).
35](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-48-320.jpg)
![5 FG-SPA Implementation Framework
This Chapter aims with the implementation issue of the sum product algorithm on factor graphs (FG-SPA)
standing for one of the main results of this work. We first propose a methodical way of a canonical message
representation obtaining for a general continuously valued messages in FG-SPA [44]. This method is based
on Karhunen Loève Transform (KLT) exploiting the stochastic nature of the messages in the FG-SPA1
.
The orthogonal canonical kernels serving for the message representation are evaluated. As a consequence
of the KLT, the mean square error of the message approximation caused by dropping a given set of kernels
can be easily controlled. This can be used for such a message representation that the complexity/fidelity
trade-off can be easily found for a given fidelity goal. We further propose an update rule design for a general
family of linear canonical message representations with orthogonal kernels [47]. Since the eigenfunctions
resulting from the KLT are orthogonal, a generic FG-SPA implementation framework can be composed
from the KLT-message representation and the generic update rules design.
The wide introduction including the problem statement and fundamental definitions are formally stated
in Chapter 4, nevertheless this Chapter is written to be self-contained so that the introduction and defini-
tions is briefly repeated.
5.1 Introduction
5.1.1 Background
The sum product algorithm on factor graphs (FG-SPA) provides a popular generic framework usable in a
wide spectrum of problems across many fields such as communications, signal processing, etc. [27,33]. The
aim of the FG-SPA consists mainly in an efficient evaluation of the decision and estimation problems in
Bayesian networks.
The FG-SPA serves for an exact efficient inference evaluation algorithm, whenever the FG underlaying
the problem is cycle-free. If the FG-SPA contains loop, it can still serve as a high-efficient evaluation
algorithm (e.g. LDPC decoding), but the loops cause that the inference evaluation is no longer guaranteed
to be the exact one. The FG-SPA becomes to be an iterative algorithm and its convergence issue arises.
This convergence issue stands for a principal nature problem and its investigation is out of scope of this
work (see e.g. [36] for an exhaustive overview of the looped FG-SPA convergence investigation possibilities).
Even if a message passing directed by the update rules [27] leads to a correct (not necessarily exact)
marginal evaluation (inference), an efficient FG-SPA implementation is often not straightforward even if
the update rules are clearly defined. The difficulty consists mainly in an infinite memory requested to
store a general continuously valued messages and also in an evaluation often intractable marginalization
integrals required in the update rules evaluation.
A commonly used way of the FG-SPA implementation is to represent (even approximately) the messages
by several discrete parameters which are easy to be stored within the iteration process and to develop the
update rules upon these parameters instead of the update rules upon the original messages. A natural goal
of the FG-SPA implementation is to have
• the simplest possible message representation,
• a low-complex evaluation of the update rules upon the message representation,
• a minimal inaccuracy caused by the use of the message representation and the corresponding update
rules.
1The stochastic process is formed by messages in different realizations and iterations.
36](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-49-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
Obtaining a proper message representation and the corresponding update rules is a crucial point in the
FG-SPA implementation, because a wrong choice of the message representation easily leads to a need for
a very complex messages and the update rules required for a given fidelity goal.
Since the message is represented by a pdf or pmf, it is conventional that messages are properly scaled
such that the marginalization of the message is normed to 1. This normalization is can be easily done for
pdf or pmf. The message representation has to somehow cope this normalization.
5.1.2 Related Work
We can find several message representation - update rules design frameworks in literature. Some of them
are discussed in greater detail in Sec. 4.2.5. An example of the FG-SPA implementation improvement in
case of LDPC decoding over GF(2) consists in the use of single-valued LLR values instead of the binary
pmf as the message representation and the corresponding update rules can be found in [27]. In this case,
the LLR-based FG-SPA implementation gives equal2
results as the direct FG-SPA implementation with
better efficiency.
In the previous case, we had still choice to directly implement the FG-SPA, since all considered mes-
sages were discretely valued. If the messages are continuously valued (represented by a pdf), a message
parametrization is unavoidable. Nevertheless, there exists a straightforward parameterization for a family
of continuously-valued messages (e.g. Gaussian message fully represented by its mean and variance). The
update rule upon the Gaussian messages remains the Gaussian one, whenever the update is linear (only
superposition and addition factor nodes are allowed). In such a case, it is sufficient to find a relation
between means and variances. An exhaustive overview of the Gaussian messages use in the Linear models
can be found in [32].
The easily parameterizable case is, however, a very specific one. Whenever the FG contains both the
discretely and continuously valued messages, an appearance of a mixed-type message becomes unavoidable
(see Fig. 4.3 for an illustration). The mixed-type message is a continuously valued message arising from
the mixed marginalization, i.e. marginalization over both discrete and continuously valued messages. The
problem is with the arising number of parameters in the iterative process requested for the mixed-type
message parameterization. An example of this problem is joint data-detection phase-estimation, where a
discrete-nature data are mixed with a continuous-nature phase shift.
The implementation of the FG-SPA containing the mixed type message appears several time in the liter-
ature. The most straightforward method is sampling of the continuously valued message. The update rules
provided upon the discretely valued sampled messages stand for a numerical integration with rectangular
rule (e.g. [12,13]). An another approach consists in the representation of the message by a Kronecker-delta
function posed into maximum of the message. The update rules are (e.g. gradient methods) are proposed
in [12,13]. These update rules are built on the non-linear optimization basis. We can also mention particle
method, where the message is represented by (weighted) particles (see e.g. [13] for details).
Several works (e.g. [28,52,55]) solve the increasing number of parameters required for the mixed-type
message representation by an approximation of the mixed-type message by a properly selected well param-
eterizable function. The largest attention attracts the Gaussian-form pdf because of its implementation
simplicity.
A unified framework to the message representation issue can be found in [63]. The authors propose
that a general message may be represented using a set of kernel functions and some coefficients related to
the kernel (canonical) functions. This framework does, however, not solve a way of the kernel functions
obtaining and also the update rules design is not solved there.
We can found several works deriving the canonical kernels and related update rules for a particular
system model. As an example, the authors in [5] propose Fourier and Tikhonov message parameterization
2Excepting numerical inaccuracies.
37](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-50-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
used components
unused components
→ MSE
λi
i
ε
Dε
Figure 5.1: Demonstration of the KLT message representation fidelity-complexity trade-off handling. The
residual MSE is indeed easily controlled by the sum of the eigenvalues corresponding to the unused (trun-
cated) components.
where we assume the message to be a real valued function of argument t ∈ I ⊆ R. This message can be
approximated using a canonical kernel set {χi(t)}i by
µx,k(t) ≈ ˆµx,k(t) = Φ(q(x, k)) =
i
qi(x, k)χi(t). (5.1)
The expansion coefficients [. . . , qi(x, k), . . . ] fully describing the approximated message ˆµk,x(t) ∈ ˆM are
random.
The KLT of the process formed by the messages µx,k(t) for random iteration and observation input uses
the second order statistics - the autocorrelation function rxx(s, t) to create an orthogonal kernel set with
uncorrelated expansion coefficients and the second order moments are directly related with the residual
mean square error.
The autocorrelation is function given by
rxx(s, t) = Ex,k [(µx,k(s) − Ex,k [µx,k(s)])(µx,k(t) − Ex,k [µx,k(t)])] , (5.2)
where Ex[·] denotes expectation over x and (s, t) ∈ I2
. The solution of the eigenequation given by
I
rxx(s, t)χ(s)ds = λχ(t) (5.3)
produces the eigenfunctions {χi(t)}i serving for the orthogonal canonical kernel set and the eigenvalues
{λi}i serving for the importance weight of the corresponding message representation component in terms
of the mean square error.
The expansion coefficients can be evaluated using the orthogonal property of the canonical kernel set
by
qi(x, k) = Φ−1
(µx,k(t)) =
I
µx,k(t)χi(t)dt. (5.4)
These coefficients serve for the approximated message description by (5.1).
The complexity (dimensionality) of the message representation can be further reduced by omitting
several components {qiχi(t)}. The components with index i > Dε are dropped to reduce the representation
complexity. We denoted Dε the number of used components (dimensionality of the message) that refers to
mean square error ε. The resulting mean square error of the approximated message
ˆµx,k =
Dε
i=0
qi(x, k)χi(t) (5.5)
is indeed given by the term ε = i>Dε
λi as one can check in Fig. 5.1.
39](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-52-320.jpg)
![5.2. FG-SPA IMPLEMENTATION FRAMEWORK
Note 5.1 (Linear Canonical Message Representation). Thanks to the KLT, the kernels are orthogonal and
the representation is linear. It means that the KLT message representation is a linear canonical message
representation with orthogonal kernels as we discussed in Note 4.11. Therefore both mappings determining
the parameterization Φ, Φ−1
are clearly given.
5.2.2 Evaluation of the KLT Approximated Message in Real Applications
We have clearly described the procedure of the KLT message representation obtaining. Nevertheless the
evaluation of the autocorrelation function (5.2) can be very complex. This can be simplified considering
the approximation of the continuously valued message given by a discrete vector resulting from a sufficient
sampling of the message6
. The estimated autocorrelation function is given by
Rxx = Ex,k[µx,kµT
x,k] ≈
1
KM
K
k=1
M
x=1
µx,kµT
x,k, (5.6)
where K is the number of iteration, M is the number of realizations and µ = [. . . , µx,k[i], . . . ]T
is a discrete
vector referring to the sampled message µx,k(t).
The eigenequation (5.3) of the discrete message is given by
Rxxχ = λχ. (5.7)
The vector χ refers to the sampled kernel functions χ(t). The expansion coefficients are given by
qi(x, k) =
j
µx,k[j]χi[j]. (5.8)
Finally the sampled version of the approximated message is given by
µx,k ≈ ˆµx,k =
i
qi(x, k)χi. (5.9)
The truncation of the coefficients can be performed equally as in the continuous case.
5.2.3 Generic Update Rules of the Canonical Message Representation with
Orthogonal Kernels
We call a Linear Canonical Message Representation (LCMR) such a message (see Note 4.10) that the
message can be written in the form
µF →x
(x) ≈ ˆµF →x
(x) =
D(F →x)
i=1
qF →x
i χF →x
i (x), (5.10)
We remind that the proposed KLT message representation (Sec. 5.2.1) is LCMR as we already mentioned
in Note. 5.1.
We now derive the FG-SPA update rules for a general LCMR (5.10) with orthogonal canonical kernels
upon the parameterization, that is a direct relation between the parameters of the message as we discussed
in Sec. 4.2.3. We derive the FG-SPA update rules for the LCMR assumed in a whole or in a part of the
system. The coexistence of more message representations in the FG-SPA requires interfaces with another
message representations. Particularly, we investigate
6The complexity issue of the autocorrelation function evaluation is not complication at all, because it is sufficient to
estimate the autocorrelation function just once prior the processing. The required number of samples enabling a full message
recovery from samples is given by the sampling theorem.
40](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-53-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
x
y
F z
µx→F
(x)
µy→F
(y)
µF →z
(z)
Figure 5.2: Visualization of the FN updated message µF →z
.
• variable node (VN) update rules with the LCMR,
• factor node (FN) update rules
– with the LCMR only,
– with the LCMR and the representation by samples,
– with the LCMR and an another representation.
We also address our attention to a proper scaling of the approximated messages and we point out the exact-
form updated-message rectification problem, when the approximated message is negative (non-positive) at
some points.
For simplicity, we will assume only three neighboring nodes. It means that for a FN F update, we
consider just three neighboring variable nodes x, y and z (see Fig. 5.2) and for a VN x update, we consider
just three neighboring factor nodes F, G and H (see Fig. 5.3). An extension to a general case with an
arbitrary number of neighbors is straightforward.
Factor Node Update Rules
We first investigate the update rules related to factor nodes. We start with a conventional FN update rule
formula (4.6), [27]
µF →z
(z) = p(z|y, x)µx→F
(x)µy→F
(y)dxdy
≈ p(z|y, x)
i
qx
i χx
i (x)
j
qy
j χy
j (y)dxdy
=
i j
qx
i qy
j p(z|y, x)χx
i (x)χy
j (y)dxdy,
where we used (5.10) to express the approximated messages.
Our target is to obtain the expansion coefficients qz
k of the approximated message ˆµF →z
(z) = k qz
kχz
k(z).
We exploit the orthogonality of the kernels to evaluate the expansion coefficient by means of the inner prod-
uct as
qz
k = ˆµF →z
(z), χz
k(z) =
i j
qx
i qy
j wF
i,j,k, (5.11)
where7
wF
i,j,k = p(z|y, x)χx
i (x)χy
j (y)χz
k(z)dxdzdy is a table determined by given kernels and a conditional
probability density function (pdf) p(z|y, x). The table is parameterized by i, j, k and it can be precomputed
prior the processing.
7Since we assume real-valued kernel functions and real expansion coefficients, we drop the complex conjugation related to
the inner product.
41](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-54-320.jpg)
![5.2. FG-SPA IMPLEMENTATION FRAMEWORK
F
G
x H
µF →x
(x)
µG→x
(x)
µx→H
(x)
Figure 5.3: Visualization of the VN updated message µx→H
.
We stress that (5.11) indeed relates the parameterization of the input and output messages as we
requested in Sec. 4.2.4 to the implementation framework.
Variable Node Update Rules
The investigation of the VN update rules can be conducted similarly. We start again with the definition
of the VN update rule according to [27]
µx→H
(x) = µF →x
(x)µG→x
(x)
≈
i
qF
i χF
i (x)
j
qG
j χG
j (x). (5.12)
The expansion coefficients of the approximated message are evaluated again by means of the inner product
by
qH
k = ˆµx→H
(x), χH
k (x) =
i j
qF
i qG
j wV
i,j,k, (5.13)
where the update table wV
i,j,k = χF
i (x)χG
j (x)χH
k (x)dx is given purely by the sets of the kernel functions
and it can be precomputed prior the processing.
Factor Node Update with both Discretely and Linearly Represented Continuous Messages
We now consider a case of a mixed updating, when a FN F is neighboring both discretely and continuously
valued messages. Particularly, the variable y is a discretely valued and the message µy→F
(y) is represented
by pmf of y. The variables x and z are continuously valued.
Our goal is to evaluate the LCMR of the message µF →z
(z) from the LCMR µx→F
(x) and discretely
valued message µy→F
(y) = [. . . , qy
j , . . . ] and conversely the discrete message µF →y
(y) from the LCMR
µz→F
(z) and µx→F
(x).
The derivation is very similar as in Sec. 5.2.3. We shortly write the FN-update as (see Fig. 5.2)
µF →z
(z) =
j
p(z|yj, x)µx→F
(x)µy→F
(y)dx (5.14)
≈
j
p(z|yj, x)
i
qx
i χx
i (x)qy
j dx (5.15)
and the expansion coefficients qz
k of the approximated message ˆµF →z
(z) = k qz
kχz
k(z) are evaluated as
qz
k = ˆµF →z
(z), χz
k(z) =
i j
qx
i qy
j wF
i,j,k, (5.16)
42](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-55-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
where wF
i,j,k = p(z|yj, x)χx
i (x)χz
k(z)dxdz is a precomputed table for given kernel sets and the conditional
pdf p(z|yj, x).
After a very similar derivation, we obtain the update of the discrete message µF →y
(y) by
qy
j =
i k
qx
i qz
kwF
i,j,k. (5.17)
This update is, however, not guaranteed to be a non-negative number for all j as one requests to pmf. A
rectification of the updated message is briefly discussed in Sec. 5.2.3.
Interface with Other Message Representations
We have shown a closed update-relation for linearly represented continuously valued messages and discrete
messages represented by their pmf. Nevertheless, it may be still useful to have two different representations
of a continuously valued message. In such a case, we split this VN to two variable nodes and insert an
interface factor node between them.
Assume a VN ¯x connected to two parts of a FG-SPA. The first part of the FG-SPA operates with an
arbitrary representation of the continuously valued message (e.g. Gaussian) and the second one is assumed
to be represented by the LCMR with orthogonal kernels. The VN ¯x is split to variable nodes x and y,
where a FN F is inserted between them. It is assumed that the VN x is represented by the LCMR with
orthogonal kernels and the VN y by an arbitrary another message representation capable to describe the
continuously valued message.
The node F is described by the conditional (pdf) given by p(y|x) = δ(y − x) and the expansion
coefficients qx
i of the LCMR message ˆµF →x
(x) = i qx
i χx
i (x) are evaluated as
qx
i = ˆµF →x
(x), χx
i (x) = µy→F
(x)χx
i (x)dx. (5.18)
The message ˆµy→F
(x) is used if µy→F
(x) is not available. If the message ˆµy→F
(x) is parameterized
by a vector p, one can avoid the integration during the processing by precomputing a set of functions
Ψi(q) = ˆµy→F
(x; q)χx
i (x)dx i
.
Conversely, the corresponding message representation is evaluated from the LCMR approximated mes-
sage ˆµx→F
(x) = i qx
i χx
i (x) by means of the particular message representation. We can mention e.g.
Gaussian representation given by
[my
, σy
] = Ex[ˆµx
(x)], Ex[(ˆµx
(x))2
] − Ex[ˆµx
(x)]2
, (5.19)
where Ex[µ(x)] = xµ(x)dx denotes expectation over x. One can find out that
Ex[(ˆµx
(x))n
] =
i
(qi)n
Ex[(χi(x))n
]
and Ex[(χi(x))n
], ∀i, n can be again precomputed.
Norming and Rectifying the Approximated Message
Since the original message µ(t) is supposed to represent a pdf, we may not care about the message scaling
during the update rules. When the message is approximated, we need to be able to achieve ˆµ(t)dt = 1.
Suppose that ˆµ
¯
(t) = k q
¯
k
χk(t) is a non-normed LCMR. We consider a norming constant α ∈ R such that
αˆµ
¯
(t)dt = ˆµ(t)dt = 1.
43](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-56-320.jpg)
![5.3. APPLICATIONS AND NUMERICAL VERIFICATION
ˆµ(t) < 0
t
negative pmf
UR
Figure 5.4: Demonstration of the FG-SPA updated exact-form negative message problem.
ˆµ
θ
˜µ
Figure 5.5: Rectification of the negative message appearance.
We can express α = 1/ ˆµ
¯
(t)dt = 1/ i Pq
¯
i
Ωi, where we defined Ωi = χi(t)dt. Since αˆµ
¯
(t) = ˆµ(t)
surely guarantees a proper norming, we can express the norming expression as
qi = αq
¯
i
=
q
¯
i
j q
¯
j
Ωj
, ∀i. (5.20)
As a consequence of the message approximation, the approximated message ˆµ(t) = k qkχk(t) can be
negative at some points. The messages are conventionally assumed to be non-negative in the FG-SPA.
The negativeness of the approximated message can be ignored, when the message is used to evaluate
an another approximated message, because the non-negative request is casted to the true (exact-form)
messages. The approximated (or the updated) message must, however, be somehow rectified in case of the
exact-form updated message (see Fig. 5.4). There is a number of ways of the negative message rectification
(e.g. subtracting mint(ˆµ(t)) from the approximated message and norming it properly as ad-hoc suggested
in [44]).
Motivated by our simulation results in Sec. 5.3.5, it is beneficial to avoid a hard decision provided
by the FN-update rule upon the approximated message. It means that the updated exact-form message
should not contain any zero element (e.g. discrete message represented by pmf should satisfy pj > 0, ∀j,
if it is a result of the FN-update with an approximated message input). Following this simple idea, we
add-hoc suggest to define a softness threshold θ and the rectified updated exact-form message ˜µ(t) = ˆµ(t)
if ˆµ(t) ≥ θ and ˜µ(t) = θ otherwise (see Fig. 5.5).
We numerically find out a very high sensitivity of the performance with respect to the updated-message
rectification in a specific scenario (Sec. 5.3.5). With the increasing precision of the message representation
the impact of this phenomenon will decrease.
5.3 Applications and Numerical Verification
We exemplify the proposed FG-SPA implementation framework on the well-know case of the joint phase
estimation data detection. Various FG-SPA implementation frameworks suited for this case can be found
in literature (e.g. [10,12,52,53,55]). We stress that while all mentioned frameworks are designed for this
particular scenario, the proposed implementation framework is more generic.
The verification consists from two parts. First, we obtain the canonical kernel functions of the KLT
message representation. This evaluation is based on the stochastic analysis of the messages in the FG-
SPA (Sec. 5.2.2). The impact of this approximation is investigated and compared with other message
representations mentioned in Sec. 4.2.5. The second part investigates behavior of a whole implementation
framework.
44](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-57-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
d C c M s ×
exp(jϕ)
u +
w
x
Figure 5.6: The system models in signal space.
d CPE q NMM s ×
exp(jϕ)
u +
w
x LPD θ
Figure 5.7: The MSK modulation in phase space.
5.3.1 System Model
We consider several system model variants to achieve a more complex view into the system. This is
desired mainly for the stochastic analysis of the messages in the phase model. We consider several linear
modulations models in signal space space and a non-linear minimal shift keying (MSK) modulation in the
phase space model. We also consider two phase models, the constant phase and the random walk (RW).
Signal Space Models
We assume a binary i.i.d. data vector d = [d1, . . . , dn]T
of length N as an input. The coded symbols are
given by c = C(d). The modulated signal vector is given by s = M(c) , where M is a signal space mapper.
The channel is selected to be the AWGN channel with phase shift modeled by the random walk (RW) phase
model. The phase as the function of time index is described by ϕj+1 = mod(ϕj +wϕ)2π, where wϕ is a zero
mean real Gaussian random value (RV) with variance σ2
ϕ. Thus the received signal is x = s exp(jϕ) + w,
where w stands for the vector of complex zero mean Gaussian RV with variance σ2
w = 2N0. The model is
depicted in Fig. 5.6.
MSK in Phase Space
We again assume the vector d as an input into the minimum-shift keying (MSK) modulator. The modulator
is supposed to be in the canonical form, i.e. it is modelled by the continuous phase encoder (CPE) and non-
linear memoryless modulator (NMM) as shown in [50]. The modulator is implemented in the discrete time
with two samples per symbol. The phase of the MSK signal is given by φj = π/2 (ςi + di(j − 2i)/2) mod4,
where φj is the jth
sample of the phase function, ςi is the ith
state of the CPE and the sampled modulated
signal is given by sj = exp(jφj). The communication channel is selected to be the AWGN channel with
constant phase shift, i.e. x = s exp(jϕ) + w, where x stands for the received vector, ϕ is the constant
phase shift of the channel and w is the AWGN vector. The nonlinear Limiter Phase Discriminator (LPD)
captures the phase of the vector x, i.e. θj = ∠(xj). The whole system is shown in Fig. 5.7.
5.3.2 Factor Graph of the Joint Phase Estimator Channel Decoder
The FG-SPA is used as a joint synchronizer-decoder (see Fig. 5.8) for all mentioned models. Note that
the FG for the considered models might be found in the literature (phase space model in [55] and signal
space models in [10,12]).
Prior the description itself, we found a notation to enable a common description of the models. We
define
45](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-58-320.jpg)
![CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK
AWGN (W):
p(ζj|γj) = pΞ(σ2
W , ζj − γj)
• Factor nodes common for both signal and phase space random walk (RW):
p(ϕj+1|ϕj) =
∞
l=−∞
ρσ2
ϕ
(ϕj+1 − ϕj + 2πl)
Other factor nodes such as the coder, CPE, and signal space mappers FN are situated in the discrete
part of the model and their description is obvious from the definition of the related components (see
e.g. [27] for an example of such a description).
Message Types Presented in the FG-SPA
The FG-SPA contains both discretely and continuously valued messages. The channel encoder and the
signal space mapper contains only the discretely valued messages. The messages are represented by their
pmf the update rules can be directly implemented according their definitions.
The AWGN part contains well parameterizable continuously valued messages and the phase model con-
tains the mixed-type messages. These mixed type messages are represented by means of the implementation
framework (KLT message representation in our case).
5.3.3 Concrete Scenarios
We specify several concrete scenarios for the analysis purposes. All of the scenarios may be seen as a
special case of the system model described in the section 5.3.1. The first four configurations are used for the
stochastic analysis of the mixed type message (Sec. 5.3.4) and the fifth one is used for the implementation of
the whole implementation framework including the KLT message representation and the proposed update
rules implementation (Sec.5.3.5).
Uncoded QPSK modulation
The Uncoded QPSK modulation scheme assumes the QPSK modulation, which is situated in the AWGN
channel with RW phase model. The length of the frame is N = 24 data symbols, the length of the preamble
is 4 symbols and the preamble is situated at the beginning of the frame. The variance of the phase noise
equals σ2
ϕ = 0.001. This scenario leads to a cycle-free FG-SPA and thus only inaccuracies caused by
the imperfect implementation are presented. The information needed to resolve the phase ambiguity is
contained in the preamble and, by a proper selection of the analyzed message, we can maximize the
approximation impact to the key metrics such as BER or MSE of the phase estimation. We thus select the
message µRW−→ϕ3
to be analyzed.
Coded 8PSK modulation
The coded 8PSK modulation contains in addition to the previous scenario the (7, 5, 7)8 convolutional
encoder8
. The length of the frame is N = 12 data symbols, the length of the preamble is 2 symbols. The
same message is selected to be analyzed (µRW−→ϕ3
).
MSK modulation
The MSK modulation with constant-phase model of the phase shift assumes the length of the frame is
N = 20 data symbols. The analyzed message is µϕ→P S
. These messages are nearly equal for all possible
PS factor nodes (e.g. [13]).
8We mean by (7, 5, 7)8 a conventional octal definition of the convolutional encoder.
47](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-60-320.jpg)
![5.3. APPLICATIONS AND NUMERICAL VERIFICATION
models do) from the numerical complexity and it is therefore unsuitable for a direct implementation.
We classify the messages appearing in the reference FG-SPA model (Fig. 5.8) and their update rules.
• Discrete messages are all messages on left from the PS FN (see Fig 5.8). It means all messages in
the encoder and mapper part. These discrete messages are fully represented by their pmf and the
evaluation of the update rules is performed directly according to [27].
• Unimportant messages (µP S→γj
, µγj →W
, µW →ζj
and µζj→δj
) lead to the open branch and neither an
update nor a representation of them is required, because these messages cannot affect the estimation
of the target parameters (data, phase).
• Parameterizable continuous messages (µδj →ζj
and µζj →W
) can be easily represented by a number
x0 meaning µ(x) = δ(x − x0), µW →γj
and µγj →P S
are representable by the pair {m, σ2
w} meaning
µ(x) = ρσ2
w
(x − m), µ(x) = pΞ(σ2
W , x − m) respectively. One can easily find the slightly modified
update rules derived from the standard update rules. Examples of those may be seen in [13].
• The mixed-type messages situated in the phase model (µP S→ϕj
, µϕj →P S
, µRW+→ϕj
, µϕj →RW+
,
µRW−→ϕj
and µϕj →RW+
) are10
discretized and the marginalization is performed using the numerical
integration with the rectangle rule [13, 32] in the update rules. The number of samples is chosen
sufficiently large so that the impact of the approximation can be neglected. The mentioned mixture
messages are real valued one-dimensional functions for all considered models.
Eigensystem Analysis
We first provide the analysis of the eigensystem. For the investigated messages, we collect the sample
realizations to obtain the second order statistics (the empirical autocorrelation function) according to
(5.6). The eigenfunctions solving the eigenequation (5.7) result in harmonic functions independently of the
parameters of the simulation as one can see in Fig. 5.10 and 5.11.
The related eigenvalues are shown11
in Fig. 5.12. We can see several artefacts. The eigenvalues provide
an important information about the importance of a particular message approximation component. The
MSE resulting by dropping the component is given directly by the corresponding eigenvalue as discussed
in Sec.5.2.1. The floor in Fig. 5.12 is caused by the finite floating precision. As one can see, the higher
SNR, the slower is the descent of the eigenvalues with the dimension index. The curves in the plots point
out to the fact that the eigenvalues are descending in pairs e.g. λ1 − λ2 λ2 − λ3.
Relation of the MSE of the Approximated Message with the Target Criteria Metrics
The KLT message representation provides the best approximation in the MSE sense for a given dimen-
sionality. This is, unfortunately, not the target metric to be optimized in almost all cases. This relation
is moreover not easy to analytically obtain. We therefore provide a numerical simulation to inspect the
relation between the approximation error and the target metrics (MSE and BER) at least in several distinct
system model configurations.
Few notes are addressed before going over the results. The MSE of the phase estimation is computed as
an average over all MSE of the phase estimates in the model. The measurement of the MSE is limited by
the granularity of the reference model. The simulations of the stochastic analysis are numerically intensive.
We are limited by the computing resources. The simulations of the BER might suffer from this, especially
for small error rates. The threshold of the detectable error rate is about Pmin
be ≈ 10−6
for the Uncoded
QPSK model and Pmin
be ≈ 2 · 10−5
for the BICM model.
10We have denoted RW+ the factor node RW, which lies between jth and (j + 1)th section according to the Fig. 5.9.
Analogously, RW− stands for the FN between (j − 1)th and (j)th section.
11The results in other model seems very similarly and they are therefore not explicitly shown.
50](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-63-320.jpg)
![5.3. APPLICATIONS AND NUMERICAL VERIFICATION
si PS ui RW
ϕi
ϕi+1
si+1 PS ui+1
Coder and
Mapper
Figure 5.16: The factor graph of the decoder for the random walk phase model implemented in the signal
space. This can be seen as a special case of Fig. 5.8, where we recognize the discrete type of the messages
(green edges), parameterizable (Gaussian) continuous messages (blue edges) and mixed type messages (red
edges).
5.3.5 Proposed Generic Update Rules Implementation
Within the implementation of the update rules, we consider the system model described in Sec.5.3.1 with
the BICM modulation with RW phase model implemented in signal space. The factor graph of the receiver
for this particular model can be found in Fig 5.16 as a special case of 5.8.
Canonical Kernel Functions Assignment
The results of eigenanalysis (Sec. 5.3.4 or Fig. 5.11 and Fig. 5.10) suggest to use the harmonic functions
as the canonical kernel basis system for the mixed-type messages. Using the Fourier representation offers
the best message approximation in the MSE sense for a given dimensionality. We therefore employ the
Fourier message representation, where the approximated message is given by (4.22).
A proper scaling of the Fourier representation can be done using Sec. 5.2.3. One can check that
Ωk =
2π
0
χk(t)dt = 0, ∀k = 0 and Ω0 =
2π
0
χ0(t)dt = 1/
√
2π. The scaling factor α is thus given by
α =
√
2π/q
¯
0
. The scaling of the approximated message to achieve k qkχk(t)dt = 1 can be easily done
using (5.20), concretely by setting qk = q
¯
k
√
2π/q
¯
0
. One can easily check that αˆµ
¯
(t)dt = αΩ0q
¯
0
= 1.
Update Rules Derivation for the Assigned Canonical Kernel Functions
According to the set of kernel functions, we can prepare the table of weights and the functions needed for
the efficient evaluation of the update rules12
. We start with the RW update.
RW-FN Update
Recalling results from Sec. 5.2.3, we can write
wi,j = p(ϕi+1|ϕi)χi(ϕi)χj(ϕi+1)dϕidϕi+1. (5.21)
12We can note that the update rules design for the Fourier representation can be found also in [10] for this model. In
contrast with this work, where the authors propose the implementation framework suited for this concrete scenario, we use
this model just to verify the proposed generic design.
54](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-67-320.jpg)
![5.3. APPLICATIONS AND NUMERICAL VERIFICATION
1e-05
0.0001
0.001
0.01
0.1
-4 -2 0 2 4 6 8 10 12
Pbe
SNR[dB]
BER for different number of used canonical functions
D=3-KLT
D=5-KLT
D=3-Sampling
D=5-Sampling
D=9-Sampling
D=11-Sampling
D=256-Sampling
Figure 5.17: The bit error rate comparison of the proposed implementation framework base on the KLT
approximated message with the representation by samples updated using the numerical integration with
rectangular rule.
0.01
0.1
1
-2 0 2 4 6 8 10 12
MSE
SNR[dB]
MSE for different number of used canonical functions
D=3-KLT
D=5-KLT
D=11-KLT
D=5-Sampling
D=11-Sampling
D=256-Sampling
Figure 5.18: The mean square error of the phase estimation of the proposed implementation framework
base on the KLT approximated message with the representation by samples updated using the numerical
integration with rectangular rule.
56](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-69-320.jpg)
![6 Physical Wireless Network Communication
This Chapter provides a gentle introduction to wireless communication networks. The goal of this Chapter
is to provide some basic overview and classification of the wireless networks to prepare a ground for the
next Chapter, where we propose a decoder suited for the networks with wireless network coding. The
reader familiar with the wireless network coding (physical layer network coding) [26, 30, 39, 54] can skip
this Chapter directly to Chapter 7.
The physical layer wireless communication is often viewed as an extension of the conventional commu-
nication, where the higher layers direct the basic communication elements (physical layer) containing only
P2P channels and MAC channels with SD-IC. The higher layers (e.g. routing) are optimized independently
to the physical layer. The optimization of several layers (OSI-model) independently is intuitively not op-
timal from the global point of view. The global optimization is addressed as the cross-layer design solving
the optimization across more (all) layers jointly.
The main problem of the cross-layer design is complexity of the cross-layer optimization. In this work,
we rather focus on some basic phenomena (network coding paradigm [1,67]) that can be applied in wireless
networks and that promises a significant gain against the conventional independent optimization.
The network coding is however designed in the discrete world and it still uses the physical layer as a
system of orthogonal pipes. The adoption of the physical layer taking into account the network coding
paradigm (wireless network coding) then brings another potential improvement. Particularly the perfor-
mance beyond the higher (e.g. second)-order cut-set bound can be achieved avoiding the conventional
method such as either joint decoding or orthogonal P2P approach. Instead, a hierarchical decoding is
introduced. The hierarchical decoding roughly means that the node decodes directly a function of data
streams instead of decoding of all streams followed by a network operation upon the decoded streams.
The decoding of a function of data can be also viewed as reduction of interference that unavoidably
corrupts the performance in the P2P based communication (MAC region limitation [11]). The largest
potential gain of WNC is therefore in dense network, where the interference presents the main limitation
of the system. Since still arising number of wireless devices communicates in a given (constant) area with
given (constant) resources (wireless medium) with increasing demand on the transmission speed nowadays,
the method of non-orthogonal interference reduction obviously attains a lot of attention of the research
community in the past few years [68].
The WNC can be viewed as a cooperative diverse scheme [29]. The MIMO system is in fact a multi-
source multi-destination(relay) wireless network with a perfect cooperation. It can be used as a benchmark
for many applications. The full awareness of the system allows to adjust the precoding matrix with respect
to the received signal. If the links between the individual antennas are not perfect, that is some constraints
are assumed on these links (e.g. wireless channel), we can consider the wireless network as a generalized
MIMO.
Prior we dive into the topic of the wireless network coding, we provide some basic classification of a
general wireless network from the topology, constraints and processing point of view within this Chapter to
get a better categorization of our work. Since the topic of detection in the multi-source wireless is closely
and formally described in the following Chapter 7, this Chapter is written in intro-based style without any
formal notation and definitions.
6.1 Wireless Network
6.1.1 Basic Elements
The wireless network is formed by a set of nodes that are located in an area and that are equipped by
devices enabling a wireless communication. The nodes are supposed to be connected only through the
wireless medium (wireless channels). The nodes can be classified either as terminal nodes or relay nodes.
60](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-73-320.jpg)
![CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION
The relay nodes are neither data (information) source nor data sink. The terminal nodes are divided to
sources (data source) and destinations (data sinks). Some examples of the models for wireless networks
that are significant for our work are discussed in Sec. 6.1.6.
We can describe the network by a graph, where the edges are given by wireless channels between the
nodes and the nodes are given by the physical nodes. The information flow is given by some requirements
from destinations. That is each destination is interested in information from a certain set of sources. A
crucial difference between the conventional wireless networks based on OSI layer is that the information
content in the pipes is correlated and therefore some description is dedicated also for some cuts of the net-
work. According to the connections in the network and potentially further criteria (constraints), something
(e.g. node itself or genie entity) decides about behaviour of each node.
6.1.2 Assumptions
Within this part of the work, we will explicitly assume the following constrains and assumptions.
Half Duplex Constraint
The half duplex constraint is a consequence of real hardware limitation, particularly the interference of the
transmitted and received signal arises in the antenna if the signals are passed simultaneously (full-duplex).
Due to this, it is assumed that the each node can either transmit or receive the signal at one time instance.
One can found many papers aiming with consequences of the half-duplex constraint investigation. A
straightforward natural time division simplifying the network design is the same time portion dedicated
for both the transmission and the reception of the signal in each node. On the other hand, a desired power
allocation can be achieved adjusting the time division properly.
Uniform Unitary Transmit Power Distribution
A unit transmit power of all nodes is assumed from the simplicity reasons. An unequal power distribution
of nodes within the network can be still adjusted by means of the flat fading coefficient in necessary.
Perfect Time Synchronization
The imperfect time synchronization basically leads to the inter-symbol interference. Since the network
contains more nodes, various levels of synchronization imperfectness can be assumed. We consider a
perfect time synchronization and channel estimation among all nodes in the network within this part. We
refer e.g. [30] as a tutorial considering also the synchronization issue.
Acyclic Wireless Networks
From the complexity reasons, we will consider exclusively cycle-free networks within this work. It means
that the directed graph representing the wireless network and the flow of the information is cycle free, that
is if a node transmits some information, this information cannot be delivered back to the node in some
later stage.
6.1.3 Network Processing Classification
Within the network processing, we recognize basically two types of the processing, particularly (see e.g. [41])
• centralized processing,
• distributed processing.
61](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-74-320.jpg)
![CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION
R
x0 sR
Figure 6.1: Relay-node processes the signal space observation vector x0. The output of the relay is again
a signal space vector sR depending on the relay processing strategy.
Hierarchical Side Information Link
The H-SI link presents some complementary information to desired data. This is in sense of the network
coding paradigm, where the information can be temporary mixed in the network to increase the overall
throughput. The H-SI is then used to reveal the desired data from the temporary mixed information that
does not fully contain the desired data in general.
6.1.5 Relaying Strategies
The relaying strategy is called all the processing in the relay between the observation x0 and the transmission
of sR (see Fig. 6.1). We omit a dummy relaying strategy, where the relay is silent or its behaviour does
not depend on its observation x0.
Amplify and Forward
The Amplify and Forward (AF) strategy is the simplest possible relaying strategy. It works just as a
multiplier, that is sR = βx0, where β is a constant securing that sR has unit energy per symbol. If we
consider a MIMO channel received in the relay, β becomes to be a matrix.
This strategy preserves all information contained in x0, but the power of relay is wasted on the am-
plifying of the noise and unwanted interference collected in the wireless channel(s). The AF strategy is
latency free. A description of the AF strategy in 2-WRC with half duplex constraint can be found e.g.
in [48].
Hierarchical Decode and Forward
The Hierarchical Decode and Forward (HDF) is a strategy that provides a decision about a function of the
received data (hierarchical data). The entropy of the hierarchical data is less or equal to the entropy of
all received data vectors. The decoded hierarchical data are then again encoded and mapped to the signal
space vector sR.
This strategy will be described later more precisely. We can mention that the HDF strategy includes
(among others) as e.g. Compute and Forward strategy [40] or Denoise and Forward [26]. The HDF strategy
suffers from the latency, because the full decoding requires to operate with whole sequence.
Note 6.1 (X-Decode and Forward). The hierarchical function is not closer specified here. It means that
the HDF strategy contains also conventional decode and forward (DF), where the function is identity for a
given data stream or joint decode and forward (JDF), where the function is joint identity for all streams.
Compress and Forward
The Compress and Forward (CF) strategy is called such a strategy that somehow process x0 to provide a
metric γ. This metric should be a sufficient statistics about the (hierarchical) data. A special case of this
metric is the hierarchical data vector considered in HDF.
The CF strategy covers more approaches, typically quantize and forward, where the metric γ is given
by somehow sampled version of x0.
63](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-76-320.jpg)
![6.1. WIRELESS NETWORK
6.1.6 Physical Network Topologies
We browse some basic topologies that are important for this work. We also exploit the particular topologies
to demonstrate some principles of a given strategy (mainly HDF) that can be clearly seen in the given
topology that are related with the sequel Chapter.
Two-Way Relay Channel
The two-way relay channel (TWRC - Fig. 6.2) is the simplest multi-source relay wireless network containing
three nodes. The terminals want to exchange the data using the auxiliary relay. This model is also often
used to demonstrate the power of the wireless network coding, since it is clearly sufficient to consider two
time slots to exchange the information1
. It nicely demonstrates the difference between the conventional
P2P orthogonal channels, network coding provided upon orthogonal P2P models and the wireless network
coding.
• Orthogonal P2P based communication without interference, where one needs four time-slots to ex-
change the information between the terminal nodes, particularly
1. source SA transmits to R data bA that are decoded in relay, i.e. SA(bA) → R(bA),
2. SB(bB) → R(bB),
3. R(bA) → DA(bA),
4. R(bB) → DB(bB).
Each time-slot is occupied with a conventional P2P transmission without any interference. The
transmission can thus operate on the rates corresponding to the capacity of the channels.
• P2P communication with network coding requiring three time slots for the full exchange, i.e.
1. SA(bA) → R(bA),
2. SB(bB) → R(bB),
3. R(bA ⊕ bB) → DA(bA ⊕ bB), DB(bA ⊕ bB).
The destinations are able to decode the desired data utilizing the identity bA = bA ⊕ bB ⊕ bB and
analogously for bB. The first two time slots are equal to the previous case and the third one can also
operate on the channel capacity considering symmetric channels.
• Wireless network coding demanding only two time-slots for full data exchange, particularly
1. SA(bA), SB(bB) → R(bA ⊕ bB)
2. R(bA ⊕ bB) → DA(bA ⊕ bB), DB(bA ⊕ bB).
The first time slot is occupied by a multiple-sources transmission (MAC) channel. The second order
cut-set bound is avoided by decoding of bA ⊕ bB instead of the joint pair (bA, bB) decoding. The
performance within half bit below the first order cut-set bound was proven to be achievable in [38]
for AWGN channels.
One can see that the main advantage of the wireless network coding in TWRC is in the bidirectional
communication, where the symmetry of the information amount exchange is assumed.
1With some further assumptions.
64](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-77-320.jpg)
![6.1. WIRELESS NETWORK
SA
SB
R1
R2
D
Figure 6.4: Two source, two relay, single destination network without direct source-destination H-SI links.
SourceTerminals
1stRelayLayer
...
LastRelayLayer
DestinationTerminals
Figure 6.5: Acyclic layered network.
in relay R1 and ¯bAB = ¯Xb(bA, bB) in relay R2. These hierarchical streams must jointly contain all infor-
mation about the tuple (bA, bB) to keep the decoding possible in the final destination, i.e. some decoding
function [bA, bB] = X−1
b (bAB, ¯bAB) must exists.
The overall hierarchical function (formed by Xb and ¯Xb) can provide either the minimal possible in-
formation, i.e. the vectors bAB, ¯bAB are independent and the system has no diversity. If the information
is larger than the minimal one, the hierarchical data streams are bAB, ¯bAB correlated and the system
might utilize the correlation of the received symbols in the final destinations. A typical example of the
minimal possible information is TWRC or the butterfly network with perfect H-SI. On the other hand, the
correlation occurs if both relays jointly decode both data streams bA, bB.
General Acyclic Network
A wireless network coding generalization brinks couple of complications. The hierarchical functions must
be defined in each relay such that each destination receives a proper information for its purposes. Due
to the complexity of the design, we proposed a unified framework [58], where the problem of the overall
network design is decomposed to several sub-task. The core sub-task is the wireless aware network coding
determining the hierarchical functions according to both the desired end-to-end information flows and the
constrains given by the wireless network. A simplified model of the general acyclic network is a layered
model (Fig. 6.5), where we assume a radio visibility just among the neighboring layers of nodes.
Multi-Hop Relay Channel
The multi-hop relay channel model (see Fig. 6.6 for two hops example) is a single-source scenario. It is
typically associated with the latency problem. The point is that the relay must wait for a whole sequence
followed by the decoding process in case of DF strategy, while the AF strategy works symbol-wisely and
therefore no latency is accumulated in relay. The performance difference between the DF and AF presents
a playground for some strategies (CF-based) providing better performance than AF strategy, but with
lower latency than DF strategy.
66](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-79-320.jpg)
![CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION
S R D
Figure 6.6: Multi-hop relay channel with two hops.
6.1.7 Challenging Problems in Wireless Networks
Within this Chapter, we briefly summarized the principle of the wireless network coding up to now taking
accent mainly on the hierarchical decoding. Some further challenges in the wireless networks will be pointed
out. These challenges are given by the tasks required for the cross-layer design in general. Nevertheless,
we point out the challenges that are related directly with the goal of the thesis.
Wireless Aware joint Network Coding
The utilization of the network coding paradigm in the wireless network extension is available only in simple
networks. The challenge is to design a method solving an ideal hierarchical maps distribution within a
general wireless network such that some criteria are optimized (e.g. maximal utilization of the network
coding paradigm still enabling desired decoding in final destinations). This solution should be moreover
achieved in a distributive way. Note that the proposed block structured layered design (Chapter 7) can be
applied to provide the hierarchical maps and the network-consistent channel coding design for some given
coding rates, which can be derived from particular channel capacities. The next challenge is to derive a
staging algorithm assigning properly stages when and how long each of the node should receive/transmit
such that the power resources are utilized as efficiently as possible.
Channel Parameterization
The pure network coding directly shows how to utilize the network coding paradigm. Nevertheless, the
utilization is quite complicated in the signal space, because the network coding is no longer provided upon
the finite alphabets, but upon the complex-valued signal space. The channel is moreover parameterized by
further constants (e.g. complex valued fading). Since the parameterization is related to more sources, it is
not possible to avoid some crucial combination of the channel parameters purely by the processing in relay.
It means that the constellation seen in a node receiving more signals (typically relay) is parameterized
by a channel parameter, which can cause a constellation distortion decreasing the (hierarchical) channel
capacity (available rate).
The way leading to solve this problem is twofold. One can consider some feedback to sources (e.g.
channel pre-rotation [26]) or to design some alphabet that is resistant to the vulnerability of the performance
with respect to the channel parameterization [59].
Synchronization and Other Real Implementation Problems
Of cause, there is plenty of problems that are beyond the scope of this thesis. As an instance, we assume
a signal space description of the signal in network, which means that both the time and frequency syn-
chronization are available in each node. It is however quite challenging problem to achieve this without a
central unit.
67](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-80-320.jpg)
![7 Network Channel Code Design
The relay strategies based on a decoding (hard decision) of a function of the source data streams (e.g. bitwise
XOR) were shown to move ahead the system performance in wireless networks. A proper implementation
of these relay strategies needs to consistently incorporate channel coding across the network. Such an
implementation is available only for special cases (mainly symmetric rates and equal alphabet cardinalities)
in the current state of art. This Chapter presents a composition of the channel encoders in sources as
well as the corresponding relay decoder for an arbitrary (asymmetric) individual source rates and for the
corresponding (hierarchical) function of the source data streams. The proposed design consists of a proper
mixture of the already known symmetric-rate based decoder design and the conventional joint decoding.
We develop a unifying (linear) description of the processing in wireless networks. Utilizing this description,
the proposed design results in a block-diagonal matrix composition preserving the (good) properties of the
block elements (e.g. LDPC codes). The proposed design is demonstrated in a simple example and verified
by the numerical simulation.
7.1 Introduction to Wireless Network Coding
The coding and modulation design in a general wireless physical network arose as a huge challenge recently.
A pioneering work [1] showed that the information content can be temporarily mixed (network coded)
within the network to increase either the overall network throughput (capacity) by means of reduced wireless
channel use or the system diversity. An extension of the network coding (NC) to include also the wireless
environment properties (wireless network coding (WNC) a.k.a. physical layer network coding [26,30,39])
then brings additionally further potential performance improvement. The idea of WNC consists in decoding
(providing hard decision) of a (hierarchical) function of source data streams directly in constellation space
[40] and forwarding it further, thus it is named Hierarchical Decode (Compute) and Forward strategy
(HDF). The hierarchical function of source data streams (hierarchical stream) contains less information
than the joint source data streams. Providing a hard-decision on the hierarchical stream, the classical
Multiple Access Channel (MAC) region [11,15] restrictions limiting the conventional joint-decoding based
strategies does not longer limit the performance.
Since the hierarchical stream contains less information than the joint source data streams, the requested
complement of this information must be delivered into the final destination by alternative paths in order
to secure a reliable end to end transmission. This request is reflected e.g. by a condition that the final
destination must collect enough number of equations from all paths in [40]. The information amount
flowing by the alternative paths is called Hierarchical Side Information (H-SI).
A proper utilization of the WNC with HDF across the network potentially takes advantage of both
reduced wireless channel use and operation rates beyond the classical MAC regions. This utilization is,
however, not trivial in a general wireless network. The network design contains a joint optimization over
immense degrees of freedom that has to be solved. Assuming a given known network state information and
a perfect time synchronization among all nodes in the network, the overall optimization can be partitioned
to some interacting subtasks [58], particularly (1) network information flow determining the hierarchical
functions in each relay, i.e. the information routing that is aware of the wireless environment, (2) network-
consistent channel encoders design respecting properly the network information flow as well as the maximal
possible achievable coding rates resulting from the network state information and (3) signal space mappers
connecting the codewords with the wireless environment. One can also consider various metrics to be
optimized, e.g. the overall throughput maximization. The closed-form solution for such an optimization is
not available in the current state-of-art.
The available literature offers the solution either of the cases with some very specific circumstances,
typically small wireless networks with fixed network information flows, or a deeper insight in some particular
sub-task without taking into account the others. One can recognize two major approaches. The first one
68](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-81-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
serves mainly for proving the achievability of a performance by means of the information theory tools,
typically the lattice codes [17]. Some works based on this approach show very nice results, e.g. [38], where
the authors proof the achievability of the performance within a half bit under the first order cut-set bound
in Gaussian Two-Way Relay Channel (TWRC). The lattice code based models are, however, not suitable for
a direct implementation. The second approach is therefore more constructive and directly implementable.
It is based mainly on conventional capacity achieving channel codes and signal space mappers for a given
network information flow design. Design of the directly implementable consistent channel encoders across
the wireless network for arbitrary channel code rates, signal space mappers and network information flow
stands for the main focus of this work. To best of our knowledge, no such design enabling arbitrary
asymmetric rates is available in the current state of art (excepting our initial approach [45] and traditional
joint decoding followed by a network code).
7.1.1 Related Work
Throughout the literature, it is very common to assume a special case of a uniform hierarchical ratio1
.
The uniform hierarchical ratio allows (among others) a popular bit-wise XOR of the source data streams
as the hierarchical function [7, 8, 30, 64, 65], i.e. the traditional understanding of the (wireless) network
coding. The hierarchical ratio is classified by means of the relay-alphabet cardinality in [54]. We stress
that a commonly used TWRC as well as its generalization the butterfly network without a direct link
from sources to destinations does not allow arbitrary hierarchical ratio in relay, because the H-SI cannot
contain an arbitrary information amount (maximally one data stream). The case of maximal information
flow through the H-SI (implicitly satisfied in TWRC with symmetric rates) allows the uniform hierarchical
ratio case referred as the minimal hierarchical cardinality (meaning minimal information flow through the
relay) in [54].
The consistent channel encoders design in the network is basically joint channel-wireless network coding
that is able to benefit from both reduced channel use and the operational rates beyond the classical MAC
region. Several works focus on the joint network-channel encoding (see e.g. [4, 20, 22]). These works are
typically provided upon some graphical models (e.g. Tanner graph in [22] or factor graph in [20]). The low
density parity check (LDPC) channel encoding technique is commonly used as the channel encoding part
(e.g. in [20]). Linearity of both network and channel coding is standardly assumed. None of these works
allows to work with arbitrary channel rates and hierarchical ratios.
The joint channel-wireless network coding design appears in literature only for very simple networks with
further restrictions. These restrictions are typically symmetric channel rates and the uniform hierarchical
ratio. For example, the authors of [30] propose both optimal and suboptimal soft metric based decoder relay
design for TWRC, where the source encoders are assumed to be equal. The optimal design relies on the
repeat-and accumulate codes in [30]. A similar design is proposed in [64], where the optimal design is built
upon higher-order LDPC codes and the turbo codes based optimal relay decoder design is proposed in [8].
All of these optimal decoder designs rely on equal channel encoders in sources and uniform hierarchical
ratio.
It is common to assume memoryless wireless channels connecting the nodes as well as a symbol by
symbol mapping of the coded symbols to signal space symbols across a whole network2
. In such a case, a
symbol-wise processing can be applied in soft output demodulator (SoDeM) in each node to produce a soft
metric for the channel decoder providing hard decision about a required hierarchical stream for a given
node. The symbol-by-symbol metric evaluation enables a separate investigation of the channel symbol
1Notice that the hierarchical ratio determines the information amount flowing through the relay and it will be formally
defined later in Definition 7.1.
2Notice that some works assume more general assumptions, e.g. more generic channels with inter-symbol interference caused
by imperfect time synchronization, or the signal space mapping in sources that is over more than one symbol. Nevertheless
none of these works allows the joint network-channel encoders design for an arbitrary source rates.
69](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-82-320.jpg)
![7.2. NOTATION & DEFINITIONS
processing, where channel coding cannot be straightforwardly added (e.g. [26]). The symbol-by-symbol
evaluated soft metric can be either optimal, i.e. a sufficient statistic for the hierarchical data (e.g. joint
likelihood), or suboptimal (typically some marginalization of the optimal metric). The use of a suboptimal
soft decoding metric is motivated by simplicity of the relay decoder implementation (see e.g. [30,54]).
A construction of the relay decoder for an optimized (even non-linear) hierarchical function and LDPC
encoders in sources in TWRC model is proposed in [31]. The assumption strictly limiting this work is that
the non-zero elements of the LDPC matrix must be on the same position in all sources and relay LDPC
matrices. This design thus does not allow arbitrary individual rates of sources.
7.1.2 Goals & Contribution
The main goal of this part of the work is to propose a network-consistent channel encoders design for
a generic class of network structures respecting arbitrary network information flows, arbitrary (including
asymmetric) rates of the channel encoders and arbitrary signal space mappers.
We develop a unifying robust network description based on linearity of the information flow as well the
linearity of the channel encoders across a whole network. The available hierarchical data stream decoding
options are revised by means of the proposed description. The advantages of the available strategies
are combined in order to fulfill the overall goal, that is the network-consistent channel encoders design
capable to incorporate arbitrary rates, information flows given by the hierarchical ratio and signal space
mappers. We investigate some basic properties of the proposed network-consistent channel encoders design
and numerically verify its performance.
7.1.3 Structure of Chapter
The rest of this Chapter is structured as follows. Notation and basic definitions are found in Sec. 7.2. The
system description is addressed into Sec. 7.3. The following Section 7.4 aims with classification of the relay
decoding options from the optimality, robustness and complexity point of view. The core of this chapter is
presented in Sec. 7.5, where we describe a consistent directly implementable channel encoders construction
concept across the whole network allowing to process arbitrary channel code rates as well as arbitrary
hierarchical ratio. This construction is built upon the description in Sec. 7.3, where the building blocks
are described in Sec. 7.4. We discuss also some properties of the proposed design. The proposed design
is exemplified in Sec. 7.6 and some numerical verifications inspecting the performance of the proposed
design can be found in Sec. 7.7. Some generalizations are addressed into Sec. 7.8 and finally the work is
concluded in Sec. 7.9.
7.2 Notation & Definitions
We remind the notation that is used within this Chapter. A vector is denoted by a bold-face lower-case
letter and it is implicitly assumed to be in a column form. It means that a vector [b1, . . . , bk]T
is denoted
b. A zero vector with a single non-zero unit value on the ith position is denoted by ei, for example e3 is a
vector given by [0, 0, 1, 0, . . . ]T
. If the order of a particular symbol in the vector is not important or clear
from the context, the subscript is dropped, i.e. b instead of bi.
Similarly a matrix is denoted by a bold upper-case letter. For example Xk×n = [x1 . . . xn] is k × n
matrix composed from n vectors x1, . . . , xn, where each of them is k symbols long. If the matrix is square,
its dimension is denoted by a single index, that is Xk×k = Xk. The matrix dimension in the subscript is
omitted whenever the matrix dimension is clear from the context. A zero matrix 0 is a matrix containing
all zero elements and a matrix E denotes the unitary matrix that can be written as E = [e1, e2, . . . ]. A
matrix Xk×n is said full-rank if rank(X) = min(k, n).
70](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-83-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
Source data are conventionally associated with letter b and their length with k, similarly the codewords
are denoted by c with associated length denoted n. The domain of both source data and codewords is
an extended Galois Field GF[qm
], where the base q may differ for data and codewords as well as m.
Throughout this work, we assume binary data, but an arbitrary extended GF can be also employed. The
signal space points are associated with s and the soft metric outputting the SoDeM is denoted by µ.
The operator Sb stands for a set of all possible vectors b and its size is denoted Mb = |Sb|. In case of
binary independent and identically distributed vector b ∈ GF[2k
], the size is Mb = 2k
7.3 System Description
We partly follow the partitioning suggested in [58] to describe the system. It means that we start with the
information flow description, then we add the channel coding and signal space connection.
A considerable part of the investigated design is demonstrable in a simple 2-source 2-path network
scenario. It means two sources, one stream flowing through a relay and one complementary stream flowing
by an another way to a final destination. The complementary link, i.e. H-SI, is not closer specified, it may
either flow though a complex network or be directly presented in the final destination. We will consider the
simple 2-source 2-path network scenario in the main part of this work. Some generalizations are addressed
into Sec. 7.8.
7.3.1 Information Flow Description
Assuming two informational sources A and B. The source data are independent and identically distributed
(iid. ) vectors bA and bB of lengths kA and kB that is bA ∈ GF[2kA
] and bB ∈ GF[2kB
].
We assume linear only hierarchical function. It means that the hierarchical data stream bAB of length
kAB flowing through the relay can be expressed as
bAB = Xb,AbA + Xb,BbB, (7.1)
where the hierarchical function is described by kAB × (kA + kB) matrix Xb = Xb,A Xb,B . The
complementary data stream (H-SI) is formally denoted by an over-bar symbol and the H-SI data stream
is given similarly by
¯bAB = ¯Xb,AbA + ¯Xb,BbB.
The overall information flowing through the whole network (both relay and its complement) can be
written as
bAB
¯bAB
=
Xb
¯Xb
bA
bB
=
Xb,A Xb,B
¯Xb,A
¯Xb,B
bA
bB
, (7.2)
where the overall matrix must be invertible to keep the source data fully decodable in the final destination.
7.3.2 Channel Coding upon the Information Flow
Due to the error protection reason, we consider a channel encoding in the network. The source data
vectors are encoded by linear encoders cA = GAbA with coding rate rA = kA/nA and cB = GBbB with
coding rate rB = kB/nB. The hierarchical codewords cAB are related to the hierarchical data by a virtual
(generally non-linear) encoder such that cAB = CAB(bAB) with rate rAB = kAB/nAB. We stress that
domains of bA, bB and bAB are not equal in general and therefore it is not straightforward how to find
CAB given GA and GB.
The relation between the source and hierarchical codewords is also given by a vector codeword hier-
archical function Xc as cAB = Xc(cA, cB). In order to enable a simplified soft-metric evaluation, we also
71](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-84-320.jpg)
![7.3. SYSTEM DESCRIPTION
bA
bB
GA SA
GB SB
+
w
·
hB
·
hA
SoDeM,
Decoder
cA sA
cB sB
x ˆbAB
ˆbA
ˆbB
bAB
¯bAB
Destination
Figure 7.1: 2-Source 2-path scenario. The goal of this work consists in a joint design of both source
encoders and relay decoder for arbitrary source rates as well as for an arbitrary hierarchical ratio.
define a symbol-wise hierarchical codeword function Xc(cA, cB) as the ith element of the vector hierarchical
codeword function Xc(eicAi, eicBi). The hierarchical symbol is then given by3
cAB = Xc(cA, cB).
The hierarchical codeword satisfies
cAB = Xc(cA, cB) = Xc(GAbA, GBbB) (7.3)
= CAB(bAB) = CAB(Xb,AbA + Xb,BbB), (7.4)
where the codeword hierarchical function is used in (7.3) and the virtual hierarchical encoder in (7.4).
If both the codeword hierarchical mapping Xc and the virtual encoder CAB are linear, the equations
(7.3) and (7.4) can be written in the matrix form
cAB = Xc,AcA + Xc,BcB = Xc,AGAbA + Xc,BGBbB (7.5)
= GABbAB = GAB(Xb,AbA + Xb,BbB). (7.6)
Our goal to have the consistent channel encoders respecting properly the information flow can be now
stated as follows. For arbitrary, but given source rates and hierarchical ratio, find consistent generator
matrices4
with good5
corresponding parity check matrices and the inner structure of the hierarchical
functions (hierarchical matrices). Dimensions and ranks of the matrices must be in accordance with the
requested rates.
7.3.3 Connection with Signal Space
The hierarchical data stream is stochastically connected though the hierarchical codewords with the source
codewords. The connection can be expressed by conditional probability p(x|cAB), where x is an observation
available in the relay.
The signal space space symbols sA(cA), sB(cB) are given by signal space mappers in sources. We assume
a memoryless channel, i.e. the relay receives a noisy superposition of these x = hAsA + hBsB + w, where
hA, hB is flat fading channel parametrization and w is complex zero mean additive white Gaussian noise
(AWGN) with unit variance. The variance of the AWGN can be adjusted by a proper scaling of the channel
gains γA = |hA|2
and γB = |hB|2
. The overall signal to noise ratio (SNR) available in relay is thus given
by SNR = γA + γB assuming E[|sA|2
] = E[|sB|2
] = 1. See Fig. 7.1.
3This relation is valid only for a memoryless channel, where the symbol cABi depends only on cAi and cBi. Otherwise,
this symbol-wise hierarchical mapping has no sense.
4The consistent generator matrices are channel encoders in sources and decoder in relay designed in such a way that the
decoder in relay properly decodes the desired hierarchical data stream in relay. See Def. 7.2 for a formal definition.
5E.g. LDPC matrix with proper degrees distribution and maximized girth.
72](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-85-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
bA
bB
bAB
¯bAB
ˆbA, ˆbB
Cut #1
Cut #2
Cut #3
Figure 7.2: Visualization of the information flow within the network. The information amount flowing
through the relay (bAB) is visualized by the Cut #1, the information flowing through H-SI is denoted Cut
#2 and the overall information that needs to be delivered into a final destination(s) is denoted by Cut
#3. One can see that the Cuts #1 and #2 can be seen as a multi-source broadcast channel. Setting the
hierarchical ratios λ and ¯λ, one can adjust the information amount flowing though particular Cuts.
7.3.4 Classification of Information Flow in Relay
The information flow through the relay can be expressed by means of entropy of the hierarchical stream,
that is H(bAB) = rank(Xb) for binary uniform iid. bA and bB (see Cut #1 in Fig. 7.2). Note that
for full-rank XB, rank(Xb) = kAB. We can also consider the overall information amount related to the
number of channel use measured by length of the hierarchical codeword cAB, i.e. rank(Xb)/nAB.
The wireless network is desired to deliver H([bT
AbT
B]) = kA + kB bits into the final destination (see Cut
#3 in Fig. 7.2). We therefore define the hierarchical ratio determining a portion of the overall information
to be processed by the relay.
The ratio
λ =
rank(Xb)
kA + kB
(7.7)
is called the hierarchical ratio for a given relay.
Definition 7.1 (Hierarchical Ratio).
If the matrix rank(Xb) is full-rank, we can express the hierarchical ratio as
λ =
kAB
kA + kB
(7.8)
In oder to reliably deliver kA + kB bits into the final destination, the H-SI must process at least the
complement of the information amount flowing through the relay. This amount is given by rank( ¯Xb) ≥
kA + kB − rank(Xb) bits (Cut #2 in Fig. 7.2). We can also define complementary hierarchical ratio as
¯λ =
rank( ¯Xb)
kA + kB
≥ 1 −
rank(Xb)
kA + kB
. (7.9)
Now the diversity of the system is given by λ + ¯λ − 1 and it takes values from the interval 0, 1 , where 0
denotes no diversity and 1 refers to full diversity. Notice that in case of non-zero diversity, the correlation
of the streams bAB and ¯bAB is implied.
73](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-86-320.jpg)
![7.4. RELAY DECODING CLASSIFICATION
A special case of full-rank hierarchical matrices and kA = kB = kAB refers to the uniform information
flow, because the information flowing through both the relay and the H-SI is equal assuming no diversity.
This case enables an elegant solution of the joint network channel encoder, where GA = GB = GAB and
this will be discussed later.
The paper [54] classifies the information flow through the relay according to the relay alphabet cardi-
nality, which is equal to our entropy-based classification for iid. data. They assume a symmetric butterfly
network, where the H-SI is given maximally by one data stream. It means that the minimal information
flowing though the relay must be the second data stream. The classification is concretely given by:
• minimal cardinality: MbAB
= max(MbA
, MbB
),
• extended cardinality: max(MbA
, MbB
) < MbAB
< MbA
MbB
• full cardinality: MbAB
= MbA
MbB
.
7.4 Classification of the Relay Decoding in Wireless Networks
with HDF
The HDF technique requires a decision about the hierarchical stream bAB. The main area of interest of
this work lies in all processing (decoding) between the vector x received in relay (the superimposed signal
from several sources parameterized by the sources-relay channels) and the estimation of the hierarchical
stream bAB in relay.
The goal of the hierarchical decoder in HDF strategy is to obtain the hierarchical stream bAB from the
observation x. The decoder can be classified as the joint-network channel decoder. The objectives of this
decoder can be summarized as:
• Performance properties.
• Implementation complexity.
• Robustness of design, that is an ability to properly incorporate arbitrary rates and hierarchical ratio..
There are basically three major approaches differing in the metric used for the decoder design.
7.4.1 Symbol-Wise Metric Evaluation for they Decoder in Relay
As it was stated, the goal is to obtain the hierarchical data stream bAB. The hierarchical data are related
with the hierarchical codewords by a virtual hierarchical channel encoder6
CAB. Assuming a memoryless
channel, it is reasonable to evaluate a symbol-wise soft metric in demodulator due to the complexity
reasons.
Optimal Symbol-Wise Metric
The MAP rule gives
P(bAB|x) = P(cAB|x) ∼ p(x|cAB)p(cAB). (7.10)
6We assume just that such an encoder exists. A particular form of the encoder will be discussed later.
74](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-87-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
SoDeM HAB
x µ(cAB) = p(x|cAB) bAB
Figure 7.5: The relay SoDeM-decoder scheme for the hierarchical decoding. Soft output demodulator
evaluates symbol-wisely p(xi|cABi) for all i. This metric is served to one hierarchical decoder HAB. This
decoder directly provides a hard decision bAB.
To formally describe this solution, we use a similar formalism as in [64] that is both codewords cAB and
joint data pair (bA, bB) take values from the extended GF with four elements8
in {0, 1, D, D + 1}. The
codeword hierarchical matrices are assumed to be the matrices upon the extended GF given by Xc,A = EnA
,
Xc,B = DEnB
and the decoding with (7.5) can be expressed as
cAB = EnA
cA + DEnB
cB = GAbA + DGBbB. (7.19)
The decoder aims with a joint soft metric about9
bA + DbB that serves as the soft output of the decoder,
that is p(x|bA, bB) in (7.17).
In a special symmetric case of uniform hierarchical ratio and equal encoders in sources, that is
GA = GB = G, we obtain for (7.19)
cAB = G(bA + DbB) (7.20)
and a straightforward decoder construction evaluating p(x|bA, bB) can be applied. Such a
decoder construction on LDPC codes is proposed in [64] and similar construction based on
turbo encoder is proposed in [8].
Example 7.1 (Optimal Solution - Uniform Hierarchical Ratio).
Note 7.2 (Optimality of Joint Decoding with Network Coding). The optimality of the processing is with
respect to obtain as much information as possible. It means that the soft metric p(x|bA, bB) contains all
available information for the hierarchical data stream decision. Nevertheless the optimality is paid by the
complexity of the decoder.
Direct (Hierarchical) Decoding
The direct (hierarchical) decoding uses the hierarchical metric p(xi|cABi) in the demodulator and the
hierarchical stream bAB is evaluated directly from the hierarchical metric. See Fig. 7.5. The capacity
regions are computed in [54] for the hierarchical MAC phase in case of uniform hierarchical ratio. The
metric p(x|cAB) is suboptimal, but a direct evaluation of the hierarchical stream brings an efficient solution,
because it is required only one run of the decoder. The problem of the this type decoding is to find a
consistent encoders and hierarchical maps such that both (7.5) and (7.6) are satisfied. A straightforward
construction of consistent hierarchical functions and encoders is not obvious in general.
8We assume that both cA and cB take values from binary extended finite fields.
9If the vector lengths are not equal, that is kA = kB, the shorter vector is filled by zeros such that the dimensions will
pass.
77](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-90-320.jpg)
![7.5. BLOCK-STRUCTURED LAYERED DESIGN
EkA1
Ek20
0 0
0
Xb,A
EkB1
Ek20
0
0 0
Xb,B
k2kB1kA1
kA1 k2 kB1 k2
Figure 7.6: The dimension of hierarchical matrices in the proposed block structured layered design. Notice
that the structure of the codeword hierarchical matrices is equal with dimensions n instead of k .
A simple, however very special solution, is available for uniform hierarchical ratio. Then the
design in a particular form containing equal encoders GA = GB = G and unitary the hierarchical
functions Xb,A = Xb,B = E, Xc,A = Xc,B = E implies
cAB = G(bA + bB). (7.21)
One can check that both (7.5) and (7.6) are fulfilled and therefore this network design stands
for a valid network configuration (see e.g. [54]).
Example 7.3 (Direct Decoding - Uniform Hierarchical Ratio).
It is obvious that this approach will not work for arbitrary hierarchical ratio and asymmetric codes.
7.5 Block-Structured Layered Design
Our approach aims with a maximal utilization of the methods suited for uniform hierarchical rate, because
we are able to design either optimal (Section 7.4.2, particularly Example 7.1) or hierarchical (direct)
(Section 7.4.2, particularly Example 7.3) channel encoder implementation for such a case. To do this, we
restrict slightly the structure of the encoding matrices into block-wise form. The inner matrices of the
block-structured design can be arbitrary valid capacity achieving single-user linear block codes and thus
the layered design paradigm is preserved. Those component codes can be e.g. LDPC codes and the overall
design preserves the sparsity of the overall matrices. An arbitrary hierarchical ratio as well as arbitrary
source rates can be achieved by a proper mixture of the block-matrices dimensions.
7.5.1 Formal Definition
We formally define our goal to have a valid network configuration. It means that GAB must properly
define the virtual hierarchical encoder, i.e. (7.5) must be satisfied jointly with (7.6) for all bA, bB.
78](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-91-320.jpg)
![7.5. BLOCK-STRUCTURED LAYERED DESIGN
The only block-component part of the decoder in relay is a common encoder G2 and thus all meth-
ods of the decoders design across the networks relying on the uniform hierarchical ratio can be
applied. The rest of the encoder design in sources can be provided without any restrictions, just
according to needs of H-SI. Notice that if this case is considered as H-SI to the previous case
λ ≥ (max(kA, kB))/(kA + kB), either Xb,A = 0, Xc,A = 0 or Xb,B = 0, Xc,B = 0. In such a
case the relay provides a single-stream decision to have a correct complementary information to H-
SI. We can again simplify the condition defining this case to kAB ≤ min(kA, kB) for fixed codeword
length.
• The remaining case (min(kA, kB))/(kA + kB) ≤ λ ≤ (max(kA, kB))/(kA + kB) can occur only for
kA = kB. Assuming without loss of generality that kA < kB, then k2 = kA and n2 = nA. The
hierarchical matrices securing the ratio from this range are given by
Xb,A =
0
Ek2
, Xb,B =
0 0
EkB12
0
0 Ek2
, (7.32)
Xc,A =
0
En2
, Xc,B =
0 0
EnB12
0
0 En2
, (7.33)
where the stream bB1 is further divided to [bT
B11bT
B12] according to the requested hierarchical ratio
λ. In this case, the relay decoder provides partly joint decoding of bT
B12 and partly decoding enabling
the uniform hierarchical ratio based methods.
7.5.3 Signal Space Mappers and Soft Metric for Decoder
We leave the optimization of concrete signal space mappers in sources out of its focus. Instead, we address
our attention to 1) a concrete way of the signal space symbol composition from the codeword sub-streams
and 2) the (symbol-wise) soft metric evaluation that was investigated in Sec. 7.4.1. The proposed block-
structured layered code requires just the soft metric for the block-decoders. Due to the memoryless channel,
the metric is evaluated symbol-wisely in the SoDeM.
The proposed design offers a freedom, how the particular codeword sub-streams are mapped into the
signal space. We can consider orthogonal (e.g. time) sharing methods as well as a common signal space
symbol that contains information from all streams. Of cause a mixture of these options is also available. In
the case of a common signal space symbol, one can see such signal space mapping as superposition coding
for the multiple access broadcast channel shown in Fig. 7.2.
7.5.4 Block-Structured Layered Design Convergence Properties
In this part, we investigate the overall decoder performance properties. We will be interested mainly in
case of hierarchical ratio λ ≥ (max(kA, kB))/(kA + kB), because the other cases can be seen as special
cases of this one from the decoder convergence point of view.
Due to the block-structured layered design, the block decoders HA1, HB1 and H2 work independently
and the overall performance of the hierarchical decoder HAB is limited by the weakest block decoder. The
hierarchical decoder HAB therefore operates on the rate of its weakest block sub-decoder.
This limitation can be naturally reduced, when bits from individual block encoders are combined into
one higher order channel symbol and a SoDeM is applied to evaluate the soft metric for the decoder.
A soft information exchange between the SoDeM and the decoder (see Fig. 7.8) is a conventional way
of a performance improvement. In our case, the improvement is emphasized by the fact that the soft
82](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-95-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
bA1
bA2
bB1
bB2
GA1
G2
SAi
GB1
G2
SAI
SBi
SBI
+ +
w
Di
+ +
I = {k : k = i}
DI
HA1
HB1
H2
ˆbA1
ˆbB1
ˆb2
cA1i
cA
2i
cB1i
cB
2i
sAi
sBi
sABi
+
xi
µA1i
µB1i
µ2i
Figure 7.8: 2-Source 1-Relay system example from bit-wise processing point of view. It is assumed that
nA1 = nB1 = n2 for simplicity. The hierarchical decoder is shown in the decomposed form. A soft message
µX is capable to carry the extrinsic information about cX evaluated in both the decoder and SoDeM.
Consequently, the block decoders are able to utilize the information from other block decoders carried via
SoDeMs.
SA
SB
R
¯R
D
Figure 7.9: System model for the motivational example.
information exchange between SoDeM and decoder also enables an information exchange between the
independent individual block decoders.
Even if one assumes the soft information exchange through the SoDeM, the overall decoder is still
limited by the worst block-decoder although with aid from the SoDeM. To reduce this limitation, we can
consider interleaving of the codewords in sources prior they are mapped into the signal space to average the
encoders convergence properties. The metric evaluated in the SoDeM must be then properly deinterleaved
and combined according to needs of a particular method.
The information exchange can be described by means of EXIT charts [19, 56]. We assume that the
conditions required for the EXIT chart [56] analysis hold, that is the independent input to soft in soft out
(SISO) blocks and Gaussian distribution of the systematic log-likelihood ratio (LLR).
7.6 Block Structured Layered Design in Simple Example
In order to better understand the proposed concept, we provide its demonstration in a simple wireless
network design.
We consider a simple two-source two-relay one destination wireless network without a direct link from
the sources to destination (see Fig. 7.9). The destination aims with decoding both data streams.
We distinguish 3 sub-streams referring to bAB in the relay R, which are formed by the individual streams
inputing the encoders GA1, GB1 (bA1, bB1) and a virtual stream b2 = bA2 ⊕ bB2 (upon GF(2)) encoded
by G2. The second (complementary) hierarchical stream ¯bAB in the relay ¯R must therefore contain at
least either bA2 or bB2 to secure decodability in the final destination. It means that the hierarchical ratio
in R is λ ≥ (max(kA, kB))/(kA + kB) (it means extended cardinality in terms of [54]) in our notation. To
83](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-96-320.jpg)
![7.6. BLOCK STRUCTURED LAYERED DESIGN IN SIMPLE EXAMPLE
λ rA1 rB1 r2 rAB
9/14 1/2 1/6 5/6 9/18
10/14 2/3 1/3 2/3 10/18
11/14 5/6 1/2 1/2 11/18
Table 7.1: Particular block-codes rates for different λ and fixed rA = 1/2, rB = 2/3 evaluated according
to (7.34-7.37).
make the example more straightforward, we assume given fixed coding rates in sources, i.e. rA and rB.
7.6.1 Adjusting Block Rates
Fixing the length of codewords to nA1 = nB1 = n2 = n and considering the case λ ≥ (max(kA, kB))/(kA +
kB), we can express the particular block component rates as
rA1 = 2rAλ − 2rB(1 − λ) (7.34)
rB1 = 2rBλ − 2rA(1 − λ) (7.35)
r2 = 2(rA + rB)(1 − λ). (7.36)
The virtual hierarchical rate rAB can be shown to be
rAB =
2
3
λ(rA + rB). (7.37)
Some examples of the block rates evaluation for different λ can be seen in Tab. 7.1.
As an example, we fix rA = 1/2 and rB = 2/3 and choose the middle line in Tab. 7.1. We further
construct the particular block components as conventional simple block coders on the corresponding rates.
We choose the block codeword length n as short as possible, that is n = 3 for given rates and the hierarchical
ratio. The block decoders properly respecting the requested rates can be then considered e.g.
HA1 = 1 1 1 , HB1 =
1 0 1
0 1 1
(7.38)
and
H2 = ι ι ι , (7.39)
where ι = 1 in case of GF(2) and ι = 1 + D in GF(4) implementation case.
The block composition of the hierarchical decoder according to Theorem 7.3 is then given by
HAB =
1 1 1 0 0 0 0 0 0
0 0 0 1 0 1 0 0 0
0 0 0 0 1 1 0 0 0
0 0 0 0 0 0 ι ι ι
. (7.40)
7.6.2 Signal Space Mapping
The encoders and the signal space mappers in sources including the bit mapping jointly with the adequate
receiver construction in relay are shown in Fig. 7.10, where the channel parameters are for simplicity
considered as hA = hB = 1. The codeword bits are mapped in the order [cA1, cA2, cB1, cB2] to the
superimposed modulation seen in relay, that is [blue, violet, cyan, magenta] order of colors in Fig. 7.10.
84](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-97-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
The hierarchical (symbol-wise) codeword mapping is therefore given either by10
cAB = [cA1, cB1, cA2 ⊕cB2]
(upon GF(2)) or by cAB = [cA1, cB1, cA2 + DcB2] upon GF(4). This can be written using matrices ˜Xc,A
and ˜Xc,B in a block diagonal form. The desired block form of Xc,A and Xc,B can be obtained by a column
swapping properly reflected in SoDeM.
7.6.3 Soft Metric Evaluation
The decoder in relay (see bottom part of Fig. 7.10) incorporates the soft metric symbol-wisely evaluated in
the SoDeM. One can see that the particular streams are mapped into a certain position in the superimposed
modulation (see different colors in Fig. 7.10) and thus the bits associated to different streams are unequally
resistant against the noise. The connection of the decoder and the demodulator can be seen also in Fig.
7.8, where the soft information exchange is shown.
The unequal resistance of the streams against the noise can be equalized11
by averaging using interleaver
over a whole codeword as shown in Fig. 7.10. This can be simply implemented such that both the codewords
cA and cB are interleaved by an equal interleaver of length nA = nB = 2n. The joint demodulator-decoder
shown in Fig. 7.10 illustrates the mapping connection for both interleaved and non-interleaved case.
In case, where the interleaver is not used, the first bit of the superimposed symbol is always associated
with the decoder HA1, similarly the third bit with HB1 and finally both second and fourth bit with H2 as
one can see in Fig. 7.10. On the other hand, the use of interleaver causes that all bits are associated with
the particular decoders randomly as show in Fig. 7.10 (thick dashed lines). Note that particular metrics as
shown in Fig. 7.8 are evaluated by marginalization of the joint likelihood p(x|cA, cB) in SoDeM exploiting
a priory information from other streams.
7.6.4 Overall Network Design
Up to this time, we investigated processing only in one of the relays. We set the particular block rates for
the block-component decoder in the relay. It is obvious that we will cannot independently set arbitrary rates
in the complementary relay. It means that the complementary relay seems to be forced to decode at least
either bA2 or bB2 on rate r2 in order to have an error-less minimal complementary information required in
destination. Nevertheless, we have available at least two approaches how to solve the inconvenience, when
the stand-alone rate r2 is not directly achievable in ¯R. The first one is that we have choice to increase the
power in source at expanse of the second transmitted data stream (e.g. bB1). The second choice might be
to communicate on the rate of a whole data codeword or some of its part12
(e.g. whole bB). As we will
observe (Fig. 7.12), the enabled loops between the SoDeM and decoder in case of higher order modulations
enables to operate on some averaged rate (e.g. rB < r2 in our case).
If we use this averaged rate, we have more information that is directly needed for the final destination
in the relays. Considering a decoding of the whole data vector in ¯R, that is ¯bAB = bB in our case, the
relays have available the following streams:
• Relay R: bA1, bB1, b2 = bA2 ⊕ bB2
• Relay ¯R: bB1, bB2.
It can be clearly seen that the stream bB1 is available in both relays. The relays can therefore transmit
all streams, where some power must be allocated to this redundant information. On the other hand, it is
10Notice that this concrete setup may be seen similar to the approach proposed in [22]. The fundamental difference is that
their approach is based on clear network coding, that is the aforementioned relation is only upon the hierarchical data.
11Notice that this averaging is not necessary. One can consider the case when the unequal error protection is exploited e.g.
to enable a particular block decoder operation on higher rate. Such a signal space mapping design is a challenging question
that is addressed for future work.
12Further block structuring can be considered.
85](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-98-320.jpg)
![7.7. NUMERICAL VERIFICATION
7.7.2 Simulation Setup and Outer Parameters
We compose a system to provide a numerical verification of the proposed design. We consider the equal
system model as in Sec. 7.6, where only one difference is in the codeword lengths nA1 = nB1 = n2 =
n = 2400 and the block-coders are assumed to be regular LDPC codes with maximized girth according
to [35]. We choose (ad-hoc) the signal space mappers at bit level to be QPSK for both sA and sB. The bit
mapping is ad-hoc selected to maximize the minimal hierarchical distance. The symbol received in relay
is given by x = sA + hsB + w, where h = exp(j π/4) and w is assumed to be a zero-mean AWGN. The
relay evaluates a corresponding soft metric in SoDeM. The model is in fact shown in Fig. 7.10, where some
longer encoders and interleaver are considered.
To demonstrate the system diversity, we consider the hierarchical matrices as
X =
EkA1
0 0 0
0 0 EkB1
0
0 Ek2 0 Ek2
(7.41)
and two complementary matrices,
¯X ∈ 0 0 0 Ek2 ,
0 0 EkB1
0
0 0 0 Ek2
. (7.42)
It means that the first relay R has available the streams [bA1, bB1, b2 = bA2⊕bB2] and the complementary
relay ¯R has available either stand-alone [bB2] or both [bB1 bB2] as in Sec. 7.6.4. For simplicity, we assume
that kA1 = kB1 = k2 = k = 1600 in this case (code rate is 2/3 for all streams). All streams are encoded
in the relays by equal encoder and mapped to BPSK modulation, where the mean energy of the signal
¯sB2,min(bB2) referring to the scheme without diversity is split between ¯sB1,mid(bB1) and ¯sB2,mid(bB2) in the
diverse scheme, where the "min" subscript denotes no diversity (minimal complementary information) and
"mid" subscript denotes the diversity given by a redundant stream bB1. The streams are then transmitted
orthogonally into the final destination through AWGN channel, where the decoder based on the factor
graph is performed to decode the streams.
7.7.3 Simulation Results
First of all, we can recognize the waterfall behavior of all BER curves indicating a proper functionality of
the proposed design. The performance of the individual streams is given by signal space properties and
by the rate of the referring block-decoder. We can see (Figs. 7.11 and 7.12) that the overall hierarchical
decoder is indeed limited by its weakest sub-decoder, when the iterative loop via SoDeM is not applied.
The soft information exchange between SoDeM and decoder significantly improves the performance
(see Figs. 7.11 and 7.12). The properties of the block decoders are averaged and the hierarchical encoder
definitely operates on a larger rate then its stand-alone weakest block decoder. The EXIT curves of the
hierarchical decoder are shown in Fig. 7.14.
Using the block decoder H2 working upon higher field, i.e. GF(4), we obtain some further performance
gain (Fig. 7.13), which is however paid by higher complexity of the relay decoder. Using the interleaver
to average the quality of the metric serving to the block decoders does not have any important impact to
the performance as one can check in Fig. 7.13 for given rates and the signal space mapping.
According to the provided results, we can conclude that the hierarchical (direct) decoding (H2 upon
GF(2)) achieves similar performance to the reference JDF over NC with less complexity, while the decoding
with (H2 upon GF(4)) benefits approximately half dB gain against the reference solution for given scenario
(see Fig. 7.13).
The performance comparison between diverse and no-diverse scheme can be found in Fig. 7.15. The
diversity improves the performance of the stream bB1, while the performance of the streams bA2, bB2 is
88](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-101-320.jpg)
![CHAPTER 7. NETWORK CHANNEL CODE DESIGN
4 4.5 5 5.5 6 6.5 7
10−7
10−6
10−5
10−4
10−3
10−2
10−1
100
SNR of the MAC channel
Pbe
BER for different decoder realizations for rAB=5/9.
hier
hier+int
GF(4) based H2
JDF over NC
Figure 7.13: Comparison of the relay-decoder performance decoder for different realizations. We consider
conventional decoders as GA1 and GB1 and different variants of G2 that are (1) hierarchical form, i.e.
GF(2) decoder with G2 = GA2 = GB2 with and without interleaver, (2) optimal form, i.e. GF(4) as in [64].
study of these more complex network topologies is addressed for the future work.
91](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-104-320.jpg)
![8 Overall Conclusions
The main topic of this thesis is the receiver design in a multi-source multi-node wireless network. We cope
this topic from two layers. The first one is the overall network design on the channel code level, that is to
design a joint consistent cross-network channel code that satisfies some requests. The latter consists in a
direct implementation of the generic sum product algorithm that is capable to describe among others the
receiver in the wireless network. One can see a clear relation between the mentioned topics, that is the
overall network design properly and efficiently implemented by means of the sum product algorithm.
The sum product algorithm direct implementation was solved by the implementation framework con-
taining the proposed generic KLT-message representation and the update rules design upon the message
representation that are applicable among others to the proposed generic KLT message representation. We
numerically figured out that the messages in a wide class of phase models lead to the harmonic eigen-
functions applying the KLT on their stochastic description. This proves the Fourier representation to be
the best possible linear message representation in the MSE sense. We can note that the implementation
framework relying on the Fourier message representation was independently proposed in [5]. We verified
the capability of the KLT (Fourier) representation by a numerical simulation for a given model, where the
performance results are shown in Figures 5.13, 5.14 and 5.15.
We further proposed an update rules design for an arbitrary linear message representation with orthog-
onal canonical kernels. The assumption of the linearity and orthogonality is inherently satisfied for the
KLT-message representation. The proposed update rules design jointly with the KLT-message representa-
tion therefore form a generic implementation framework applicable whenever the stochastic description of
the messages is available, i.e. KLT of the random process given by the different message realizations can
be evaluated. The framework is moreover built upon the best linear message approximation in the MSE
sense for a given dimensionality as a consequence of KLT.
The Fourier representation forms an orthogonal kernel basis system. Being aware of the eigen-analysis
results (see Figures 5.10, 5.11 and 5.12), we used the Fourier representation for the proposed implementation
framework verification in the given joint data detection-phase estimation system model. The proposed
implementation framework that results in a highly efficient implementation (check Table 5.1) for this
particular scenario.
The demand for a flexible wireless aware joint network-channel code design becomes clear after proving
the wireless network coding potential (see e.g. [38]). We started with defining a robust system description.
This description is based on the system linearity, that is the linear hierarchical function (in fact a wireless
aware network code - see Note 6.4) and the channel code. The linearity assumption is satisfied by conven-
tional capacity achieving codes (e.g. LDPC or turbo codes). We therefore form a Theorem 7.3 defining a
block structure of the channel codes across the network. The relation of the inner block sizes (diagonal
structure is assumed) determines the particular channel code rates. Arbitrary channel code rates can be
set adjusting properly the sizes of inner blocks (Theorem 7.4).
The proposed block structured layered design basically combines two known principles, where both
the requested hierarchical ratio (amount of information flowing through the relay) and the requested rates
are achieved in average over a whole codeword. We also discussed some possible structures of the decoder
properly respecting the block structured layered design. This overall decoder is basically composed from the
conventional LDPC block-decoders, where the soft information exchange between the (independent) block
parts can be conducted e.g. via soft output demodulator for higher order alphabets. The performance for
different metrics evaluated in the soft output demodulator and corresponding decoder structure is evaluated
in a particular system model (check Figures 7.12 and 7.11). These results proves the waterfall behaviour
indicating a proper decoder implementation.
93](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-106-320.jpg)
![Bibliography
[1] Rudolf Ahlswede, Ning Cai, Shuo-Yen Robert Li, and Raymond W. Yeung. Network information flow.
IEEE Trans. Inf. Theory, 46(4):1204–1216, July 2000.
[2] Hans andrea Loeliger. Some remarks on factor graphs. In Proc. 3rd Int. Symp. on Turbo Codes and
Related Topics, pages 111–115, 2003.
[3] Khoirul Anwar and Tad Matsumoto. Accumulator-assisted distributed turbo codes for relay systems
exploiting source-relay correlation. Communications Letters, IEEE, 16(7):1114–1117, 2012.
[4] Xingkai Bao and JT Li. Generalized adaptive network coded cooperation (gancc): A unified framework
for network coding and channel coding. IEEE Trans. Commun., 59(11):2934–2938, 2011.
[5] Alan Barbieri, Giulio Colavolpe, and Giuseppe Caire. Joint iterative detection and decoding in the
presence of phase noise and frequency offset. In Proc. IEEE Internat. Conf. on Commun. (ICC), 2005.
[6] Claude Berrou and Alain Glavieux. Near optimum error correcting coding and decoding: Turbo-codes.
IEEE Trans. Commun., 44(10):1261–1271, October 1996.
[7] U. Bhat and T. Duman. Decoding strategies at the relay with physical-layer network coding. IEEE
Trans. Wireless Commun., pages 4503–4513, December 2012.
[8] M.C. Castro, B.F. Uchoa-Filho, T.T.V. Vinhoza, M. Noronha-Neto, and J. Barros. Improved joint
turbo decoding and physical-layer network coding. In Proc. IEEE Inf. Theory Workshop (ITW), pages
532–536. IEEE, 2012.
[9] Keith Chugg, Achilleas Anastasopoulos, and Xiaopeng Chen. Iterative Detection: Adaptivity, Com-
plexity Reduction, and Applications. Kluwer Academic Publishers, 2001.
[10] Giulio Colavolpe, Alan Barbieri, and Giuseppe Caire. Algorithms for iterative decoding in the presence
of strong phase noise. IEEE J. Sel. Areas Commun., 23(9):1748–1757, September 2005.
[11] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley & Sons, 1991.
[12] Justin Dauwels and Hans andrea Loeliger. Phase estimation by message passing. In Proc. IEEE Int.
Conf. on Communications, pages 523–527, 2004.
[13] Justin H. G. Dauwels. On Graphical Models for Communications and Machine Learning:Algorithms,
Bounds, and Analog Implementation. PhD thesis, Swiss Federal Institute of Technology, Zürich, May
2006.
[14] M.C. Davey and D.J.C. MacKay. Low density parity check codes over gf(q). In Information Theory
Workshop, 1998, pages 70 –71, jun 1998.
[15] Abbas El Gamal and Young-Han Kim. Network Information Theory. Cambridge University Press,
2011.
[16] G. Elidan, I. McGraw, and D. Koller. Residual belief propagation: Informed scheduling for asyn-
chronous message passing. In Proceedings of the Twenty-second Conference on Uncertainty in AI
(UAI), Boston, Massachussetts, July 2006.
[17] Uri Erez and Ram Zamir. Achieving 1/2 log(1 + SNR) on the AWGN channel with lattice encoding
and decoding. IEEE Trans. Inf. Theory, 50(10):2293–2314, October 2004.
96](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-109-320.jpg)
![BIBLIOGRAPHY
[18] Gianluigi Ferrari, Giulio Colavolpe, and Riccardo Raheli. A unified framework for finite-memory
detection. Selected Areas in Communications, IEEE Journal on, 23(9):1697–1706, 2005.
[19] Alex Grant. Convergence of non-binary iterative decoding. In Proc. IEEE Global Telecommun. Conf.
(GlobeCom), 2001.
[20] Z. Guo, J. Huang, B. Wang, S. Zhou, J.H. Cui, and P. Willett. A practical joint network-channel
coding scheme for reliable communication in wireless networks. IEEE Trans. Wireless Commun.,
11(6):2084–2094, 2012.
[21] Joachim Hagenauer, Elke Offer, and Lutz Papke. Iterative decoding of binary block and convolutional
codes. IEEE Trans. Inform. Theory, 42:429–445, 1996.
[22] C. Hausl, F. Schreckenbach, I. Oikonomidis, and G. Bauch. Iterative network and channel decoding on
a tanner graph. In Proc. Allerton Conf. on Commun., Control and Computing. Citeseer, September
2005.
[23] Sheryl L Howard, Vincent C Gaudet, and Christian Schlegel. Soft-bit decoding of regular low-density
parity-check codes. Circuits and Systems II: Express Briefs, IEEE Transactions on, 52(10):646–650,
2005.
[24] Steven M. Kay. Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall,
1993.
[25] Steven M. Kay. Fundamentals of Statistical Signal Processing: Detection Theory. Prentice-Hall, 1998.
[26] Toshiaki Koike-Akino, Petar Popovski, and Vahid Tarokh. Optimized constellations for two-way
wireless relaying with physical network coding. IEEE J. Sel. Areas Commun., 27(5):773–787, June
2009.
[27] Frank R. Kschischang, Brendan J. Frey, and Hans-Andrea Loeliger. Factor graphs and the sum-product
algorithm. IEEE Trans. Inf. Theory, 47(2):498–519, February 2001.
[28] Brian M. Kurkoski and Justin Dauwels. Belief-Propagation Decoding of Lattices Using Gaussian
Mixtures. CoRR, abs/0904.4741, 2009.
[29] J.N. Laneman, D.N.C. Tse, and G.W. Wornell. Cooperative diversity in wireless networks: Efficient
protocols and outage behavior. Information Theory, IEEE Transactions on, 50(12):3062 – 3080, dec.
2004.
[30] Soung Chang Liew, Shengli Zhang, and Lu Lu. Physical-layer network coding: Tutorial, survey, and
beyond. in Phys. Commun., 2011 [Online]. Available: http://arxiv.org/abs/1105.4261, 2011.
[31] J. Liu, M. Tao, and Y. Xu. Pairwise Check Decoding for LDPC Coded Two-Way Relay Block Fading
Channels. IEEE Trans. Commun., PP(99):1–12, May 2012.
[32] H.-A. Loeliger, J. Dauwels, Junli Hu, S. Korl, Li Ping, and F.R. Kschischang. The Factor Graph
Approach to Model-Based Signal Processing. Proceedings of the IEEE, 95(6):1295–1322, June 2007.
[33] Hans-Andrea Loeliger. An introduction to factor graphs. IEEE Signal Process. Mag., 21(1):28–41,
January 2004.
[34] J.A. McGowan and R.C. Williamson. Loop Removal from LDPC Codes. In Information Theory
Workshop, pages 230 – 233, april 2003.
97](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-110-320.jpg)
![BIBLIOGRAPHY
[35] James A McGowan and Robert C Williamson. Loop removal from ldpc codes. In Information Theory
Workshop, 2003. Proceedings. 2003 IEEE, pages 230–233. IEEE, 2003.
[36] J. M. Mooij. Understanding and Improving Belief Propagation. PhD thesis, Radboud University
Nijmegen, May 2008.
[37] Joris M. Mooij and Hilbert J. Kappen. Sufficient conditions for convergence of the sum-product
algorithm. IEEE Trans. Inf. Theory, pages 4422–4437, May 2007.
[38] Wooseok Nam, Sae-Young Chung, and Yong H. Lee. Capacity bounds for two-way relay channels. In
Proc. Int. Zurich Seminar on Communications, 2008.
[39] B. Nazer and M. Gastpar. Reliable physical layer network coding. Proceedings of the IEEE, 99(3):438–
460, 2011.
[40] Bobak Nazer and Michael Gastpar. Compute-and-forward: Harnessing interference through structured
codes. IEEE Trans. Inf. Theory, 57(10):6463–6486, October 2011.
[41] O. Oyman, N.J. Laneman, and S. Sandhu. Multihop relaying for broadband wireless mesh networks:
From theory to practice. Communications Magazine, IEEE, 45(11):116 –122, November 2007.
[42] Athanasios Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, second
edition, 1984.
[43] John G. Proakis. Digital Communications. McGraw-Hill, 4th edition, 2001.
[44] Pavel Prochazka and Jan Sykora. Karhunen-Loève based reduced-complexity representation of the
mixed-density messages in SPA on Factor Graph and its impact on BER. EURASIP J. on Wireless
Comm. and Netw., 2010:11, December 2010.
[45] Pavel Prochazka and Jan Sykora. Block-structure based extended layered design of network coded
modulation for arbitrary individual using hierarchical decode & forward strategy. In In COST IC1004
MCM, Bristol, UK, pages 1–7, September 2012.
[46] Pavel Prochazka and Jan Sykora. Block-Structured Layered Design of Wireless Aware Joint Network-
Channel Coding for Asymmetric Rates using HDF Relaying Strategy. IEEE Trans. Wireless Commun.,
July 2013. Submitted for publication.
[47] Pavel Prochazka and Jan Sykora. Generic efficient fg-spa implementation framework based on the
best linear canonical message representation in mse sense. European Wireless 2013, 2013.
[48] Boris Rankov and Armin Wittneben. Spectral efficient protocols for half-duplex fading relay channels.
Selected Areas in Communications, IEEE Journal on, 25(2):379–389, 2007.
[49] Tom Richardson and Ruediger Urbanke. Modern Coding Theory. Cambridge University Press, New
York, NY, USA, 2008.
[50] Bixio E. Rimoldi. A decomposition approach to CPM. IEEE Trans. Inf. Theory, 34(2):260–270,
March 1988.
[51] Christian B. Schlegel and Lance C. Perez. Trellis and Turbo Coding. John Wiley & Sons, 2004.
[52] Frederik Simoens and Marc Moeneclaey. Code-aided estimation and detection on time-varying cor-
related MIMO channels: A factor graph approach. EURASIP J. on Advances in Signal Processing,
2006:1–11, 2006.
98](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-111-320.jpg)
![BIBLIOGRAPHY
[53] Jan Sykora. Factor graph framework for serially concatenated coded CPM with limiter phase dis-
criminator receiver. In Proc. IEEE Vehicular Technology Conf. (VTC), pages 1–6, Baltimore, USA,
October 2007.
[54] Jan Sykora and Alister Burr. Layered design of hierarchical exclusive codebook and its capacity
regions for HDF strategy in parametric wireless 2-WRC. IEEE Trans. Veh. Technol., 60(7):3241–
3252, September 2011.
[55] Jan Sykora and Pavel Prochazka. Performance evaluation of the factor graph CPM phase discriminator
decoder with canonical messages and modulo mean updates. In Proc. IEEE Asia-Pacific Conf. on
Communications (APCC), pages 1–5, Tokyo, Japan, October 2008.
[56] Stephan ten Brink. Convergence begavior of iteratively decoded parallel concatenated codes. IEEE
Trans. Commun., 49(10):1727–1737, October 2001.
[57] Stephan ten Brink, Gerhard Kramer, and Alexei Ashikhmin. Design of low-density parity-check codes
for modulation and detection. Communications, IEEE Transactions on, 52(4):670–678, 2004.
[58] Tomas Uricar, Tomas Hynek, Pavel Prochazka, and Jan Sykora. Wireless-aware network coding:solving
a puzzle in acyclic multi-stage cloud networks. In Accepted to ISWCS, pages 1–5, September 2013.
[59] Tomas Uricar and Jan Sykora. Non-uniform 2-slot constellations for relaying in butterfly network
with imperfect side information. IEEE Commun. Lett., 2012.
[60] Nenad Veselinovic, Tad Matsumoto, and Markku Juntti. Iterative receivers for sttrc-coded mimo
turbo equalization. In Vehicular Technology Conference, 2004. VTC 2004-Spring. 2004 IEEE 59th,
volume 1, pages 460–463. IEEE, 2004.
[61] Branka Vucetic and Jinhong Yuan. Turbo Codes: Principles and Applications. Kluwer Academic
Publishers, 2000.
[62] Niclas Wiberg. Codes and Decoding on General Graphs. Phd thesis, Linköping University, 1996.
[63] Andrew P. Worthen and Wayne E. Stark. Unified design of iterative receivers using factor graphs.
IEEE Trans. Inf. Theory, 47(2):843–849, February 2001.
[64] D. Wübben and Y. Lang. Generalized sum-product algorithm for joint channel decoding and physical-
layer network coding in two-way relay systems. In IEEE 2010 Global Telecommunications Conference
(Globecom 2010), Miami, FL, USA, Dec 2010.
[65] Sichao Yang and Ralf Koetter. Network coding over a noisy relay: a belief propagation approach. In
Proc. IEEE Internat. Symp. on Inf. Theory (ISIT), 2007.
[66] Jonathan S. Yedidia, William T. Freeman, and Yair Weiss. Constructing free energy approximations
and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51:2282–
2312, 2005.
[67] Raymond W. Yeung, Shuo-Yen Robert Li, Ning Cai, and Zhen Zhang. Network Coding Theory. now
Publishers, 2006.
[68] Yan Zhang, Hsiao-Hwa Chen, and Mohsen Guizani. Cooperative Wireless Communications. Auerbach
Publications, Boston, MA, USA, 2009.
99](https://image.slidesharecdn.com/2fddc85b-8abb-46a6-b0b2-a58852fea97c-160907192136/85/disertation_Pavel_Prochazka_A1-112-320.jpg)
disertation_Pavel_Prochazka_A1
- 1. Distributed Iterative Decoding and Processing in Multi-Source Cooperative Networks. PhD Thesis Pavel Procházka Czech Technical University in Prague Faculty of Electrical Engeneering Department of Radioelectronics PhD Programme: Electrical Engineering and Information Technology Branch of study: Radioelectronics (2601V010) December, 2013 (A1) Supervisor: Prof. Ing. Jan Sýkora CSc.
- 2. Annotation The wireless network coding [26,30,39,54] (wireless extension of the network coding [1,67]) was shown to have a potential to significantly improve the performance in the wireless networks against the conventional networks based on independent (or interfering) point to point channels directed by higher layers. The whole network must be however designed jointly to take advantages from the wireless network coding. The receiver processing then should take into account this global design. This thesis aims to (1) the global design of the channel encoders and corresponding decoders among all nodes within the wireless network and (2) the receiver processing of a particular node in the wireless communication network. A joint solution of the channel coders design across the network is called the block structure layered design [46]. This solution allows to incorporate arbitrary particular channel code rates. The design is based on a linear description of the wireless network containing hierarchical maps (network coding) and channel coding. The design combines already known approach suited just for symmetric channel code rates (equal channel encoders) [30] properly utilizing advances of the network coding paradigm [67] in wireless extension. This principle is mixed with traditional joint decoding such that the desired code rates and relay information flows are achieved in average over whole codeword. This mixture results in the block diagonal structure using the proposed linear system description. We numerically evaluate the performance for a particular wireless network that is designed by means of the wireless network coding to prove the design properties. The fundamental topic underlaying the whole thesis is the sum product algorithm on factor graphs [27,33]. This generic tool enables a construction of the receiver processing for a general class of scenarios including the wireless networks with properly respected global design. The thesis aims to a direct im- plementation of the sum product algorithm that is applied in the receiver within the wireless networks. Our endeavour results in a generic implementation framework of the sum-product algorithm [44,47] that is applicable in a wide class of scenarios including the receiver processing in the wireless networks. The framework is based on the proposed KLT message representation. The KLT message representation is capable to methodically describe an arbitrary continuous message in FG-SPA, whenever the stochastic description (even numerical) is available and the KLT of the random process formed by random message realizations can be evaluated. The description is moreover the best one among all linear message repre- sentations for a given message dimension as a consequence of the KLT. We stress that the uniqueness of the proposed message representation consists in its methodical nature and applicability on the continuous messages (the FG-SPA definition straightforwardly enables only representation of discrete messages). We further derive the update rules upon linear message representations with orthogonal kernels. The KLT message representation satisfies both the linearity and the orthogonality of the kernels and therefore the proposed update rules design is applicable also for the KLT message representation. The KLT message representation jointly with the update rules design therefore form the generic FG-SPA implementation framework. The proposed implementation framework is verified in a particular scenario (a receiver aiming with joint phase estimation data detection) and compared with a conventional solution. i
- 3. Acknowledgement I would like to thank everybody supporting me and my research in both working and personal live during my studies. Particularly, to my supervisor for his suggestions and ideas that are reflected in this work, mainly in the topic related to Chapter 5, especially the KLT message representation. I would like to thank also the members of DiRaC lab working on similar topic for mutual discussions leading to improve my (hopefully our) knowledge related not only to this thesis. The next acknowledgement is dedicated to the anonymous reviewers of both journal and conference papers, editors and the interacting audience in different venues (conferences, meetings, workshops, etc. ) for their often apt comments and suggestions that helped to improve this work. An inseparable part enabling me to concentrate to the research beyond this thesis is the environment of my personal live, mainly my family and friends. I would like to thank all of them for assisting me to relax and to overcome the desperate whiles resulting from numerous deadlocks that sometimes crossed my research. Namely, I would like to thank my wife Marketa among others for her support and tolerance of my asocial behaviour during long nights before numerous of deadlines. Last but not the least, the work was supported by the following grants: • EU COST-IC1004: Cooperative Radio Communications for Green Smart Environments, 2011-2015, • EU COST 2100: Pervasive Mobile and Ambient Wireless Communications, 2006-2010, • FP7-ICT/STREP (INFSO-ICT-248001): SAPHYRE - Sharing Physical Resources Mechanisms and Implementations for Wireless Networks, 2010-2012, • FP7-ICT-2011-8/ICT-2009.1.1: DIWINE - Dense Cooperative Wireless Cloud Network, 2013-2015, • Grant Agency of the Czech Republic (GACR 102/09/1624): Mobile Radio Communication Systems with Distributed, Cooperative and MIMO processing, 2009-2012, • Ministry of Education, Youth and Sports (OC 188): Signal Processing and Air-Interface Technique for MIMO radio communication systems, 2007-2010, • Ministry of Education, Youth and Sports (LD12062): Wireless Network Coding and processing in Cooperative and Distributed Multi-Terminal and Multi-Node Communication Systems, 2012-2015, • Grant Agency of the CTU in Prague: SGS10/287/OHK3/3T/13, 2010-2012, • Grant Agency of the CTU in Prague: SGS13/083/OHK3/1T/13, 2013. The main contribution of this work can be also found in the final (concurrent) reports of the aforementioned grants. ii
- 4. Contents 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Goals and Summary of Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 I Background 5 2 Theoretical Background 6 2.1 Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.1 Capacity of Scalar Point to Point AWGN Channel . . . . . . . . . . . . . . . . . . . 6 2.1.2 Capacity of Multiple Access AWGN Channel . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Estimation Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.2 Maximum Likelihood Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.3 Bayesian Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 Error Protection Techniques 10 3.1 Encoding Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.1 FSM/Block Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.2 Turbo Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1.3 LDPC Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1.4 Other Encoding Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Receiver Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2.1 FSM Decoding - Viterbi Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2.2 FSM decoding - FBA Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2.3 Graph Based Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 Convergence Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3.1 Log-likelihood Arithmetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3.2 Binary EXIT Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 II FG-SPA Processing and its Implementation 17 4 Factor Graphs and Related Algorithms 18 4.1 Cycle-Free FG-SPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.1.1 Formal Definition of the Factor Graph . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.1.2 Message Passing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.1.3 Sum Product Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.1.4 MAP Decoding and MMSE Estimation Using FG-SPA . . . . . . . . . . . . . . . . . 24 4.2 FG-SPA Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.2.1 Message Types in FG-SPA and their Representation . . . . . . . . . . . . . . . . . . 25 4.2.2 Message Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2.3 Update Rules Upon the Message Representation . . . . . . . . . . . . . . . . . . . . 29 4.2.4 FG-SPA Implementation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2.5 Instances of the FG-SPA Implementation Frameworks . . . . . . . . . . . . . . . . . 32 4.3 SPA on Looped Factor Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 iii
- 5. CONTENTS 4.3.1 Message Scheduling in FG-SPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.3.2 Convergence of the Looped FG-SPA . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5 FG-SPA Implementation Framework 36 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.1.3 Summary of Results - Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.4 Summary of Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2 FG-SPA Implementation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2.1 KLT Message Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2.2 Evaluation of the KLT Approximated Message in Real Applications . . . . . . . . . 40 5.2.3 Generic Update Rules of the Canonical Message Representation with Orthogonal Kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.3 Applications and Numerical Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.3.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3.2 Factor Graph of the Joint Phase Estimator Channel Decoder . . . . . . . . . . . . . 45 5.3.3 Concrete Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.3.4 Stochastic Analysis of the Mixed-Density Messages in FG-SPA Run . . . . . . . . . 48 5.3.5 Proposed Generic Update Rules Implementation . . . . . . . . . . . . . . . . . . . . 54 5.4 FG-SPA Implementation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 III Decoding in Multi-Source Wireless Networks 59 6 Physical Wireless Network Communication 60 6.1 Wireless Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.1.1 Basic Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.1.2 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.1.3 Network Processing Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.1.4 Type and Nature of Information Appearing in the Wireless Network . . . . . . . . . 62 6.1.5 Relaying Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.1.6 Physical Network Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 6.1.7 Challenging Problems in Wireless Networks . . . . . . . . . . . . . . . . . . . . . . . 67 7 Network Channel Code Design 68 7.1 Introduction to Wireless Network Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 7.1.2 Goals & Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 7.1.3 Structure of Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 7.2 Notation & Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 7.3 System Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 7.3.1 Information Flow Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 7.3.2 Channel Coding upon the Information Flow . . . . . . . . . . . . . . . . . . . . . . . 71 7.3.3 Connection with Signal Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 7.3.4 Classification of Information Flow in Relay . . . . . . . . . . . . . . . . . . . . . . . 73 7.4 Relay Decoding Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 7.4.1 Symbol-Wise Metric Evaluation for they Decoder in Relay . . . . . . . . . . . . . . . 74 7.4.2 Decoding Strategies in Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.5 Block-Structured Layered Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 iv
- 6. CONTENTS 7.5.1 Formal Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 7.5.2 Block-Structured Layered Design Properties . . . . . . . . . . . . . . . . . . . . . . . 81 7.5.3 Signal Space Mappers and Soft Metric for Decoder . . . . . . . . . . . . . . . . . . . 82 7.5.4 Block-Structured Layered Design Convergence Properties . . . . . . . . . . . . . . . 82 7.6 Block Structured Layered Design in Simple Example . . . . . . . . . . . . . . . . . . . . . . 83 7.6.1 Adjusting Block Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.6.2 Signal Space Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.6.3 Soft Metric Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 7.6.4 Overall Network Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 7.7 Numerical Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.7.1 Models to be Numerically Simulated . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.7.2 Simulation Setup and Outer Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.7.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.8 Generalization and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 7.9 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 8 Overall Conclusions 93 v
- 7. List of Figures 4.1 Simple Factor Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.2 Simple FG-SPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.3 Mixed-Density Parameterization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.4 Message Approximation Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.5 Canonical Message Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.6 Update Rules Upon Exact Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.7 Update Rules Upon Approximated Message . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.8 Ilustration of Message Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1 KLT Fidelity-Complexity Trade-Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.2 FN Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.3 VN Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.4 Negative Message Appearance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.5 Negative Message Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.6 Signal Space System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.7 MSK Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.8 Factor Graph of Joint Phase Estimator-Data Detector . . . . . . . . . . . . . . . . . . . . . 46 5.9 Phase Shift Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.10 KLT Kernel Functions - Phase Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.11 KLT Kernel Functions - MSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.12 KLT Eigenvalues - MSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.13 Performance of Uncoded QPSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.14 Performance of BICM with 8PSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.15 BER as Function of Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.16 FG of Receiver in Signal Space with RW Phase Model . . . . . . . . . . . . . . . . . . . . . 54 5.17 FG-SPA Implementation Framework - BER Comparison . . . . . . . . . . . . . . . . . . . . 56 5.18 FG-SPA Implementation Framework - MSE Comparison . . . . . . . . . . . . . . . . . . . . 56 6.1 Relay Input Output Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.2 Two-Way Relay Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.3 Butterfly Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.4 Two-Source Two-Relay Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.5 Acyclic Layered Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.6 Multi-Hop Relay Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.1 Two-Source Two-Path Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 7.2 Infromation Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 7.3 Demodulator and Decoder - JDF Over NC . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 7.4 Demodulator and Decoder - Optimal Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 76 7.5 Demodulator and Decoder - Direct Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.6 Hierarchical Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 7.7 Block Structured Channel Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 7.8 Bit Mapping in Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.9 Motivation Example - System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.10 Bit Mapping in Demodulator-Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 7.11 Impact of Demodulator Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 7.12 Impact of the Demodulator Use - Averaging Impact . . . . . . . . . . . . . . . . . . . . . . 90 7.13 Comparison of Different Decoder Realizations . . . . . . . . . . . . . . . . . . . . . . . . . . 91 vi
- 8. LIST OF FIGURES 7.14 EXIT Chart Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.15 Diversity Impact Investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 vii
- 9. List of Tables 5.1 FG-SPA Implementation Framework - Complexity Consideration . . . . . . . . . . . . . . . 57 7.1 Block Structured Layered Design - Rates Assignment Examples . . . . . . . . . . . . . . . . 84 viii
- 10. Nomenclature ˆM domain of approximated messages in FG-SPA R set of real numbers C set of complex numbers M domain of messages in FG-SPA N set of natural numbers R+ set of non-negative real numbers AF Amplify and Forward AWGN Additive White Gaussian Channel BER Bit Error Rate BICM Bit Interleaved Coded Modulation CF Compress and Forward CPE Continuous Phase Encoder DF Decode and Forward EXIT Extrinsic Information Transfer FBA Forward backward Algorithm FG Factor Graphs FG-SPA Sum Product Algorithm on Factor Graphs FN Factor Node FSA Flood Schedule Algorithm FSM Finite State Machine GF(q) Galois Field Modulo q H-SI Hierarchical-Site Information HDF Hierarchical Decode and Forward iid independent and identically distributed (random vector) JDF Joint Decode and Forward KLT Karhunen Loeve Transform (principal component analysis) LCMR Linear Canonical Message Representation LD Loop Disabled LDPC Low Density Parity Check ix
- 11. NOMENCLATURE LE Loop Enabled LF Likelihood Function LLF Log-Likelihood Function LLR Log Likelihood Ratio LPD Limited Phase Discriminator MAC Multiple Access Channel MAP Maximum A Posteriori (probability) MIMO Multiple Input Multiple Output (channel) ML Maximum Likelihood MMSE Minimum Mean Square Error MPSK M-ary Phase Shift Keying MSE Mean Square Error MSK Minimum Shit Keying NC Network Coding NMM Non-linear Memoryless Modulator OSI Open Systems Interconnection (model) P2P Point to Point (channel) pdf probability density function pf probability function pmf probability mass function Pr Probability PS Phase Shift RV Random Variable (Value) RW Random Walk (phase shift model) SD-IC Successive Decoding-Interference Cancellation SINR Signal to Interference plus Noise Ratio SISO Soft In Soft Out SNR Signal to Noise Ratio SoDeM Soft Output Demodulator SPA Sum Product Algorithm x
- 12. NOMENCLATURE TWRC Two-Way Relay Channel VA Viterbi Algorithm VN Variable Node WN Wireless Network WNC Wireless Network Coding XDF Whatever Decode and Forward xi
- 13. NOMENCLATURE xii
- 14. 1 Introduction 1.1 Motivation The wireless network coding attained a large popularity recently, because it stands for one of the possible flows improving the current state of art of the digital communications. The gaps between the fundamental limits and the current state of art implementations shrank to negligible values for almost all utility metrics in conventional point to point channels. Nevertheless, if we consider a wireless network as a collection of independent point to point channels that are directed by higher layers (conventional OSI model) the situation is no longer optimal from the global point of view. The situation in wireless networks is therefore fundamentally different. Considering the whole wireless network as one unit to be optimized opens a whitespace for new fun- damental limits related to the wireless network communication as well as for novel procedures improving the network performance. The main part of the research community in communication aims to fill this whitespace, which can be proven by majority of such topics in the communication journals and conferences nowadays. A lot of work has been done, especially in simple network models such as two-way relay channel containing two nodes that want to exchange the data using an auxiliary relay. A performance very close (half bit below) to the first order cut-set bound was proven in [38] for the two way relay channel upon AWGN channels. It means that the relay receives a superimposed signal from two sources simultaneously and the performance of the multiple access channel is limited almost only by the Gaussian noise, the interference limitation is therefore almost avoided. The proof of the achievability is, however, based on the lattice code approach that does not allow a straightforward implementation. There is therefore need for a directly implementable channel coding strategy allowing to approach the performance close to this limit. Many papers suggest to adopt the conventional channel coding strategies to properly respect the wireless network coding, but only limited results are available so far. Particularly a natural benchmark, i.e. the joint decoding followed by the pure network coding, can be obviously implemented for arbitrary channel code rates in sources, nevertheless the advantage of the wireless network coding is not fully exploited. On the other hand, for a special case, one can compose a compound decoder processing jointly the channel decoding and the network operation. This approach is however limited for a particular class of scenarios, where the equal source rates are assumed. A solution utilizing the advantage of the wireless network coding and allowing arbitrary channel code rates is an obvious target. Suppose now that we have already described the receiver in a mathematical way. The point is how to properly implement the processing in the receiver. This processing should be efficient to avoid waisting the resources and also it should be as exact as possible. One can feel that this is another crucial step for the system implementation. As an example from the history, we can mention the conventional capacity achieving channel codes that were known (LDPC codes developed by Gallager) for a long time, but their implementation was trackable since the iterative processing was employed. Nevertheless, this took over twenty years. The sum product algorithm on factor graph [27, 33] was shown as a tool that successfully solves a wide family of problems. The generic nature of the sum product algorithm allows its use also to describe the receiver processing in a wireless network properly taking into account the global design. The sum product algorithm is, however, exactly described in systems leading to a cycle-free factor graph. The capacity achieving channel codes that are supposed to be utilized in the receiver typically lead to the factor graphs containing loops. One should take attention about the system convergence in such a case, which is rather analytical than the implementation problem. Another problem is that the sum product algorithm is described as an iterative algorithm that is directed by some update rules. These update rules provide basically marginalization of a soft information (message) describing a particular local variable in the system. The problem arising from the implementation of the sum product algorithm is therefore given 1
- 15. 1.2. GOALS AND SUMMARY OF MAIN RESULTS by a description of a general continuously valued message for the processing purposes and the numerical intensity of the continuous message marginalization (integration) required by the update rules. 1.2 Goals and Summary of Main Results The goal of this thesis is twofold. The first goal aims to the overall channel codes construction within the network that utilizes the wireless network coding. The second goal is dedicated to the implementation issue of generic sum product algorithm that can be used for the receiver processing design. The generic nature of the sum product algorithm enables to incorporate a various number of flavours resulting from the overall network design. The answer to the first goal is contained in the block structured layered design describing the network design for an arbitrary channel code rates and simultaneously profiting from the wireless network coding advantage. This contribution is situated in a self-contained form in Chapter 7 as well as in [46]. The block structured layered design is build upon the system linearity assumption, that is linear channel codes and hierarchical functions. The channel codes can be then described by matrices (LDPC codes) that are properly composed within a global matrix in each node. Such a (block) construction indeed allows to incorporate arbitrary source rates as well as an arbitrary information flow division between the hierarchical streams (paths of the mixed information) in the network. To best of our knowledge, no such a design is available in the current state of art literature. The second goal is an implementation framework for the generic sum product algorithm applicable among others also in the wireless networks. This framework should efficiently and methodically solve the message representation as well as an efficient implementation of the update rules upon the given message parameterization. We therefore formally describe the problem of the message representation in Chapter 4. The solution is then situated into the sequel Chapter 5, where we first serve a generic KLT message representation that is based on the stochastic description of the message. The message representation (approximation) obtaining is both methodical and efficient, where the optimized criterion is the MSE of the approximated message (Sec. 5.2.1 or [44]). The update rules upon the message representation for a special type of linear canonical message representation with orthogonal kernels are then proposed in Section 5.2.3 or [47]. An example demonstrating the power of the proposed implementation framework can be found in Section 5.3. 1.3 Structure of the Thesis The thesis is divided into three main parts, where the core of the work is situated into the second and third part. The main contribution is contained in Chapters 5 (also in [44,47]) and 7 (also in [46] and partly in [58]). These Chapters are written such that they are supposed to be self-contained. It means that the rest of the thesis is some kind of supplementary material for these two main Chapters. The first part situated immediately after introduction contains a collection of prerequisites that will be further used in the sequel parts. A common characteristic of the first part is that the prerequisites are supposed to be presented just for the completeness. The first part is therefore not written in the tutorial form at all. Instead, some basic ideas and definitions are discussed with reference to the current state of art. We often point out the relation of a particular topic with the contribution that is situated into the sequel parts. The second part contains topic related to the implementation of the sum product algorithm. We formally state the problem of the implementation framework and then we introduce the proposed imple- mentation framework containing the KLT message representation and the generic update rules design. The third part is dedicated to the channel code design across the wireless network properly respecting the global design requests (wireless aware network coding). This is presented after a short introduction to 2
- 16. CHAPTER 1. INTRODUCTION the wireless network coding. The contents of the individual chapters can be summarized as: • Part I contains the prerequisites for the sequel part. – Chapter 2 provides a collection of some basic terms that form some fundamental limits (chan- nel capacity) and starting points for the receiver algorithm derivation (maximum a posteriory estimation criterion). – Chapter 3 briefly lists conventional channel coding and decoding techniques that approach the channel capacity. Namely the turbo and LDPC codes. The decoding algorithms are also dis- cussed including the tools dedicated for the investigation of the decoder convergence properties (EXIT charts). • Part II presents a solution to the first goal, that is the generic implementation framework. – Chapter 4 is supposed to formally define the sum product algorithm on factor graph as well as what does it exactly mean the sum product algorithm implementation framework. Although the implementation framework can be explained by two sentences, we found a formalism underlaying the message representation (approximation) and update rules upon the message representation forming the implementation framework together. Some examples of implementation frameworks are considered for an illustration. For the sake of completeness, we then also discuss the issues of the sum product algorithm resulting from its use on the looped factor graphs. – Chapter 5 presents one of the main contribution of this work, that is the implementation framework based on the KLT message representation. This message representation is intro- duced as a message representation solution methodically minimizing the MSE of the message approximation for a given message dimensionality. The generic KLT message representation is supplemented by the generic update rules design that is applicable for a wide class of mes- sages (among others the KLT message representation). The proposed framework is numerically verified and compared with a conventional solution. • Part III covers the topic of the error protection in the wireless network operating with the wireless network coding. – Chapter 6 offers a basic overview of the wireless network coding and its basic classification. Some basic assumptions and constraints are discussed. The fundamental principles charac- terizing the wireless network coding are demonstrated when the basic network topologies are presented. – Chapter 7 serves the second main contribution of this work. This chapter basically corresponds to [46]. We describe both hierarchical operation and channel codes by a linear operation. The block structured layered design is then introduced upon this description. The block structured layered design enables to design jointly the hierarchical functions and the channel codes such that arbitrary desired rates can be incorporated. The principle is finally verified in a concrete wireless network example. 1.4 Thesis Organization The aim of this Section is to guide through some common rules that are used within this thesis (e.g. some global notation). We try to avoid overloading of all symbols (at least locally). 3
- 17. 1.4. THESIS ORGANIZATION Vector Notation and Operations A vector is denoted by a bold-face lower-case letter and it is implicitly assumed to be in a column form. It means that a vector [b1, . . . , bk]T is denoted b. A zero vector with a single non-zero unit value on the ith position is denoted by ei, for example e3 is a vector given by [0, 0, 1, 0, . . . ]T . If the order of a particular symbol in the vector is not important or clear from the context, the subscript is dropped, i.e. b instead of bi. The vector can be also written in the form bA, where A is an ordered set or vector of indexes, e.g. A = {1, 2, 5} gives bA = b1,2,5 = [b1, b2, b5]T . We further allow to remove a particular dimension by the tilde operation1 ∼ . As an example, we consider a vector b = [b1, b2, b3]T . The vector b ∼ b2 is therefore given by b ∼ b2 = [b1, b3]T . Matrix Notation and Operations A matrix is conventionally denoted by a bold upper-case letter. For example Xk×n = [x1 . . . xn] is k × n matrix composed from n vectors x1, . . . , xn, where each of them is k symbols long. If the matrix is square, its dimension is denoted by a single index, that is Xk×k = Xk. The matrix dimension in the subscript is omitted whenever the matrix dimension is clear from the context. A zero matrix 0 is a matrix containing all zero elements and a matrix E denotes the unitary matrix that can be written as E = [e1, e2, . . . ]. A matrix Xk×n is said full-rank if rank(X) = min(k, n). We denote R to be a set of real numbers. An n-dimensional set An is denoted by A∗ , whenever the number of dimension is not important for that time. We further implicitly assume a symbol-wise operations upon given sets excepting conventional vector and matrix multiplication. Random Variables and Density Functions The random variables and random processes [42] are widely used within this thesis. We assume a random variable X described by its probability function (either pdf or pmf) pX(x) = ∂Pr(X < x)/∂x. This is denoted just by p(x) for simplicity if it is clear which probability function is meant. The conditional probability px|y(x, y) = ∂Pr(X < x ∩ Y < y)/∂xPr(Y < y) will be simply denoted by p(x|y). We denote a random realization of the random process in subscript, i.e. µx(t) as a random process of a variable t. We will consider basic moment operations upon the random variables and process. Particularly mean value given by Ex[·] = ·p(x)dx for a real domain of x. Note that the mean value is sometimes denoted by an over-bar symbol or by m letter, that is Ex[·] = m· = ¯·. Similarly the variance is given by the second moment, i.e. var[x] = Ex[(x − Ex[x])2 ]. The variance is sometimes denoted by σ letter, i.e. var[x] = σx. The most notable distribution used within this thesis is the Gaussian distribution given by ρσ(y − x) = 1 σ2π exp −|y − x|2 σ2 , (1.1) for complex random variable and ρσ(y − x) = 1 √ 2σ2π exp −|y − x|2 2σ2 , (1.2) for real random variable. 1The tilde operation is also used for the proportionality relation. The concrete meaning of a given tilde operation can be recognized from the context within this thesis. 4
- 19. 2 Theoretical Background This chapter aims with the theories essentially affecting the decoding procedure in wireless communication. We will consider the information and estimation theory. Although some results from other fields can be also applied, we focus just to these. This Chapter is dedicated as a reference and therefore it is not written in the tutorial form. We point out some important definitions and theorems that will be used in later part of the work. We refer excellent tutorials [11] for the information theory and [24] for the estimation theory. 2.1 Information Theory The information theory answers some principal questions from the communication design, mainly regarding the fundamental limits that can be used as a benchmark for a real application, i.e. the gap between the fundamental limit and the real application refers to the space for an improvement for a given realization. The fundamental limits are ordinarily connected with some utility metric. These limits state the best possible performance of the system from a given metric point of view for given constraints. The main utility metric that is considered within this work is the channel capacity in a power constraint wireless channel, because it answers an essential natural question, how many information can be reliably transmitted through a noisy channel for a given transmit power. We also touch the latency issue, because the capacity optimization leads to processing of long sequences unavoidably causing the latency. 2.1.1 Capacity of Scalar Point to Point AWGN Channel The channel capacity is given by the stochastic description of the channel. The most important channel for our consideration is the Additive White Gaussian Channel (AWGN) described by the Gaussian conditional pdf, that is p(y|x) = 1 √ 2πσ2 exp −|y − x|2 2σ2 = ρσ(y − x), x, y ∈ R. (2.1) The power constraint channel capacity for such a channel is given by CAW GN = 1/2 log(1 + γ), (2.2) where γ = P/N is the signal to noise ratio and the unit is bit per transmission in case of binary base of the logarithm, which is implicitly assumed. The capacity of complex channel, i.e. x, y ∈ R becomes to CAW GN = log(1 + γ). The channel capacity for AWGN is shown to be achievable in [11, Theorem 9.1.1]. The proof of the capacity theorem also reveals how the codeword should look like to attain the capacity. Particularly, the distribution p(x) maximizing the mutual information I(X, Y ) is the Gaussian one. The channel code approaching the channel capacity moreover relies on the asymptotic arguments and therefore the channel codeword is expected to be infinitely long to achieve the channel capacity. Practically, the codeword cannot be infinitely long, so one must choose some trade-off between the codeword sequence length and the gap from the channel capacity. The processing of a long sequence brings two issues: (1) a time required for the long sequence processing unavoidably brings latency and (2) one need to design a channel code which is implementable with a reasonable complexity (e.g. linear with respect to the codeword length) and that approaches the channel capacity. The random coding used in [11, Theorem 9.1.1] cannot be directly implemented due to its exponential complexity with respect to the codeword length. We dedicate Chapter 3 as an overview of the conventional coding strategies including their implementation in P2P channels. 6
- 20. CHAPTER 2. THEORETICAL BACKGROUND 2.1.2 Capacity of Multiple Access AWGN Channel If more nodes transmit simultaneously to one receiver, the situation is referred as a Multiple Access Channel (MAC). The receiver hears a superposition of the signals from more nodes corrupted by a noise. This can be described by a conditional pdf p(y|x1, . . . , xn). We assume that no other1 information about the particular streams is available in the receiver (the information is passed only through the channel) for this moment. We also assume that the receiver wishes to fully decode all of the received signals. The channel capacity (2.2) does not refer only to a single rate, but now it is given by the rates of all transmitting nodes. We therefore speak about the capacity region, which is given by a union of all achievable rates for a given MAC channel [11]. In special case of two sources (operating at rates R1 and R2), the capacity region (complex channels assumed) is given by R1 ≤ log(1 + P1/N) and R2 ≤ log(1 + P2/N), where P1 and P2 stand for the power of individual signals. The achievable single rate cannot be larger than in case, where the second signal is not present . This is referred as the first order cut-set bound of the classical MAC channel. The second order cut-set bound is given by R1 + R2 ≤ log(1 + (P1 + P2)/N) and if refers to the joint decoding, i.e. full decoding of data from both sources. Note 2.1 (Moving Beyond the Second Order Cut Set Bound in More Complex Networks). We will discuss the wireless network coding in sequel part of this work. This strategy is able to operate beyond the second order cut-set bound thanks to avoiding of the full decoding in auxiliary relays. Instead a function of the pair is decoded and one relies that a complement of this function is delivered by an alternative path into the final destination. The final destination then solves a system of equations. This capacity region can be achieved e.g. by a Successive Decoding-Interference Cancellation (SD-IC) method, where the receiver decodes one of the signal threating the latter as noise and after decoding, this signal is re-encoded and subtracted from the superimposed signal. In case of error-less decoding of the first stream, the subtracted signal is equal to the ordinary P2P channel as one can find e.g. in [11]. 2.2 Estimation Theory Estimation and detection theory stands for one of the fundamental cornerstones for the data communica- tion. This theory provides some criteria of a communication receiver design optimality. A nice tutorial for the estimation theory can be found e.g. in [24]. In this section, we briefly point out the most important criteria (ML, MAP, MMSE estimation) for the receiver design within the wireless communication. 2.2.1 Problem Statement Assume a random parameter ξ = [. . . , ξi, . . . ] to be estimated (e.g. data vector, phase shift, etc.) that is observed through a (noisy) observation θ = [. . . , θi, . . . ]. A system description defines a joint pdf p(ξi, θ), ∀i. We can now consider several approaches of the desired vector ξ estimation. We will distinguish symbol estimation ˆξi and the whole vector estimation ˆξ. 1Breaking this assumption opens a way to the wireless network coding significantly favouring this work. The wireless network coding will be discussed in Chapters 6 and 7 7
- 21. 2.2. ESTIMATION THEORY 2.2.2 Maximum Likelihood Estimator The likelihood function (LF) is defined2 by Λ(ξ) = p(θ|ξ). (2.3) The maximization of (2.3), i.e. ˆξ = arg max ˇξ p(θ|ˇξ) (2.4) is called maximum likelihood (ML) estimator. From the monotonicity of the logarithm operation, the maximization of log-likelihood function (LLF) Λ (ξ) = ln Λ(ξ) in (2.4) instead of the LF leads to the same result. Note 2.2 (Optimality of ML Estimator). One can show that [24, Theorem 7.3] the estimation is asymptot- ically minimal variance unbiased (MVU), that is the estimated vector ˆξ asymptotically obeys a Gaussian distribution ρ(ξ, I−1 (ξ))), where the variance matrix is given by the inverse of the Fisher n×n information matrix given by −E ∂2 ln Λ(ξ) ∂ξi∂ξj , ∀i, j. (2.5) The ML estimator therefore asymptotically achieves the Crammer-Rao bound [24]. 2.2.3 Bayesian Estimators An another branch of estimators is based on a Bayesian approach. According to nature of the estimated parameter ξ, a proper type of the estimator is chosen. The general Bayesian estimator is defined by a cost function C( ), where = ξi − ˆξi denotes an error in the estimation. For our purposes, we consider • hit or miss cost function defined by C( ) = 0 | | ≤ δ 1 | | > δ , (2.6) where δ defines a threshold, • vector hit or miss function defined by C( ) = 0 ≤ δ 1 > δ , (2.7) where = ξ − ˆξ and 2 = i 2 i and • mean square error cost function defined by C( ) = 2 . (2.8) A minimization of the Bayes risk given by R = E[C( )] = C(ξi − ˆξi)p(ξi, θ)dξidθ for a given observation θ provides [24]: 2One can consider a symbol-LF Λ(ξi) = p(θ|ξi). 8
- 22. CHAPTER 2. THEORETICAL BACKGROUND • Minimal mean square error (MMSE) estimation for the cost function (2.8) is given by ˆξi = ξip(ξi|θ)dξi = E[ξi|θ], (2.9) where the vector θ may contain additionally the whole vector ξ excepting ξi. Note that the vector form of the MMSE estimator, that is ˆξ = E[ξ|θ] minimizes the MSE of each component ˆξi. The MMSE criterion is commonly used e.g. in the phase estimation. • Maximal a posterior probability (MAP) estimation for the cost function (2.6) is given by ˆξi = arg max ξi p(ξi|θ). (2.10) The MAP criterion is suitable to use for the data detection, because it is equivalent to the minimal average bit error rate (BER) criterion standing for a natural goal of the reliable communication. A considerable part of this work aims with the evaluation of p(ξi, θ) serving for the MAP estimation (2.10). • Vector maximal a posterior probability estimation for the cost function (2.7) is given by ˆξ = arg max ξ p(ξ|θ). (2.11) Note that the vector MAP estimation does not generally give the same result as the classical MAP estimator (2.10). If a priory distribution of ξi is uniform (as data in the digital communication are conventionally as- sumed), the MAP estimator is equal to the ML estimator. As a consequence, the likelihood metric is a sufficient statistics for the minimal average BER criterion for a given observation. 9
- 23. 3 Error Protection Techniques This Chapter shortly reviews the channel coding techniques with accent to the decoding processing and the investigation of the decoding behaviour. Since the core of this work is basically the decoding processing, the basic coding theory forms one of the prerequisites for the sequel part serving the main contribution of this work. The main purpose of the channel code is a protection of the data for a reliable transmission through an unreliable channel (wireless medium). The fundamental limits are given by means of the information theory, mainly the channel capacity. This channel code design (Sec. 3.1) aims to approach the channel capacity. The receiver then needs to properly incorporate the received signal taking into account the con- sidered channel encoder in the source. A direct application of the optimal solution leads to the intrackable implementation. We therefore focus on some advanced techniques (iterative detection) incorporating a slightly suboptimal, but trackable, approach to the code design and the corresponding receiver processing in Sec. 3.2. The suboptimality (convergence) investigation techniques are the topic of Sec. 3.3. Throughout this Chapter, we will consider a memoryless only channel, that is the channel described by a conditional pdf p(x|c) = i p(xi|ci). All memory is therefore situated in the channel code. 3.1 Encoding Techniques We briefly list several encoding techniques attempting to approach the channel capacity (e.g. [49]). We will consider exclusively linear codes. As it was shown in the lattice code case [17], the linearity of the code still enables to achieve the channel capacity. 3.1.1 FSM/Block Codes The block and FSM based codes are the oldest ways of the channel coding techniques. The block code is described by a matrix G such that the codeword is given by c = Gd, where d is a data vector and c is a codeword vector. The rate of this code is given by ratio of dimension of the matrix G in case of full-rank generator matrix. The trellis (FSM) based codes [51] are described by a FSM. We will use the conventional octal notation to identify the code. The analysis of the FSM-based codes from the performance point of view can be found e.g. in [51]. Since the instant (ith) codeword symbol is given by the data vector up to the ith symbol, the trellis codes are causal in contrast to the block codes, where one needs a whole data vector for obtaining the instant codeword symbol in general1 . The block code can be also described by a FSM, but both output and state functions are generally time variant in the corresponding FSM. 3.1.2 Turbo Codes In contrast with the FSM/block codes described above having a significant gap from the channel capacity, the turbo codes [6,61] attain the performance very close to the capacity. The main idea is in concatenation of FSM encoders separated by an interleaver to reduce the probability of terminating sequences for the recursive FSM code and therefore to almost meet the assumption of the infinite codeword length. The concatenation can be considered either parallel or serial. We will again not investigate the performance of the turbo coding here, since it is not directly related with the core of this work. Instead we refer some literature containing the details on the turbo codes including the performance analysis, e.g. [51, 61]. We note here, that the "turbo" principle is quite more 1Using a systematic form of the code, the whole data vector is needed only for the parity part of the codeword. 10
- 24. CHAPTER 3. ERROR PROTECTION TECHNIQUES general. The "turbo principle" basically means two correlated entities (e.g. FSM), that are able to some- how exchange a randomized (interleaved) soft information such that the information input improve the information output for each entity. One can find some other systems, where the turbo principle is applied (e.g. MIMO equalization [60] or decoding of correlative sources [3]). The decoding of the turbo code is based on an iterative algorithm that will be described later. Note that the use of the interleaver corrupts the causality of the overall encoding process. 3.1.3 LDPC Codes The LDPC codes [14] are also popular easy to implement encoding technique that attains a close-capacity performance. The LDPC code is basically a block code described by a large dense parity check matrix H such that HG = 0. Due to the matrix description, the LDPC code is linear and it will form the basic building block in Chapter 7. The LDPC decoding is a nice example of the FG-SPA implementation that will be discussed in Example 4.13. Although the FG-SPA can be directly applied for the LDPC decoding, it is much more efficient to employ an implementation framework, where the message is represented by soft bit and the corresponding update rules are shown e.g. in [23]. Note 3.1 (Enconding Procedure). The encoding of the LDPC codes is not straightforward, because they are defined using the parity check matrix. One can obtain the corresponding generator matrix directly using the Gaussian elimination from the parity check matrix. The problem is in the numerical intensity for large matrices. We refer [49, Appendix A] for an efficient way of the LDPC encoding. Note 3.2 (Parity Check Matrix Obtaining). Design of good parity check matrix is a nutshell, because the matrix should be very large to respect the very long codeword assumption for the capacity achieving code. Particularly the properties of the LDPC code are given by degree distribution as well as by the girth, that is the length of the shortest loop in the corresponding graph [49]. The size of the matrix makes numerically intensive to provide some additional loops removing algorithms from a given code as suggested e.g. in [34]. 3.1.4 Other Encoding Techniques One can find much more channel codes with various utility targets (channel capacity, latency). This is however not directly related with this thesis. We therefore point out mainly the lattice codes [17] from the other encoding techniques. This family of codes is very nice tool used mainly in the information theory. The problem of the lattice codes is that their direct implementation is not straightforward and therefore they serve rather as a tool proving some performance than a manual for a direct implementation. 3.2 Receiver Processing The receiver is assumed to obtain a noisy observation of the encoded vector. It aims to recover the original data [25,43]. First a (symbol-wise) metric is evaluated, since we assume a memoryless channel. This metric is served to the decoder. We will shortly discuss some fundamental algorithms capable to provide the MAP estimation (or some approximation) for the aforementioned encoding procedures [9]. 3.2.1 FSM Decoding - Viterbi Algorithm The Viterbi Algorithm (VA) [9] is a well known algorithm providing the vector-MAP estimation of the FSM-coded data vector in with a linear complexity. The algorithm optimizes a metric across an underlaying FSM trellis in the forward part keeping sur- viving paths active, that is a hard decision about a path in the trellis according to the metric increments 11
- 25. 3.2. RECEIVER PROCESSING is performed. In the backward part, one resulting path is selected according to the overall accumulated optimized metric. We start with the vector MAP description to derive a metric for the VA. The causality of the FSM coding and ability to be described by the trellis [18] gives p(d|x) ∼ p(x|d)p(d) (3.1) = i p(xi|x0,...,i−1, d)p(di) (3.2) = i p(xi|x0,...,i−1, d0,...,i)p(di) (3.3) = i p(xi|x0,...,i−1, ςi−1, di)p(di), (3.4) where the causality was used in (3.3) and the FSM description in (3.4). Using the memoryless property of the channel, we finally obtain p(d|x) ∼ i p(xi|ςi−1, di)p(di) (3.5) The trellis transition is unambiguously given by a pair Ti = (ςi−1, di) and the probability of the transition with respect to a given received vector x refers to (3.5), namely γ(Ti) = p(xi|ςi−1, di)p(di) (3.6) for memoryless channel. The term γ(Ti) can be viewed as an increment of the metric for a corresponding transition Ti since p(d|x) ∼ i γ(Ti). Thanks to the monotonicity of the logarithm operation, the equation (3.4) can be also written as log(p(d|x)) ∼ i log(γ(Ti)) (3.7) and the corresponding vector MAP estimation can be written in logarithm form as ˆd = arg maxd log(p(d|x)). The VA now presents an efficient way of this maximization. The VA relies on a fact that each possible data vector corresponds unambiguously with a path in the whole trellis. The main simplification is that one is interested only is the path with the maximal probability leading to a given state in the ith step. All of the other pats can be trashed, because they surely does not refer to the part of data vector maximizing the vector MAP estimation. We define a metric ρi ∈ R as an accumulated metric in the ith state and ∆ρ as the metric incre- ment referred to a transition Ti, e.g. ∆ρ(Ti) → γ(Ti) and ρi = i j=1 γ(Tj). The accumulated metric ρi corresponds to a certain path 2 in the trellis, i.e. {Tj}i j=0. The Viterbi Algorithm is then given by: 1. initialization: set the worst possible metric to all states excepting the initial one ς0 2. the forward part of the VA also know as add-compare select unit: In each possible state ςi provide the following: (a) assuming that the metric is already evaluated in the i − 1th step, evaluate the metric in the next state by ρi = ∆ρ(Ti)(add)ρi−1 for all possible transitions leading to a given state Ti : S(di, ςi−1) = ςi, where the add operation stands for plus in the log-domain and product in the probability domain. 2The certain means is the path optimizing ρi among all possible paths in VA. 12
- 26. CHAPTER 3. ERROR PROTECTION TECHNIQUES (b) compare all evaluated metrics evaluated in the previous step, (c) select the transition with the best ρi as the surviving path, save that and discard all other transitions. 3. backward part: select the final state with optimal metric and by the surviving paths select the overall path in the trellis. The final selected path refers to the vector MAP estimation of the data vector. 3.2.2 FSM decoding - FBA Algorithm The Forward Backward Algorithm (FBA) is suited for the turbo codes decoding. The FBA aims with the symbol-MAP estimation for a given FSM. The algorithm consists from a recursive forward and backward part. Using a similar argumentation as in the VA derivation, we can derive [18] the symbol MAP estimation with the FSM description: p(di|x) = ςi−1 p(ςi−1, di|x) (3.8) ∼ ςi−1 p(x|ςi−1, di)p(ςi−1, di) (3.9) = ςi−1 p(x0,...,i−1|ςi−1, di)p(xi|ςi−1, di, x0,...,i−1) p(xi+1,...,N−1|ςi−1, di, x0,...,i)p(ςi−1, di) (3.10) = ςi−1 p(x0,...,i−1|ςi−1)p(ςi−1) α(ςi−1) p(xi|ςi−1, di, x0,...,i)p(di) γ(Ti) p(xi+1,...,N−1|ςi) β(ςi) , (3.11) where (3.10) follows from the chain rule and (3.11) is given by causality of the FSM. Since ςi = S(ςi−1, d), we can rewrite (3.11) to p(di|x) ∼ ςi−1 α(ςi−1)γ(T(di, ςi−1))β(S(di, ςi−1)), (3.12) where T(di, ςi−1) = Ti. One can derive (e.g. [18]) a recursive formula α(ςi−1) = Ti−1:ςi−1 α(ςi−2)γ(Ti−1) (3.13) β(ςi−1) = Ti:ςi β(ςi)γ(Ti), (3.14) for α and β metrics evaluation serving jointly with (3.12) for an efficient evaluation of the symbol-wise MAP estimation which is called the Forward Backward Algorithm. Note 3.3 (Turbo Decoding). The main purpose of the FBA lies in the turbo decoding. The FBA is applied to the SISO blocks referring to the concatenated FSM based encoders. The algorithm is iterative and the convergence can be described by EXIT-charts. The decoder consists of two either parallel or serial concatenation of Soft In Soft Out (SISO) blocks connected by (de)interleaver. The FBA is iteratively performed in the SISO blocks, where the output is evaluated according to an input from the complementary SISO block and the output serves as the input for the second SISO block and conversely. 13
- 27. 3.3. CONVERGENCE CRITERIA 3.2.3 Graph Based Solution A common point of the aforementioned decoding algorithms is that the system can be described using a graph, that is • FSM trellis in the VA case, • compound FSM trellis with interleaver in the FBA-based turbo decoding. One can consider a generalization of these algorithms. The factor graph describing a structure of an arbitrary system can be also used for the aforementioned decoding techniques. Some algorithms can be then defined upon the factor graph. Since the algorithms operate upon the decomposed system, they are much more efficient than the brute-force solution optimizing the global goal at the same time. One of such algorithm (sum product algorithm) will be described later in greater detail. The LDPC codes can be decoded using this algorithm as it will be shown later. 3.3 Convergence Criteria The turbo decoding as well as the graph based decoding have an iterative nature. The essential question is a convergence of such algorithms. The solution is achieved in a suboptimal way, that is the prize for the system trackability. The main tool that we will discuss are the EXIT charts [19,56] enabling to describe the convergence properties by means of the mutual information transfer investigation between some entities. Some other convergence criteria are discussed in Sec. 4.3.2. 3.3.1 Log-likelihood Arithmetics The log-likelihood algebra [21] is used with advantage for a description of a random variable. This de- scription is best suited in the binary case, that is random variable with two possible states. This refers to binary data in the digital communications. This algebra simplifies the description of the random variable. Assume a random variable S with pmf given by P(S = −1) = p−1 and P(S = 1) = p1. The domain of S is a binary GF with zero element −1. The Log-Likelihood Ratio (LLR) is defined as λ = log(p1/p−1). One can see that the binary random variable is stochastically described just by one number. Using the natural identity p−1 + p1 = 1, we can express the pmf as p−1 = 1 exp(λ) + 1 = exp(−λ/2) exp(λ/2) + exp(−λ/2) p1 = exp(λ) exp(λ) + 1 = exp(λ/2) exp(λ/2) + exp(−λ/2) . (3.15) The a posteriori probability after receiving an observation x is p(s|x) ∼ p(x|s)p(s), which can be written using LLR as λs|x = log p(s = 1|x) p(s = −1|x) = log p(x|s = 1) p(x|s = −1 xλC + log p(s = 1) p(s = −1 λs . (3.16) If we consider Gaussian channel p(x|s), one can show that λC = 4γ. We further note that λs|x1,x2 = λC1 x1 + λC2 x2 + λs. (3.17) for conditionally independent x1 and x2. 14
- 28. CHAPTER 3. ERROR PROTECTION TECHNIQUES The GF(2) addition denoted by ⊕ is a cornerstone operation for the coding. We now assume a random variable given by x = x1 ⊕ x2, where x1 and x2 are described by their LLR λx1 and λx2 . It can be shown that λx = 2 tanh−1 (tanh(λx1 /2) tanh(λx2 /2)). (3.18) 3.3.2 Binary EXIT Charts The EXIT charts developed in [56] bring a semi-analytical tool well describing the convergence of the paral- lel turbo codes. The principle consists in measuring of the extrinsic information exchange of concatenated soft in soft out blocks (SISO). Nevertheless this principle is more general and it can be extended also for LDPC codes, where the SISO blocks are given by check nodes and variable nodes. We assume a SISO block C. A mutual information between systematic data d and a prior input of the SISO block A is called an intrinsic information I(d, A) = IA. The mutual information between the data and the SISO output I(d, E) = IE is called an extrinsic information. A function T given by IE = T(IA) is called extrinsic transfer function. If the prior information is in the Gaussian form, the mutual information is given only by variance of A denoted σA. In case of a concatenation of more SISO blocks, where the extrinsic information of the first one becomes to the intrinsic information of an another block and conversely, the convergence of the system as whole can be described using the extrinsic transfer functions of individual blocks. We demonstrate the principle on the parallel turbo code example [56]. We assume that the received signal is given by x = s + w, where w is an AWGN and s ∈ {−1, 1} is a systematic bit. The likelihood is then given by Λ(s) = p(x|s) = 1 2πσ2 w exp − (x − s)2 2σ2 w . (3.19) We use the log-likelihood form X = log p(x|s = 1) p(x|s = −1) = (x + 1)2 − (x − 1)2 2σ2 w = 2x σ2 w = 2 σ2 w (s + w), (3.20) which can be rewritten in the form X = µXs + nX with mean µX = 2 σ2 w (3.21) and a zero-mean Gaussian random variable nX with variance σX = 4 σ2 w . (3.22) The variance and mean of X are connected through σ2 w as µX = σ2 X 2 . (3.23) The log-likelihood form of A in case of SISO block in AWGN channel was shown to have the Gaussian distribution. Therefore it can be written in the form A = µAs + nA, where s is the systematic bit. The mean is given by µA = σ2 A 2 . (3.24) 15
- 29. 3.3. CONVERGENCE CRITERIA Using this mean, we can write the likelihood (3.19) as pA(a|s) = 1 2πσ2 A exp − (a − µAs)2 2σ2 A (3.25) = 1 2πσ2 A exp − (a − σ2 A 2 s)2 2σ2 A . (3.26) A joint pdf p(a, s) used to evaluate the intrinsic mutual information is given by pA(a, s) = pA(a|s = −1) + pA(a|s = 1) (3.27) and the intrinsic information is therefore given by IA = s pA(a, s) pA(a)p(s) pA(a, s) da (3.28) = 1 2 s=±1 ∞ −∞ pA(a|S = s) log 2pA(a|S = s) pA(a|s = 1) + pA(a|s = −1) da. (3.29) The extrinsic information can be evaluated in the same way, where the extrinsic distribution is used, that is IE = s pE(a, s) pE(a)p(s) pE(a, s) da (3.30) = 1 2 s=±1 ∞ −∞ pE(a|S = s) log 2pE(a|S = s) pE(a|s = 1) + pE(a|s = −1) da. (3.31) Note 3.4 (Investigation of turbo decoding). The EXIT function is evaluated for the SISO blocks given by the FSMs. During the evaluation of the EXIT function, one must pay attention that the probability serving for the evaluation (3.29) and (3.31) is in a proper form. Note 3.5 (Investigation of LDPC decoding). The first SISO block is formed by variable nodes with connection to observation and the second SISO block is given by the check node. One can find the EXIT chart application on LDPC codes e.g. [57]. 16
- 30. Part II FG-SPA Processing and its Implementation 17
- 31. 4 Factor Graphs and Related Algorithms The aim of this Chapter is to introduce the sum product algorithm on factor graphs and to formally describe the issues that should be taken into account during the implementation of this algorithm. We formally provide motivation for the sequel Chapter 5, where a concrete implementation framework methodically solving the issues is introduced as one of the main contribution of this work. Finally, we briefly touch the extension to looped FG-SPA and corresponding problems within this Chapter. The factor graph (FG) is a tool graphically describing an arbitrary function (system) called the global function using its inner structure. The sum-product algorithm on factor graph (FG-SPA) is a highly- efficient algorithm capable to solve a wide family of problems. The most important one for this work is the MAP-detection. The FG-SPA evolved as a generalization of trellis algorithms, where a general graph is considered instead of a trellis [62]. The framework of the FG-SPA is introduced in [27, 33], where the authors also show a number of applications such as the FSM/decoding, Kalman filtering, etc. The problem of the inference (marginalization of the global function serving for e.g. the MAP-detection) is complex when it is applied on the global function. The point of the FG-SPA is that it marginalizes local functions in the decomposed system instead of the global function with typically significantly less complexity. A crucial assumption is a system decomposability. If some variables represented in the FG are random and independent, their joint probability (global function) can be composed as a product of some parts referring to the independent variables. The independence implies a strict limitation of the FG-SPA, that is the FG representing the global function must be cycle-free1 . It was shown that the FG-SPA often works well also in the systems leading to a looped-graph, where the FG-SPA was not originally designed for. We stress that the inference is only approximation in the FG-SPA operating on the looped graphs (called also the loopy belief propagation [36,66]). The algorithms very similar to the looped FG-SPA were re-discovered independently in more fields. Many practical problems across many fields lead to the looped-graphs. Most notably for this work, we again mention the MAP-detection of the capacity achieving codes such as turbo or LDPC codes. We stress that the FG-SPA (or some of its variant) is capable to cope among others all decoding algorithms discussed in Chapter 3. Furthermore, the global function need not to be restricted only to the decoder itself. We can consider e.g. a joint decoder and phase estimator as one global function. Therefore all decoding and signal processing can be covered by one global function. The advantage is that the individual tasks are processed jointly and therefore with a potentially better performance than solving the tasks independently. The major problems that are presented in the looped FG-SPA are: • convergence of the FG-SPA issue, • FG-SPA message representation. The convergence of the FG-SPA issue stands for a principle-nature issue. The cycled graph implies unknown fixed points existence and their uniqueness. The convergence of the FG-SPA to a proper fixed point is not also guaranteed in general. We refer [36] and references from there (e.g. [66]) for the reader interested in the investigation of the looped FG-SPA convergence topic. While the issue of the FG-SPA convergence is a principle-nature problem, the message representation stands for the implementation nature problem. The problem is, roughly said, how to represent a general continuously valued function by means of several parameters (the less the better), because the messages are passed through the corresponding factor graph and therefore their description is unavoidable for the implementation. A methodical solution for this problem is one of the main contribution of this work. We refer Section 5.1.2 or e.g. [10, 13, 28] and references from there as an overview of the current state of art for the FG-SPA message representation problem. 1According to the original definition presented in [27,33]. 18
- 32. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS 4.1 Cycle-Free Factor Graph and Sum Product Algorithm We will first consider a cycle-free FG-SPA and present a fundamental theorem 4.1 stating the possibility of an exact inference evaluation in the cycle-free FG-SPA. A discussion of the implementation issues will follow. 4.1.1 Formal Definition of the Factor Graph We start with some formal description of the system. A motivation of the following notation is to describe a collection of functions {fI}I forming some local parts of the global function. The factor graph for such a system is then defined. Let x ∈ Ax be a multi-dimensional variable with dimensions [x1, . . . , xN ] ∈ Ax1 × · · · × AxN = Ax, where a set2 given by the individual dimensions is denoted by X, i.e. X = {x1, . . . xN }. Further let F = {f1, . . . fM }, fI : Aψ(fI ) → CI, ∀I ∈ {1, . . . , M} be a set of functions, where Aψ(fI ) is a domain of fI and CI is its codomain. We defined a mapping ψ(fI) : F → XnI assigning a vector of arguments to a given function fI and nI denoting the number of arguments of fI. Similarly, we also define a mapping υ(i) giving a vector of functions, where xi as the argument. We consider two functions f1(x1, x3, x4) and f2(x2, x4). The domain of f2(x2, x4) = f2(x2,4) = f2(ψ(f2)) is Aψ(f2) = Ax2,x4 = Ax2 × Ax4 , the codomain is C2, the mapping ψ(f2) = x2,4 = [x2, x4] and n2 = 2. Similarly domain of f1(x1, x3, x4) = f1(x1,3,4) = f1(ψ(f1)) is Aψ(f1) = Ax1,x3,x4 = Ax1 × Ax3 × Ax4 , the codomain is C1, the mapping ψ(f1) = [x1, x3, x4] and n1 = 3. Finally υ(x1) = f1, υ(x2) = f2, υ(x3) = f1 and υ(x4) = [f1, f2]. Example 4.1 (Simple Demonstration of Notation). Using the introduced notation, we can formally define the factor graph as a visualization of a global function, where the global function (whole factor graph) is composed from local functions as in Example 4.1. The global function g(x) can be composed from two local functions f1(x1, x3, x4) and f2(x2, x4) as shown in Fig. 4.1. One can see that the factor graph respects inner structure of the global function (see Example 4.2), where the functions are visualized by factor (check) nodes and the variables by variable nodes. The arguments of the function are represented by edge between the corresponding factor and variable node. If C = I CI forms a commutative semi-ring, the factor graph is defined as a bipartite graph G = (N, V, E), where N = {F1, . . . FM } is a set of FN referring one to one to the set of functions F, V = {ν1, . . . , νN } is a set of VN referring one to one to the set X and E = {e1, . . . , eL} is a set of edges connecting FN and VN if and only if the variable represented by the VN is argument of the local function referring to the FN. It means that an edge connects a FN FI ∈ N referring to fI and a VN νi ∈ V if the variable xi represented by the VN νi is an argument of fI. This can be written using the mapping ψ as νi is connected with FI if and only if xi is in the vector given by ψ(fI). Definition 4.1 (Factor Graph). 2We mean an ordered set by default. 19
- 33. 4.1. CYCLE-FREE FG-SPA x1 F1 x3 x4 F2 x2 F1 → f1(x1, x3, x4) = f1(ψ(f1)) F2 → f2(x2, x4) = f2(ψ(f2)) Figure 4.1: Simple demonstration of a factor graph corresponding to Example 4.1. We will further not recognize between a VN or FN and the corresponding variable or function itself. It means that both a VN (νi) and the corresponding variable are denoted by xi and both a FN and the corresponding function (fI) are denoted by FI. We will also use the mappings υ and ψ also for the factor nodes and variable nodes, since the sets V and X are one to one as well as the sets F and N. It means that we will not recognize between ψ(fI) and ψ(FI). Both of them give a vector of either variables or variable nodes according to the context. We form a convention to visualize the FN by a rectangle and the VN by a circle. We will also use upper case letters for FN and lower case letters for VN. A global function g : Ax → C represents a joint pf of x. The global function can be composed3 from the local functions as g(x) = 1/Z M I=1 fI(ψ(fI)), where Z = X M I=1 fI(ψ(fI)) is basically a norming of the pf. Note that this norming constant is not a function of any x ∈ X. The constant is conventionally normed such as Z = 1, because g(x) is supposed to be the pf. We will further slightly abuse the notation such that we denote M I=1 simply by I. A pf of a particular set of variables A, where A ⊆ X can be evaluated by a marginalization of the global function, i.e. p(xA) = XA g(x) = XA I fI(ψ(fI)). (4.1) The following example shows a basic factor graph and also it demonstrates the idea that is beyond the implementation efficiency. We can also find there a basic motivation for the message passing that will be discussed later. 3Note that the operation stands for a composition operation and for the marginalization operation in general. Since this work aims with the sum product algorithm that will be formally defined in Sec. 4.1.3, we consider the summation and product as the marginalization and composition operations. 20
- 34. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS A factor graph corresponding to Example 4.1 is shown in Fig. 4.1. The global function can be de- composed to two local functions such that g(x) = f1(x1, x3, x4)f2(x2, x4) = f1(ψ(f1))f2(ψ(f2)). If we are interested in the marginalization of e.g. x2, we can see that x1,x3,x4 g(x) = x1,x3,x4 f1(x1, x3, x4)f2(x2, x4). (4.2) Nevertheless, this can be written as x1,x3,x4 g(x) = x4 x1,x3 f1(x1, x3, x4) µ(x4) f2(x2, x4) = x4 µ(x4)f2(x2, x4). (4.3) If we carefully check 4.3, we can see that we decomposed the marginalization into two steps. In the other words, it is not necessary to provide marginalizations over three parameters at one time, i.e. 3 encapsulated loops, but it is sufficient to first evaluate µ(x4) using marginalization over two variables and the last marginalization x4 µ(x4)f2(x2, x4) can be provided extra. It means that we save the third encapsulated loop in the evaluation, which is no longer encapsulated when (4.3) is used. One can feel that this simplification becomes to be more crucial in complex systems. In addition, if we express similarly the marginalization of x1 or x3, we obtain x1(or x3),x2,x4 g(x) = x1(or x3) f1(x1, x3, x4) x2,x4 f2(x2, x4) µ(x1,x3) (4.4) = x1(or x3) f1(x1, x3, x4)µ(x1, x3). (4.5) It is therefore sufficient to evaluate the marginalization µ(x1, x3) = x2,x4 f2(x2, x4) just once and to use it for the both x1 and x3 marginalization evaluations. Example 4.2 (Simple Factor Graph). 4.1.2 Message Passing We shown how to visualise a global function by means of the FG using its inner structure and also how to express a marginalized probability function (4.1). The motivation is now to efficiently evaluate the marginalized function. We partly reveal the principle in Example 4.2. We defined a marginalized sub-result µ(x4) that is then used to evaluate the marginal p(x2) = x1,3,4 g(x). The sub-result µ(x4) contains all information about x1 and x3 required for evaluation of p(x2) contained in a message. We now define this message more formally and form the fundamental Theorem 4.1 stating the way of the marginal evaluation using a message passing. 21
- 35. 4.1. CYCLE-FREE FG-SPA For each edge given by a FN XI and a VN xi, we assign two messages. The first one is from the VN to the FN (µxi→XI (xi)) and the second one with a converse direction (µXI →xi (xi)). The message is a function of the variable xi. A domain of the message is a set µ ∈ Aµ = M. Definition 4.2 (Message in FG-SPA). Note 4.3 (Notation comments to messages). The domain inherits the identification of the message, i.e. µxi→XI (xi) ∈ Mxi→XI . Each message can be parameterized by some further parameters which are situated into the subscript. We drop (simplify) the message parameters or its identification, whenever the this information is clear or not important. Motivated by the marginal evaluation using the messages, we define the update rules defining a relation of the messages. Note that the definition corresponds to a commonly used update rules [27,33]. The FN update rule of a given messagea µFI →xi is a mapping UF : Mψ(FI )∼xi→FI → MFI →xi , where µψ(FI )∼xi→FI ∈ Mψ(FI )∼xi→FI are the messages inputing the node FI excepting µxi→FI and similarly for the variable node, i.e. UV : Mυ(xi)∼FI →xi → Mxi→FI . The particular form of the update rules is given by µFI →xi k+1 (xi) = ψ(fI )∼xi fI(ψ(fI)) xj :ψ(fI )∼xi µ xj →FI k (xj) (4.6) µxi→FI k+1 (xi) = FJ :υ(xi)∼fI µFJ →xi k+1 (xi), (4.7) where (4.6) defines the update rule in FN and (4.7) defines the VN-update. Note that the sum operator becomes to integral whenever the domain Axi is a continuous interval. The subscript k denotes the iteration. aThe unary tilde operation means the whole vector expecting the argument. See Section 1.4 for details. Definition 4.3 (Update Rules). The marginal of a variable xi given by I∈υ(i) µXI →xi (xi) (4.8) and it represents a marginalized global function serving for further purposes. We state a fundamental theorem of the FG-SPA serving for the marginal evaluation using the message passing directed by the update rules. Let x be a multi-dimensional random variable described by a joint pf g(x). If the FG G referring to g is cycle-free, the message passing directed by the update rules defined by (4.6) and (4.7) provides messages enabling an exact inference evaluation (4.8) with an arbitrary scheduling algorithm containing all messages. Theorem 4.1 (Fundamental FG-SPA Theorem). 22
- 36. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS x1 F1 x3 x4 F2 x2 A2A3Oθ µx1→F1 (x1) µx3→F1 (x3) µF1→x4 (x4) µF2→x2 (x2) µx4→F2 (x4) µA2→x2 (x2) µA3→x3 (x3) µO→x1 (x1) Figure 4.2: Simple demonstration of FG-SPA corresponding to Example 4.5. The messages required for the marginalization of p(x2, θ) are denoted. The theorem is so fundamental that the proof can be found in many tutorials to FG-SPA e.g. in [27]. 4.1.3 Sum Product Algorithm Although the FG can be established for any commutative semi-ring codomain, we consider Ax to be a support of a multi-dimensional random variable x. Providing further specification, we can formally define the sum product algorithm. We consider a factor graph G. The sum-product algorithm on the factor graph is the message passing algorithm directed by the update rules (Def. 4.3), where • the local functions fI are conditional pf. • the marginal (4.8) is a marginalized pf p(xi), i.e. p(xi) = I∈υ(xi) µXI →xi (xi). We recognize forward messages having a prior form and backward messages in likelihood form. Definition 4.4 (Sum Product Algorithm). The local functions fI are some probability functions (pf - pdf, pmf or their mixture) up to a non-negative scale factor describing a multi-dimensional random variable ψ(fI). Consequently we state the codomain CI ∈ 0, ∞), ∀fI. Note 4.4 (Other Algorithms on FG). The composition and marginalization operations are more general. One can consider e.g. max-sum algorithm, where the marginalization is given by maximization and the composition by sum operation. The Viterbi Algorithm in the log-domain can be mentioned as an instance of the max-sum algorithm. 23
- 37. 4.1. CYCLE-FREE FG-SPA We return to Example 4.2. We specify that x1 is connected to an observation θ. Further, we assume an a prior information x2 and x3. This is expressed by adding further factor node to the graph as one can see in Fig. 4.2 The application of the update rules gives µF1→x4 (x4) = x1,x3 p(x1|x3, x4) f1(x1,x3,x4) p(x3) µx3→F1 p(θ|x1) µx1→F1 = x1,x3 p(θ, x1, x3|x4) = p(θ|x4). (4.9) Using the VN update, we can see that µF1→x4 (x4) = µx4→F2 (x4) and the second FN update is given by µF2→x2 (x2) = x4 p(x4|x2) f2(x2,x4) p(θ|x4) µx4→F2 (x4) = p(θ|x2). (4.10) Using a prior information about x2 (message µA→x2 (x2) = p(x2)), we evaluate the marginal as µF2→x2 (x2)µA→x2 (x2) = p(x2)p(θ|x2) = p(θ, x2). Note that we can recognize forward form of the messages (a prior form) and backward messages (likelihood). Example 4.5 (Message passing in a simple example). One can recognize the message µ(x4) evaluated in Example 4.2 in (4.9). 4.1.4 MAP Decoding and MMSE Estimation Using FG-SPA The MAP decoding is a fundamental capability of the FG-SPA for our purposes. The maximum a posterior probability of a parameter ξi given by an observation θ is given by (2.10) ˆξi = arg maxξi p(ξi|θ). This can be rewritten as ˆξi = arg max ξi p(ξi|θ) (4.11) = arg max ξi p(ξi, θ) (4.12) = arg max ξi x∼ξi g(x) (4.13) = arg max ξi x∼ξi I fI(ψ(fI)). (4.14) Assuming a cycle-free FG G referring to the global function g, we can use fundamental SPA Theorem 4.1 to evaluate exactly the marginal pf p(ξi, θ). An evaluation of the MAP estimation is then straightforward with a reasonable complexity. The MMSE estimation can be done in a similar manner, i.e. using (2.9), we have ˆξi = E[p(ξi|θ)] (4.15) = E x∼ξi I fI(ψ(fI)) (4.16) for a given observation θ. 24
- 38. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS 4.2 FG-SPA Implementation One of the main advantage of the FG-SPA is that it defines powerful algorithms. These algorithms are described precisely using the FG-SPA. Nevertheless if one would like to implement these algorithms, some difficulties arise. We browse these difficulties including their formal description. According to this descrip- tion, we form a formal requirement for the implementation framework serving for a direct implementation of the FG-SPA. 4.2.1 Message Types in FG-SPA and their Representation We start with description of the message representation. We have to somehow cope the representation of the message for the processing purposes. We are able to store and process a finite number of discrete values. This gives as a basic constraint, which is not important in case of discrete messages. The FG-SPA is, however, defined also for messages describing continuously valued variables (pdf) requiring uncountable number of parameters for their direct exact representation in general. This brinks a question about a parameterization of these messages. Since we are able to store a finite number of parameters, the message is therefore parameterized by an n-dimensional vector q ∈ Aq ⊆ Rn . The message µ ∈ M is then given by µ = Φ(q), where Φ : Aq → M is a function defining the parameterization. Obviously, the less number n the more efficient parameterization. On the other hand some messages may need an infinite number of parameters to be exactly represented. We will also use an inverse mapping Φ−1 : M → Aq to determine the parameter vector for a given message, i.e. q = Φ−1 (µ). The parameterization of the message is therefore given by the mappings Φ : Aq → M and Φ−1 : M → Aq determining the relation between the message µ ∈ M and its parameterization q ∈ Aq ⊆ Rn . We further define a dimensionality of the message as number of parameters D required for the message representation, that is the length of the vector q. Within this work, we assume all variables xi to be scalar variables in complex domain or some of its subset, i.e. xi ∈ Axi ⊆ C, ∀xi ∈ X. According to the domain Axi of a variable xi, we recognize the following types of the messages connected to the VN xi: Discrete Messages The discretely valued message is a message connected to xi, if Axi has a finite number of elements. A straightforward representation of such a message is q = [. . . , qj, . . . ], where the components qj refers to the elements of Axi . This refers to the pmf of the corresponding variable xi. The minimal dimensionality over all parameterizations of this message is obviously upper-bounded by the number of parameters, i.e. min(D) ≤ |Aq| = Mq. 25
- 39. 4.2. FG-SPA IMPLEMENTATION A typical example is a binary message, where xi ∈ Axi = {−1, 1} and µ(xi) = [q0, q1] = [p−1, p1]. An another form of the binary message representation is log-likelihood ratio (LLR). Since xi is a binary random value, it can be described by a log likelihood ratio given by λ = log(p1/p−1) standing for mapping Φ−1 . Since p−1 + p1 = 1, we can express the message by (3.15), i.e. µ(xi) = [p−1, p1] = Φ(λ) = 1 1 + exp(λ) , exp(λ) 1 + exp(λ) . (4.17) Note that the dimension of the message is D = 1, because the message can be fully described by a single real number (λ). We therefore found a parameterization of a pmf with lower dimension then the description by the pmf itself, where D = 2 for the binary variable. Example 4.6 (Log-likelihood message representation). Parameterizable Continuous Messages The continuous parameterizable message is a message connected to xi, if Axi is an infinite set, e.g. a continuous interval I ⊆ R. This message is continuous, but directly parameterizable. The direct parame- terization is a concrete form of the function Φ. The number of parameters again forms the upper-bound on the message dimensionality. As an example, we mention the real Gaussian message fully described by its mean and variance q = [¯xi, σx] and the message is given by µ(xi) = Φ(¯xi, σx) = 1 2πσ2 x exp |xi − ¯xi|2 2σ2 x . (4.18) The inverse mapping Φ−1 is given by q = Φ−1 (µ(x)) = [Ex[µ(x)], var[µ(x)]]. Note that it can be shown that the mapping Φ is bijection in this case. Example 4.7 (Gaussian message representation). General Continuous Message A general continuous message is a message connected to xi, if Axi is an infinite set and there is not any straightforward parameterization of the message. This means that the dimension of the message is either infinite, or we are not able to find a parameterization with a reasonable dimension. The message appears whenever both discretely and continuously valued messages are presented in one FG-SPA. We mention the phase model of the joint decoder phase estimator as an instance. The problem is increasing number of parameters required for the message parametrization with increasing iteration. See a demonstration in Fig. 4.3. Example 4.8 (Mixed-Type Message). 26
- 40. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS x F y z µ(x) µ(y) µ(z) µ(x) = 1√ 2πσ2 exp −|x−m|2 2σ2 p → {m, σ2 } m q0 q1 q2 q3 µ(y) = {qi}i p → {qi}i µ(z) = . . . p →??? Figure 4.3: Illustration of the mixed-density message parameterization problem. 4.2.2 Message Approximation Motivated by the representation of the general continuous message, we consider some parameterization that does not exactly determine the message, but instead some approximation of the true message is provided. Introducing of the approximated message, we bring some kind of inaccuracy to the FG-SPA, which must be somehow handled. On the other hand, we are able to (at least) approximately describe all messages with a given number of parameters. More formally, we define the message approximation using a parameterization, which does not fully describe the exact message. Suppose that the exact message µ ∈ M is parametrized by q = Φ−1 (µ) as we considered so far. But up to now, we silently assumed that the message can be then obtained by µ = Φ(q), as in Example 4.7. We now suppose that no such a mapping Φ that µ = Φ(Φ−1 (µ)) , ∀µ ∈ M exists. Instead, the mapping Φ is given by Φ : Aq → ˆM, where the approximated message must not be in ˆM as one can see in Fig. 4.4. The messages ˇµ ∈ M ˆM are therefore not exactly parameterizable by Φ. The approximated message ˆµ ∈ ˆM is thus given by a parameterization ˆµ = Φ(q). The principle of the parameterization causing the message approximation is shown in Fig. 4.4, where we also considered a hypothetical vector qE ∈ AqE exactly representing the true message. Particularly, we assume: • The parameterization assigns a parameter vector q = Φ−1 (µ) to all messages, i.e. Φ−1 (µ) ∈ Aq , ∀µ ∈ M ∪ ˆM. This mapping is not injective in general and therefore more messages are mapped to the same parameterization as shown in Fig. 4.4. We should also note that the mapping Φ−1 is formally given by Φ−1 : M ∪ ˆM → Aq. • The approximated message can be unambiguously assigned for all parameter vectors, i.e. Φ(q) ∈ ˆM, ∀q ∈ Aq. The mapping Φ is supposed to be a bijection4 , that is the parameter vector and the approximated message are one to one mapping (see Fig. 4.4). • The last assumption is that the mapping Φ−1 is also bijection, if it is provided upon the approximated message5 . As a consequence, the approximated message referring to the parameter vector of the approximated message is the same, i.e. ˆµ = Φ(Φ−1 (ˆµ)) , ∀ˆµ ∈ ˆM. 4One can consider a parameterization such that more parameters can represent the same (approximated) message. Nev- ertheless it is not necessary for our purposes, so we will assume Φ to be clearly a bijection within this work. 5The mapping Φ−1 is in fact composed from two mappings. The first one is the inverse of Φ, which is clearly given in ˆM, since Φ is bijection. The second part is a mapping from M ˆM that is not invertible (see Fig. 4.4). 27
- 41. 4.2. FG-SPA IMPLEMENTATION M ˆM AqE Aq Φ−1 (µ) Φ(q) Figure 4.4: Principle of the message approximation given by the parameterization ˆµ = Φ(q). The true message µ ∈ M is mapped through a mapping Φ−1 (dashed arrows) to a parameter vector q ∈ Aq. The vector q ∈ Aq is unambiguously related with an approximated message ˆµ ∈ ˆM by a mapping Φ (dotted arrows). We denoted a hypothetical vector qE ∈ AqE exactly representing the true message µ (dash-dotted set). If a domain of the variable xi that is represented by a message µ(xi) is some real interval, i.e. xi ∈ Axi = I ⊆ R, the message can be surely parameterized by samples. It means that an original true continuously valued message µ(xi) ∈ I = a, b) in the pdf form is represented a vector q = [q1, . . . , qD] T = [µ(a), . . . , µ(a + (D − 1)∆)]] T , where ∆ = (b − a)/D. This formally defines the mapping Φ−1 (µ). And the approximated continuous message is then composed (mapping Φ) as a piecewise function from the samples ˆµ(t) = D−1 i=0 qi+1ν(t − i∆), (4.19) where ∆ = (b − a)/D, ν(t) = 1 for t ∈ 0; ∆) and ν(t) = 0 otherwise. Using a finite number of samples (see Fig.4.8a), we are able to represent an arbitrary message in a finite interval. Intuitively, the more samples the more exact representation can be expected but at expanse of higher computational complexity. Example 4.9 (Representation of a Continuous Message by Samples). The number of samples in Example 4.9 suggests a crucial natural question about quality of the approxi- mation. One can feel a trade-off between the fidelity of the message representation and the dimension of the parameterization. The fidelity can be measured by means of the overall FG-SPA performance. Neverthe- less, it would be useful to define a metric M(µ, ˆµ) that assigns some quality to the message representation computable just from the message approximation and the true message. As an example of such a metric, we mention the mean square error of the approximated message. This metric will be optimized by the proposed KLT message representation in Sec. 5.2.1. 28
- 42. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS χ1(x) χ2(x) χ3(x) µ(x) ≈ c1χ1(x) + c2χ2(x) + c3χ3(x) Figure 4.5: Demonstration of the canonical message representation. Canonical Message Representation A general concept of the message representation is proposed in [63], where the authors propose the canonical message representation consisting in a set of kernel function representing the message as a superposition of some kernel functions. The message is then defined by a parameter vector qj. The message is written in the form ˆµ(xi) = j χj(qj, xi), (4.20) where χj are the kernel canonical functions and qj is a set of parameters (see Fig 4.5 for an illustration). This is, however, rather a generic concept how to describe a certain class of message representation, because no way how to obtain methodically the kernel functions is not given. Furthermore, the canonical representation solves only the parameterization ˆµ = Φ(q). No way of the evaluation of the parameter q is given in general (function Φ−1 ). Note 4.10 (Linear Canonical Message Representation). If the mapping Φ can be written as a superposition a some kernel set, we refer such a representation given by Φ as a Linear Canonical Message Representation (LCMR). It means that a message can be written in form µ(x) ≈ ˆµ(x) = D i=1 qiχi(x), (4.21) where q is the vector parameterizing the message and {χi(x)}D i=1 is a collection of (canonical) kernel functions. Note that in this case, we still have available only the mapping Φ. In the other words, we no way of the parameterization obtaining from the true message is given in general. An example with three canonical kernels is shown in Fig. 4.5. Note 4.11 (Linear Canonical Message Representation with Orthogonal Kernel). If we assume a LCMR, where the kernel set {χi(x)}D i=1 is orthogonal, we are able to construct also the mapping Φ−1 by means of the inner product, since χi, χj = 0, ∀i = j and therefore the i-th component of the parameter vector is given by qi = µ, χi . 4.2.3 Update Rules Upon the Message Representation We introduced the update rules upon the messages in (4.6) and (4.7) as mapping6 U : Mn → M, for some n. That is the update rules defined upon the true messages. Nevertheless during the processing, the message 6We assume an equal domain for all messages corresponding to the update for the notation simplicity. Nevertheless there is no reason for this assumption in general. 29
- 43. 4.2. FG-SPA IMPLEMENTATION x F y µ(x) µ(y) µ(x) = . . . qx →??? µ(y) = . . . qy →??? µ(y) = p(y|x)µ(x)dx Figure 4.6: Demonstration of the update rules (Def. 4.3) provided upon the exact form of the message, which is however difficult to be directly handled in the processing. Moreover the integration of a continuous function must be evaluated within the update rule in case of continuous messages. x F y ˆµ(x) ˆµ(y) ˆµ(x) = i cx i χi(x) qx → cx ˆµ(y) = j cy j χj(x) qy → cy ˆµ(y) = p(y|x)ˆµ(x)dx Figure 4.7: Demonstration of the update rules (Def. 4.3) provided upon the (approximated) message representation. This case solves the problem with message representation, nevertheless, the integration of a continuous function must be evaluated within the update rule in case of continuous messages. are supposed to be represented rather by their parameterization. It would be therefore useful to find the corresponding update rules that will be applied directly to the message representation. More formally, the implementation consisting from the message representation and the update rules can be given either directly by the definition of the FG-SPA or some more sophisticated (efficient) message representation and the corresponding update rules, particularly 1. Messages are given µ ∈ M and the update rules U : Mn → M in conventional FG-SPA. 2. Messages are given by their parameterization q ∈ Aq and the UR are defined upon the parameteriza- tion, i.e. U : An q → Aq, where the mapping U properly respects both the message parameterization Φ, Φ−1 and the update rules U defined in Def. 4.3. We particularly recognize the following update rules according to description of the messages. Update Rules Upon the Exact Messages The update rules provided directly according to the definition of the update rules (Def. 4.3) face two problems in case of general continuous messages. The fist one is the representation of the general contin- uously valued message and the second on is the integration required for the update rules, which is often numerically intensive when it is provided numerically (see Fig. 4.6 for a demonstration). Note that no mapping Φ is used for the message parameterization, the update rules are given directly by the corresponding pf. This case is rather a starting point for some further analytical derivations then a direct manual of the FG-SPA implementation excepting the update rules upon the discrete messages. Update Rules Upon the Parameterized (Approximated) Messages In this case, we solved the problem with the message parameterization, because we have available mappings Φ and Φ−1 . Nevertheless we still use the update rules according to Def. 4.3 containing the integration that is not suitable for an efficient direct implementation in general. It means that the update rules are 30
- 44. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS provided by a chain of mappings given by Φ ◦ U ◦ Φ−1 : An q → ˆMn → ˆM → Aq for some n, where we assume the parameterization of the message as both input and output to the update rules. See Fig. 4.7 for demonstration. This is some middle-step between the direct definition of the update rules and the desired implemen- tation framework. Update Rules Directly Upon the Parameterization This case refers to the target of the derivation of the update rules for a general message representation. It provides a direct relation between the message parameterizations. It means that the chain of mappings from the previous case is reduced to Aqn → Aq and the integration required in Def. 4.3 is avoided for finite length of vector q (dimension D). We will closely focus on this topic in Section 5.2.1, where we introduce the update rules for a generic LCMR with orthogonal kernels introduced in Note. 4.11. As an example, we mention the update rules upon sampling of the continuously valued message showed in Example 4.9. One can show [13] that the update rules (Def. 4.3) applied on the samples refer to the numerical integration with rectangular rule and therefore a natural solution for the update rules. The samples (parameterization) of pdf (true message) are taken as pmf inputting into the update rules. Example 4.12 (Update Rules Upon Sampling). Another example is the update rules design for the LLR (or rather soft bit) message (4.17) suited for the decoding of LDPC codes. The factor nodes refer to the check nodes proving GF-sum, i.e. ⊕ operation. The solution can be found e.g. in [23]. Example 4.13 (Update LLR for LDPC). 4.2.4 FG-SPA Implementation Framework The FG-SPA can be easily (and exactly) implemented for the discrete (especially binary) messages. The messages are exactly represented by their pmf and the update rules are performed directly according to their definition (4.6) and (4.7). This is a straightforward way, which is not necessarily the most efficient. As an example, we can mention the soft bit implementation of LDPC. This implementation attains less complexity, because both the message representation and the update rules process a single-valued messages for the binary case as we discussed in Example 4.13. To unify the ways of the FG-SPA, we define the implementation framework as a message representation and the corresponding update rules upon the given message representation. The FG-SPA implementation framework is a tuple (Φ, U ), where Φ defines a message parame- terization (even approximately) and the corresponding update rules U are defined directly upon the given message representation. Definition 4.5 (Implementation FG-SPA Framework). 31
- 45. 4.2. FG-SPA IMPLEMENTATION (a) Discretization of the continuously valued message. (b) Representation using the coefficients of the Fourier decomposition. (c) Gaussian representation given by mean and variance of the original message. (d) Representation by Dirac-delta function posed into the argument of maximum of the original message. Figure 4.8: Different message representations of general continuously valued messages. Note 4.14 (Scheduling Algorithm). We said that the implementation framework is given by the message representation and the corresponding update rules upon the message representation. We should remind that in case of looped FG-SPA, we have to also consider some scheduling algorithm. The scheduling algorithms will be discussed in Sec. 4.3.1. Criteria of the FG-SPA Implementation Framework We have already defined the implementation framework, but we did not consider any way, how to compare the frameworks from some quality point of view. A natural requests of the FG-SPA implementation are: • efficiency, • exactness. Moreover, we are often by a limited memory available to processing. These requests are naturally in contradiction and therefore some trade-off is often needed. The efficiency request basically means a sim- ple message representation (represented by a few parameters) that leads to efficient update rules design (relation between the few parameters). 4.2.5 Instances of the FG-SPA Implementation Frameworks We show some possible FG-SPA message representation frameworks that can be found in literature in greater detail (see Fig 4.8). Some of these representations serve later as reference implementation frame- works for the numerical verification of the proposed design. Sample Representation with Numerical Integration The discretization of the continuous message is the most straightforward method of the practically feasible representation as it is discussed in [12,13]. An arbitrary precision might be achieved using this represen- tation by increasing the number of samples (complexity of the approximated message). This framework is given by merging Examples 4.9 and 4.12. A straightforward evaluation of the update rules makes this representation case very important, because it is principally usable for every case, where we do not care about the numerical intensity of the FG-SPA. A direct application of the marginalization upon the message representation (the discrete vector p taking 32
- 46. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS as pmf) leads to an approximation of the marginalization upon pdf, where the integration is performed by the numerical integration with rectangular rule (see e.g. [13]). Fourier Representation The well known Fourier decomposition enables to parameterize the message by the Fourier’s series as ˆµ(t) = Φ(α, β) = α0 2 √ π + 1 √ π D i=1 αi cos(it) + βi sin(it), (4.22) where αi = 1/ √ π µ(t) cos(it)dt and βi = 1/ √ π µ(t) sin(it)dt gives the inverse mapping Φ−1 . Dimen- sionality is given by D = 2N + 1. In [10], the authors have derived update rules for the Fourier coefficients in a special case of the random-walk phase model. This message representation is the LCMR with orthogonal kernels as we stated in Note 4.11. The Fourier representation also exemplifies the case when ˆM is not a subset of M as one may wonder, when we discussed the message approximation (see Fig. 4.4). Consider that the set ˆM is given by all pdfs, that is all non-negative functions that satisfies µ(t)dt = 1. The Fourier approximation, however, does not guarantee a non-negative message as one can see in Fig. 4.8b. Dirac-Delta Representation The message is represented by a single number situated in the maximum of the message (see Fig. 4.8d). The approximated message is given by ˆµ(t) = δ(t − arg max ˇt µ(ˆt)). (4.23) The update rules upon the Dirac-Delta message approximation can be found e.g. in [12] for the phase estimation problem. The update rules are based on the gradient methods Gaussian Representation Gaussian representation and its modifications are widely used message representations (e.g. [52]). This is given by simplicity of the message representation given just by mean and variance (m, σ) with the interpretation ˆµ(t) = 1 √ 2πσ2 exp − |t − m|2 2σ2 , (4.24) where m = E[µ(t)] and σ2 = var[µ(t)] (see Fig. 4.8c) for a real message. We have already discussed this case in Example 4.7. The update rules for linear only factor nodes can be found in [32]. The Gaussian representation is moreover often used to approximate such a message representation that is difficult to be parametrized directly (e.g. [52,53,55]). 4.3 SPA on Looped Factor Graphs This section is written as an overview what happens if the factor graph contains loop. Since the FG-SPA implementation framework as we discussed so far does not explicitly need a cycle-free corresponding FG, we can freely extend the terms defined in Sec. 4.2 to the looped FG-SPA. Nevertheless, we must pay attention to the conventional problems of the looped FG-SPA. 33
- 47. 4.3. SPA ON LOOPED FACTOR GRAPHS The FG-SPA as defined for the cycle-free FG can be used also in the looped case. The main differences are summarized bellow: • The FG-SPA becomes to be an iterative algorithm and a convergence uncertainty occurs. • Even, if the looped FG-SPA converges, the inference is evaluated only approximately and a further result investigation is required. Moreover, the stationary point must not be unique. • The performance of the FG-SPA depends on a message scheduling. • The message representation issue often becomes to be hardly trackable for mixed-type messages (Example 4.8). Although the mentioned issues, the looped FG-SPA presents a tool to solve much more problems then the cycle-free FG-SPA. We have already mentioned some examples, but the most important application is capability of the algorithms described in Chapter 3. 4.3.1 Message Scheduling in FG-SPA We have several choices of the scheduling. There are three fundamentally different approaches, concretely: 1. Random scheduling selects randomly the messages to be updated. 2. Predefined fixed scheduling strictly defines the scheduling prior the SPA starts. It can be parallel, i.e. all messages are updated once per iteration, e.g. flood schedule algorithm (FSA), or sequential, i.e. there is a given order and the updated messages are used directly after evaluation. A notable sequential scheduling algorithm is the FBA used in the turbo decoding. 3. Adaptive scheduling selects message to be updated according to a comparison between a potentially updated message with the non-updated message. It means that a metric M(µk, µk+1) is selected and then the message with the metric satisfying some properties e.g. [16] is updated. We can mix the mentioned principles. For example, one part of messages are passed randomly and the second one by a predefined scheduling. This approach is reasonable, when the system is composed from a smaller sub-systems and each subsystem has its own well defined scheduling order. 4.3.2 Convergence of the Looped FG-SPA The convergence of the looped FG-SPA becomes a huge complication. We cannot use the fundamental theorem 4.1 and thus we have no guarantee of an approximated inference evaluation. We can mention several approaches of the looped FG-SPA convergence that can be found in literature. • Analytical approach [36,37], where the authors describe the update function by an analytical function of the all messages in one iteration and then they investigate the properties of the function by the Banach theorem. Their analysis gives a very practical method for binary messages and relatively small graphs. The FG-SPA with more complex (even non-binary) messages suffers from very difficult analytical formulas and the convergence test is dropped absolutely for FG-SPA containing continuous messages in [37]. • Statistical physics approach presented in [66] is completely different to [36, 37]. The authors have used methods of statistical physics. They found a correspondence between the fixed points of the FG-SPA and the stationary points of Bethe approximation of the free energy. 34
- 48. CHAPTER 4. FACTOR GRAPHS AND RELATED ALGORITHMS • EXIT chart convergence description 3.3.2 [56] was designed as a description of the turbo decoding. However, the principle is more general, it describes the convergence according to the mutual infor- mation transfer of some entities. As it was stated in [2], the FG automatically handle correctly the extrinsic and intrinsic information. Nevertheless the difficulty occurs with the numerical evaluation of the mutual information, since we cannot generally assume that it can be fully determined by a single parameter as in case of binary EXIT charts. We can finally mention that it is known that the correlation decays exponentially with the loop length in the FG.. Therefore (if possible) it is convenient to maximize the FG-loops length (girth) for the FG-SPA performance optimization, because the long loops with reduced correlation are closer to the crucial FG-SPA assumption, that is the decomposability of the system (i.e. independence of the parameters). 35
- 49. 5 FG-SPA Implementation Framework This Chapter aims with the implementation issue of the sum product algorithm on factor graphs (FG-SPA) standing for one of the main results of this work. We first propose a methodical way of a canonical message representation obtaining for a general continuously valued messages in FG-SPA [44]. This method is based on Karhunen Loève Transform (KLT) exploiting the stochastic nature of the messages in the FG-SPA1 . The orthogonal canonical kernels serving for the message representation are evaluated. As a consequence of the KLT, the mean square error of the message approximation caused by dropping a given set of kernels can be easily controlled. This can be used for such a message representation that the complexity/fidelity trade-off can be easily found for a given fidelity goal. We further propose an update rule design for a general family of linear canonical message representations with orthogonal kernels [47]. Since the eigenfunctions resulting from the KLT are orthogonal, a generic FG-SPA implementation framework can be composed from the KLT-message representation and the generic update rules design. The wide introduction including the problem statement and fundamental definitions are formally stated in Chapter 4, nevertheless this Chapter is written to be self-contained so that the introduction and defini- tions is briefly repeated. 5.1 Introduction 5.1.1 Background The sum product algorithm on factor graphs (FG-SPA) provides a popular generic framework usable in a wide spectrum of problems across many fields such as communications, signal processing, etc. [27,33]. The aim of the FG-SPA consists mainly in an efficient evaluation of the decision and estimation problems in Bayesian networks. The FG-SPA serves for an exact efficient inference evaluation algorithm, whenever the FG underlaying the problem is cycle-free. If the FG-SPA contains loop, it can still serve as a high-efficient evaluation algorithm (e.g. LDPC decoding), but the loops cause that the inference evaluation is no longer guaranteed to be the exact one. The FG-SPA becomes to be an iterative algorithm and its convergence issue arises. This convergence issue stands for a principal nature problem and its investigation is out of scope of this work (see e.g. [36] for an exhaustive overview of the looped FG-SPA convergence investigation possibilities). Even if a message passing directed by the update rules [27] leads to a correct (not necessarily exact) marginal evaluation (inference), an efficient FG-SPA implementation is often not straightforward even if the update rules are clearly defined. The difficulty consists mainly in an infinite memory requested to store a general continuously valued messages and also in an evaluation often intractable marginalization integrals required in the update rules evaluation. A commonly used way of the FG-SPA implementation is to represent (even approximately) the messages by several discrete parameters which are easy to be stored within the iteration process and to develop the update rules upon these parameters instead of the update rules upon the original messages. A natural goal of the FG-SPA implementation is to have • the simplest possible message representation, • a low-complex evaluation of the update rules upon the message representation, • a minimal inaccuracy caused by the use of the message representation and the corresponding update rules. 1The stochastic process is formed by messages in different realizations and iterations. 36
- 50. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK Obtaining a proper message representation and the corresponding update rules is a crucial point in the FG-SPA implementation, because a wrong choice of the message representation easily leads to a need for a very complex messages and the update rules required for a given fidelity goal. Since the message is represented by a pdf or pmf, it is conventional that messages are properly scaled such that the marginalization of the message is normed to 1. This normalization is can be easily done for pdf or pmf. The message representation has to somehow cope this normalization. 5.1.2 Related Work We can find several message representation - update rules design frameworks in literature. Some of them are discussed in greater detail in Sec. 4.2.5. An example of the FG-SPA implementation improvement in case of LDPC decoding over GF(2) consists in the use of single-valued LLR values instead of the binary pmf as the message representation and the corresponding update rules can be found in [27]. In this case, the LLR-based FG-SPA implementation gives equal2 results as the direct FG-SPA implementation with better efficiency. In the previous case, we had still choice to directly implement the FG-SPA, since all considered mes- sages were discretely valued. If the messages are continuously valued (represented by a pdf), a message parametrization is unavoidable. Nevertheless, there exists a straightforward parameterization for a family of continuously-valued messages (e.g. Gaussian message fully represented by its mean and variance). The update rule upon the Gaussian messages remains the Gaussian one, whenever the update is linear (only superposition and addition factor nodes are allowed). In such a case, it is sufficient to find a relation between means and variances. An exhaustive overview of the Gaussian messages use in the Linear models can be found in [32]. The easily parameterizable case is, however, a very specific one. Whenever the FG contains both the discretely and continuously valued messages, an appearance of a mixed-type message becomes unavoidable (see Fig. 4.3 for an illustration). The mixed-type message is a continuously valued message arising from the mixed marginalization, i.e. marginalization over both discrete and continuously valued messages. The problem is with the arising number of parameters in the iterative process requested for the mixed-type message parameterization. An example of this problem is joint data-detection phase-estimation, where a discrete-nature data are mixed with a continuous-nature phase shift. The implementation of the FG-SPA containing the mixed type message appears several time in the liter- ature. The most straightforward method is sampling of the continuously valued message. The update rules provided upon the discretely valued sampled messages stand for a numerical integration with rectangular rule (e.g. [12,13]). An another approach consists in the representation of the message by a Kronecker-delta function posed into maximum of the message. The update rules are (e.g. gradient methods) are proposed in [12,13]. These update rules are built on the non-linear optimization basis. We can also mention particle method, where the message is represented by (weighted) particles (see e.g. [13] for details). Several works (e.g. [28,52,55]) solve the increasing number of parameters required for the mixed-type message representation by an approximation of the mixed-type message by a properly selected well param- eterizable function. The largest attention attracts the Gaussian-form pdf because of its implementation simplicity. A unified framework to the message representation issue can be found in [63]. The authors propose that a general message may be represented using a set of kernel functions and some coefficients related to the kernel (canonical) functions. This framework does, however, not solve a way of the kernel functions obtaining and also the update rules design is not solved there. We can found several works deriving the canonical kernels and related update rules for a particular system model. As an example, the authors in [5] propose Fourier and Tikhonov message parameterization 2Excepting numerical inaccuracies. 37
- 51. 5.2. FG-SPA IMPLEMENTATION FRAMEWORK and corresponding update rules for the joint data decoding phase estimation problem. This way of the canonical kernels obtaining is however derived for the concrete model, no methodical way is proposed. 5.1.3 Summary of Results - Contribution The additional value of this work can be summarize as follows: 1. We propose a methodical way of the canonical kernels obtaining for an arbitrary FG-SPA implemen- tation, where the KLT is defined upon the process formed by different message realizations. 2. The KLT-based procedure serves as a connection between the message description complexity and the fidelity criterion in terms of the mean square error of the message approximation. 3. We design a generic efficient implementation of the update rules design for linear canonical message representations with orthogonal kernels. 4. The orthogonality of the canonical kernel functions guaranteed by the KLT allows to state a generic FG-SPA implementation framework, where the messages are represented by the KLT-message repre- sentation3 and the update rules are given4 by the proposed generic efficient implementation. 5. We numerically verify the proposed design. 5.1.4 Summary of Notation We briefly remind the basic notation. See Chapter 4 for full description. We denote a message from a node F to x by µF →x (x) ∈ MF →x . The notation of this message and its domain can be simplified to µF (x) ∈ MF , µx (x) ∈ Mx or just to µ(x) ∈ M if the notation remains unambiguous. The superscripts are simplified whenever it is possible without lost of unambiguity or they are not important for that moment. Other parameters of a message are shown in subscript e.g. µk(t) as the kth iteration of the message µ(t). The upper-case letters denote the factor nodes and the lower-case letters denote the variable nodes. We assume a set of square integrable real-valued functions defined on an interval I ⊆ R to be a domain of the continuously-valued messages. The domain of the discretely-valued messages is supposed to be a D-tuple, where D denotes dimensionality of the message. 5.2 FG-SPA Implementation Framework We now propose the FG-SPA implementation framework, where we employ a LCMR with orthonormal kernels resulting from the KLT. We then derive the update rules for a general LCMR with otrhogonal kernels that forms a complete implementation framework with the proposed KLT message representation. 5.2.1 KLT Message Representation We assume a FG-SPA without any implementation issues with a given scheduling algorithm. The particular shape of each message depends on (1) the iteration number k and (2) a random observation input x. The true message5 describing a variable T in a given iteration and with given observation is denoted by µx,k(t), 3Other representations (Gaussian, Discrete, ...) can be also considered within the FG-SPA. 4As in case of the message representation, the update rules can be also performed by an arbitrary other way. 5Note that we assume a continuously valued message within this derivation, concretely the message from L2 space. When the message is discretely valued, the proposed results can be achieved by equal steps respecting properly the discrete domain. 38
- 52. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK used components unused components → MSE λi i ε Dε Figure 5.1: Demonstration of the KLT message representation fidelity-complexity trade-off handling. The residual MSE is indeed easily controlled by the sum of the eigenvalues corresponding to the unused (trun- cated) components. where we assume the message to be a real valued function of argument t ∈ I ⊆ R. This message can be approximated using a canonical kernel set {χi(t)}i by µx,k(t) ≈ ˆµx,k(t) = Φ(q(x, k)) = i qi(x, k)χi(t). (5.1) The expansion coefficients [. . . , qi(x, k), . . . ] fully describing the approximated message ˆµk,x(t) ∈ ˆM are random. The KLT of the process formed by the messages µx,k(t) for random iteration and observation input uses the second order statistics - the autocorrelation function rxx(s, t) to create an orthogonal kernel set with uncorrelated expansion coefficients and the second order moments are directly related with the residual mean square error. The autocorrelation is function given by rxx(s, t) = Ex,k [(µx,k(s) − Ex,k [µx,k(s)])(µx,k(t) − Ex,k [µx,k(t)])] , (5.2) where Ex[·] denotes expectation over x and (s, t) ∈ I2 . The solution of the eigenequation given by I rxx(s, t)χ(s)ds = λχ(t) (5.3) produces the eigenfunctions {χi(t)}i serving for the orthogonal canonical kernel set and the eigenvalues {λi}i serving for the importance weight of the corresponding message representation component in terms of the mean square error. The expansion coefficients can be evaluated using the orthogonal property of the canonical kernel set by qi(x, k) = Φ−1 (µx,k(t)) = I µx,k(t)χi(t)dt. (5.4) These coefficients serve for the approximated message description by (5.1). The complexity (dimensionality) of the message representation can be further reduced by omitting several components {qiχi(t)}. The components with index i > Dε are dropped to reduce the representation complexity. We denoted Dε the number of used components (dimensionality of the message) that refers to mean square error ε. The resulting mean square error of the approximated message ˆµx,k = Dε i=0 qi(x, k)χi(t) (5.5) is indeed given by the term ε = i>Dε λi as one can check in Fig. 5.1. 39
- 53. 5.2. FG-SPA IMPLEMENTATION FRAMEWORK Note 5.1 (Linear Canonical Message Representation). Thanks to the KLT, the kernels are orthogonal and the representation is linear. It means that the KLT message representation is a linear canonical message representation with orthogonal kernels as we discussed in Note 4.11. Therefore both mappings determining the parameterization Φ, Φ−1 are clearly given. 5.2.2 Evaluation of the KLT Approximated Message in Real Applications We have clearly described the procedure of the KLT message representation obtaining. Nevertheless the evaluation of the autocorrelation function (5.2) can be very complex. This can be simplified considering the approximation of the continuously valued message given by a discrete vector resulting from a sufficient sampling of the message6 . The estimated autocorrelation function is given by Rxx = Ex,k[µx,kµT x,k] ≈ 1 KM K k=1 M x=1 µx,kµT x,k, (5.6) where K is the number of iteration, M is the number of realizations and µ = [. . . , µx,k[i], . . . ]T is a discrete vector referring to the sampled message µx,k(t). The eigenequation (5.3) of the discrete message is given by Rxxχ = λχ. (5.7) The vector χ refers to the sampled kernel functions χ(t). The expansion coefficients are given by qi(x, k) = j µx,k[j]χi[j]. (5.8) Finally the sampled version of the approximated message is given by µx,k ≈ ˆµx,k = i qi(x, k)χi. (5.9) The truncation of the coefficients can be performed equally as in the continuous case. 5.2.3 Generic Update Rules of the Canonical Message Representation with Orthogonal Kernels We call a Linear Canonical Message Representation (LCMR) such a message (see Note 4.10) that the message can be written in the form µF →x (x) ≈ ˆµF →x (x) = D(F →x) i=1 qF →x i χF →x i (x), (5.10) We remind that the proposed KLT message representation (Sec. 5.2.1) is LCMR as we already mentioned in Note. 5.1. We now derive the FG-SPA update rules for a general LCMR (5.10) with orthogonal canonical kernels upon the parameterization, that is a direct relation between the parameters of the message as we discussed in Sec. 4.2.3. We derive the FG-SPA update rules for the LCMR assumed in a whole or in a part of the system. The coexistence of more message representations in the FG-SPA requires interfaces with another message representations. Particularly, we investigate 6The complexity issue of the autocorrelation function evaluation is not complication at all, because it is sufficient to estimate the autocorrelation function just once prior the processing. The required number of samples enabling a full message recovery from samples is given by the sampling theorem. 40
- 54. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK x y F z µx→F (x) µy→F (y) µF →z (z) Figure 5.2: Visualization of the FN updated message µF →z . • variable node (VN) update rules with the LCMR, • factor node (FN) update rules – with the LCMR only, – with the LCMR and the representation by samples, – with the LCMR and an another representation. We also address our attention to a proper scaling of the approximated messages and we point out the exact- form updated-message rectification problem, when the approximated message is negative (non-positive) at some points. For simplicity, we will assume only three neighboring nodes. It means that for a FN F update, we consider just three neighboring variable nodes x, y and z (see Fig. 5.2) and for a VN x update, we consider just three neighboring factor nodes F, G and H (see Fig. 5.3). An extension to a general case with an arbitrary number of neighbors is straightforward. Factor Node Update Rules We first investigate the update rules related to factor nodes. We start with a conventional FN update rule formula (4.6), [27] µF →z (z) = p(z|y, x)µx→F (x)µy→F (y)dxdy ≈ p(z|y, x) i qx i χx i (x) j qy j χy j (y)dxdy = i j qx i qy j p(z|y, x)χx i (x)χy j (y)dxdy, where we used (5.10) to express the approximated messages. Our target is to obtain the expansion coefficients qz k of the approximated message ˆµF →z (z) = k qz kχz k(z). We exploit the orthogonality of the kernels to evaluate the expansion coefficient by means of the inner prod- uct as qz k = ˆµF →z (z), χz k(z) = i j qx i qy j wF i,j,k, (5.11) where7 wF i,j,k = p(z|y, x)χx i (x)χy j (y)χz k(z)dxdzdy is a table determined by given kernels and a conditional probability density function (pdf) p(z|y, x). The table is parameterized by i, j, k and it can be precomputed prior the processing. 7Since we assume real-valued kernel functions and real expansion coefficients, we drop the complex conjugation related to the inner product. 41
- 55. 5.2. FG-SPA IMPLEMENTATION FRAMEWORK F G x H µF →x (x) µG→x (x) µx→H (x) Figure 5.3: Visualization of the VN updated message µx→H . We stress that (5.11) indeed relates the parameterization of the input and output messages as we requested in Sec. 4.2.4 to the implementation framework. Variable Node Update Rules The investigation of the VN update rules can be conducted similarly. We start again with the definition of the VN update rule according to [27] µx→H (x) = µF →x (x)µG→x (x) ≈ i qF i χF i (x) j qG j χG j (x). (5.12) The expansion coefficients of the approximated message are evaluated again by means of the inner product by qH k = ˆµx→H (x), χH k (x) = i j qF i qG j wV i,j,k, (5.13) where the update table wV i,j,k = χF i (x)χG j (x)χH k (x)dx is given purely by the sets of the kernel functions and it can be precomputed prior the processing. Factor Node Update with both Discretely and Linearly Represented Continuous Messages We now consider a case of a mixed updating, when a FN F is neighboring both discretely and continuously valued messages. Particularly, the variable y is a discretely valued and the message µy→F (y) is represented by pmf of y. The variables x and z are continuously valued. Our goal is to evaluate the LCMR of the message µF →z (z) from the LCMR µx→F (x) and discretely valued message µy→F (y) = [. . . , qy j , . . . ] and conversely the discrete message µF →y (y) from the LCMR µz→F (z) and µx→F (x). The derivation is very similar as in Sec. 5.2.3. We shortly write the FN-update as (see Fig. 5.2) µF →z (z) = j p(z|yj, x)µx→F (x)µy→F (y)dx (5.14) ≈ j p(z|yj, x) i qx i χx i (x)qy j dx (5.15) and the expansion coefficients qz k of the approximated message ˆµF →z (z) = k qz kχz k(z) are evaluated as qz k = ˆµF →z (z), χz k(z) = i j qx i qy j wF i,j,k, (5.16) 42
- 56. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK where wF i,j,k = p(z|yj, x)χx i (x)χz k(z)dxdz is a precomputed table for given kernel sets and the conditional pdf p(z|yj, x). After a very similar derivation, we obtain the update of the discrete message µF →y (y) by qy j = i k qx i qz kwF i,j,k. (5.17) This update is, however, not guaranteed to be a non-negative number for all j as one requests to pmf. A rectification of the updated message is briefly discussed in Sec. 5.2.3. Interface with Other Message Representations We have shown a closed update-relation for linearly represented continuously valued messages and discrete messages represented by their pmf. Nevertheless, it may be still useful to have two different representations of a continuously valued message. In such a case, we split this VN to two variable nodes and insert an interface factor node between them. Assume a VN ¯x connected to two parts of a FG-SPA. The first part of the FG-SPA operates with an arbitrary representation of the continuously valued message (e.g. Gaussian) and the second one is assumed to be represented by the LCMR with orthogonal kernels. The VN ¯x is split to variable nodes x and y, where a FN F is inserted between them. It is assumed that the VN x is represented by the LCMR with orthogonal kernels and the VN y by an arbitrary another message representation capable to describe the continuously valued message. The node F is described by the conditional (pdf) given by p(y|x) = δ(y − x) and the expansion coefficients qx i of the LCMR message ˆµF →x (x) = i qx i χx i (x) are evaluated as qx i = ˆµF →x (x), χx i (x) = µy→F (x)χx i (x)dx. (5.18) The message ˆµy→F (x) is used if µy→F (x) is not available. If the message ˆµy→F (x) is parameterized by a vector p, one can avoid the integration during the processing by precomputing a set of functions Ψi(q) = ˆµy→F (x; q)χx i (x)dx i . Conversely, the corresponding message representation is evaluated from the LCMR approximated mes- sage ˆµx→F (x) = i qx i χx i (x) by means of the particular message representation. We can mention e.g. Gaussian representation given by [my , σy ] = Ex[ˆµx (x)], Ex[(ˆµx (x))2 ] − Ex[ˆµx (x)]2 , (5.19) where Ex[µ(x)] = xµ(x)dx denotes expectation over x. One can find out that Ex[(ˆµx (x))n ] = i (qi)n Ex[(χi(x))n ] and Ex[(χi(x))n ], ∀i, n can be again precomputed. Norming and Rectifying the Approximated Message Since the original message µ(t) is supposed to represent a pdf, we may not care about the message scaling during the update rules. When the message is approximated, we need to be able to achieve ˆµ(t)dt = 1. Suppose that ˆµ ¯ (t) = k q ¯ k χk(t) is a non-normed LCMR. We consider a norming constant α ∈ R such that αˆµ ¯ (t)dt = ˆµ(t)dt = 1. 43
- 57. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION ˆµ(t) < 0 t negative pmf UR Figure 5.4: Demonstration of the FG-SPA updated exact-form negative message problem. ˆµ θ ˜µ Figure 5.5: Rectification of the negative message appearance. We can express α = 1/ ˆµ ¯ (t)dt = 1/ i Pq ¯ i Ωi, where we defined Ωi = χi(t)dt. Since αˆµ ¯ (t) = ˆµ(t) surely guarantees a proper norming, we can express the norming expression as qi = αq ¯ i = q ¯ i j q ¯ j Ωj , ∀i. (5.20) As a consequence of the message approximation, the approximated message ˆµ(t) = k qkχk(t) can be negative at some points. The messages are conventionally assumed to be non-negative in the FG-SPA. The negativeness of the approximated message can be ignored, when the message is used to evaluate an another approximated message, because the non-negative request is casted to the true (exact-form) messages. The approximated (or the updated) message must, however, be somehow rectified in case of the exact-form updated message (see Fig. 5.4). There is a number of ways of the negative message rectification (e.g. subtracting mint(ˆµ(t)) from the approximated message and norming it properly as ad-hoc suggested in [44]). Motivated by our simulation results in Sec. 5.3.5, it is beneficial to avoid a hard decision provided by the FN-update rule upon the approximated message. It means that the updated exact-form message should not contain any zero element (e.g. discrete message represented by pmf should satisfy pj > 0, ∀j, if it is a result of the FN-update with an approximated message input). Following this simple idea, we add-hoc suggest to define a softness threshold θ and the rectified updated exact-form message ˜µ(t) = ˆµ(t) if ˆµ(t) ≥ θ and ˜µ(t) = θ otherwise (see Fig. 5.5). We numerically find out a very high sensitivity of the performance with respect to the updated-message rectification in a specific scenario (Sec. 5.3.5). With the increasing precision of the message representation the impact of this phenomenon will decrease. 5.3 Applications and Numerical Verification We exemplify the proposed FG-SPA implementation framework on the well-know case of the joint phase estimation data detection. Various FG-SPA implementation frameworks suited for this case can be found in literature (e.g. [10,12,52,53,55]). We stress that while all mentioned frameworks are designed for this particular scenario, the proposed implementation framework is more generic. The verification consists from two parts. First, we obtain the canonical kernel functions of the KLT message representation. This evaluation is based on the stochastic analysis of the messages in the FG- SPA (Sec. 5.2.2). The impact of this approximation is investigated and compared with other message representations mentioned in Sec. 4.2.5. The second part investigates behavior of a whole implementation framework. 44
- 58. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK d C c M s × exp(jϕ) u + w x Figure 5.6: The system models in signal space. d CPE q NMM s × exp(jϕ) u + w x LPD θ Figure 5.7: The MSK modulation in phase space. 5.3.1 System Model We consider several system model variants to achieve a more complex view into the system. This is desired mainly for the stochastic analysis of the messages in the phase model. We consider several linear modulations models in signal space space and a non-linear minimal shift keying (MSK) modulation in the phase space model. We also consider two phase models, the constant phase and the random walk (RW). Signal Space Models We assume a binary i.i.d. data vector d = [d1, . . . , dn]T of length N as an input. The coded symbols are given by c = C(d). The modulated signal vector is given by s = M(c) , where M is a signal space mapper. The channel is selected to be the AWGN channel with phase shift modeled by the random walk (RW) phase model. The phase as the function of time index is described by ϕj+1 = mod(ϕj +wϕ)2π, where wϕ is a zero mean real Gaussian random value (RV) with variance σ2 ϕ. Thus the received signal is x = s exp(jϕ) + w, where w stands for the vector of complex zero mean Gaussian RV with variance σ2 w = 2N0. The model is depicted in Fig. 5.6. MSK in Phase Space We again assume the vector d as an input into the minimum-shift keying (MSK) modulator. The modulator is supposed to be in the canonical form, i.e. it is modelled by the continuous phase encoder (CPE) and non- linear memoryless modulator (NMM) as shown in [50]. The modulator is implemented in the discrete time with two samples per symbol. The phase of the MSK signal is given by φj = π/2 (ςi + di(j − 2i)/2) mod4, where φj is the jth sample of the phase function, ςi is the ith state of the CPE and the sampled modulated signal is given by sj = exp(jφj). The communication channel is selected to be the AWGN channel with constant phase shift, i.e. x = s exp(jϕ) + w, where x stands for the received vector, ϕ is the constant phase shift of the channel and w is the AWGN vector. The nonlinear Limiter Phase Discriminator (LPD) captures the phase of the vector x, i.e. θj = ∠(xj). The whole system is shown in Fig. 5.7. 5.3.2 Factor Graph of the Joint Phase Estimator Channel Decoder The FG-SPA is used as a joint synchronizer-decoder (see Fig. 5.8) for all mentioned models. Note that the FG for the considered models might be found in the literature (phase space model in [55] and signal space models in [10,12]). Prior the description itself, we found a notation to enable a common description of the models. We define 45
- 59. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION dj dj+1 αj αj+1 PS PS γj γj+1 W W ζj ζj+1 δ(xj − x0 j ) δ(xj+1 − x0 j+1) ϕi ϕi+1 DecoderandDemaper Phasemodel Figure 5.8: The factor graph of the receiver covering both phase models. PS ϕj ϕj+1 PS jth section (j + 1)th section RW PS PS jth section (j + 1)th section ϕ Figure 5.9: Phase shifts models: Random walk model (left) and the constant phase shift model (right). • αj = sj, γj = αj exp(jϕj) and ζj = xj, where αj, γj, ζj ∈ C for the signal space models with the RW phase model, • αj = ∠(sj), γj = (αj + ϕj)mod2π and ζj = θj, where αj, γj, ζj ∈ R for the phase space model with the constant phase model. One can see, that ϕj, ϕ ∈ R for both models. The FG is depicted in Fig. 5.8. We shortly describe the presented factor nodes and message types presented in the FG. Factor Nodes We denote ρσ2 (x−y) = exp(−|x−y|2 /(2σ2 ))/ √ 2πσ2 and then we use pΞ(σ2 , ξ−κ) as the phase distribution of the RV given by Ξ = exp(j(ξ − κ)) + w, where w stands for the zero mean complex Gaussian RV with variance σ2 w; x, y ∈ C and ξ, κ ∈ R. The phase models are shown in Fig. 5.9. • Factor nodes in the signal space models Phase Shift (PS): p(γj|ϕj, αj) = δ(γj − (αj exp(jϕj))) AWGN (W): p(ζj|γj) = ρσ2 w (ζj − γj) • Factor nodes in the phase space model Phase Shift (PS): p(γj|ϕ, αj) = δ(γj − (αj + ϕ)mod2π) 46
- 60. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK AWGN (W): p(ζj|γj) = pΞ(σ2 W , ζj − γj) • Factor nodes common for both signal and phase space random walk (RW): p(ϕj+1|ϕj) = ∞ l=−∞ ρσ2 ϕ (ϕj+1 − ϕj + 2πl) Other factor nodes such as the coder, CPE, and signal space mappers FN are situated in the discrete part of the model and their description is obvious from the definition of the related components (see e.g. [27] for an example of such a description). Message Types Presented in the FG-SPA The FG-SPA contains both discretely and continuously valued messages. The channel encoder and the signal space mapper contains only the discretely valued messages. The messages are represented by their pmf the update rules can be directly implemented according their definitions. The AWGN part contains well parameterizable continuously valued messages and the phase model con- tains the mixed-type messages. These mixed type messages are represented by means of the implementation framework (KLT message representation in our case). 5.3.3 Concrete Scenarios We specify several concrete scenarios for the analysis purposes. All of the scenarios may be seen as a special case of the system model described in the section 5.3.1. The first four configurations are used for the stochastic analysis of the mixed type message (Sec. 5.3.4) and the fifth one is used for the implementation of the whole implementation framework including the KLT message representation and the proposed update rules implementation (Sec.5.3.5). Uncoded QPSK modulation The Uncoded QPSK modulation scheme assumes the QPSK modulation, which is situated in the AWGN channel with RW phase model. The length of the frame is N = 24 data symbols, the length of the preamble is 4 symbols and the preamble is situated at the beginning of the frame. The variance of the phase noise equals σ2 ϕ = 0.001. This scenario leads to a cycle-free FG-SPA and thus only inaccuracies caused by the imperfect implementation are presented. The information needed to resolve the phase ambiguity is contained in the preamble and, by a proper selection of the analyzed message, we can maximize the approximation impact to the key metrics such as BER or MSE of the phase estimation. We thus select the message µRW−→ϕ3 to be analyzed. Coded 8PSK modulation The coded 8PSK modulation contains in addition to the previous scenario the (7, 5, 7)8 convolutional encoder8 . The length of the frame is N = 12 data symbols, the length of the preamble is 2 symbols. The same message is selected to be analyzed (µRW−→ϕ3 ). MSK modulation The MSK modulation with constant-phase model of the phase shift assumes the length of the frame is N = 20 data symbols. The analyzed message is µϕ→P S . These messages are nearly equal for all possible PS factor nodes (e.g. [13]). 8We mean by (7, 5, 7)8 a conventional octal definition of the convolutional encoder. 47
- 61. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t Conv code, 12dB -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t QPSK, 8dB -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0 1 2 3 4 5 6 χ(t) t BICM, 8dB -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0 1 2 3 4 5 6 χ(t) t QPSK 15dB χ1 χ2 χ3 χ4 χ5 Figure 5.10: Shape of canonical kernel functions for different system configuration in phase space. BICM 1 The bit-interleaved coded modulation (BICM) model employs the BICM consisting of (7, 5)8 convolutional encoder and QPSK signal-space mapper. The phase is modeled by the RW model with σ2 ϕ = 0.005. The length of the frame is N = 1500 data symbols and 150 of those are pilot symbols. This model slightly changes our concept. Instead of the investigation of the single message, we analyze all µP S→ϕ and µϕ→P S messages jointly. It means that all of the analyzed messages are approximated in the simulations and the stochastic analysis is performed over all investigated messages. BICM 2 We assume again the BICM. The model contains (7, 5)8 convolutional encoder, QPSK signal-space mapper, RW phase model with σ2 ϕ = 0.004, frame length N = 120 and preamble of length 15 pilot symbols. 5.3.4 Stochastic Analysis of the Mixed-Density Messages in FG-SPA Run Once the FG-SPA is described, the stochastic analysis described in Sec.5.2.2 can be applied. We describe the reference model that aims with the collection of the mixed-type messages for the stochastic analysis. The FG-SPA Reference Model The empirical stochastic analysis requires a sample of the message realizations. Thus we ideally need a perfect implementation of the FG-SPA for each model. We call this perfect or better said almost perfect FG-SPA implementation 9 as the reference model. Note that the reference model might suffer (and our 9Note that even if the implementation of the FG-SPA is perfect, the convergence of the FG-SPA is still not secured in the looped cases. We call the perfect implementation such a model that does not suffer from the implementation related issues such as an update rules design and a messages representation. The flood schedule message passing algorithm is assumed. 48
- 62. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t SNR = 4dB -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t SNR = 8dB -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t SNR = 12dB -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0 1 2 3 4 5 6 χ(t) t SNR = 16dB χ1 χ2 χ3 χ4 χ5 Figure 5.11: Shape of canonical kernel functions for the MSK modulation. 1e-14 1e-12 1e-10 1e-08 1e-06 0.0001 0.01 20 40 60 80 100 120 λi i Eigenvalues for the MSK modulation - log scale SNR=4dB SNR=8dB SNR=12dB SNR=16dB 0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 1 2 3 4 5 6 7 8 9 10 λi i Eigenvalues for the MSK modulation SNR=4dB SNR=8dB SNR=12dB SNR=16dB Figure 5.12: Eigenvalues descending for the MSK modulation. 49
- 63. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION models do) from the numerical complexity and it is therefore unsuitable for a direct implementation. We classify the messages appearing in the reference FG-SPA model (Fig. 5.8) and their update rules. • Discrete messages are all messages on left from the PS FN (see Fig 5.8). It means all messages in the encoder and mapper part. These discrete messages are fully represented by their pmf and the evaluation of the update rules is performed directly according to [27]. • Unimportant messages (µP S→γj , µγj →W , µW →ζj and µζj→δj ) lead to the open branch and neither an update nor a representation of them is required, because these messages cannot affect the estimation of the target parameters (data, phase). • Parameterizable continuous messages (µδj →ζj and µζj →W ) can be easily represented by a number x0 meaning µ(x) = δ(x − x0), µW →γj and µγj →P S are representable by the pair {m, σ2 w} meaning µ(x) = ρσ2 w (x − m), µ(x) = pΞ(σ2 W , x − m) respectively. One can easily find the slightly modified update rules derived from the standard update rules. Examples of those may be seen in [13]. • The mixed-type messages situated in the phase model (µP S→ϕj , µϕj →P S , µRW+→ϕj , µϕj →RW+ , µRW−→ϕj and µϕj →RW+ ) are10 discretized and the marginalization is performed using the numerical integration with the rectangle rule [13, 32] in the update rules. The number of samples is chosen sufficiently large so that the impact of the approximation can be neglected. The mentioned mixture messages are real valued one-dimensional functions for all considered models. Eigensystem Analysis We first provide the analysis of the eigensystem. For the investigated messages, we collect the sample realizations to obtain the second order statistics (the empirical autocorrelation function) according to (5.6). The eigenfunctions solving the eigenequation (5.7) result in harmonic functions independently of the parameters of the simulation as one can see in Fig. 5.10 and 5.11. The related eigenvalues are shown11 in Fig. 5.12. We can see several artefacts. The eigenvalues provide an important information about the importance of a particular message approximation component. The MSE resulting by dropping the component is given directly by the corresponding eigenvalue as discussed in Sec.5.2.1. The floor in Fig. 5.12 is caused by the finite floating precision. As one can see, the higher SNR, the slower is the descent of the eigenvalues with the dimension index. The curves in the plots point out to the fact that the eigenvalues are descending in pairs e.g. λ1 − λ2 λ2 − λ3. Relation of the MSE of the Approximated Message with the Target Criteria Metrics The KLT message representation provides the best approximation in the MSE sense for a given dimen- sionality. This is, unfortunately, not the target metric to be optimized in almost all cases. This relation is moreover not easy to analytically obtain. We therefore provide a numerical simulation to inspect the relation between the approximation error and the target metrics (MSE and BER) at least in several distinct system model configurations. Few notes are addressed before going over the results. The MSE of the phase estimation is computed as an average over all MSE of the phase estimates in the model. The measurement of the MSE is limited by the granularity of the reference model. The simulations of the stochastic analysis are numerically intensive. We are limited by the computing resources. The simulations of the BER might suffer from this, especially for small error rates. The threshold of the detectable error rate is about Pmin be ≈ 10−6 for the Uncoded QPSK model and Pmin be ≈ 2 · 10−5 for the BICM model. 10We have denoted RW+ the factor node RW, which lies between jth and (j + 1)th section according to the Fig. 5.9. Analogously, RW− stands for the FN between (j − 1)th and (j)th section. 11The results in other model seems very similarly and they are therefore not explicitly shown. 50
- 64. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK 1e-07 1e-06 1e-05 0.0001 0.001 0.01 0.1 1 10 2 3 4 5 6 7 Pbe Number of used components (D) BER to D cycle-free FG with SNR=15dB Fourier KLT Sampling Dirac Gauss 0.001 0.01 0.1 1 10 2 3 4 5 6 7 MSE Number of used components (D) MSE to D cycle-free FG with SNR=15dB Fourier KLT Sampling Reference Dirac Gauss Figure 5.13: MSE and BER for the cycle-free FG-SPA (uncoded QPSK modulation with RW phase model). 51
- 65. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION 1e-06 1e-05 0.0001 0.001 0.01 0.1 1 10 2 3 4 5 6 Pbe Number of used components (Dε) Comparison of the BER for BICM model on SNR=8dB Fourier KLT Sampling Gauss 0.01 0.1 1 10 2 3 4 5 6 MSE Number of used components (D) Comparison of the MSE with the BICM model, SNR=8dB Fourier KLT Sampling Reference Dirac Gauss Figure 5.14: MSE and BER for the BICM with 8PSK modulation and RW phase model. 52
- 66. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK 1e-06 1e-05 0.0001 0.001 0.01 0.1 1 0 20 40 60 80 100 120 Pbe Iteration number (k) Dependence BER on k - BICM model, SNR=8dB Fourier D=5 Fourier D=6 KLT D=5 KLT D=6 Dirac Reference Figure 5.15: BER for the BICM with 8PSK modulation and RW phase model with dependence on the iterations considering the flood-schedule message passing algorithm. • The results of the uncoded QPSK modulation are shown in Fig. 5.13. The capabilities of the proposed KLT message representation are equal with the capabilities of the Fourier representation. This is not any surprise taking into account the results of the eigenanalysis (Sec.5.3.4), but it means that the Fourier representation provides the best linear message representation in the MSE sense. The expected proportional relationship between the MSE of the approximated message and the target criteria does not strictly hold (compare BER of the five-component and six-component approximated message in Fig. 5.13). The discretization does not work well for small number of samples, because the higher SNR (15dB) brinks narrower shapes of the messages. Consequently the messages suffers by a wider spectral bandwidth and a higher amount of samples is required to satisfy the sampling theorem. • The results of the BICM model at SNR=8dB are shown in Fig. 5.14 and 5.15. We can see that the KLT message representation lost a little bit against the Fourier representation. The KLT ap- proximated message converges a little bit faster than the Fourier representation up to approx. 45th iteration, where the KLT approximated message achieves the error floor. We suggest two possible explanations of the error floor appearance (see Fig. 5.15). First, the eigenvectors constituting the basis system are not evaluated with a sufficient precision. Second, the evaluated KLT basis is the best linear approximation in average through all iterations and this basis is not capable to describe the messages appearing in the higher iterations sufficiently. The relationship between the MSE of the approximated message and the target criteria is not again strictly proportional (compare the 3-component and 4-component messages in Fig. 5.14). One can suggest that the eigenfunctions need to be in pair to increase a precision to the results. 53
- 67. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION si PS ui RW ϕi ϕi+1 si+1 PS ui+1 Coder and Mapper Figure 5.16: The factor graph of the decoder for the random walk phase model implemented in the signal space. This can be seen as a special case of Fig. 5.8, where we recognize the discrete type of the messages (green edges), parameterizable (Gaussian) continuous messages (blue edges) and mixed type messages (red edges). 5.3.5 Proposed Generic Update Rules Implementation Within the implementation of the update rules, we consider the system model described in Sec.5.3.1 with the BICM modulation with RW phase model implemented in signal space. The factor graph of the receiver for this particular model can be found in Fig 5.16 as a special case of 5.8. Canonical Kernel Functions Assignment The results of eigenanalysis (Sec. 5.3.4 or Fig. 5.11 and Fig. 5.10) suggest to use the harmonic functions as the canonical kernel basis system for the mixed-type messages. Using the Fourier representation offers the best message approximation in the MSE sense for a given dimensionality. We therefore employ the Fourier message representation, where the approximated message is given by (4.22). A proper scaling of the Fourier representation can be done using Sec. 5.2.3. One can check that Ωk = 2π 0 χk(t)dt = 0, ∀k = 0 and Ω0 = 2π 0 χ0(t)dt = 1/ √ 2π. The scaling factor α is thus given by α = √ 2π/q ¯ 0 . The scaling of the approximated message to achieve k qkχk(t)dt = 1 can be easily done using (5.20), concretely by setting qk = q ¯ k √ 2π/q ¯ 0 . One can easily check that αˆµ ¯ (t)dt = αΩ0q ¯ 0 = 1. Update Rules Derivation for the Assigned Canonical Kernel Functions According to the set of kernel functions, we can prepare the table of weights and the functions needed for the efficient evaluation of the update rules12 . We start with the RW update. RW-FN Update Recalling results from Sec. 5.2.3, we can write wi,j = p(ϕi+1|ϕi)χi(ϕi)χj(ϕi+1)dϕidϕi+1. (5.21) 12We can note that the update rules design for the Fourier representation can be found also in [10] for this model. In contrast with this work, where the authors propose the implementation framework suited for this concrete scenario, we use this model just to verify the proposed generic design. 54
- 68. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK According to the numerical evaluation, we observed that the weights wi,j are non-zero valued only on the main diagonal in this case. This makes the evaluation of the expansion coefficients very efficient, that is the complexity grows linearly with the dimensionality of the messages. The update is given by qRW→ϕi j = wjjq ϕi+1→RW j (5.22) q RW→ϕi+1 j = wjjqϕi→RW j . (5.23) PS-FN Update The PS-FN is a hybrid node, because all adjacent variable nodes differ in their message representation types. We need two updates, concretely µP S→ϕ and µP S→s for the decoding purposes. The message µP S→u leads to the open branch and its update is not required within the receiver. We start with the discrete message µP S→s , which is represented by a discrete message qs i = p(u|s, ϕ)µϕ (ϕ)µu (u)dudϕ (5.24) ≈ 1 √ 2πσ2 k qkΨi,k(u0), (5.25) where we used the sampling property of δ function, the conditional pdf describing the PS FN and a set of functions Ψi,k(u0) = exp − |u0 − si exp(jϕ)|2 2σ2 χk(ϕ)dϕ. (5.26) The set {Ψi,k(u0)}i,k is formed by some real-valued functions of a complex argument that are parameterized by the harmonic kernel functions χk(ϕ), constellation points si and AWGN variance σ2 . The set can be precomputed prior the processing to make the main FG-SPA run more efficient. Similarly, one can derive qk = 1 √ 2πσ2 i qs i Ψi,k(u0). (5.27) Note that the update (5.25) is subjected to the rectification mentioned in Sec. (5.2.3), if ˆµϕ→P S (t) < 0 for any t. VN Update The VN {ϕi}i updates are given by (5.13). FG-SPA Implementation Framework Numerical Verification We adopt the model 5.3.3 and implemented the KLT-message representation with the proposed update rules. The sample representation with numerical integration as the update rules is selected as a reference implementation framework. The results are shown in Fig. 5.17. As a reference case, we have chosen the sample representation of the mixed-type density messages with numerical integration as the update rules. The results are shown in Figs. 5.17 and 5.18 for different dimensions of the messages (D). Observing Fig. 5.17, we can identify the "waterfall" behavior of the proposed model and thus the curves confirm the framework functionality. One can also see that the proposed implementation framework 55
- 69. 5.3. APPLICATIONS AND NUMERICAL VERIFICATION 1e-05 0.0001 0.001 0.01 0.1 -4 -2 0 2 4 6 8 10 12 Pbe SNR[dB] BER for different number of used canonical functions D=3-KLT D=5-KLT D=3-Sampling D=5-Sampling D=9-Sampling D=11-Sampling D=256-Sampling Figure 5.17: The bit error rate comparison of the proposed implementation framework base on the KLT approximated message with the representation by samples updated using the numerical integration with rectangular rule. 0.01 0.1 1 -2 0 2 4 6 8 10 12 MSE SNR[dB] MSE for different number of used canonical functions D=3-KLT D=5-KLT D=11-KLT D=5-Sampling D=11-Sampling D=256-Sampling Figure 5.18: The mean square error of the phase estimation of the proposed implementation framework base on the KLT approximated message with the representation by samples updated using the numerical integration with rectangular rule. 56
- 70. CHAPTER 5. FG-SPA IMPLEMENTATION FRAMEWORK Sampling KLT D = 3 D = 11 D = 256 D = 3 D = 5 Table 0 0 0 0.8 3 + 1.4 14.1 6746 0.7 2.8 × 2.2 18.1 7640 1.7 6 exp(·) 0.6 5.5 2543 0 0 abs(·) 0.2 0.8 19.7 0 0 Ψi,k(u) 0 0 0 0.01 0.02 Table 5.1: Number of symbol by symbol operations of some selected representations during the evaluation of the phase model - red color part of the receiver FG shown in Fig. 5.8. All numbers are divided by 106 . The function Ψi,k(u) is given in (5.26). considerably outperforms the reference sample-based implementation FG-SPA framework in both MSE and BER cases for a given dimensionality. We have (ad-hoc) chosen a threshold (softness coefficient) θ = 10−4 to avoid a hard decision about a symbol in codeword for the message µP S→si . The reason is that the discrete soft in soft out decoder much better incorporates a potentially wrong soft decision then the hard one with θ = 0. The reference sample-based implementation of the phase-model FG-SPA suffers by error floors for small number of samples. A particular reason can be seen in Fig. 5.18, where one can see a considerably worse phase estimation capabilities comparing to the proposed framework. The error-floor in the reference case can be explained by the presented narrow-shaped high-SNR Gaussian-like messages that are often below the resolution of a few (equidistant) samples. The performance is,however, not only one measure of the FG-SPA implementation framework quality. We should also consider the complexity of the implementation, which is shown in Tab. 5.1. One can see that the complexity of the proposed framework is lower then the complexity of the reference solution. We should note that the complexity results in Tab. 5.1 are quite rough estimate, because they are just snapped from the simulation run without any further complexity-optimization in both methods. In summary, the presented FG-SPA implementation framework is very generic as well as the sample- based framework. The KLT based framework requires a lot of evaluations (numerically intensive) prior the processing, while the reference model does not require any pre-computed calculation. The pre-computation effort is, however, rewarded by a significant gain in both BER and MSE performance metrics as well as in a lower implementation complexity for a given message dimensionality as discussed above. 5.4 KLT Based Implementation Framework Summary We have proposed a methodical way for the canonical message representation based on the KLT. The method itself is not restricted for a particular scenario. It is sufficient to have a stochastic description of the message or at least a satisfactory number of message realizations and the KLT upon this process must be defined. The method, as it is presented, is restricted to real-valued one-dimensional messages in the FG-SPA. We further derived a generic update rule design suited for an arbitrary linear canonical message rep- resentation with orthogonal canonical kernels. This generic update rule design is combined with the pro- posed KLT-message representation providing inherently the linear canonical message representation with orthogonal canonical kernels to form a generic framework. This linear canonical message representation is moreover the best one among all linear approximations in the MSE sense for a given number of repre- senting parameters. Its fidelity can be well controlled by the sum of eigenvalues referring to the unused components. 57
- 71. 5.4. FG-SPA IMPLEMENTATION SUMMARY We numerically verified the proposed framework in a concrete application - joint data detection - phase estimation in a conventional point-to-point AWGN channel with. We shown by simulations that the methodical way of the canonical kernel functions obtaining leads to the harmonic functions independently to the tested phase models, the channel encoders and the signal space mappers. We investigated the capabilities of the KLT approximated message and compared them with the con- ventional message representations appearing in the literature. Furthermore, we compared the results of the proposed generic FG-SPA implementation framework including the proposed update rules design with the conventional sampling method. The reason of the sampled message selection as the reference case is that it stands also for a generic framework as well as the proposed design. 58
- 72. Part III Decoding in Multi-Source Wireless Networks 59
- 73. 6 Physical Wireless Network Communication This Chapter provides a gentle introduction to wireless communication networks. The goal of this Chapter is to provide some basic overview and classification of the wireless networks to prepare a ground for the next Chapter, where we propose a decoder suited for the networks with wireless network coding. The reader familiar with the wireless network coding (physical layer network coding) [26, 30, 39, 54] can skip this Chapter directly to Chapter 7. The physical layer wireless communication is often viewed as an extension of the conventional commu- nication, where the higher layers direct the basic communication elements (physical layer) containing only P2P channels and MAC channels with SD-IC. The higher layers (e.g. routing) are optimized independently to the physical layer. The optimization of several layers (OSI-model) independently is intuitively not op- timal from the global point of view. The global optimization is addressed as the cross-layer design solving the optimization across more (all) layers jointly. The main problem of the cross-layer design is complexity of the cross-layer optimization. In this work, we rather focus on some basic phenomena (network coding paradigm [1,67]) that can be applied in wireless networks and that promises a significant gain against the conventional independent optimization. The network coding is however designed in the discrete world and it still uses the physical layer as a system of orthogonal pipes. The adoption of the physical layer taking into account the network coding paradigm (wireless network coding) then brings another potential improvement. Particularly the perfor- mance beyond the higher (e.g. second)-order cut-set bound can be achieved avoiding the conventional method such as either joint decoding or orthogonal P2P approach. Instead, a hierarchical decoding is introduced. The hierarchical decoding roughly means that the node decodes directly a function of data streams instead of decoding of all streams followed by a network operation upon the decoded streams. The decoding of a function of data can be also viewed as reduction of interference that unavoidably corrupts the performance in the P2P based communication (MAC region limitation [11]). The largest potential gain of WNC is therefore in dense network, where the interference presents the main limitation of the system. Since still arising number of wireless devices communicates in a given (constant) area with given (constant) resources (wireless medium) with increasing demand on the transmission speed nowadays, the method of non-orthogonal interference reduction obviously attains a lot of attention of the research community in the past few years [68]. The WNC can be viewed as a cooperative diverse scheme [29]. The MIMO system is in fact a multi- source multi-destination(relay) wireless network with a perfect cooperation. It can be used as a benchmark for many applications. The full awareness of the system allows to adjust the precoding matrix with respect to the received signal. If the links between the individual antennas are not perfect, that is some constraints are assumed on these links (e.g. wireless channel), we can consider the wireless network as a generalized MIMO. Prior we dive into the topic of the wireless network coding, we provide some basic classification of a general wireless network from the topology, constraints and processing point of view within this Chapter to get a better categorization of our work. Since the topic of detection in the multi-source wireless is closely and formally described in the following Chapter 7, this Chapter is written in intro-based style without any formal notation and definitions. 6.1 Wireless Network 6.1.1 Basic Elements The wireless network is formed by a set of nodes that are located in an area and that are equipped by devices enabling a wireless communication. The nodes are supposed to be connected only through the wireless medium (wireless channels). The nodes can be classified either as terminal nodes or relay nodes. 60
- 74. CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION The relay nodes are neither data (information) source nor data sink. The terminal nodes are divided to sources (data source) and destinations (data sinks). Some examples of the models for wireless networks that are significant for our work are discussed in Sec. 6.1.6. We can describe the network by a graph, where the edges are given by wireless channels between the nodes and the nodes are given by the physical nodes. The information flow is given by some requirements from destinations. That is each destination is interested in information from a certain set of sources. A crucial difference between the conventional wireless networks based on OSI layer is that the information content in the pipes is correlated and therefore some description is dedicated also for some cuts of the net- work. According to the connections in the network and potentially further criteria (constraints), something (e.g. node itself or genie entity) decides about behaviour of each node. 6.1.2 Assumptions Within this part of the work, we will explicitly assume the following constrains and assumptions. Half Duplex Constraint The half duplex constraint is a consequence of real hardware limitation, particularly the interference of the transmitted and received signal arises in the antenna if the signals are passed simultaneously (full-duplex). Due to this, it is assumed that the each node can either transmit or receive the signal at one time instance. One can found many papers aiming with consequences of the half-duplex constraint investigation. A straightforward natural time division simplifying the network design is the same time portion dedicated for both the transmission and the reception of the signal in each node. On the other hand, a desired power allocation can be achieved adjusting the time division properly. Uniform Unitary Transmit Power Distribution A unit transmit power of all nodes is assumed from the simplicity reasons. An unequal power distribution of nodes within the network can be still adjusted by means of the flat fading coefficient in necessary. Perfect Time Synchronization The imperfect time synchronization basically leads to the inter-symbol interference. Since the network contains more nodes, various levels of synchronization imperfectness can be assumed. We consider a perfect time synchronization and channel estimation among all nodes in the network within this part. We refer e.g. [30] as a tutorial considering also the synchronization issue. Acyclic Wireless Networks From the complexity reasons, we will consider exclusively cycle-free networks within this work. It means that the directed graph representing the wireless network and the flow of the information is cycle free, that is if a node transmits some information, this information cannot be delivered back to the node in some later stage. 6.1.3 Network Processing Classification Within the network processing, we recognize basically two types of the processing, particularly (see e.g. [41]) • centralized processing, • distributed processing. 61
- 75. 6.1. WIRELESS NETWORK One can also combine these types of processing to form a hybrid processing that is directed partly in both distributed and centralized manner. Centralized Processing The centralized processing is called whatever direction of the processing that uses a central entity (genie) for the management of the processing in each node. It means that the processing in all nodes is directed by some genie. This solution is often taken as a reference, due to the possibility of the optimal solution upon the global description of the system. The problem is the communication overhead required to manage the network, since all communication between the central entity and a given node is supposed to be passed through the wireless medium (at expanse of data). Distributed Processing The distributed processing does not contain any central entity managing the network behaviour. It means that the nodes decide about their processing according to their own. The distributed processing saves the resource overhead required for the communication with the genie. On the other hand, only a local information is available for a decision about the behaviour of the node and therefore the optimality cannot be achieved from the global point of view in general. Mixture Processing The near-optimal fully-distributed processing of nodes is obviously the holy grail of the wireless network designers. This endeavor is, however, often difficult considering the fully-distributed system. We therefore classify also a mixture of the distributed and centralized processing that defines an genie entity for a given set of nodes. The decision about the behaviour is still provided in a distributed level from the global point of view, but the individual nodes are managed by a sub-genie entity. 6.1.4 Type and Nature of Information Appearing in the Wireless Network A transmitted signal can be recognized only as a useful signal or nuisance interference in the conventional higher layer directed P2P approach. A fundamental difference between the WN and the higher layer directed P2P channels is that the WNC recognizes a Hierarchical-Side Information (H-SI). This information does not contain directly the desired data for the receiver, but instead, it somehow helps to decode the desired data. To be more specific, we provide the following classification: Direct Link The receiver requires the data directly contained in the link from the given source. This is the same as in the conventional P2P communication. Interference Link The receiver does not need the data from the given source directly and also the hierarchical function is not a function of the given data. The interference link is again something that is well know from the conventional solution. 62
- 76. CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION R x0 sR Figure 6.1: Relay-node processes the signal space observation vector x0. The output of the relay is again a signal space vector sR depending on the relay processing strategy. Hierarchical Side Information Link The H-SI link presents some complementary information to desired data. This is in sense of the network coding paradigm, where the information can be temporary mixed in the network to increase the overall throughput. The H-SI is then used to reveal the desired data from the temporary mixed information that does not fully contain the desired data in general. 6.1.5 Relaying Strategies The relaying strategy is called all the processing in the relay between the observation x0 and the transmission of sR (see Fig. 6.1). We omit a dummy relaying strategy, where the relay is silent or its behaviour does not depend on its observation x0. Amplify and Forward The Amplify and Forward (AF) strategy is the simplest possible relaying strategy. It works just as a multiplier, that is sR = βx0, where β is a constant securing that sR has unit energy per symbol. If we consider a MIMO channel received in the relay, β becomes to be a matrix. This strategy preserves all information contained in x0, but the power of relay is wasted on the am- plifying of the noise and unwanted interference collected in the wireless channel(s). The AF strategy is latency free. A description of the AF strategy in 2-WRC with half duplex constraint can be found e.g. in [48]. Hierarchical Decode and Forward The Hierarchical Decode and Forward (HDF) is a strategy that provides a decision about a function of the received data (hierarchical data). The entropy of the hierarchical data is less or equal to the entropy of all received data vectors. The decoded hierarchical data are then again encoded and mapped to the signal space vector sR. This strategy will be described later more precisely. We can mention that the HDF strategy includes (among others) as e.g. Compute and Forward strategy [40] or Denoise and Forward [26]. The HDF strategy suffers from the latency, because the full decoding requires to operate with whole sequence. Note 6.1 (X-Decode and Forward). The hierarchical function is not closer specified here. It means that the HDF strategy contains also conventional decode and forward (DF), where the function is identity for a given data stream or joint decode and forward (JDF), where the function is joint identity for all streams. Compress and Forward The Compress and Forward (CF) strategy is called such a strategy that somehow process x0 to provide a metric γ. This metric should be a sufficient statistics about the (hierarchical) data. A special case of this metric is the hierarchical data vector considered in HDF. The CF strategy covers more approaches, typically quantize and forward, where the metric γ is given by somehow sampled version of x0. 63
- 77. 6.1. WIRELESS NETWORK 6.1.6 Physical Network Topologies We browse some basic topologies that are important for this work. We also exploit the particular topologies to demonstrate some principles of a given strategy (mainly HDF) that can be clearly seen in the given topology that are related with the sequel Chapter. Two-Way Relay Channel The two-way relay channel (TWRC - Fig. 6.2) is the simplest multi-source relay wireless network containing three nodes. The terminals want to exchange the data using the auxiliary relay. This model is also often used to demonstrate the power of the wireless network coding, since it is clearly sufficient to consider two time slots to exchange the information1 . It nicely demonstrates the difference between the conventional P2P orthogonal channels, network coding provided upon orthogonal P2P models and the wireless network coding. • Orthogonal P2P based communication without interference, where one needs four time-slots to ex- change the information between the terminal nodes, particularly 1. source SA transmits to R data bA that are decoded in relay, i.e. SA(bA) → R(bA), 2. SB(bB) → R(bB), 3. R(bA) → DA(bA), 4. R(bB) → DB(bB). Each time-slot is occupied with a conventional P2P transmission without any interference. The transmission can thus operate on the rates corresponding to the capacity of the channels. • P2P communication with network coding requiring three time slots for the full exchange, i.e. 1. SA(bA) → R(bA), 2. SB(bB) → R(bB), 3. R(bA ⊕ bB) → DA(bA ⊕ bB), DB(bA ⊕ bB). The destinations are able to decode the desired data utilizing the identity bA = bA ⊕ bB ⊕ bB and analogously for bB. The first two time slots are equal to the previous case and the third one can also operate on the channel capacity considering symmetric channels. • Wireless network coding demanding only two time-slots for full data exchange, particularly 1. SA(bA), SB(bB) → R(bA ⊕ bB) 2. R(bA ⊕ bB) → DA(bA ⊕ bB), DB(bA ⊕ bB). The first time slot is occupied by a multiple-sources transmission (MAC) channel. The second order cut-set bound is avoided by decoding of bA ⊕ bB instead of the joint pair (bA, bB) decoding. The performance within half bit below the first order cut-set bound was proven to be achievable in [38] for AWGN channels. One can see that the main advantage of the wireless network coding in TWRC is in the bidirectional communication, where the symmetry of the information amount exchange is assumed. 1With some further assumptions. 64
- 78. CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION SA/DB R SB/DA Figure 6.2: Two way relay channel. SA SB R DB DA Figure 6.3: Butterfly network. Note 6.2 (Hierarchical Function). We used operation ⊕, which is viewed as addition upon a given finite field, typically bitwise XOR upon GF(2), relying that the destination is able to decode the mixed stream bA ⊕ bB using its data. This can be generalized to an arbitrary function enabling the decoding exploiting own data. Such a function is called hierarchical function or also hierarchical map. Note 6.3 (Hierarchical Decoding). The possibilities of the direct decoding of the hierarchical function (hierarchical stream) bAB = Xb(bA, bB) = bA ⊕ bB in relay are the main topic of the this part (Chapter 7). Note 6.4 (Relation of Hierarchical Function and Pure Network Coding). Since the hierarchical function is a function of the data streams, one can be confused with the conventional network coding basically providing a function upon data as well. The difference is that the conventional network code is provided upon an estimation of whole data packets that are jointly decoded, i.e. the joint decision about all data streams was first performed and consequently the higher order cut-set bound limits the performance. The hierarchical function gives the function of data that are to be decoded (no joint decision is required). It is assumed only a decision about the hierarchical data that are some function of the source data streams and the higher order cut-set bound can be therefore overcome. Butterfly Network The butterfly network is a generalization of the TWRC, where we assume that the destinations do not have available the complementary information directly, but the information is received via a wireless channel as shown in Fig. 6.3. The complementary information flowing from the sources directly to the destinations called Hierarchical Side Information (H-SI) can be imperfect. The hierarchical function provided in relay must properly respect the H-SI such that the desired data should be computable from the hierarchical stream bAB = Xb(bA, bB) and H-SI (some function of bA in DB and bB in DA) in the final destinations. Two-Relay Single-Destination Network The two-relay single-destination network (Fig. 6.4) is a further generalization of the butterfly network. It is often assumed that the direct links are not available for simplicity. The extension against the butterfly network consists in a generalization of the H-SI from one relay point of view, which is formed by the hier- archical operation in the complementary relay. Indeed we have two hierarchical streams bAB = Xb(bA, bB) 65
- 79. 6.1. WIRELESS NETWORK SA SB R1 R2 D Figure 6.4: Two source, two relay, single destination network without direct source-destination H-SI links. SourceTerminals 1stRelayLayer ... LastRelayLayer DestinationTerminals Figure 6.5: Acyclic layered network. in relay R1 and ¯bAB = ¯Xb(bA, bB) in relay R2. These hierarchical streams must jointly contain all infor- mation about the tuple (bA, bB) to keep the decoding possible in the final destination, i.e. some decoding function [bA, bB] = X−1 b (bAB, ¯bAB) must exists. The overall hierarchical function (formed by Xb and ¯Xb) can provide either the minimal possible in- formation, i.e. the vectors bAB, ¯bAB are independent and the system has no diversity. If the information is larger than the minimal one, the hierarchical data streams are bAB, ¯bAB correlated and the system might utilize the correlation of the received symbols in the final destinations. A typical example of the minimal possible information is TWRC or the butterfly network with perfect H-SI. On the other hand, the correlation occurs if both relays jointly decode both data streams bA, bB. General Acyclic Network A wireless network coding generalization brinks couple of complications. The hierarchical functions must be defined in each relay such that each destination receives a proper information for its purposes. Due to the complexity of the design, we proposed a unified framework [58], where the problem of the overall network design is decomposed to several sub-task. The core sub-task is the wireless aware network coding determining the hierarchical functions according to both the desired end-to-end information flows and the constrains given by the wireless network. A simplified model of the general acyclic network is a layered model (Fig. 6.5), where we assume a radio visibility just among the neighboring layers of nodes. Multi-Hop Relay Channel The multi-hop relay channel model (see Fig. 6.6 for two hops example) is a single-source scenario. It is typically associated with the latency problem. The point is that the relay must wait for a whole sequence followed by the decoding process in case of DF strategy, while the AF strategy works symbol-wisely and therefore no latency is accumulated in relay. The performance difference between the DF and AF presents a playground for some strategies (CF-based) providing better performance than AF strategy, but with lower latency than DF strategy. 66
- 80. CHAPTER 6. PHYSICAL WIRELESS NETWORK COMMUNICATION S R D Figure 6.6: Multi-hop relay channel with two hops. 6.1.7 Challenging Problems in Wireless Networks Within this Chapter, we briefly summarized the principle of the wireless network coding up to now taking accent mainly on the hierarchical decoding. Some further challenges in the wireless networks will be pointed out. These challenges are given by the tasks required for the cross-layer design in general. Nevertheless, we point out the challenges that are related directly with the goal of the thesis. Wireless Aware joint Network Coding The utilization of the network coding paradigm in the wireless network extension is available only in simple networks. The challenge is to design a method solving an ideal hierarchical maps distribution within a general wireless network such that some criteria are optimized (e.g. maximal utilization of the network coding paradigm still enabling desired decoding in final destinations). This solution should be moreover achieved in a distributive way. Note that the proposed block structured layered design (Chapter 7) can be applied to provide the hierarchical maps and the network-consistent channel coding design for some given coding rates, which can be derived from particular channel capacities. The next challenge is to derive a staging algorithm assigning properly stages when and how long each of the node should receive/transmit such that the power resources are utilized as efficiently as possible. Channel Parameterization The pure network coding directly shows how to utilize the network coding paradigm. Nevertheless, the utilization is quite complicated in the signal space, because the network coding is no longer provided upon the finite alphabets, but upon the complex-valued signal space. The channel is moreover parameterized by further constants (e.g. complex valued fading). Since the parameterization is related to more sources, it is not possible to avoid some crucial combination of the channel parameters purely by the processing in relay. It means that the constellation seen in a node receiving more signals (typically relay) is parameterized by a channel parameter, which can cause a constellation distortion decreasing the (hierarchical) channel capacity (available rate). The way leading to solve this problem is twofold. One can consider some feedback to sources (e.g. channel pre-rotation [26]) or to design some alphabet that is resistant to the vulnerability of the performance with respect to the channel parameterization [59]. Synchronization and Other Real Implementation Problems Of cause, there is plenty of problems that are beyond the scope of this thesis. As an instance, we assume a signal space description of the signal in network, which means that both the time and frequency syn- chronization are available in each node. It is however quite challenging problem to achieve this without a central unit. 67
- 81. 7 Network Channel Code Design The relay strategies based on a decoding (hard decision) of a function of the source data streams (e.g. bitwise XOR) were shown to move ahead the system performance in wireless networks. A proper implementation of these relay strategies needs to consistently incorporate channel coding across the network. Such an implementation is available only for special cases (mainly symmetric rates and equal alphabet cardinalities) in the current state of art. This Chapter presents a composition of the channel encoders in sources as well as the corresponding relay decoder for an arbitrary (asymmetric) individual source rates and for the corresponding (hierarchical) function of the source data streams. The proposed design consists of a proper mixture of the already known symmetric-rate based decoder design and the conventional joint decoding. We develop a unifying (linear) description of the processing in wireless networks. Utilizing this description, the proposed design results in a block-diagonal matrix composition preserving the (good) properties of the block elements (e.g. LDPC codes). The proposed design is demonstrated in a simple example and verified by the numerical simulation. 7.1 Introduction to Wireless Network Coding The coding and modulation design in a general wireless physical network arose as a huge challenge recently. A pioneering work [1] showed that the information content can be temporarily mixed (network coded) within the network to increase either the overall network throughput (capacity) by means of reduced wireless channel use or the system diversity. An extension of the network coding (NC) to include also the wireless environment properties (wireless network coding (WNC) a.k.a. physical layer network coding [26,30,39]) then brings additionally further potential performance improvement. The idea of WNC consists in decoding (providing hard decision) of a (hierarchical) function of source data streams directly in constellation space [40] and forwarding it further, thus it is named Hierarchical Decode (Compute) and Forward strategy (HDF). The hierarchical function of source data streams (hierarchical stream) contains less information than the joint source data streams. Providing a hard-decision on the hierarchical stream, the classical Multiple Access Channel (MAC) region [11,15] restrictions limiting the conventional joint-decoding based strategies does not longer limit the performance. Since the hierarchical stream contains less information than the joint source data streams, the requested complement of this information must be delivered into the final destination by alternative paths in order to secure a reliable end to end transmission. This request is reflected e.g. by a condition that the final destination must collect enough number of equations from all paths in [40]. The information amount flowing by the alternative paths is called Hierarchical Side Information (H-SI). A proper utilization of the WNC with HDF across the network potentially takes advantage of both reduced wireless channel use and operation rates beyond the classical MAC regions. This utilization is, however, not trivial in a general wireless network. The network design contains a joint optimization over immense degrees of freedom that has to be solved. Assuming a given known network state information and a perfect time synchronization among all nodes in the network, the overall optimization can be partitioned to some interacting subtasks [58], particularly (1) network information flow determining the hierarchical functions in each relay, i.e. the information routing that is aware of the wireless environment, (2) network- consistent channel encoders design respecting properly the network information flow as well as the maximal possible achievable coding rates resulting from the network state information and (3) signal space mappers connecting the codewords with the wireless environment. One can also consider various metrics to be optimized, e.g. the overall throughput maximization. The closed-form solution for such an optimization is not available in the current state-of-art. The available literature offers the solution either of the cases with some very specific circumstances, typically small wireless networks with fixed network information flows, or a deeper insight in some particular sub-task without taking into account the others. One can recognize two major approaches. The first one 68
- 82. CHAPTER 7. NETWORK CHANNEL CODE DESIGN serves mainly for proving the achievability of a performance by means of the information theory tools, typically the lattice codes [17]. Some works based on this approach show very nice results, e.g. [38], where the authors proof the achievability of the performance within a half bit under the first order cut-set bound in Gaussian Two-Way Relay Channel (TWRC). The lattice code based models are, however, not suitable for a direct implementation. The second approach is therefore more constructive and directly implementable. It is based mainly on conventional capacity achieving channel codes and signal space mappers for a given network information flow design. Design of the directly implementable consistent channel encoders across the wireless network for arbitrary channel code rates, signal space mappers and network information flow stands for the main focus of this work. To best of our knowledge, no such design enabling arbitrary asymmetric rates is available in the current state of art (excepting our initial approach [45] and traditional joint decoding followed by a network code). 7.1.1 Related Work Throughout the literature, it is very common to assume a special case of a uniform hierarchical ratio1 . The uniform hierarchical ratio allows (among others) a popular bit-wise XOR of the source data streams as the hierarchical function [7, 8, 30, 64, 65], i.e. the traditional understanding of the (wireless) network coding. The hierarchical ratio is classified by means of the relay-alphabet cardinality in [54]. We stress that a commonly used TWRC as well as its generalization the butterfly network without a direct link from sources to destinations does not allow arbitrary hierarchical ratio in relay, because the H-SI cannot contain an arbitrary information amount (maximally one data stream). The case of maximal information flow through the H-SI (implicitly satisfied in TWRC with symmetric rates) allows the uniform hierarchical ratio case referred as the minimal hierarchical cardinality (meaning minimal information flow through the relay) in [54]. The consistent channel encoders design in the network is basically joint channel-wireless network coding that is able to benefit from both reduced channel use and the operational rates beyond the classical MAC region. Several works focus on the joint network-channel encoding (see e.g. [4, 20, 22]). These works are typically provided upon some graphical models (e.g. Tanner graph in [22] or factor graph in [20]). The low density parity check (LDPC) channel encoding technique is commonly used as the channel encoding part (e.g. in [20]). Linearity of both network and channel coding is standardly assumed. None of these works allows to work with arbitrary channel rates and hierarchical ratios. The joint channel-wireless network coding design appears in literature only for very simple networks with further restrictions. These restrictions are typically symmetric channel rates and the uniform hierarchical ratio. For example, the authors of [30] propose both optimal and suboptimal soft metric based decoder relay design for TWRC, where the source encoders are assumed to be equal. The optimal design relies on the repeat-and accumulate codes in [30]. A similar design is proposed in [64], where the optimal design is built upon higher-order LDPC codes and the turbo codes based optimal relay decoder design is proposed in [8]. All of these optimal decoder designs rely on equal channel encoders in sources and uniform hierarchical ratio. It is common to assume memoryless wireless channels connecting the nodes as well as a symbol by symbol mapping of the coded symbols to signal space symbols across a whole network2 . In such a case, a symbol-wise processing can be applied in soft output demodulator (SoDeM) in each node to produce a soft metric for the channel decoder providing hard decision about a required hierarchical stream for a given node. The symbol-by-symbol metric evaluation enables a separate investigation of the channel symbol 1Notice that the hierarchical ratio determines the information amount flowing through the relay and it will be formally defined later in Definition 7.1. 2Notice that some works assume more general assumptions, e.g. more generic channels with inter-symbol interference caused by imperfect time synchronization, or the signal space mapping in sources that is over more than one symbol. Nevertheless none of these works allows the joint network-channel encoders design for an arbitrary source rates. 69
- 83. 7.2. NOTATION & DEFINITIONS processing, where channel coding cannot be straightforwardly added (e.g. [26]). The symbol-by-symbol evaluated soft metric can be either optimal, i.e. a sufficient statistic for the hierarchical data (e.g. joint likelihood), or suboptimal (typically some marginalization of the optimal metric). The use of a suboptimal soft decoding metric is motivated by simplicity of the relay decoder implementation (see e.g. [30,54]). A construction of the relay decoder for an optimized (even non-linear) hierarchical function and LDPC encoders in sources in TWRC model is proposed in [31]. The assumption strictly limiting this work is that the non-zero elements of the LDPC matrix must be on the same position in all sources and relay LDPC matrices. This design thus does not allow arbitrary individual rates of sources. 7.1.2 Goals & Contribution The main goal of this part of the work is to propose a network-consistent channel encoders design for a generic class of network structures respecting arbitrary network information flows, arbitrary (including asymmetric) rates of the channel encoders and arbitrary signal space mappers. We develop a unifying robust network description based on linearity of the information flow as well the linearity of the channel encoders across a whole network. The available hierarchical data stream decoding options are revised by means of the proposed description. The advantages of the available strategies are combined in order to fulfill the overall goal, that is the network-consistent channel encoders design capable to incorporate arbitrary rates, information flows given by the hierarchical ratio and signal space mappers. We investigate some basic properties of the proposed network-consistent channel encoders design and numerically verify its performance. 7.1.3 Structure of Chapter The rest of this Chapter is structured as follows. Notation and basic definitions are found in Sec. 7.2. The system description is addressed into Sec. 7.3. The following Section 7.4 aims with classification of the relay decoding options from the optimality, robustness and complexity point of view. The core of this chapter is presented in Sec. 7.5, where we describe a consistent directly implementable channel encoders construction concept across the whole network allowing to process arbitrary channel code rates as well as arbitrary hierarchical ratio. This construction is built upon the description in Sec. 7.3, where the building blocks are described in Sec. 7.4. We discuss also some properties of the proposed design. The proposed design is exemplified in Sec. 7.6 and some numerical verifications inspecting the performance of the proposed design can be found in Sec. 7.7. Some generalizations are addressed into Sec. 7.8 and finally the work is concluded in Sec. 7.9. 7.2 Notation & Definitions We remind the notation that is used within this Chapter. A vector is denoted by a bold-face lower-case letter and it is implicitly assumed to be in a column form. It means that a vector [b1, . . . , bk]T is denoted b. A zero vector with a single non-zero unit value on the ith position is denoted by ei, for example e3 is a vector given by [0, 0, 1, 0, . . . ]T . If the order of a particular symbol in the vector is not important or clear from the context, the subscript is dropped, i.e. b instead of bi. Similarly a matrix is denoted by a bold upper-case letter. For example Xk×n = [x1 . . . xn] is k × n matrix composed from n vectors x1, . . . , xn, where each of them is k symbols long. If the matrix is square, its dimension is denoted by a single index, that is Xk×k = Xk. The matrix dimension in the subscript is omitted whenever the matrix dimension is clear from the context. A zero matrix 0 is a matrix containing all zero elements and a matrix E denotes the unitary matrix that can be written as E = [e1, e2, . . . ]. A matrix Xk×n is said full-rank if rank(X) = min(k, n). 70
- 84. CHAPTER 7. NETWORK CHANNEL CODE DESIGN Source data are conventionally associated with letter b and their length with k, similarly the codewords are denoted by c with associated length denoted n. The domain of both source data and codewords is an extended Galois Field GF[qm ], where the base q may differ for data and codewords as well as m. Throughout this work, we assume binary data, but an arbitrary extended GF can be also employed. The signal space points are associated with s and the soft metric outputting the SoDeM is denoted by µ. The operator Sb stands for a set of all possible vectors b and its size is denoted Mb = |Sb|. In case of binary independent and identically distributed vector b ∈ GF[2k ], the size is Mb = 2k 7.3 System Description We partly follow the partitioning suggested in [58] to describe the system. It means that we start with the information flow description, then we add the channel coding and signal space connection. A considerable part of the investigated design is demonstrable in a simple 2-source 2-path network scenario. It means two sources, one stream flowing through a relay and one complementary stream flowing by an another way to a final destination. The complementary link, i.e. H-SI, is not closer specified, it may either flow though a complex network or be directly presented in the final destination. We will consider the simple 2-source 2-path network scenario in the main part of this work. Some generalizations are addressed into Sec. 7.8. 7.3.1 Information Flow Description Assuming two informational sources A and B. The source data are independent and identically distributed (iid. ) vectors bA and bB of lengths kA and kB that is bA ∈ GF[2kA ] and bB ∈ GF[2kB ]. We assume linear only hierarchical function. It means that the hierarchical data stream bAB of length kAB flowing through the relay can be expressed as bAB = Xb,AbA + Xb,BbB, (7.1) where the hierarchical function is described by kAB × (kA + kB) matrix Xb = Xb,A Xb,B . The complementary data stream (H-SI) is formally denoted by an over-bar symbol and the H-SI data stream is given similarly by ¯bAB = ¯Xb,AbA + ¯Xb,BbB. The overall information flowing through the whole network (both relay and its complement) can be written as bAB ¯bAB = Xb ¯Xb bA bB = Xb,A Xb,B ¯Xb,A ¯Xb,B bA bB , (7.2) where the overall matrix must be invertible to keep the source data fully decodable in the final destination. 7.3.2 Channel Coding upon the Information Flow Due to the error protection reason, we consider a channel encoding in the network. The source data vectors are encoded by linear encoders cA = GAbA with coding rate rA = kA/nA and cB = GBbB with coding rate rB = kB/nB. The hierarchical codewords cAB are related to the hierarchical data by a virtual (generally non-linear) encoder such that cAB = CAB(bAB) with rate rAB = kAB/nAB. We stress that domains of bA, bB and bAB are not equal in general and therefore it is not straightforward how to find CAB given GA and GB. The relation between the source and hierarchical codewords is also given by a vector codeword hier- archical function Xc as cAB = Xc(cA, cB). In order to enable a simplified soft-metric evaluation, we also 71
- 85. 7.3. SYSTEM DESCRIPTION bA bB GA SA GB SB + w · hB · hA SoDeM, Decoder cA sA cB sB x ˆbAB ˆbA ˆbB bAB ¯bAB Destination Figure 7.1: 2-Source 2-path scenario. The goal of this work consists in a joint design of both source encoders and relay decoder for arbitrary source rates as well as for an arbitrary hierarchical ratio. define a symbol-wise hierarchical codeword function Xc(cA, cB) as the ith element of the vector hierarchical codeword function Xc(eicAi, eicBi). The hierarchical symbol is then given by3 cAB = Xc(cA, cB). The hierarchical codeword satisfies cAB = Xc(cA, cB) = Xc(GAbA, GBbB) (7.3) = CAB(bAB) = CAB(Xb,AbA + Xb,BbB), (7.4) where the codeword hierarchical function is used in (7.3) and the virtual hierarchical encoder in (7.4). If both the codeword hierarchical mapping Xc and the virtual encoder CAB are linear, the equations (7.3) and (7.4) can be written in the matrix form cAB = Xc,AcA + Xc,BcB = Xc,AGAbA + Xc,BGBbB (7.5) = GABbAB = GAB(Xb,AbA + Xb,BbB). (7.6) Our goal to have the consistent channel encoders respecting properly the information flow can be now stated as follows. For arbitrary, but given source rates and hierarchical ratio, find consistent generator matrices4 with good5 corresponding parity check matrices and the inner structure of the hierarchical functions (hierarchical matrices). Dimensions and ranks of the matrices must be in accordance with the requested rates. 7.3.3 Connection with Signal Space The hierarchical data stream is stochastically connected though the hierarchical codewords with the source codewords. The connection can be expressed by conditional probability p(x|cAB), where x is an observation available in the relay. The signal space space symbols sA(cA), sB(cB) are given by signal space mappers in sources. We assume a memoryless channel, i.e. the relay receives a noisy superposition of these x = hAsA + hBsB + w, where hA, hB is flat fading channel parametrization and w is complex zero mean additive white Gaussian noise (AWGN) with unit variance. The variance of the AWGN can be adjusted by a proper scaling of the channel gains γA = |hA|2 and γB = |hB|2 . The overall signal to noise ratio (SNR) available in relay is thus given by SNR = γA + γB assuming E[|sA|2 ] = E[|sB|2 ] = 1. See Fig. 7.1. 3This relation is valid only for a memoryless channel, where the symbol cABi depends only on cAi and cBi. Otherwise, this symbol-wise hierarchical mapping has no sense. 4The consistent generator matrices are channel encoders in sources and decoder in relay designed in such a way that the decoder in relay properly decodes the desired hierarchical data stream in relay. See Def. 7.2 for a formal definition. 5E.g. LDPC matrix with proper degrees distribution and maximized girth. 72
- 86. CHAPTER 7. NETWORK CHANNEL CODE DESIGN bA bB bAB ¯bAB ˆbA, ˆbB Cut #1 Cut #2 Cut #3 Figure 7.2: Visualization of the information flow within the network. The information amount flowing through the relay (bAB) is visualized by the Cut #1, the information flowing through H-SI is denoted Cut #2 and the overall information that needs to be delivered into a final destination(s) is denoted by Cut #3. One can see that the Cuts #1 and #2 can be seen as a multi-source broadcast channel. Setting the hierarchical ratios λ and ¯λ, one can adjust the information amount flowing though particular Cuts. 7.3.4 Classification of Information Flow in Relay The information flow through the relay can be expressed by means of entropy of the hierarchical stream, that is H(bAB) = rank(Xb) for binary uniform iid. bA and bB (see Cut #1 in Fig. 7.2). Note that for full-rank XB, rank(Xb) = kAB. We can also consider the overall information amount related to the number of channel use measured by length of the hierarchical codeword cAB, i.e. rank(Xb)/nAB. The wireless network is desired to deliver H([bT AbT B]) = kA + kB bits into the final destination (see Cut #3 in Fig. 7.2). We therefore define the hierarchical ratio determining a portion of the overall information to be processed by the relay. The ratio λ = rank(Xb) kA + kB (7.7) is called the hierarchical ratio for a given relay. Definition 7.1 (Hierarchical Ratio). If the matrix rank(Xb) is full-rank, we can express the hierarchical ratio as λ = kAB kA + kB (7.8) In oder to reliably deliver kA + kB bits into the final destination, the H-SI must process at least the complement of the information amount flowing through the relay. This amount is given by rank( ¯Xb) ≥ kA + kB − rank(Xb) bits (Cut #2 in Fig. 7.2). We can also define complementary hierarchical ratio as ¯λ = rank( ¯Xb) kA + kB ≥ 1 − rank(Xb) kA + kB . (7.9) Now the diversity of the system is given by λ + ¯λ − 1 and it takes values from the interval 0, 1 , where 0 denotes no diversity and 1 refers to full diversity. Notice that in case of non-zero diversity, the correlation of the streams bAB and ¯bAB is implied. 73
- 87. 7.4. RELAY DECODING CLASSIFICATION A special case of full-rank hierarchical matrices and kA = kB = kAB refers to the uniform information flow, because the information flowing through both the relay and the H-SI is equal assuming no diversity. This case enables an elegant solution of the joint network channel encoder, where GA = GB = GAB and this will be discussed later. The paper [54] classifies the information flow through the relay according to the relay alphabet cardi- nality, which is equal to our entropy-based classification for iid. data. They assume a symmetric butterfly network, where the H-SI is given maximally by one data stream. It means that the minimal information flowing though the relay must be the second data stream. The classification is concretely given by: • minimal cardinality: MbAB = max(MbA , MbB ), • extended cardinality: max(MbA , MbB ) < MbAB < MbA MbB • full cardinality: MbAB = MbA MbB . 7.4 Classification of the Relay Decoding in Wireless Networks with HDF The HDF technique requires a decision about the hierarchical stream bAB. The main area of interest of this work lies in all processing (decoding) between the vector x received in relay (the superimposed signal from several sources parameterized by the sources-relay channels) and the estimation of the hierarchical stream bAB in relay. The goal of the hierarchical decoder in HDF strategy is to obtain the hierarchical stream bAB from the observation x. The decoder can be classified as the joint-network channel decoder. The objectives of this decoder can be summarized as: • Performance properties. • Implementation complexity. • Robustness of design, that is an ability to properly incorporate arbitrary rates and hierarchical ratio.. There are basically three major approaches differing in the metric used for the decoder design. 7.4.1 Symbol-Wise Metric Evaluation for they Decoder in Relay As it was stated, the goal is to obtain the hierarchical data stream bAB. The hierarchical data are related with the hierarchical codewords by a virtual hierarchical channel encoder6 CAB. Assuming a memoryless channel, it is reasonable to evaluate a symbol-wise soft metric in demodulator due to the complexity reasons. Optimal Symbol-Wise Metric The MAP rule gives P(bAB|x) = P(cAB|x) ∼ p(x|cAB)p(cAB). (7.10) 6We assume just that such an encoder exists. A particular form of the encoder will be discussed later. 74
- 88. CHAPTER 7. NETWORK CHANNEL CODE DESIGN Using the vector codeword hierarchical function, we can evaluate the conditional likelihood by p(x|cAB) = cA,cB:Xc(cA,cB)=cAB p(x|cA, cB) (7.11) = cA,cB:Xc(cA,cB)=cAB i p(xi|cAi, cBi). (7.12) Therefore the symbol-wise metric p(xi|cAi, cBi) is optimal with respect to the ML as well as the MAP detection7 . Hierarchical Symbol-Wise Metric The hierarchical metric p(x|cAB) is given by a marginalization p(x|cAB) = cA,cB:Xc(cA,cB)=cAB p(x|cA, cB). (7.13) This metric is, however, not the exact one for the data vector decoding purposes. The probability of the observation conditioned by the hierarchical codeword is not generally independent for different time instances. The use of the symbol-wise hierarchical metric is therefore unavoidably connected with the approximation p(x|cAB) ≈ i p(xi|cABi). (7.14) Note that in case of uncoded system and uniform hierarchical ratio, the approximation in (7.14) becomes to equality. Network Coding over Joint Decoding An another approach is to evaluate first hard decisions ˆbA, ˆbB and using these decisions to apply the classical network code. The metrics used in the demodulator are p(x|cA) and p(x|cB) obtained by a proper marginalization from p(x|cA, cB). Of cause, one can consider a conventional Successive Decoding - Interference Cancellation (SD-IC) method to improve the performance (to achieve the conventional MAC region). 7.4.2 Decoding Strategies in Relay The different soft metric evaluations in relay enables different decoding strategies. Joint Decoding with Classical Network Coding This is a natural sub-optimal basic solution, where no advantage of the wireless network coding is used. The decoder uses a conventional joint decoding in MAC. Upon this hard decision, the network operation described by the matrices Xb,A and Xb,B is performed separately. This approach can be straightforwardly extended to both more sources and more paths. The perfor- mance is limited by the conventional MAC region. Given matrices Xb,A and Xb,B, this strategy works regardless both the hierarchical ratio and the rates of the encoders in sources. Formally, the codeword hierarchical matrices are unitary matrices Xc,A = EnA , 7Assuming a uniform distribution of cAB. 75
- 89. 7.4. RELAY DECODING CLASSIFICATION SoDeM HA HB NC x µ(cA) = p(x|cA) µ(cB ) = p(x|cB ) ˆbA ˆbB bAB Figure 7.3: The relay SoDeM-decoder scheme for the joint decoding over network code. Soft output demodulator evaluates symbol-wisely p(xi|cAi) and p(xi|cBi) for all i. This metric is served to the separated decoders HA and HB. These decoders provide a hard decision about ˆbA and ˆbB. This is then used to compute the hierarchical stream in NC block by ˆbAB = Xb,A ˆbA + Xb,B ˆbB. SoDeM HA,B NC x µ(cA, cB) = p(x|cA, cB) p(x|bA, bB) bAB Figure 7.4: The optimal relay SoDeM-decoder scheme for the joint decoding and soft network code. Soft output demodulator evaluates symbol-wisely a sufficient statistic metric p(xi|cAi, cBi) for all i. This metric is served to joint decoder HA,B providing a soft metric p(x|bA, bB). This metric is then used to compute the hierarchical stream in soft NC block by (7.17). Xc,B = EnB and the decoding taking into account the codeword hierarchical relation (7.5) is solved separately according to cA = Xc,AcA = EnA cA = cA = GAbA, (7.15) cB = Xc,BcB = EnB cB = cB = GBbB, (7.16) where the marginalized metrics p(xi|cAi) and p(xi|cBi) are evaluated by SoDeM. This separated decoding leads to decision upon the both data streams, that is ˆbA and ˆbB. Finally, the hierarchical data stream is given by ˆbAB = Xb,A ˆbA + Xb,B ˆbB. This setup is shown in Fig. 7.3. Optimal Solution - Joint Decoding with Network Coding This method stands for the optimal one (Fig. 7.4), where the meaning of the optimality is explained in Note 7.2. The demodulator in relay evaluates symbol-wisely the metric p(x|cA, cB) (7.12) as a sufficient statistics for the hierarchical data. Using the evaluated optimal metric p(x|cA, cB) ,the joint decoder evaluates the metric p(x|bA, bB) for the soft network decoder. The soft network decoder provides a marginalization p(x|bAB) = 1 p(bAB) bA,bB:bAB=Xb,AbA+Xb,BbB p(x|bA, bB) (7.17) which is then passed to the hard decision as ˆbAB = arg max bAB p(x|bAB). (7.18) The problem of this method is in evaluation p(x|bA, bB), which is often numerically intensive. Moreover, this brute force solution needs to design all encoders and decoders jointly, because the relay-decoder properties will be determined by different source encoders. The complexity of the optimal decoder will rapidly grow with more considered sources. 76
- 90. CHAPTER 7. NETWORK CHANNEL CODE DESIGN SoDeM HAB x µ(cAB) = p(x|cAB) bAB Figure 7.5: The relay SoDeM-decoder scheme for the hierarchical decoding. Soft output demodulator evaluates symbol-wisely p(xi|cABi) for all i. This metric is served to one hierarchical decoder HAB. This decoder directly provides a hard decision bAB. To formally describe this solution, we use a similar formalism as in [64] that is both codewords cAB and joint data pair (bA, bB) take values from the extended GF with four elements8 in {0, 1, D, D + 1}. The codeword hierarchical matrices are assumed to be the matrices upon the extended GF given by Xc,A = EnA , Xc,B = DEnB and the decoding with (7.5) can be expressed as cAB = EnA cA + DEnB cB = GAbA + DGBbB. (7.19) The decoder aims with a joint soft metric about9 bA + DbB that serves as the soft output of the decoder, that is p(x|bA, bB) in (7.17). In a special symmetric case of uniform hierarchical ratio and equal encoders in sources, that is GA = GB = G, we obtain for (7.19) cAB = G(bA + DbB) (7.20) and a straightforward decoder construction evaluating p(x|bA, bB) can be applied. Such a decoder construction on LDPC codes is proposed in [64] and similar construction based on turbo encoder is proposed in [8]. Example 7.1 (Optimal Solution - Uniform Hierarchical Ratio). Note 7.2 (Optimality of Joint Decoding with Network Coding). The optimality of the processing is with respect to obtain as much information as possible. It means that the soft metric p(x|bA, bB) contains all available information for the hierarchical data stream decision. Nevertheless the optimality is paid by the complexity of the decoder. Direct (Hierarchical) Decoding The direct (hierarchical) decoding uses the hierarchical metric p(xi|cABi) in the demodulator and the hierarchical stream bAB is evaluated directly from the hierarchical metric. See Fig. 7.5. The capacity regions are computed in [54] for the hierarchical MAC phase in case of uniform hierarchical ratio. The metric p(x|cAB) is suboptimal, but a direct evaluation of the hierarchical stream brings an efficient solution, because it is required only one run of the decoder. The problem of the this type decoding is to find a consistent encoders and hierarchical maps such that both (7.5) and (7.6) are satisfied. A straightforward construction of consistent hierarchical functions and encoders is not obvious in general. 8We assume that both cA and cB take values from binary extended finite fields. 9If the vector lengths are not equal, that is kA = kB, the shorter vector is filled by zeros such that the dimensions will pass. 77
- 91. 7.5. BLOCK-STRUCTURED LAYERED DESIGN EkA1 Ek20 0 0 0 Xb,A EkB1 Ek20 0 0 0 Xb,B k2kB1kA1 kA1 k2 kB1 k2 Figure 7.6: The dimension of hierarchical matrices in the proposed block structured layered design. Notice that the structure of the codeword hierarchical matrices is equal with dimensions n instead of k . A simple, however very special solution, is available for uniform hierarchical ratio. Then the design in a particular form containing equal encoders GA = GB = G and unitary the hierarchical functions Xb,A = Xb,B = E, Xc,A = Xc,B = E implies cAB = G(bA + bB). (7.21) One can check that both (7.5) and (7.6) are fulfilled and therefore this network design stands for a valid network configuration (see e.g. [54]). Example 7.3 (Direct Decoding - Uniform Hierarchical Ratio). It is obvious that this approach will not work for arbitrary hierarchical ratio and asymmetric codes. 7.5 Block-Structured Layered Design Our approach aims with a maximal utilization of the methods suited for uniform hierarchical rate, because we are able to design either optimal (Section 7.4.2, particularly Example 7.1) or hierarchical (direct) (Section 7.4.2, particularly Example 7.3) channel encoder implementation for such a case. To do this, we restrict slightly the structure of the encoding matrices into block-wise form. The inner matrices of the block-structured design can be arbitrary valid capacity achieving single-user linear block codes and thus the layered design paradigm is preserved. Those component codes can be e.g. LDPC codes and the overall design preserves the sparsity of the overall matrices. An arbitrary hierarchical ratio as well as arbitrary source rates can be achieved by a proper mixture of the block-matrices dimensions. 7.5.1 Formal Definition We formally define our goal to have a valid network configuration. It means that GAB must properly define the virtual hierarchical encoder, i.e. (7.5) must be satisfied jointly with (7.6) for all bA, bB. 78
- 92. CHAPTER 7. NETWORK CHANNEL CODE DESIGN GA1 G20 0 GA kA1 k2 n2nA1 GA1 GB1 G20 0 0 0 0 0 GAB GB1 G20 0 GB kB1 k2 n2nB1 Figure 7.7: Diagonal block encoders composition. We call a set of full-rank generator matrices {GAB, GA, GB} jointly with full-rank hierarchical matrices {Xb,A, Xb,B, Xc,A, Xc,B} satisfying GABXb = GABXb,A GABXb,B = Xc,AGA Xc,BGB (7.22) a consistent generator set. Definition 7.2 (Consistent Generator Set). We extend the consistent generator set by valid parity check matrices for decoding purposes. The full-rank parity check matrices {HAB, HA, HB} jointly with the consistent generator set are called the consistent parity check set if HA, HB and HAB are valid parity check matrices of GA, GB and GAB. Definition 7.3 (Consistent Parity Check Set). The consistent parity check set formally states our overall goal. Among a number of approaches, we choose a joint network-channel encoding construction that produces a valid network implementation. We introduce orthogonal matrix-blocks GA1, GB1 and G2. One part of both individual data streams is encoded by a joint encoder G2 and the second part of them by separated encoders GA1 and GB1. 79
- 93. 7.5. BLOCK-STRUCTURED LAYERED DESIGN We assume full-rank matrices GA1 : nA1 × kA1, GB1 : nB1 × kB1 and G2 : n2 × k2. If the generator matrices can be written as GA = diag(GA1, G2), GB = diag(GB1, G2), GAB = diag(GA1, GB1, G2) (see Fig. 7.7) and the HNC matrices are in the form (see Fig. 7.6) Xb,A = EkA1 0 0 0 0 Ek2 and Xb,B = 0 0 EkB1 0 0 Ek2 , (7.23) Xc,A = EnA1 0 0 0 0 En2 and Xc,B = 0 0 EnB1 0 0 En2 . (7.24) The matrices are called block structured layered code. Definition 7.4 (Block Structured Layered Code). Note 7.4 (Mixturing Joint Decoding and Uniform Hierarchical Ratio). Notice that E(.)(.)1 matrices define the joint decoding over NC mapping part and E(.)2 matrices define the uniform hierarchical ratio mapping part. The overall map is thus a mixture of the strategies. A proper choice of matrix sizes allows to achieve an arbitrary hierarchical ratio as well as arbitrary source rates We state some basic properties of the block structured layered code. Lemma 7.1. Assuming a block structured layered code, all matrices GAB, GA, GB,Xb,A,Xb,B, Xc,A and Xc,B are full-rank. Proof. The proof of this Lemma is straightforward, because the block-matrices are full-rank from definition and the diagonal composition of the full-rank block matrices as shown in Fig. 7.7 surely preserves the full- rank property. Lemma 7.2. An arbitrary block structured layered code forms a consistent generator set. Proof. The full-rank property is guaranteed by Lemma 7.1. One can verify that (7.22) is also satisfied by substituting Definition 7.4 into (7.5) and (7.6). We show a construction of the parity check matrices based on the block structured layered code satisfying the consistency parity check conditions. We assume an arbitrary block structured layered code and denote HA1 : (nA1 − kA1) × nA1, HB1 : (nB1 − kB1) × nB1 and H2 : (n2 − k2) × n2 parity check matrices of the generator blocks GA1, GB1 and G2 in a full-rank form. Then the parity check matrices HA = diag(HA1, H2), HB = diag(HB1, H2) and HAB = diag(HA1, HB1, H2) form a consistent parity check set. Theorem 7.3 (Block-Structured Layered Code Construction). Proof. The full-rank property of the parity check matrices can be shown similarly as in the generator matrices case in proof of Lemma 7.1. The consistent generator set is formed according to Lemma 7.2. 80
- 94. CHAPTER 7. NETWORK CHANNEL CODE DESIGN To verify that the matrices are valid parity check matrices, we write e.g. for HA HAGA = diag(HA1, H2)diag(GA1, G2) (7.25) = diag(HA1GA1, H2G2) (7.26) = diag(0nA1−kA1,kA1 , 0n2−k2,k2 ) (7.27) = 0nA−kA,kA . (7.28) The same procedure can be applied to both HB and HAB. 7.5.2 Block-Structured Layered Design Properties We now show that the proposed block-structured layered design is indeed able to work with arbitrary source rates as well as with an arbitrary hierarchical ratio. We assume some desired given channel code rates rA, rB and rAB. The channel encoders satisfying the desired coding rates can be composed according to Theorem 7.3 with rates of individual block-parts given by rA1 = kAB − kB nAB − nB , rB1 = kAB − kA nAB − nA , r2 = kA + kB − kAB nA + nB − nAB . Theorem 7.4 (Block-Component Size Adjustment). Proof. Since GA = diag(GA1, G2), kA = kA1+k2 as well as nA = nA1+n2. Similarly GB = diag(GB1, G2) implies that kB = kB1 + k2 jointly with nB = nB1 + n2 and GAB = diag(GA1, GB1, G2) determines kAB = kA1 + kB1 + k2 and nAB = nA1 + nB1 + n2. The particular rates can be obtained expressing kA1 = kAB −kB, kB1 = kAB −kA, k2 = kA +kB −kAB and similarly for codeword lengths n . See Figures 7.6 and 7.7. According to Theorem 7.4, we are able to compose the block decoders for arbitrary source rates rA = kA/nA, rB = kB/nB and for the hierarchical rate rAB = kAB/nAB. In case of hierarchical rate, we have not absolute freedom of choice, because nAB ≤ nA + nB. The rates are connected (for fixed codeword length) through the hierarchical ratio by the relation kAB = λ(kA + kB) (7.29) for full-rank Xb. This relation is a straightforward consequence of (7.8). We now focus how to achieve arbitrary hierarchical ratio. It seems to be useful to classify the hierarchical ratio λ according to some intervals, which will be investigated separately. Particularly • λ ≥ (max(kA, kB))/(kA + kB): The encoders can be composed according to Theorem 7.3 and the requested source rates are adjusted by using Theorem 7.4 for given rA, rB and λ. Notice that this condition can be rewritten for fixed codeword length using (7.29) to kAB ≥ max(kA, kB). • λ ≤ (min(kA, kB))/(kA + kB): The hierarchical ratio in this interval can be achieved adjusting k2 = min(kA, kB) = kAB and set the hierarchical matrices to Xb,A = 0 Ek2 , Xb,B = 0 Ek2 , (7.30) Xc,A = 0 En2 , Xc,B = 0 En2 . (7.31) 81
- 95. 7.5. BLOCK-STRUCTURED LAYERED DESIGN The only block-component part of the decoder in relay is a common encoder G2 and thus all meth- ods of the decoders design across the networks relying on the uniform hierarchical ratio can be applied. The rest of the encoder design in sources can be provided without any restrictions, just according to needs of H-SI. Notice that if this case is considered as H-SI to the previous case λ ≥ (max(kA, kB))/(kA + kB), either Xb,A = 0, Xc,A = 0 or Xb,B = 0, Xc,B = 0. In such a case the relay provides a single-stream decision to have a correct complementary information to H- SI. We can again simplify the condition defining this case to kAB ≤ min(kA, kB) for fixed codeword length. • The remaining case (min(kA, kB))/(kA + kB) ≤ λ ≤ (max(kA, kB))/(kA + kB) can occur only for kA = kB. Assuming without loss of generality that kA < kB, then k2 = kA and n2 = nA. The hierarchical matrices securing the ratio from this range are given by Xb,A = 0 Ek2 , Xb,B = 0 0 EkB12 0 0 Ek2 , (7.32) Xc,A = 0 En2 , Xc,B = 0 0 EnB12 0 0 En2 , (7.33) where the stream bB1 is further divided to [bT B11bT B12] according to the requested hierarchical ratio λ. In this case, the relay decoder provides partly joint decoding of bT B12 and partly decoding enabling the uniform hierarchical ratio based methods. 7.5.3 Signal Space Mappers and Soft Metric for Decoder We leave the optimization of concrete signal space mappers in sources out of its focus. Instead, we address our attention to 1) a concrete way of the signal space symbol composition from the codeword sub-streams and 2) the (symbol-wise) soft metric evaluation that was investigated in Sec. 7.4.1. The proposed block- structured layered code requires just the soft metric for the block-decoders. Due to the memoryless channel, the metric is evaluated symbol-wisely in the SoDeM. The proposed design offers a freedom, how the particular codeword sub-streams are mapped into the signal space. We can consider orthogonal (e.g. time) sharing methods as well as a common signal space symbol that contains information from all streams. Of cause a mixture of these options is also available. In the case of a common signal space symbol, one can see such signal space mapping as superposition coding for the multiple access broadcast channel shown in Fig. 7.2. 7.5.4 Block-Structured Layered Design Convergence Properties In this part, we investigate the overall decoder performance properties. We will be interested mainly in case of hierarchical ratio λ ≥ (max(kA, kB))/(kA + kB), because the other cases can be seen as special cases of this one from the decoder convergence point of view. Due to the block-structured layered design, the block decoders HA1, HB1 and H2 work independently and the overall performance of the hierarchical decoder HAB is limited by the weakest block decoder. The hierarchical decoder HAB therefore operates on the rate of its weakest block sub-decoder. This limitation can be naturally reduced, when bits from individual block encoders are combined into one higher order channel symbol and a SoDeM is applied to evaluate the soft metric for the decoder. A soft information exchange between the SoDeM and the decoder (see Fig. 7.8) is a conventional way of a performance improvement. In our case, the improvement is emphasized by the fact that the soft 82
- 96. CHAPTER 7. NETWORK CHANNEL CODE DESIGN bA1 bA2 bB1 bB2 GA1 G2 SAi GB1 G2 SAI SBi SBI + + w Di + + I = {k : k = i} DI HA1 HB1 H2 ˆbA1 ˆbB1 ˆb2 cA1i cA 2i cB1i cB 2i sAi sBi sABi + xi µA1i µB1i µ2i Figure 7.8: 2-Source 1-Relay system example from bit-wise processing point of view. It is assumed that nA1 = nB1 = n2 for simplicity. The hierarchical decoder is shown in the decomposed form. A soft message µX is capable to carry the extrinsic information about cX evaluated in both the decoder and SoDeM. Consequently, the block decoders are able to utilize the information from other block decoders carried via SoDeMs. SA SB R ¯R D Figure 7.9: System model for the motivational example. information exchange between SoDeM and decoder also enables an information exchange between the independent individual block decoders. Even if one assumes the soft information exchange through the SoDeM, the overall decoder is still limited by the worst block-decoder although with aid from the SoDeM. To reduce this limitation, we can consider interleaving of the codewords in sources prior they are mapped into the signal space to average the encoders convergence properties. The metric evaluated in the SoDeM must be then properly deinterleaved and combined according to needs of a particular method. The information exchange can be described by means of EXIT charts [19, 56]. We assume that the conditions required for the EXIT chart [56] analysis hold, that is the independent input to soft in soft out (SISO) blocks and Gaussian distribution of the systematic log-likelihood ratio (LLR). 7.6 Block Structured Layered Design in Simple Example In order to better understand the proposed concept, we provide its demonstration in a simple wireless network design. We consider a simple two-source two-relay one destination wireless network without a direct link from the sources to destination (see Fig. 7.9). The destination aims with decoding both data streams. We distinguish 3 sub-streams referring to bAB in the relay R, which are formed by the individual streams inputing the encoders GA1, GB1 (bA1, bB1) and a virtual stream b2 = bA2 ⊕ bB2 (upon GF(2)) encoded by G2. The second (complementary) hierarchical stream ¯bAB in the relay ¯R must therefore contain at least either bA2 or bB2 to secure decodability in the final destination. It means that the hierarchical ratio in R is λ ≥ (max(kA, kB))/(kA + kB) (it means extended cardinality in terms of [54]) in our notation. To 83
- 97. 7.6. BLOCK STRUCTURED LAYERED DESIGN IN SIMPLE EXAMPLE λ rA1 rB1 r2 rAB 9/14 1/2 1/6 5/6 9/18 10/14 2/3 1/3 2/3 10/18 11/14 5/6 1/2 1/2 11/18 Table 7.1: Particular block-codes rates for different λ and fixed rA = 1/2, rB = 2/3 evaluated according to (7.34-7.37). make the example more straightforward, we assume given fixed coding rates in sources, i.e. rA and rB. 7.6.1 Adjusting Block Rates Fixing the length of codewords to nA1 = nB1 = n2 = n and considering the case λ ≥ (max(kA, kB))/(kA + kB), we can express the particular block component rates as rA1 = 2rAλ − 2rB(1 − λ) (7.34) rB1 = 2rBλ − 2rA(1 − λ) (7.35) r2 = 2(rA + rB)(1 − λ). (7.36) The virtual hierarchical rate rAB can be shown to be rAB = 2 3 λ(rA + rB). (7.37) Some examples of the block rates evaluation for different λ can be seen in Tab. 7.1. As an example, we fix rA = 1/2 and rB = 2/3 and choose the middle line in Tab. 7.1. We further construct the particular block components as conventional simple block coders on the corresponding rates. We choose the block codeword length n as short as possible, that is n = 3 for given rates and the hierarchical ratio. The block decoders properly respecting the requested rates can be then considered e.g. HA1 = 1 1 1 , HB1 = 1 0 1 0 1 1 (7.38) and H2 = ι ι ι , (7.39) where ι = 1 in case of GF(2) and ι = 1 + D in GF(4) implementation case. The block composition of the hierarchical decoder according to Theorem 7.3 is then given by HAB = 1 1 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 ι ι ι . (7.40) 7.6.2 Signal Space Mapping The encoders and the signal space mappers in sources including the bit mapping jointly with the adequate receiver construction in relay are shown in Fig. 7.10, where the channel parameters are for simplicity considered as hA = hB = 1. The codeword bits are mapped in the order [cA1, cA2, cB1, cB2] to the superimposed modulation seen in relay, that is [blue, violet, cyan, magenta] order of colors in Fig. 7.10. 84
- 98. CHAPTER 7. NETWORK CHANNEL CODE DESIGN The hierarchical (symbol-wise) codeword mapping is therefore given either by10 cAB = [cA1, cB1, cA2 ⊕cB2] (upon GF(2)) or by cAB = [cA1, cB1, cA2 + DcB2] upon GF(4). This can be written using matrices ˜Xc,A and ˜Xc,B in a block diagonal form. The desired block form of Xc,A and Xc,B can be obtained by a column swapping properly reflected in SoDeM. 7.6.3 Soft Metric Evaluation The decoder in relay (see bottom part of Fig. 7.10) incorporates the soft metric symbol-wisely evaluated in the SoDeM. One can see that the particular streams are mapped into a certain position in the superimposed modulation (see different colors in Fig. 7.10) and thus the bits associated to different streams are unequally resistant against the noise. The connection of the decoder and the demodulator can be seen also in Fig. 7.8, where the soft information exchange is shown. The unequal resistance of the streams against the noise can be equalized11 by averaging using interleaver over a whole codeword as shown in Fig. 7.10. This can be simply implemented such that both the codewords cA and cB are interleaved by an equal interleaver of length nA = nB = 2n. The joint demodulator-decoder shown in Fig. 7.10 illustrates the mapping connection for both interleaved and non-interleaved case. In case, where the interleaver is not used, the first bit of the superimposed symbol is always associated with the decoder HA1, similarly the third bit with HB1 and finally both second and fourth bit with H2 as one can see in Fig. 7.10. On the other hand, the use of interleaver causes that all bits are associated with the particular decoders randomly as show in Fig. 7.10 (thick dashed lines). Note that particular metrics as shown in Fig. 7.8 are evaluated by marginalization of the joint likelihood p(x|cA, cB) in SoDeM exploiting a priory information from other streams. 7.6.4 Overall Network Design Up to this time, we investigated processing only in one of the relays. We set the particular block rates for the block-component decoder in the relay. It is obvious that we will cannot independently set arbitrary rates in the complementary relay. It means that the complementary relay seems to be forced to decode at least either bA2 or bB2 on rate r2 in order to have an error-less minimal complementary information required in destination. Nevertheless, we have available at least two approaches how to solve the inconvenience, when the stand-alone rate r2 is not directly achievable in ¯R. The first one is that we have choice to increase the power in source at expanse of the second transmitted data stream (e.g. bB1). The second choice might be to communicate on the rate of a whole data codeword or some of its part12 (e.g. whole bB). As we will observe (Fig. 7.12), the enabled loops between the SoDeM and decoder in case of higher order modulations enables to operate on some averaged rate (e.g. rB < r2 in our case). If we use this averaged rate, we have more information that is directly needed for the final destination in the relays. Considering a decoding of the whole data vector in ¯R, that is ¯bAB = bB in our case, the relays have available the following streams: • Relay R: bA1, bB1, b2 = bA2 ⊕ bB2 • Relay ¯R: bB1, bB2. It can be clearly seen that the stream bB1 is available in both relays. The relays can therefore transmit all streams, where some power must be allocated to this redundant information. On the other hand, it is 10Notice that this concrete setup may be seen similar to the approach proposed in [22]. The fundamental difference is that their approach is based on clear network coding, that is the aforementioned relation is only upon the hierarchical data. 11Notice that this averaging is not necessary. One can consider the case when the unequal error protection is exploited e.g. to enable a particular block decoder operation on higher rate. Such a signal space mapping design is a challenging question that is addressed for future work. 12Further block structuring can be considered. 85
- 99. 7.6. BLOCK STRUCTURED LAYERED DESIGN IN SIMPLE EXAMPLE bA1 + GA1 bA2 + GA2 01 00 1011 bB1 GB1 bB2 + GB2 1101 0010 + + + w1 w2 w3 0100 0000 1101 1001 0011 1011 0110 1110 10101111 01110010 1100 1000 00010101 x1 x3 Soft Output Demodulator HA1 HB1 H2 Figure 7.10: The bit mapping visualization including the source encoders and relay decoder. The connection of SoDeM and decoders from the soft information transfer point of view can be seen in Fig. 7.8 for the case without interleaver. 86
- 100. CHAPTER 7. NETWORK CHANNEL CODE DESIGN possible to save this power and to transmit only the required information (no diversity). A good question is, if the gain of the proper correlated sources utilization, i.e. the distributed source coding in the relays R, ¯R and the corresponding decoding in the final destination, is more attractive than the saved power required for the system diversity. 7.7 Numerical Verification Since our work is modular, there is not an obvious reference, what should be numerically verified to demonstrate any gain of the proposed work13 . We rather focus on the waterfall-behavior of BER of the proposed decoders proving proper functionality. 7.7.1 Models to be Numerically Simulated We consider several ways of the hierarchical stream bAB evaluation from the received superimposed vector x. We assume that we have given requested asymmetric code rates for the source channel encoders like in the example in Sec. 7.6. As a reference case, we choose the conventional joint decoding strategy with successive interference cancellation followed by the network coding (Sec. 7.4.2). We focus on the following features to be verified by the numerical simulations. 1. Performance of the proposed framework. 2. Performance (BER) difference of the relay receiver, where the hierarchical decoder and SoDeM work separately (i.e. loops between SoDeM and decoder are disabled) against decoding with enabled loops. (Figs. 7.11, 7.12) 3. Performance difference of different decoder construction, particularly • hierarchical decoder operating whole on GF(2) without interleaver, • hierarchical decoder operating whole on GF(2) with interleaver, • hierarchical decoder operating partly on GF(4) with interleaver, that is the decoder H2 works on GF(4). The results for a given hierarchical ratio are shown in Fig. 7.13. 4. Trade-off between the diversity of the processing and stronger broadcast phase as discussed in Sec. 7.6.4 at least in a particular scenario (Fig. 7.15). The first three features can be shown on 2-source 1-relay scenario and the last one is suited into a whole network as shown in Fig. 7.9. Nevertheless to keep the things simple and better analyzable, we consider a subset of the whole model containing only the relays and the final destination for investigation of the fourth feature. The relays are assumed to be able to correctly decode the requested data streams, i.e. they can be seen as (correlated) sources. 13We remind that the main contribution of this work consists in the relay decoder construction for arbitrary source rates, where the decoder takes advantage the wireless network coding. 87
- 101. 7.7. NUMERICAL VERIFICATION 7.7.2 Simulation Setup and Outer Parameters We compose a system to provide a numerical verification of the proposed design. We consider the equal system model as in Sec. 7.6, where only one difference is in the codeword lengths nA1 = nB1 = n2 = n = 2400 and the block-coders are assumed to be regular LDPC codes with maximized girth according to [35]. We choose (ad-hoc) the signal space mappers at bit level to be QPSK for both sA and sB. The bit mapping is ad-hoc selected to maximize the minimal hierarchical distance. The symbol received in relay is given by x = sA + hsB + w, where h = exp(j π/4) and w is assumed to be a zero-mean AWGN. The relay evaluates a corresponding soft metric in SoDeM. The model is in fact shown in Fig. 7.10, where some longer encoders and interleaver are considered. To demonstrate the system diversity, we consider the hierarchical matrices as X = EkA1 0 0 0 0 0 EkB1 0 0 Ek2 0 Ek2 (7.41) and two complementary matrices, ¯X ∈ 0 0 0 Ek2 , 0 0 EkB1 0 0 0 0 Ek2 . (7.42) It means that the first relay R has available the streams [bA1, bB1, b2 = bA2⊕bB2] and the complementary relay ¯R has available either stand-alone [bB2] or both [bB1 bB2] as in Sec. 7.6.4. For simplicity, we assume that kA1 = kB1 = k2 = k = 1600 in this case (code rate is 2/3 for all streams). All streams are encoded in the relays by equal encoder and mapped to BPSK modulation, where the mean energy of the signal ¯sB2,min(bB2) referring to the scheme without diversity is split between ¯sB1,mid(bB1) and ¯sB2,mid(bB2) in the diverse scheme, where the "min" subscript denotes no diversity (minimal complementary information) and "mid" subscript denotes the diversity given by a redundant stream bB1. The streams are then transmitted orthogonally into the final destination through AWGN channel, where the decoder based on the factor graph is performed to decode the streams. 7.7.3 Simulation Results First of all, we can recognize the waterfall behavior of all BER curves indicating a proper functionality of the proposed design. The performance of the individual streams is given by signal space properties and by the rate of the referring block-decoder. We can see (Figs. 7.11 and 7.12) that the overall hierarchical decoder is indeed limited by its weakest sub-decoder, when the iterative loop via SoDeM is not applied. The soft information exchange between SoDeM and decoder significantly improves the performance (see Figs. 7.11 and 7.12). The properties of the block decoders are averaged and the hierarchical encoder definitely operates on a larger rate then its stand-alone weakest block decoder. The EXIT curves of the hierarchical decoder are shown in Fig. 7.14. Using the block decoder H2 working upon higher field, i.e. GF(4), we obtain some further performance gain (Fig. 7.13), which is however paid by higher complexity of the relay decoder. Using the interleaver to average the quality of the metric serving to the block decoders does not have any important impact to the performance as one can check in Fig. 7.13 for given rates and the signal space mapping. According to the provided results, we can conclude that the hierarchical (direct) decoding (H2 upon GF(2)) achieves similar performance to the reference JDF over NC with less complexity, while the decoding with (H2 upon GF(4)) benefits approximately half dB gain against the reference solution for given scenario (see Fig. 7.13). The performance comparison between diverse and no-diverse scheme can be found in Fig. 7.15. The diversity improves the performance of the stream bB1, while the performance of the streams bA2, bB2 is 88
- 102. CHAPTER 7. NETWORK CHANNEL CODE DESIGN 0 2 4 6 8 10 12 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 rA1 = 2/3 rB1=1/3 r2=2/3 SNR of the MAC channel Pbe BER for the rate 5/9 with and without loop via SoDeM bA1,LE bB1,LE b2,LE bAB,LE bA1,LD bB1,LD b2,LD bAB,LD EnabledloopsviaSoDeM DisabledloopsviaSoDeM Figure 7.11: BER of the hierarchical stream bAB and its individual sub-streams bA1,bB1 and b2 in case of pure decoder (loop disabled-LD) and decoder with enabled information exchange with SoDeM (LE) for different overall rates. worse due to the weaker relay to destination link (transmission power must be shared with the stream bB1). 7.8 Generalization and Future Work The area that is covered in this work presents a part of the overall wireless network design. Particularly, the proposed design is able to incorporate: • Arbitrary network information flow expressed by means of hierarchical ratio. • Arbitrary source rates. • Arbitrary symbol-wise signal space mapping, where a white space in both the modulation design and a concrete bit mapping optimization arises. The generalization can be considered in several ways, particularly: • general channels across the network, • complex network (more sources and streams), • allowing non-linear coding and hierarchical functions. The first way of generalization is relatively straightforward, because the core of this work needs just a metric for decoders. In case of linear system description, we can easily generalize the relation of the virtual decoder in relay and the codeword hierarchical function (7.5) and (7.6). We denote R = {A, B, . . . } as a set of all sources, then cR = i∈R Xc,ici = i∈R Xc,iGibi (7.43) = GRbR = GR i∈R Xb,ibi. (7.44) 89
- 103. 7.9. DISCUSSION AND CONCLUSIONS 0 2 4 6 8 10 12 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 SNR of the MAC channel Pbe BER for different rates with and without loop via SoDeM b2, rAB = 1/2 b2, rAB = 1/2 bA1, rAB = 5/9 bA1, rAB = 5/9 bAB, rAB = 5/9 bAB, rAB = 5/9 bAB, rAB = 1/2 bAB, rAB = 1/2 LE LD rAB = 1/2 rAB = 5/9 Figure 7.12: Bit error rate verification of the proposed design for the overall hierarchical stream bAB and its weakest individual sub-streams from bA1,bB1 and b2 for different hierarchical rates (as well as hierarchical ratios). The BER simulation is performed in two cases: 1) pure decoder, where the hierarchical decoder works as three independent decoders and therefore it operates on the rate of its weakest block-part and 2) decoder with enabled information exchange with SoDeM. As one can expect, the SoDeM usage results in a performance gain. Moreover, the enabled information exchange between the SoDeM and decoder causes an averaging of the sub-decoders properties such that the hierarchical decoder is not further limited by the rate of its weakest block sub-decoder and it operates on the averaged (hierarchical) rate rAB. Generalization towards more relays is relatively straightforward, because we just considered a comple- mentary H-SI, but our approach does not rely how many such H-SI links are presented. The information streams can be further separated in order to comply the requested source coding rates. Nevertheless, in case of more sources, the information flow can request either a decision upon more streams (e.g. bA + bB + bC) and/or decision upon doublets (triples, etc.) of data streams (e.g. {bA + bB}, {bB + bC}). In such a case a revision of the block structured layered should be performed, but the basic building blocks based on uniform hierarchical ratio for N sources with equal rates such that λ = 1/n are preserved. The encoder design based on equal source and relay decoder can be applied accordingly, where the order of finite field is arising in case of the optimal decoding. 7.9 Discussion and Conclusions We proposed a design of the joint network-channel encoding for linear encoders and hierarchical functions enabling to incorporate an arbitrary individual rates. This design is based on a predefined block-structure of both the channel encoders and the hierarchical functions. We proved the network consistency of the proposed design. We shown that the overall hierarchical decoder inherits the properties of its sub-decoders. We numerically verified the performance of such a design. We further verified the implementation of the proposed design, where we consider a decoding with both enabled and disable iterative loops through the soft output demodulator. The loops connecting the block-diagonal parts enable the information exchange among the block-component parts causing averaging of the particular block-rates. Finally, we shown that the proposed linear description enables an extension of the principle to more complex network topologies, where further block structuring can be considered. Nevertheless a detailed 90
- 104. CHAPTER 7. NETWORK CHANNEL CODE DESIGN 4 4.5 5 5.5 6 6.5 7 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 SNR of the MAC channel Pbe BER for different decoder realizations for rAB=5/9. hier hier+int GF(4) based H2 JDF over NC Figure 7.13: Comparison of the relay-decoder performance decoder for different realizations. We consider conventional decoders as GA1 and GB1 and different variants of G2 that are (1) hierarchical form, i.e. GF(2) decoder with G2 = GA2 = GB2 with and without interleaver, (2) optimal form, i.e. GF(4) as in [64]. study of these more complex network topologies is addressed for the future work. 91
- 105. 7.9. DISCUSSION AND CONCLUSIONS 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 IA(VN), IE(CN) IE(VN),IA(CN) EXIT analysis of the hierarchical decoder CN-EXIT function VN-EXIT function,SNR=8dB VN-EXIT function,SNR=7dB VN-EXIT function,SNR=6dB VN-EXIT function,SNR=5dB Figure 7.14: EXIT behaviour of the hierarchical decoder operation on rate 5/9 for different SNR of the MAC channel. Comparing to Fig. 7.13, we can see that the 5 dB level of the SNR in the MAC channel indeed does not work well. In case of 6 dB, one can expect an error-less behaviour according to the EXIT chart. Nevertheless the cliff appears approximately in the level 6.7 dB. This gentle inaccuracy is probably caused by a slight dependency of the SISO block inputs violating the assumptions for the EXIT chart measurement. −4 −3 −2 −1 0 1 2 3 4 5 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 SNR of the BC channel Pbe BER of the considered data streams in the final destination bB1,min b2,min b2,mid bB1,mid Figure 7.15: The performance of the particular streams in the final destination. We dropped the perfor- mance of the stream bA1, because it is equal to the performance of the stream bB1 in case of no diversity. This case is denoted by the min subscript, because only minimal information is passed towards the des- tination. The diversity case is denoted by the mid subscript, because the information passed towards the destination is somewhere in middle between the minimal information and the full information. Concretely, we assume that bB1 is transmitted from both relays in the "middle case". 92
- 106. 8 Overall Conclusions The main topic of this thesis is the receiver design in a multi-source multi-node wireless network. We cope this topic from two layers. The first one is the overall network design on the channel code level, that is to design a joint consistent cross-network channel code that satisfies some requests. The latter consists in a direct implementation of the generic sum product algorithm that is capable to describe among others the receiver in the wireless network. One can see a clear relation between the mentioned topics, that is the overall network design properly and efficiently implemented by means of the sum product algorithm. The sum product algorithm direct implementation was solved by the implementation framework con- taining the proposed generic KLT-message representation and the update rules design upon the message representation that are applicable among others to the proposed generic KLT message representation. We numerically figured out that the messages in a wide class of phase models lead to the harmonic eigen- functions applying the KLT on their stochastic description. This proves the Fourier representation to be the best possible linear message representation in the MSE sense. We can note that the implementation framework relying on the Fourier message representation was independently proposed in [5]. We verified the capability of the KLT (Fourier) representation by a numerical simulation for a given model, where the performance results are shown in Figures 5.13, 5.14 and 5.15. We further proposed an update rules design for an arbitrary linear message representation with orthog- onal canonical kernels. The assumption of the linearity and orthogonality is inherently satisfied for the KLT-message representation. The proposed update rules design jointly with the KLT-message representa- tion therefore form a generic implementation framework applicable whenever the stochastic description of the messages is available, i.e. KLT of the random process given by the different message realizations can be evaluated. The framework is moreover built upon the best linear message approximation in the MSE sense for a given dimensionality as a consequence of KLT. The Fourier representation forms an orthogonal kernel basis system. Being aware of the eigen-analysis results (see Figures 5.10, 5.11 and 5.12), we used the Fourier representation for the proposed implementation framework verification in the given joint data detection-phase estimation system model. The proposed implementation framework that results in a highly efficient implementation (check Table 5.1) for this particular scenario. The demand for a flexible wireless aware joint network-channel code design becomes clear after proving the wireless network coding potential (see e.g. [38]). We started with defining a robust system description. This description is based on the system linearity, that is the linear hierarchical function (in fact a wireless aware network code - see Note 6.4) and the channel code. The linearity assumption is satisfied by conven- tional capacity achieving codes (e.g. LDPC or turbo codes). We therefore form a Theorem 7.3 defining a block structure of the channel codes across the network. The relation of the inner block sizes (diagonal structure is assumed) determines the particular channel code rates. Arbitrary channel code rates can be set adjusting properly the sizes of inner blocks (Theorem 7.4). The proposed block structured layered design basically combines two known principles, where both the requested hierarchical ratio (amount of information flowing through the relay) and the requested rates are achieved in average over a whole codeword. We also discussed some possible structures of the decoder properly respecting the block structured layered design. This overall decoder is basically composed from the conventional LDPC block-decoders, where the soft information exchange between the (independent) block parts can be conducted e.g. via soft output demodulator for higher order alphabets. The performance for different metrics evaluated in the soft output demodulator and corresponding decoder structure is evaluated in a particular system model (check Figures 7.12 and 7.11). These results proves the waterfall behaviour indicating a proper decoder implementation. 93
- 107. Index Algorithm FBA, 13 Min Sum, 23 Sum Product, 23 Viterbi, 11 Block Structured Layered Code, 80 Layered Design, 78, 80 Capacity Region, 7 Channel Parameterization, 67 Code Block, 10 Block Structured, 80 FSM, 10 LDPC, 11 Turbo, 10 Contribution, 38, 70 Cost Function, 8 Hit or Miss, 8 Mean Square Error, 8 Vector Hit or Miss, 8 Cut-Set Bound, 7 First Order, 7 Second Order, 7 Decoder Hierarchical, 72 Decoding FSM, 11, 13 Hierarchical, 64, 65, 68, 74 Direct, 77 JDF over NC, 75 Optimal, 76 Turbo, 13 Demodulator Metric, 74 Hierarchical, 75 Joint Decoding, 75 Optimal Symbol-Wise, 74 Estimation, 7 MAP, 9, 24 ML, 8 MMSE, 9, 24 Vector MAP, 9 EXIT Chart, 15 Factor Graph, 18 Cycle-Free, 19 Global Fuction, 18 Looped, 33 Message Passing, 21 Min Sum Algorithm, 23 Sum Product Algorithm, 23 Hierarchical Function, 65, 71 Upon Codeword Stream, 72 Upon Data Stream, 71 Hierarchical Map, 65, 71 Hierarchical Ratio, 69, 73 Uniform, 74 Inference, 18 Information Type, 62 Direct Link, 62 Hierarchical Side, 63, 68 Hierarchical-Side, 62, 71 Interference Link, 62 Interference Cancellation, 7 Likelihood Function, 8 Log-Likelihood Arithmetics, 14 Log-Likelihood Ratio, 14 Loopy Belief Propagation, 33 MAP Decoding, 24 Marginal, 22 Message in FG-SPA, 22 Message Approximation, 27 Message Dimension, 25 Message Parameterization, 25, 27 Message Representation, 18, 25, 27 Canonical, 29 Dirac-Delta, 33 Discrete, 32 Fourier, 33 Gaussian, 26, 33 KLT, 38 Linear Canonical, 29, 40 Samples, 28 Message Type, 25 Discrete, 25, 50 General Continuous, 26 Mixed, 26, 50 Parameterizable Continuous, 26, 50 Norming, 43 Rectification, 43 Multiple Access Channel, 7 94
- 108. INDEX Network Processing, 61 Centralized, 62 Distributed, 62 Hybrid, 62 Network Topology, 64 Butterfly Network, 65 General Acyclic Network, 66 Multi-Hop Relay Channel, 66 Two-Relay Single-Destination, 65 Two-Way Relay Channel, 64 Point to Point AWGN Channel, 6 Relation Hierarchical and Network Coding, 65 Relaying Strategies, 63 Amplify and Forward, 63 Compress and Forward, 63 Decode and Forward, 63 Hierarchical Decode and Forward, 63, 68 Joint Decode and Forward, 63 Sum Product Algorithm, 23 Convergence, 18, 34 Implementation Framework, 31, 38, 57 Scheduling Algorithm, 32, 34 Adaptive, 34 Fixed, 34 Flood Schedule Algorithm, 34 Random, 34 Update Rules, 22 Exact, 22, 30 FN Update, 22, 41 Linear Canonical Representation, 40 Mixed FN Update, 42 Upon Parameterization, 30 VN Update, 22, 42 Virtual MIMO, 60 Wireless Aware Network Coding, 66 Wireless Network, 60 Destination Node, 60 Half Duplex Constraint, 61 Information Flow, 61, 68, 71 Relay Node, 60 Source Node, 60 Time Synchronization Assumption, 61 Wireless Network Coding, 68 95
- 109. Bibliography [1] Rudolf Ahlswede, Ning Cai, Shuo-Yen Robert Li, and Raymond W. Yeung. Network information flow. IEEE Trans. Inf. Theory, 46(4):1204–1216, July 2000. [2] Hans andrea Loeliger. Some remarks on factor graphs. In Proc. 3rd Int. Symp. on Turbo Codes and Related Topics, pages 111–115, 2003. [3] Khoirul Anwar and Tad Matsumoto. Accumulator-assisted distributed turbo codes for relay systems exploiting source-relay correlation. Communications Letters, IEEE, 16(7):1114–1117, 2012. [4] Xingkai Bao and JT Li. Generalized adaptive network coded cooperation (gancc): A unified framework for network coding and channel coding. IEEE Trans. Commun., 59(11):2934–2938, 2011. [5] Alan Barbieri, Giulio Colavolpe, and Giuseppe Caire. Joint iterative detection and decoding in the presence of phase noise and frequency offset. In Proc. IEEE Internat. Conf. on Commun. (ICC), 2005. [6] Claude Berrou and Alain Glavieux. Near optimum error correcting coding and decoding: Turbo-codes. IEEE Trans. Commun., 44(10):1261–1271, October 1996. [7] U. Bhat and T. Duman. Decoding strategies at the relay with physical-layer network coding. IEEE Trans. Wireless Commun., pages 4503–4513, December 2012. [8] M.C. Castro, B.F. Uchoa-Filho, T.T.V. Vinhoza, M. Noronha-Neto, and J. Barros. Improved joint turbo decoding and physical-layer network coding. In Proc. IEEE Inf. Theory Workshop (ITW), pages 532–536. IEEE, 2012. [9] Keith Chugg, Achilleas Anastasopoulos, and Xiaopeng Chen. Iterative Detection: Adaptivity, Com- plexity Reduction, and Applications. Kluwer Academic Publishers, 2001. [10] Giulio Colavolpe, Alan Barbieri, and Giuseppe Caire. Algorithms for iterative decoding in the presence of strong phase noise. IEEE J. Sel. Areas Commun., 23(9):1748–1757, September 2005. [11] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley & Sons, 1991. [12] Justin Dauwels and Hans andrea Loeliger. Phase estimation by message passing. In Proc. IEEE Int. Conf. on Communications, pages 523–527, 2004. [13] Justin H. G. Dauwels. On Graphical Models for Communications and Machine Learning:Algorithms, Bounds, and Analog Implementation. PhD thesis, Swiss Federal Institute of Technology, Zürich, May 2006. [14] M.C. Davey and D.J.C. MacKay. Low density parity check codes over gf(q). In Information Theory Workshop, 1998, pages 70 –71, jun 1998. [15] Abbas El Gamal and Young-Han Kim. Network Information Theory. Cambridge University Press, 2011. [16] G. Elidan, I. McGraw, and D. Koller. Residual belief propagation: Informed scheduling for asyn- chronous message passing. In Proceedings of the Twenty-second Conference on Uncertainty in AI (UAI), Boston, Massachussetts, July 2006. [17] Uri Erez and Ram Zamir. Achieving 1/2 log(1 + SNR) on the AWGN channel with lattice encoding and decoding. IEEE Trans. Inf. Theory, 50(10):2293–2314, October 2004. 96
- 110. BIBLIOGRAPHY [18] Gianluigi Ferrari, Giulio Colavolpe, and Riccardo Raheli. A unified framework for finite-memory detection. Selected Areas in Communications, IEEE Journal on, 23(9):1697–1706, 2005. [19] Alex Grant. Convergence of non-binary iterative decoding. In Proc. IEEE Global Telecommun. Conf. (GlobeCom), 2001. [20] Z. Guo, J. Huang, B. Wang, S. Zhou, J.H. Cui, and P. Willett. A practical joint network-channel coding scheme for reliable communication in wireless networks. IEEE Trans. Wireless Commun., 11(6):2084–2094, 2012. [21] Joachim Hagenauer, Elke Offer, and Lutz Papke. Iterative decoding of binary block and convolutional codes. IEEE Trans. Inform. Theory, 42:429–445, 1996. [22] C. Hausl, F. Schreckenbach, I. Oikonomidis, and G. Bauch. Iterative network and channel decoding on a tanner graph. In Proc. Allerton Conf. on Commun., Control and Computing. Citeseer, September 2005. [23] Sheryl L Howard, Vincent C Gaudet, and Christian Schlegel. Soft-bit decoding of regular low-density parity-check codes. Circuits and Systems II: Express Briefs, IEEE Transactions on, 52(10):646–650, 2005. [24] Steven M. Kay. Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall, 1993. [25] Steven M. Kay. Fundamentals of Statistical Signal Processing: Detection Theory. Prentice-Hall, 1998. [26] Toshiaki Koike-Akino, Petar Popovski, and Vahid Tarokh. Optimized constellations for two-way wireless relaying with physical network coding. IEEE J. Sel. Areas Commun., 27(5):773–787, June 2009. [27] Frank R. Kschischang, Brendan J. Frey, and Hans-Andrea Loeliger. Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory, 47(2):498–519, February 2001. [28] Brian M. Kurkoski and Justin Dauwels. Belief-Propagation Decoding of Lattices Using Gaussian Mixtures. CoRR, abs/0904.4741, 2009. [29] J.N. Laneman, D.N.C. Tse, and G.W. Wornell. Cooperative diversity in wireless networks: Efficient protocols and outage behavior. Information Theory, IEEE Transactions on, 50(12):3062 – 3080, dec. 2004. [30] Soung Chang Liew, Shengli Zhang, and Lu Lu. Physical-layer network coding: Tutorial, survey, and beyond. in Phys. Commun., 2011 [Online]. Available: http://arxiv.org/abs/1105.4261, 2011. [31] J. Liu, M. Tao, and Y. Xu. Pairwise Check Decoding for LDPC Coded Two-Way Relay Block Fading Channels. IEEE Trans. Commun., PP(99):1–12, May 2012. [32] H.-A. Loeliger, J. Dauwels, Junli Hu, S. Korl, Li Ping, and F.R. Kschischang. The Factor Graph Approach to Model-Based Signal Processing. Proceedings of the IEEE, 95(6):1295–1322, June 2007. [33] Hans-Andrea Loeliger. An introduction to factor graphs. IEEE Signal Process. Mag., 21(1):28–41, January 2004. [34] J.A. McGowan and R.C. Williamson. Loop Removal from LDPC Codes. In Information Theory Workshop, pages 230 – 233, april 2003. 97
- 111. BIBLIOGRAPHY [35] James A McGowan and Robert C Williamson. Loop removal from ldpc codes. In Information Theory Workshop, 2003. Proceedings. 2003 IEEE, pages 230–233. IEEE, 2003. [36] J. M. Mooij. Understanding and Improving Belief Propagation. PhD thesis, Radboud University Nijmegen, May 2008. [37] Joris M. Mooij and Hilbert J. Kappen. Sufficient conditions for convergence of the sum-product algorithm. IEEE Trans. Inf. Theory, pages 4422–4437, May 2007. [38] Wooseok Nam, Sae-Young Chung, and Yong H. Lee. Capacity bounds for two-way relay channels. In Proc. Int. Zurich Seminar on Communications, 2008. [39] B. Nazer and M. Gastpar. Reliable physical layer network coding. Proceedings of the IEEE, 99(3):438– 460, 2011. [40] Bobak Nazer and Michael Gastpar. Compute-and-forward: Harnessing interference through structured codes. IEEE Trans. Inf. Theory, 57(10):6463–6486, October 2011. [41] O. Oyman, N.J. Laneman, and S. Sandhu. Multihop relaying for broadband wireless mesh networks: From theory to practice. Communications Magazine, IEEE, 45(11):116 –122, November 2007. [42] Athanasios Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, second edition, 1984. [43] John G. Proakis. Digital Communications. McGraw-Hill, 4th edition, 2001. [44] Pavel Prochazka and Jan Sykora. Karhunen-Loève based reduced-complexity representation of the mixed-density messages in SPA on Factor Graph and its impact on BER. EURASIP J. on Wireless Comm. and Netw., 2010:11, December 2010. [45] Pavel Prochazka and Jan Sykora. Block-structure based extended layered design of network coded modulation for arbitrary individual using hierarchical decode & forward strategy. In In COST IC1004 MCM, Bristol, UK, pages 1–7, September 2012. [46] Pavel Prochazka and Jan Sykora. Block-Structured Layered Design of Wireless Aware Joint Network- Channel Coding for Asymmetric Rates using HDF Relaying Strategy. IEEE Trans. Wireless Commun., July 2013. Submitted for publication. [47] Pavel Prochazka and Jan Sykora. Generic efficient fg-spa implementation framework based on the best linear canonical message representation in mse sense. European Wireless 2013, 2013. [48] Boris Rankov and Armin Wittneben. Spectral efficient protocols for half-duplex fading relay channels. Selected Areas in Communications, IEEE Journal on, 25(2):379–389, 2007. [49] Tom Richardson and Ruediger Urbanke. Modern Coding Theory. Cambridge University Press, New York, NY, USA, 2008. [50] Bixio E. Rimoldi. A decomposition approach to CPM. IEEE Trans. Inf. Theory, 34(2):260–270, March 1988. [51] Christian B. Schlegel and Lance C. Perez. Trellis and Turbo Coding. John Wiley & Sons, 2004. [52] Frederik Simoens and Marc Moeneclaey. Code-aided estimation and detection on time-varying cor- related MIMO channels: A factor graph approach. EURASIP J. on Advances in Signal Processing, 2006:1–11, 2006. 98
- 112. BIBLIOGRAPHY [53] Jan Sykora. Factor graph framework for serially concatenated coded CPM with limiter phase dis- criminator receiver. In Proc. IEEE Vehicular Technology Conf. (VTC), pages 1–6, Baltimore, USA, October 2007. [54] Jan Sykora and Alister Burr. Layered design of hierarchical exclusive codebook and its capacity regions for HDF strategy in parametric wireless 2-WRC. IEEE Trans. Veh. Technol., 60(7):3241– 3252, September 2011. [55] Jan Sykora and Pavel Prochazka. Performance evaluation of the factor graph CPM phase discriminator decoder with canonical messages and modulo mean updates. In Proc. IEEE Asia-Pacific Conf. on Communications (APCC), pages 1–5, Tokyo, Japan, October 2008. [56] Stephan ten Brink. Convergence begavior of iteratively decoded parallel concatenated codes. IEEE Trans. Commun., 49(10):1727–1737, October 2001. [57] Stephan ten Brink, Gerhard Kramer, and Alexei Ashikhmin. Design of low-density parity-check codes for modulation and detection. Communications, IEEE Transactions on, 52(4):670–678, 2004. [58] Tomas Uricar, Tomas Hynek, Pavel Prochazka, and Jan Sykora. Wireless-aware network coding:solving a puzzle in acyclic multi-stage cloud networks. In Accepted to ISWCS, pages 1–5, September 2013. [59] Tomas Uricar and Jan Sykora. Non-uniform 2-slot constellations for relaying in butterfly network with imperfect side information. IEEE Commun. Lett., 2012. [60] Nenad Veselinovic, Tad Matsumoto, and Markku Juntti. Iterative receivers for sttrc-coded mimo turbo equalization. In Vehicular Technology Conference, 2004. VTC 2004-Spring. 2004 IEEE 59th, volume 1, pages 460–463. IEEE, 2004. [61] Branka Vucetic and Jinhong Yuan. Turbo Codes: Principles and Applications. Kluwer Academic Publishers, 2000. [62] Niclas Wiberg. Codes and Decoding on General Graphs. Phd thesis, Linköping University, 1996. [63] Andrew P. Worthen and Wayne E. Stark. Unified design of iterative receivers using factor graphs. IEEE Trans. Inf. Theory, 47(2):843–849, February 2001. [64] D. Wübben and Y. Lang. Generalized sum-product algorithm for joint channel decoding and physical- layer network coding in two-way relay systems. In IEEE 2010 Global Telecommunications Conference (Globecom 2010), Miami, FL, USA, Dec 2010. [65] Sichao Yang and Ralf Koetter. Network coding over a noisy relay: a belief propagation approach. In Proc. IEEE Internat. Symp. on Inf. Theory (ISIT), 2007. [66] Jonathan S. Yedidia, William T. Freeman, and Yair Weiss. Constructing free energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51:2282– 2312, 2005. [67] Raymond W. Yeung, Shuo-Yen Robert Li, Ning Cai, and Zhen Zhang. Network Coding Theory. now Publishers, 2006. [68] Yan Zhang, Hsiao-Hwa Chen, and Mohsen Guizani. Cooperative Wireless Communications. Auerbach Publications, Boston, MA, USA, 2009. 99