Cycle’s topological optimizations and the iterative decoding problem on general graphs

www.huawei.com
Security Level:
HUAWEI TECHNOLOGIES CO., LTD.
Cycle’s topological optimizations
and
the iterative decoding problem
on general graphs
Author/ Email: Usatyuk Vasiliy, Usatyuk.Vasily@huawei.com, L@Lcrypto.com
Coding Competence Center, Moscow Research Center

Soft decoding algorithms
Wireless channels
Optical channels
Consider MAP under codeword - Maximum likelihood regularization using AP data(LLRs)
to estimate probability of frame error rate. Bit error rate require using linear model and it
subject of next discussion.

General graph for linear codes
Parity-check matrix of size :
Codes:  TT
HcFcC 0|2 











11100
11010
00111
H
Bipartite graph (Tanner graph):
54321 XXXXX
Direct Problem - Statistical Inference: find the "most likely” transmitted word
using soft a posterior probability log-likelihood rations
 
 ii
ii
i
yc
yc
where
|1Pr
|0Pr
ln



 ,,...,, 110  n
ML problem is NP-hard!
nk 
4)3(5)2(4 XfXfX xorxor Cycle 4:
)3(xorf
)2(xorf
)1(xorf
c
Variable
nodes
Check
nodes

We can approximate using Message-passing iterative decoding
algorithm(Belief Propagation, Sum Product algorithm)
All operations become local related
to exchange messages between columns
(variable nodes) and check nodes
without any loops in algorithm.
iX
)(iXORf
ML problem became O(N)-time!
If the graph is finite and cycle-free,
then the algorithm is finite (finish after finite
number of iterations) and exact.
If the graph has cycles, then the algorithm
becomes iterative and approximate.
:)(iXORi fX 
:)( iiXOR Xf 

Asymptotical properties of graph girth(shortest cycles) and
hamming distance grown from graph cardinally
To have enough error correcting capability (code distance) necessary to have a
lot of short cycles in Tanner graph

After decoding iterations Belief Propagation algorithm under the tanner graph with girth
produce wrong decoding result due cycles:
.1
4
 m
g
m
Computation tree of Belief propagation (BP) under graph with cycles :
m g
Tanner QC-LDPC code [155,64,20] computation(Wiberg) tree
after 4 iterations(4 generations) of BP decoder
Circle is variable nodes, black square – check nodes, gray variable nodes show “weak nodes”
which produce uncorrected error under BP which can corrected according code distance
Example of Subgraph
for which "" MAPBP 

After decoding iterations Belief Propagation algorithm under the tanner graph with girth
produce wrong decoding result due Trapping sets:
.1
4
 m
g
m
Trapping sets - subgraphs formed by cycles or it’s union:
m g
),( baTrapping set is a sub-graph with a variable nodes and b odd degree checks.
For example, TS(5,3) produced by three 8-cycles;
TS(a,0) is most harmfulness pseudocodeword of weight a formed by cycles 2*a.

BER and FER performance of the original and modified DVB-S2 QC-
LDPC codes of information length K=43200 and rate 2/3.
TS(9,0)
TS(10,1)
Without TS(9,0).
Without TS(9,0)
And TS(10,1).
Inverse problem – Learning (construct graph) from samples
with restrictions

Problem related to cycles eliminate:
Based on current cycle optimizations algorithm we can construct structured QC-LDPC codes :
Could it be better cycle broke algorithm?
*Y. Wang, S. C. Draper and J. S. Yedidia, "Hierarchical and High-Girth QC LDPC Codes," in IEEE Transactions on Information Theory, vol. 59, no. 7,
pp. 4553-4583, July 2013.
**M. Diouf, D. Declercq, M. Fossorier, S. Ouya and B. Vasić, "Improved PEG construction of large girth QC-LDPC codes," 2016 9th International
Symposium on Turbo Codes and Iterative Information Processing (ISTC), Brest, 2016, pp. 146-150.
***M.E. O'Sullivan. "Algebraic construction of sparse matrices with large girth". IEEE Trans. In! Theory, vo1.S2, no.2, pp.718-727, Feb. 2006.
Column
number,
L
Our
approach
Hill
Climbin
g*
Improved
QC-PEG**
4 37 39 37
5 61 63 61
6 91 103 91
7 155 160 155
8 215 233 227
9 304 329 323
10 412 439 429
11 545 577 571
12 709 758 -
Minimal value of circulant L for regular mother
matrix with row number m=3 and column
number n with girth 10
Column
number,
L
Our
approach
Improv
ed QC-
PEG**
TableV***
4 73 73 97
5 160 163 239
6 320 369 479
7 614 679 881
8 1060 1291 1493
9 1745 1963 2087
10 2734 - -
11 4083 - -
12 5964 - -
Minimal value of circulant L for regular mother
matrix with row number m=3 and column
number n with girth 12
Does better Poly-time complexity general algorithm for cycles eliminating exist?

Problem related to Trapping sets search:
For Trapping sets search we use “nauty” graph algorithms library.
Enumerating of all cycles and it union is NP-hard problem (for example, Hamiltonian Cycle).
Using Cole method* with noise injection we can search TS with a <20-30 variable:
1. Could we search Trapping sets in more efficient way (heuristics, probability algorithms)?
2. For some structured parity-check matrix (Quasi-Cyclic) with fixed column weight
distribution we can proof some simple equation to eliminate harmful trapping sets.
How it can be generalized for any (practically) regular and irregular codes on the graph?
*Chad A. Cole and etc A General Method for Finding Low Error Rates of LDPC Codes https://arxiv.org/pdf/cs/0605051

Problem related to Trapping sets bypass using BP message scheduler:
We can bypass Trapping sets using sequentially BP decoder scheduler under general graph:
In Polar code using this method sequential BP decoder (Successive cancellation) bypass
short cycles. As result, harmfulness TS weight equal to code distance.
Girth in Polar code graph for N=8
2X







11
11
H
1X
)1(xorf
)2(xorf
1X
2X
)1(xorf
)2(xorf
and construct graph in such way that some variable more reliable.
Base on this knowledge we can sequentially decode graph by BP:
Make new unobservable variable node and
corresponding check node
212,1 XXX 
.02,121)2,1(  XXXfxor











111
100
100
H
1X 2X 2,1X
)1(xorf
)2(xorf
)2,1(xorf
1X
2X
)1(xorf
)2(xorf
2,1X
)2,1(xorf
Cycle 4 bypass by cost of sequential decoding step.
1. How this sequentially BP decoder scheduler can applied to any arbitrary graph?
2. How to make trade-off between number of sequential step and size of bypassed TS?

Graph model problems:
1. What properties of graph flows allow for variational representation?
2. When do they have unique and efficiently computable solution?
3. Can we create systematic methods to solve Mixed-Integer-Non-Linear-Programming
problems over graph flow by using Linear Programming-Belief Propagation hierarchy?
4. Is it possible to exploit the natural separation of variables that exists in graph expansion
planning problems to accelerate classical Mixed Integer Programming algorithms such as
Bender's Decomposition(divide-and-conquer)?
5. Can non-linear programming techniques be incorporated as heuristics to accelerate
mixed integer program solution of graph’s model flow optimization problems with
physical constraints?
6. What properties of graph flows enable simplification of robust optimization problems?
7. How can algorithms for computation of probabilities over graphs be efficiently
incorporated within optimization problems?

Everything is a graph(model)…
Thank you!

Cycle’s topological optimizations and the iterative decoding problem on general graphs

More Related Content

What's hot (20)

Similar to Cycle’s topological optimizations and the iterative decoding problem on general graphs (20)

Recently uploaded (20)

Cycle’s topological optimizations and the iterative decoding problem on general graphs