710402_Lecture 1.ppt

Lecture:1 Date: 7/9/2010
Rahul K. Kher
Assistant Professor,
Department of Electronics & Communication
Engg.
710402: Information Theory &
Coding

Course Contents
 Three segments: Source Coding, Channel
Coding and Cryptography
 Concept and need of coding: Entropy,
Redundancy of data etc.
 Source coding techniques:
Unique decodable codes (UDC)
Instantaneous decodable codes (IDC)
Huffman and Shannon-Fano codes
Arithmetic Coding
Lossless vs. lossy coding (compression)

 Channel coding techniques:
Linear Block codes
Cyclic codes
Hamming codes
BCH codes
Reed-Solomon codes
Convolutional Coding and decoding algorithms
 Cryptography

A simple coding Example
2-out-of-5 code

 The message "173" has the following code:
110001000101100.
 How do we decode the 2-out-of-5 code?
 Of course, the first five binary digits correspond to
the first decimal one, and after decoding them, we
proceed to the second group of five binary digits,
etc.
 A helpful mnemonic rule: use 01247 as a "weight"
of the five columns, and add the weights of all l's in
your words. Examples: 110001 0+1 = 1 and
011001 1+2 = 3.
 Unfortunately, 0 is an exception.

Unique Decoding
 Definition. For each coding K (of source
symbols), we define the coding of source
messages as the rule K*, which to each word x1,
x2, …xm in the source alphabet assigns the word
K* (x1 x2... xm) = K(x1) K(x2)... K(xm) obtained
by concatenation of the code words K(xi), i = 1,
..., m.
 The coding K is said to be uniquely decodable
provided that arbitrary two distinct source
messages have distinct codes. In other words,
provided that K* is one-to-one.

 For example, the 2-out-of-5 code is uniquely
decodable. The assignment of a binary word to
"173" is a sample of coding source messages.
 In contrast, the following coding
 a 00 ; b 10 ; c 101; d 110 ; e
1001
is not uniquely decodable: try to decode 10110.
 We now introduce two important types of uniquely
decodable codes: Block codes and Instantaneous
codes

 Instantaneous codes: codes of variable word
lengths decodable symbol per symbol.
A coding is called instantaneous provided that no
code word is a prefix of another code word; i.e., if a
source symbol has a code b1 b2 ... bn then no
other source symbol has a code
b1 b2 ... bn bn+1 ...bm.
 Block codes, are the special case of
instantaneous codes with constant word length.
A coding using only pairwise distinct code words of
a certain length n is called a block coding of length
n.

Examples
 The Morse code is an example of instantaneous
code with the code alphabet { • , — , space }.

 An important example of a block code is the octal
code:

 Suppose that we are to find a binary coding for
the alphabet {0,1,2,3}, and we observe that 0
appears much more often in source messages
than any other symbols. Then the following
coding seems reasonable:
 0 0; 1 01; 2 011; 3 111

Some important Block codes
 Octal codes:
 Hexadecimal codes:

 ASCII codes:
A very important code used for a standard binary
representation of alphabetic and numeric symbols is
the ASCII code. It has 27 = 128 source symbols
encoded into binary words of length 8: the first
seven symbols carry the information, and the eighth
one is set in such a way that the parity is even (i.e.,
each code word contains an even number of l's).
The role of this eighth symbol is to enable detection
of single errors. For example, the letter A has code
Similarly, the letter “a” has code: 1 100 001

710402_Lecture 1.ppt

More Related Content

Similar to 710402_Lecture 1.ppt (20)

More from Pratik Gohel (20)

Recently uploaded (20)

710402_Lecture 1.ppt