Applications of information theory in communication engineering
- 1. Application of information theory in Communication engineering Presented by: Abdul Razaque NEDUET KARACHI abdulrazaque15tc@gmail.com 10/4/2020 1
- 2. Contents • Introduction • Source coding • Huffman Algorithm • Image/Video Compression • Channel coding • Techniques • Transmission errors • Block codes • Convolution codes • Turbo codes • Conclusion • Reference 10/4/2020 2
- 3. Introduction • Information theory studies the quantification, storage, and communication of information. • It was proposed by Claude Shannon in 1948 to find fundamental limits on signal processing and communication operations such as data compression. • Coding theory is one of the most important and direct applications of information theory, which is subdivided into source coding theory and channel coding theory . 10/4/2020 3
- 4. Source coding • Source coding remove redundancy in the information to make the message smaller. 1. Loss less Data Compression: it is used to compress the files without losing an original file's quality and data. we can say that in this technique, file size is reduced, but the quality of data remains the same. 2. Lossy Data compression: it is used to compress larger files into smaller files. In this compression technique, some specific amount of data and quality are removed (loss) from the original file.it is useful for us when the quality of data is not our first priority. 10/4/2020 4
- 5. Huffman algorithm • It is basically used for encoding entropy and to compress data without loss. In order to choose a particular representation for each symbol, Huffman coding makes use of a particular method that leads to a prefix code. • This method uses a minimum number of bits in the form of strings to represent mostly used and common source symbols. • Huffman coding used to encode run length. For JPEG, PNG, MPEG, MP3, AAC and also used in steganography for JPEG carrier compression. 10/4/2020 5
- 6. Arithmetic algorithm • Arithmetic coding does not assign particular bit patterns to the source symbols. Instead, a single arithmetic codeword is assigned to the complete sequence of symbols. Arithmetic codewords consist of sub-intervals of the unit interval [0, 1). • Supports to general purpose applications like PPM, PAQ. For image JPEG2000, JBIG, MPEG for teleconferencing SKYPE, Flash Arithmetic coding is used. 10/4/2020 6
- 7. Image/Video compression • Intra frame coding and inter frame coding are two important concepts involved in video coding. • Intra frame coding In this, Data Reduction takes place within the frames. It exploits redundancy within the frames. This effectively means using the spatial correlations. • inter frame coding Data Reduction takes place between the frames. It Exploits redundancy between successive frames 10/4/2020 7
- 8. JPEG • JPEG is a commonly used method of lossy compression for digital images, particularly for those images produced by digital photography. • JPEG compression reduces the file size with minimum image degradation by removing the smallest possible amount of information. • The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. 10/4/2020 8
- 9. MPEG • It set standards for audio and video compression and transmission • It has produced a number of digital media standards. Examples include: • MPEG-1 – Audio/video standards designed for digital storage media (such as an MP3 file) • MPEG-2 – Standards for digital television and DVD video • MPEG-4 – Multimedia standards for the computers, mobile devices, and the web 10/4/2020 9
- 10. Channel Coding • Channel coding deals with error control techniques. If the data at the output of a communications system has errors that are too frequent for the desired use, the errors can be reduced by the use of a number of technique. • Coding allows an increased rate of information transfer at a fixed error rate for a fixed transfer rate. 10/4/2020 10
- 11. Automatic Repeat Request • Automatic Repeat Request (ARQ) when a receiver circuit detects errors in a block of data, it requests that the data is retransmitted. • A feedback channel is necessary for this retransmission. • ARQ is often used where there is a full duplex (2- way) channel because it is relatively inexpensive to implement. 10/4/2020 11
- 12. Forward error correction • the transmitted data is encoded so that the data can correct as well as detect errors caused by channel noise. • The channel encoder accepts these message bits and add redundant bits to them leaving higher bit rate for transmission. • FEC is used where the channel is not full duplex or where ARQ is not desirable because real time is required. 10/4/2020 12
- 13. Transmission errors • There are two types of transmission errors (i) random error (ii) burst error. 1. Random errors are those that occur in a purely random manner. BCH codes are useful in dealing with this sort of error. 2. Burst errors occur in forms of bunches and are not independent. Convolution codes are not effective for this sort of errors. 10/4/2020 13
- 14. Block codes • Block Codes are used to detect or correct errors. • It accepts block of k information bits and produce block of n coded bits. the n-k redundant bits are added to the k information bit to get n coded bits, these codes are also related to (n, k) block codes. 10/4/2020 14
- 15. Convolution codes • Convolution codes are error detecting code used to reliably transmit digital data over unreliable communication system to channel noise. • The code words are generated by discrete – time convolution of the input sequence with impulse response of the encoder. Unlike block codes, channel encoder accepts messages as a continuous sequence and generates a continuous sequence of encoded bits at the output. 10/4/2020 15
- 16. Hamming codes • Hamming codes can detect up to two simultaneous bit errors, and correct single- bit errors; thus reliable communication is possible when the Hamming distance between the transmitted and received bit patterns is less than or equal to one. • It is suitable where burst error does not occur like transmission medium. • It is widely used in computer memory because of simplicity of hamming codes. 10/4/2020 16
- 17. Turbo codes • Turbo coding is an repeated soft-decoding scheme that combines two or more relatively simple convolutional codes and an interleaver to produce a block code that can perform to within a fraction of a decibel of the Shannon limit. • Turbo codes are used extensively in 3G and 4G mobile telephony standards e.g. in HSPA, EV-DO and LTE. MediaFLO, terrestrial mobile television system. 10/4/2020 17
- 18. Reed Solomon codes • Reed Solomon codes designed this for the purpose to detect and correct error multiple random symbol errors. Red Solomon add t to check symbols to the data and detect combination of up to t incorrect symbols, and correct half of them as t/2 symbols. It is reliable where burst error occurs. • There are various applications such as CDs, DVDs and DSL & WiMAX in data transmissions. 10/4/2020 18
- 19. Conclusion • Information theory was created to find practical ways to make better, more efficient codes and find the limits on how fast computers could process digital signals. Every piece of digital information is the result of codes that have been examined and improved using Shannon's equation. • Coding applications have grown rapidly in the past several years with the cost effective performance demonstrated to increase the efficiency and reliability of communication. 10/4/2020 19
- 20. Reference 1. Sayood, K.,” Introduction to data compression”, Morgan Kaufmann Publishers, San Francisco, CA. (Fourth Edition in 2013). 2. Muzhir Shaban AL-Ani , Fouad Hammadi Awad,” The Jpeg Image Compression A Lgorithm”, International Journal of Advances in Engineering & Technology, May 2013. 3. Rubal Chaudhary1, Vrinda Gupta,” Error Control Techniques and Their Applications”,IJCAES, Vol 1,issue 2, June2011. 4. C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J. , vol. 27, pp. 379–423, 623–656, July–Oct-1948. 5. https://www.javatpoint.com/lossless-vs-lossy-data-compression 10/4/2020 20
Editor's Notes
- #5: There are two formulations for the compression problem