Video coding technology proposal by
- 1. Video coding technology proposal by Tandberg, Nokia, Ericsson JCTVC-A119 1 st JCT-VC meeting, Dresden Germany April 2010
- 2. Summary TENTM algorithm has less complexity than H.264/AVC Baseline Profile Class B Subjective results show that TENTM brings around 30% gain on average over H.264/AVC High Profile on CS1 more than 40% gain on average over H.264/AVC High Profile on CS2 TENTM encoder and decoder runs significantly faster than JM17.0 and KTA 2.7 TENTM represents a clean, back-to-basics design : TENTM is ideal to be used in collaborative phase to test various tools on a clean environment with good software Syntax&Semantics document and TENTM software are provided To make sure concrete work starts at this meeting
- 3. Overview Requirements of mobile, video-conferencing industries: Significant coding efficiency improvement over H.264/AVC. Faster standardization of the low complexity operating point with complexity not higher than H.264/AVC Baseline profile. TENTM was designed to fulfill those requirements: Significant subjective quality improvement over H.264/AVC with less complexity than H.264/AVC BP Clean design : Useful for working towards a high coding efficiency operating point for other use cases, such as broadcast: Design from scratch : Clean and efficient software No obsolete H.264/AVC tools
- 4. Performance – Coding Efficiency Main focus has been on producing good subjective quality - especially at high resolutions. Analysis of subjective results for Class B sequences show that TENTM achieves: around 30% gain on average over H.264/AVC High Profile on CS1 more than 40% gain on average over H.264/AVC High Profile on CS2 TENTM is highly competitive to other proposals with significantly lower complexity Very few proposals outperform TENTM consistently but with a small margin (Class B is examined only)
- 5. Performance - Complexity TENTM decoder algorithm has roughly 75% complexity of H.264/AVC Baseline Profile (educated guess on typical platforms) Low complexity deblocking and interpolation filters Low complexity VLC No motion partitions smaller than 8x8 Low complexity B frames (Integer MV for SKIP and DIRECT) TENTM decoder runs more than 2x faster than JM17.0 Clean implementation with C (no low level optimizations)
- 6. TENTM Software Clean and fast software written from scratch using C TENTM decoder is 2-3 times faster than JM17.0 decoder TENTM encoder is 25 times faster than JM17.0 encoder TENTM encoder is around 400 times faster than KTA2.7 encoder Because of its low complexity and clean design TENTM software is ideal to test various tools Easily extendable towards high coding efficiency operating point in collaborative phase TENTM encoder represents a practical encoder useful for future extensions: Entire CS2 test-set is encoded in less than 4 hours on a single core Does not utilize the full potential of the TENTM syntax
- 7. Motion Representation Following candidates for motion partitions: 64x64 and 32x32 large MBs 16x16 traditional MBs with 16x16, 16x8, 8x16, 8x8 motion partitions Improved SKIP mode Runs of SKIP are always signaled on the 16x16 MB level SKIP MB can have up to two motion vector candidates (selection indicated in the bitstream) Motion vectors are rounded to integer for SKIP in B pictures Low complexity DIRECT mode: Motion vectors are rounded to integer to avoid interpolation Reference index indicate forward, backward or bi-predictive coding: Reference index is coded predictively.
- 8. Motion Representation 2 Reference Frames are used Concept of long-term reference is used for generating CS2 bitstreams High Quality & Low Complexity Interpolation Combination of Directional Interpolation Filters (DIF) and Separable Filters (SF) 1 bit is signaled if the MV points to one of the 9 central positions DIF uses strong filter for the middle position SF rounds the intermediate values to 8-bits to achieve 16-bit operation I P P P HQR- P P P P HQR Period = 4 HQR- P
- 9. Intra Frame Prediction Following INTRA prediction alternatives: INTRA 16x16 : DC, vertical, horizontal and planar prediction. Planar prediction reconstructs smooth regions in a visually pleasing way INTRA 8x8 : DC and angular prediction with 32 directions. Angular prediction reconstructs directional structures in a visually pleasing way by defining additional prediction directions INTRA 4x4 : DC, vertical, and horizontal.
- 10. Intra Frame Prediction Planar Prediction: Used to reconstruct smooth regions in a visually pleasing way Signal the bottom-right sample and reconstruct the macroblock using upper row, left column and signalled sample Step – 1: Interpolate rightmost and bottom samples Step – 2: Bi-linear interpolation of middle samples
- 11. Transforms and Quantization 4x4, 8x8, 16x16, 32x32, 64x64 transforms 16-bit implementation is used for all the transform sizes. For coding INTER modes, a spatially varying transform (SVT) is used as additional mode: The position of transform block within macroblock is varied Additional 16x4 and 4x16 transforms are utilized
- 12. In-loop Filtering Reduced complexity compared to H.264/AVC Only applied on block boundaries of 8x8 (no 4x4 filtering) Significantly less complex logic compared to H.264/AVC filtering. Uses a combination of strong and weak filters Improved visual quality with low complexity Interpolative filtering if two macroblocks are coded in planar mode
- 13. Entropy Coding VLC based entropy coding Main features: Improved context adaptivity: Use adaptive sorting tables Adapt the VLC table based on the coding statistics. Improved coefficient coding Lower complexity than H.264/AVC but improved coding efficiency code number = 7 before sorting: after sorting: 12 3 9 3 12 9 0 6 7 0 6 7 table_index = 12
- 14. Entropy Coding All the transform sizes use the same 8x8 coefficient coding engine 16x16, 32x32, 64x64 transforms are truncated ( always the 8x8 lowest frequency coefficients are coded ) Overview of coefficient coding: Position of last non-zero coefficient and whether its magnitude is larger than 1 is signaled Runs of zeros are signaled in reverse scan order. Switch to level mode based on magnitude and position of previously coded coefficients Level mode codes each remaining coefficient one by one. Coefficient coding is less complex than H.264/AVC CAVLC
- 15. Future Extensions Several well-know tools could be added during the course of standardization: CABAC Additional reference frames 3-4 % gain Sub LMB partitions (e.g. 64x32, etc.): 1-2 % gain Improved MV coding 1-2% gain Adaptive in-loop filtering Decoder side MV derivation Larger LMBs (128x128) Adaptive Interpolation Filtering (AIF)
- 16. Conclusions We believe the information brought to this meeting attest significant coding efficiency improvement is possible with very low complexity. Many requests to MPEG and VCEG earlier for the need of low complexity operating point in a short timeframe We request JCT-VC to start a concentrated effort as soon as possible Syntax & Semantics and detailed decoder description are provided to make working on low-complexity operating point easier. Many interesting proposals are brought to JCT-VC, some of them also suitable for low-complexity operating point
- 17. Conclusions We believe TENTM is ideal also for working towards the high coding efficiency goal: Back-to-basics design, low complexity and clean software TENTM encoder is 25x faster than JM17.0 ~400-800 times faster than many high complexity proposals (rough estimate). Majority of the tools brought to JCT-VC do not overlap with tools in TENTM TENTM is ideal to be used in collaborative phase to test various competing tools on a clean environment with good software Syntax&Semantics document and TENTM software are provided Make it easier for others to understand TENTM and make further developments To make sure concrete technical work starts at this meeting