INCEPT: Intra CU Depth Prediction for HEVC
0 likes1,153 views
The document presents an algorithm called INCEPT for intra coding unit (CU) depth prediction in High Efficiency Video Coding (HEVC), aimed at improving encoding efficiency. It describes the algorithm's methodology, including feature extraction and CU depth prediction phases, and reports experimental results showing a significant reduction in encoding time while maintaining bitrate efficiency. The findings suggest that the INCEPT algorithm outperforms benchmark algorithms in both time savings and coding quality, with potential for extension to the Versatile Video Coding (VVC) standard.
![Introduction
Intra CU Depth Estimation
Start CTU
Depth i ∈
[dmin, dmax]
No
d > dmax
End CTU
PU mode decisions
i = i + 1 for next
CU
No
Yes
Yes
Figure: Quad-tree CU algorithm for partitioning of a CTU.
Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 7](https://image.slidesharecdn.com/mmsppresentationvignesh-211007192407/85/INCEPT-Intra-CU-Depth-Prediction-for-HEVC-7-320.jpg)
INCEPT: Intra CU Depth Prediction for HEVC
- 1. INCEPT: Intra CU Depth Prediction for HEVC Vignesh V Menon, Hadi Amirpour, Christian Timmerer and Mohammad Ghanbari Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), University of Klagenfurt, Austria 06 October 2021 Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 1
- 2. Outline 1 Introduction 2 INCEPT Algorithm 3 Evaluation 4 Conclusions and Future Directions Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 2
- 3. Introduction Introduction Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 3
- 4. Introduction Introduction Background of High Efficiency Video Coding (HEVC)3 The compression efficiency of HEVC has been improved greatly compared to AVC1 with the adoption of numerous advanced tools. HEVC has a flexible quad-tree coding block partitioning structure and uses the Coding Units (CUs) and Prediction Units (PUs) concept to manage the partition.2 Each video frame is divided into non-overlapping predefined size Coding Tree Unit (CTUs), and each CTU is then recursively divided into CUs. A CU can be further partitioned into PUs and Transform Units (TUs). The partitioning is recursive within a quad-tree hierarchy making the codec computation- ally intensive. 1 T. Wiegand et al. “Overview of the H.264/AVC video coding standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7 (2003), pp. 560–576. 2 Ekrem Çetinkaya et al. “CTU depth decision algorithms for HEVC: A survey”. In: Signal Processing: Image Communication 99 (2021), p. 116442. issn: 0923-5965. doi: https://doi.org/10.1016/j.image.2021.116442. url: https://www.sciencedirect.com/science/article/pii/S0923596521002113. 3 G. J. Sullivan et al. “Overview of the high efficiency video coding (HEVC) standard”. In: IEEE Transactions on circuits and systems for video technology 22.12 (2012), pp. 1649–1668. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 4
- 5. Introduction Introduction CU partitioning in HEVC 32x32 16x16 8x8 depth 1 depth 0 depth 2 depth 3 Figure: An example of the CU partitioning structure of a CTU and its corresponding quad-tree structure. The white nodes represent CUs that have been partitioned. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 5
- 6. Introduction Introduction Intra Coding of High Efficiency Video Coding (HEVC)6 Intra frames are essential to conditions such as mobile devices with limited computational power, transmission over error-prone channels, and frequent random access.4 In intra coding, PU’s size is generally equal to the corresponding CU. In addition, up to 35 prediction modes, including DC mode, Planar mode, and 33 angular modes, are provided for intra prediction in HEVC, enabling more accurate predictions. The increase in CU depth and prediction modes also causes higher coding complexity than Advanced Video Coding (AVC).5 4 Yun Zhang et al. “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. In: ACM Trans. Multimedia Comput. Commun. Appl. 15.3 (July 2019). issn: 1551-6857. doi: 10.1145/3321510. url: https://doi.org/10.1145/3321510. 5 Wiegand et al., “Overview of the H.264/AVC video coding standard”. 6 Sullivan et al., “Overview of the high efficiency video coding (HEVC) standard”. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 6
- 7. Introduction Intra CU Depth Estimation Start CTU Depth i ∈ [dmin, dmax] No d > dmax End CTU PU mode decisions i = i + 1 for next CU No Yes Yes Figure: Quad-tree CU algorithm for partitioning of a CTU. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 7
- 8. INCEPT Algorithm INCEPT Algorithm Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 8
- 9. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 1: Feature Extraction Compute texture energy per Coding Tree Unit (CTU) A DCT-based energy function is used to determine the block-wise feature of each frame defined as: EDCT = w X i=1 h X j=1 e|( ij wh )2−1| |DCT(i − 1, j − 1)| (1) where w and h are the width and height of the block, and DCT(i, j) is the (i, j)th DCT component when i + j > 2, and 0 otherwise. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 9
- 10. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 2: CU Depth Prediction For each CTU: Inputs: HL, HA, HAL : weighted DCT energy of the neighboring CTUs mL, mA, mAL : mean of the CU depths of the neighboring CTUs vL, vA, vAL : variance of the CU depths of the neighboring CTUs Output: dmin and dmax Step 1: Compute Hmin and Hmax , mmin and mmax and vmin and vmax Hmin = min(HL, HA, HAL) Hmax = max(HL, HA, HAL) mmin = min(mL, mA, mAL) mmax = max(mL, mA, mAL) vmin = min(vL, vA, vAL) vmax = max(vL, vA, vAL) Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 10
- 11. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 2: CU Depth Prediction Step 2: Compute HC . Step 3: Determine dmin and dmax if HC < Hmin then dmax = dmmin + vmin 2 + φe else dmax = dmmax + vmax 2 + φe if HC > Hmax then dmin = bmmax − vmin 2 + ψc else dmin = bmmin − vmax 2 + ψc Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 11
- 12. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Depth prediction accuracy Table: Depth prediction accuracy of INCEPT algorithm over JVET sequences and QPs. Video QP22 QP26 QP30 QP34 Average CatRobot 97.93% 95.98% 95.34% 94.76% 96.00% DaylightRoad2 98.04% 97.86% 97.02% 96.39% 97.33% FoodMarket4 99.32% 99.1% 98.79% 97.34% 98.64% ParkRunning3 96.89% 95.34% 94.71% 94.03% 95.24% Average 98.05% 97.07% 96.46% 95.63% 96.80% Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 12
- 13. Evaluation Evaluation Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 13
- 14. Evaluation Evaluation Test Methodology Test videos: (i) JVET test sequences,7 (i) MCML test sequences,8 and (ii) SJTU test sequences9 representing various types of contents. System: Dual-processor server with Intel Xeon Gold 5218R (80 cores, 2.10 GHz) The presented algorithms were implemented in x265 v3.410 and tested with the veryslow preset. Sequences were encoded with ALL intra configuration and QPs ∀{22, 26, 30, 34}. The lower resolution sources were generated from the original video source by applying bi-cubic scaling using FFmpeg.11 7 Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018. 8 Manri Cheon and Jong-Seok Lee. “Subjective and Objective Quality Assessment of Compressed 4K UHD Videos for Immersive Experience”. In: IEEE Transactions on Circuits and Systems for Video Technology 28.7 (2018), pp. 1467–1480. doi: 10.1109/TCSVT.2017.2683504. 9 L. Song et al. “The SJTU 4K Video Sequence Dataset”. In: Fifth International Workshop on Quality of Multimedia Experience (QoMEX2013) (July 2013). 10 MulticoreWare Inc. x265 HEVC Encoder/H.265 Video Codec. url: http://x265.org/. 11 FFmpeg. FFmpeg Documentation. url: https://ffmpeg.org/ffmpeg.html. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 14
- 15. Evaluation Evaluation Test Methodology Metrics: ∆T: the cumulative time savings for all bitrate representations compared with the stand- alone encoding Bjøntegaard delta rates,12 BDRP and BDRV : average increase in bitrate of the represen- tations compared with that of the stand-alone encoding to maintain the same PSNR and VMAF. BDRP ∆T and BDRV ∆T are calculated to compare the performance of the algorithms. The lower is the value of BDR ∆T , the better is the performance of the algorithm. 12 G. Bjontegaard. “Calculation of average PSNR differences between RD-curves”. In: VCEG-M33 (2001). Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 15
- 16. Evaluation Evaluation Experimental Results Table: ∆T and BDR comparison between the INCEPT algorithm and the benchmark algorithms. ADTS13 SCDP14 INCEPT Video ∆T BDRP BDRV ∆T BDRP BDRV ∆T BDRP BDRV CatRobot 13.74% 2.36% 2.02% 24.97% 3.89% 3.71% 24.75% 3.08% 3.25% DaylightRoad 16.38% 1.25% 1.19% 26.30% 3.15% 2.44% 26.20% 1.72% 1.54% FoodMarket 16.15% 1.06% 1.12% 19.00% 2.56% 1.26% 20.09% 1.40% 0.72% Basketball 13.75% 1.96% 1.68% 18.69% 4.82% 3.29% 19.13% 2.16% 1.88% Bunny 15.40% 1.98% 2.03% 18.11% 3.32% 3.09% 18.69% 1.67% 1.69% Lake 13.01% 1.08% 0.97% 22.54% 2.95% 2.12% 22.89% 1.19% -2.25% BundNightScape 16.68% 1.08% 0.99% 28.18% 2.95% 4.10% 28.02% 1.43% 1.73% CampfireParty 12.27% 0.78% 1.12% 22.11% 1.88% 2.64% 23.41% 0.82% 1.36% Fountains 17.61% 0.88% 1.07% 25.12% 2.72% 2.66% 26.90% 1.53% 1.61% Average 15.00% 1.38% 1.35% 22.78% 3.14% 2.81% 23.34% 1.67% 1.28% 13 Xin Lu, Chang Yu, and Xuesong Jin. “A fast HEVC intra-coding algorithm based on texture homogeneity and spatio-temporal correlation”. In: EURASIP Journal on Advances in Signal Processing 37 (2018). doi: https://doi.org/10.1186/s13634-018-0558-4. 14 Zhang et al., “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 16
- 17. Evaluation Evaluation Experimental Results SCDP ADTS INCEPT 0 2 4 6 8 10 12 14 BDRv / T 12.35% 9.03% 5.49% Figure: Comparison of BDRV ∆T with benchmark algorithms Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 17
- 18. Evaluation Evaluation Experimental Results QP22 QP26 QP30 QP34 0 5 10 15 20 25 30 35 T (%) 28.60% 25.54% 23.34% 21.56% Figure: Average time saving using INCEPT algorithm for various QPs Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 18
- 19. Conclusions and Future Directions Conclusions and Future Directions Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 19
- 20. Conclusions and Future Directions Conclusions We proposed fast intra CU depth prediction algorithm for HEVC encoding. We analyzed the algorithm by comparing against two benchmark algorithms after integrat- ing them into the x265 open-source HEVC encoder. Experimental results demonstrate that the proposed INCEPT algorithm decreased the over- all encoding time by 23.34% with a negligible increase in bitrate. BDR ∆T metric is better for INCEPT compared to the two benchmark algorithms. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 20
- 21. Conclusions and Future Directions Future Directions More encoding time can be saved by reducing the time taken for PU mode decisions for each CU15,16,17. The proposed INCEPT algorithm can be easily extended for the VVC standard18. 15 Lu, Yu, and Jin, “A fast HEVC intra-coding algorithm based on texture homogeneity and spatio-temporal correlation”. 16 Jinzheng Lu and Yixian Li. “Fast Algorithm for CU Partitioning and Mode Selection in HEVC Intra Prediction”. In: 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). 2019, pp. 1–5. doi: 10.1109/CISP-BMEI48845.2019.8966035. 17 Tao Zhang et al. “Fast Intra-Mode and CU Size Decision for HEVC”. In: IEEE Transactions on Circuits and Systems for Video Technology 27.8 (2017), pp. 1714–1726. doi: 10.1109/TCSVT.2016.2556518. 18 Gary Sullivan. “Versatile Video Coding (VVC) Arrives”. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP). 2020, pp. 1–1. doi: 10.1109/VCIP49819.2020.9301847. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 21
- 22. Conclusions and Future Directions Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Hadi Amirpour (hadi.amirpourazarian@aau.at) Christian Timmerer (Christian.Timmerer@aau.at) Mohammad Ghanbari (ghan@essex.ac.uk) Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 22