Improved license plate recognition for low resolution cctv forensics by integrating sparse representation-based super-resolution
3 likes2,000 views
The document presents an improved method for license plate recognition (LPR) in low-resolution CCTV forensics using sparse representation-based super-resolution. It discusses the effectiveness of this approach, which enhances image quality by utilizing high-resolution training images and specific image priors such as font and color. Experimental results show that this novel method significantly outperforms traditional LPR techniques without resolution enhancement, indicating a promising advancement for real-world applications.
![Improved License Plate Recognition for Low-Resolution CCTV Forensics by
Integrating Sparse Representation-based Super-Resolution
Sparse Representation-based
Super-Resolution for LPR (1/2)
• We enhance the quality of noisy and low-resolution
LP images through SR-based super-resolution
– to that end, we make use of a dictionary that contains a
large number of LP image patches that have been
extracted from high-quality LP images
9/20
Kd
Ki
×
= R],...,,...,[ 1 ∈xxxD
xi : the feature vector of the ith LP image patch extracted
from the set of high-resolution training images
d : the dimensionality of the feature vectors
K: the number of LP image patches in the dictionary](https://image.slidesharecdn.com/improvedlicenseplaterecognitionforlow-resolutioncctvforensicsbyintegratingsparserepresentation-based-131004110627-phpapp02/85/Improved-license-plate-recognition-for-low-resolution-cctv-forensics-by-integrating-sparse-representation-based-super-resolution-9-320.jpg)
Improved license plate recognition for low resolution cctv forensics by integrating sparse representation-based super-resolution
- 1. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution 12th International Workshop on Digital Forensics and Watermarking Auckland, New Zealand October 4, 2013 Hyun-seok Min, Seung Ho Lee, Wesley De Neve, Yong Man Ro Image and Video Systems Lab Department of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea
- 2. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Overview • Introduction • Proposed method for license plate recognition • Experimental results • Conclusions 2/20
- 3. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution License Plate Recognition • Automatic license plate recognition (LPR) – important for CCTV forensics – important for privacy protection 3/20 Large-scale Privacy Protection in Google Street View LPR in a CCTV system
- 4. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Challenges • Low effectiveness of LPR for real-world imagery – due to difficult capturing conditions • E.g., frequent use of long-distance CCTV cameras – output noisy LP images with a low resolution 4/20 Example input images captured by long-distance CCTV cameras in Korea
- 5. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Super-Resolution (1/2) • Super-resolution can be used to improve the effectiveness of current LPR systems – enhances images by producing high-resolution images from one or more low-resolution images 5/20
- 6. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Super-Resolution (2/2) • Shortcomings of current approaches – focus on maximizing the SNR between the original high-resolution image and the reconstructed image • do not focus on maximizing the effectiveness of LPR – focus on the use of general image information • do not take advantage of specific image priors, such as the font, location, and color of LP characters 6/20 Super-resolution Training images Similar?
- 7. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Proposed LPR Method (1/2) • Uses high-resolution LP images for 1) creating a dictionary for sparse representation-based (SR-based) super-resolution of the test LP images 2) the training of LP character classifiers (SVM) 7/20 Proposed LPR system License plate detection Sparse representation- based super- resolution Image patchesInput image High-quality training images ... ... ... ... ... ... ... ... Classifier for 1 Classifier for 2 ... Classifier for 0 Recognition of license plate characters ... Output CCTV
- 8. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Proposed LPR Method (2/2) • Advantages – takes into account specific image priors • e.g., the font, location, and color of LP characters – shares specific image priors among different steps • SR-based resolution enhancement of the test LP images • SVM-based LP character recognition 8/20
- 9. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Sparse Representation-based Super-Resolution for LPR (1/2) • We enhance the quality of noisy and low-resolution LP images through SR-based super-resolution – to that end, we make use of a dictionary that contains a large number of LP image patches that have been extracted from high-quality LP images 9/20 Kd Ki × = R],...,,...,[ 1 ∈xxxD xi : the feature vector of the ith LP image patch extracted from the set of high-resolution training images d : the dimensionality of the feature vectors K: the number of LP image patches in the dictionary
- 10. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Sparse Representation-based Super-Resolution for LPR (2/2) 10/20 License plate detection Sparse representation- based super- resolution Image patches High-quality training images ... ... ... ... ... ... ... ...
- 11. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Setup: Test Set (1/2) • Test set characteristics – 600 input images captured by CCTV cameras in Korea – resolution of input images: 1280×720 – we extracted 772 LP images from the 600 input images by making use of sliding concentric windows (SCW) • each input image contains one or more LP images 11/20 Example input images captured by long-distance CCTV cameras in Korea
- 12. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Setup: Test Dataset (2/2) • We divided the LP images in the test set into four groups, taking into account their spatial resolution 1) Resolution Group 1: LP image width ≤ 40 pixels 2) Resolution Group 2: 40 pixels < LP image width ≤ 55 pixels 3) Resolution Group 3: 55 pixels < LP image width ≤ 70 pixels 4) Resolution Group 4: 70 pixels < LP image width 12/20 Resolution Group 1 Resolution Group 2 Resolution Group 3 Resolution Group 4
- 13. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Setup: Dictionary Construction • Dictionary construction (& classifier training) – we made use of 100 high-resolution LP images that have been downloaded via Google Image Search – we randomly extracted 10,000 image patches from the training images • image patches have a size of 10×10 pixels – we made use of a dictionary with 10,000 image patches 13/20 Example high-resolution LP images
- 14. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Results: Influence of Resolution Enhancement (1/2) 14/20
- 15. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Results: Influence of Resolution Enhancement (2/2) • Observation – the effectiveness of the proposed LPR method is better than the effectiveness of • LPR without resolution enhancement • LPR using bicubic interpolation • Visual examples 15/20 (a) original low-resolution LP images (b) LP images that are the output of bicubic interpolation (c) LP images that are the output of SR-based super- resolution 45×22 56×25
- 16. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Results: Dictionary Influence (1/2) • We enhanced the resolution of the test LP images by means of two types of dictionaries 1) a dictionary consisting of high-resolution LP images 2) a dictionary consisting of general images 16/20 Examples of general imagesExample high-resolution LP images
- 17. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Results: Dictionary Influence (2/2) • Observation – SR-based super-resolution by means of high-quality LP training images is more effective than SR-based super-resolution by means of general images 17/20
- 18. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Experimental Results: Influence of Low-Level Visual Features • Observation – color Gabor wavelet features are most effective 18/20
- 19. Improved License Plate Recognition for Low-Resolution CCTV Forensics by Integrating Sparse Representation-based Super-Resolution Conclusions • We introduced a novel LPR method – applies SR-based super-resolution to test LP images – uses a common set of high-quality LP images to facilitate • SR-based super-resolution of LP test images • SVM-based LP character recognition • Our results indicate that the proposed method allows improving the effectiveness of LPR in real-world imagery 19/20
- 20. Any questions or comments? Contact information Hyun-seok Min: hsmin@kaist.ac.kr Web: http://ivylab.kaist.ac.kr 20/20