Multi-focus Application Presentation in ICSSE2017
0 likes48 views
This document summarizes a research paper that proposes an object-based multi-focus method implemented on mobile devices. The method detects key points and edges in near and far-focused images using algorithms like SIFT, ORB, and Canny edge detection. It then defines near and far regions based on color and detects objects across images. Finally, it combines the two source images using a fusion algorithm to generate a single multi-focused image. The goal is to improve mobile phone camera capabilities by capturing objects and scenes with clearer details across different focal regions.
![NCNU
Feature detection O r i e n t e d F A S T a n d R o t a t e d B R I E F ( O R B )
Descriptor Run time [ms.] Speed – up [-]
SIFT 448.6 1.00
SURF 117.1 3.83
BRIEF 3.8 118.05
BRISK 10.6 42.32
ORB 4.2 106.80
Table. Computation times of the different descriptors for 1000 key points.
SIFT ORB
O. Miksik and K. Mikolajczyk, “Evaluation of local detectors and descriptors for fast feature matching,” in proc. of IEEE International
Conference in Pattern Recognition (ICPR), pp. 2681-2684, Nov 2012.](https://image.slidesharecdn.com/icsse2017presentation-171002083508/85/Multi-focus-Application-Presentation-in-ICSSE2017-11-320.jpg)
Multi-focus Application Presentation in ICSSE2017
- 1. MULTI FOCUS APPLICATION IN MOBILE PHONE “ P r e s e n t e r : T r o n g - A n B u i A d v i s o r : P r o f . P e i - J u n L e e Video Processing and Application Laboratory, National Chi Nan University, Taiwan I E E E I N E R N AT I O N A L C O N F E R E N C E O N S Y S T E M S C I E N C E A N D E N G I N E E R I N G | I C S S E 2 0 1 7 July 21-23, 2017 Ho Chi Minh City, Vietnam
- 2. NCNU ABOUT US Pei-Jun Lee Title: Professor E-mail: pjlee@ncnu.edu.tw Trong-An Bui Title: Researcher E-mail: trongan93@gmail.com Video Processing and Application Laboratory, National Chi Nan University, Taiwan PAPER T I T L E : M U LT I - F O C U S A P P L I C AT I O N I N M O B I L E P H O N E This paper proposes an object based multi-focus method which is implemented on mobile device.
- 3. NCNU 3 OUTLINE IN TR OD U C TION GOAL AN D PR OC ESS KEY POIN T FEATU R E D ETEC TION ED GES D ETEC TION C OLOR D ETEC TION D EFIN E N EAR AN D FAR R EGION C OMBIN E R EGION W ITH FU SION ALGOR ITH M C ON C LU SION
- 4. NCNU INTRODUCTION AT P R E S E N T • Multi-camera are used to record an object or a landscape at different angles. • To record the information of image in term of regions and objects with the clearest and most complete detail. • To improve the resolution of the synthesized image. Multi camera
- 5. NCNU Camera – Mobile phone • Incessant improvement and development in image quality of the image captured by the mobile phone camera. • Two cameras play different roles in focusing and defining areas of the photo frame, a camera acts as a close – up focus (near focus) and create depth information in the image (far focus), so the multi-focus application promises to be one of the most powerful applications in the future. There has been an incessant improvement and development in image quality of the image captured by the mobile phone camera, especially the emergence of multi-camera technology. Technology trend INTRODUCTION AT P R E S E N T
- 6. NCNU GOALS TA R G E T
- 7. NCNU PROCESS M A I N P R O C E S S Source Image (Near/Far focus) Canny Edge detection Color detection Key points feature detection Oriented FAST and Rotated BRIEF (ORB) Detect Object Feature Matching Select Focus Pixels Apply Fusion Algorithm for combine 2 source images Touch and focus by touched object End
- 8. NCNU PROCESS M A I N P R O C E S S Oriented FAST and Rotated BRIEF (ORB)
- 9. NCNU Feature detection S I F T Scale-invariant feature transform (SIFT) is an algorithm to detect and describe local features in images. Types of invariance • Illumination • Scale • Rotation • Full perspective
- 10. NCNU SIFTA L G O R I T H M Constructing Scale space Difference of Gaussian (DoG) Finding Key point Eliminating bad key points Assigning OrientationGenerating SIFT Features • In scale Space we take the image and generate progressively blurred out images, then resize the original image to half and generate blurred images. • Images that are of same size but different scale are called octaves. 1. Removing Low Contrast features • If magnitude of intensity at current pixel is less than certain value then it is rejected. 2. Removing edges • For poorly defined peaks in the DoG function, the principal curvature across the edge would be much larger than the principal curvature along it • To determine edges Hessian matrix is used.
- 11. NCNU Feature detection O r i e n t e d F A S T a n d R o t a t e d B R I E F ( O R B ) Descriptor Run time [ms.] Speed – up [-] SIFT 448.6 1.00 SURF 117.1 3.83 BRIEF 3.8 118.05 BRISK 10.6 42.32 ORB 4.2 106.80 Table. Computation times of the different descriptors for 1000 key points. SIFT ORB O. Miksik and K. Mikolajczyk, “Evaluation of local detectors and descriptors for fast feature matching,” in proc. of IEEE International Conference in Pattern Recognition (ICPR), pp. 2681-2684, Nov 2012.
- 12. NCNU O r i e n t e d F A S T a n d R o t a t e d B R I E F ( O R B ) Key point feature detected (a) From near focus region, (b) From far focus region. Feature detection
- 13. NCNU EDGES DETECTION G R A D I E N T M A P N e a r F o c u s e d I m a g e F a r F o c u s e d I m a g e
- 14. NCNU EDGES DETECTION C a n n y d e t e c t o r 𝐸𝐷 = 𝟏, 𝑤ℎ𝑒𝑟𝑒 𝑝𝑖𝑥𝑒𝑙 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 > 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 < 𝑝𝑖𝑥𝑒𝑙 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 > 𝑝𝑖𝑥𝑒𝑙 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑝𝑖𝑥𝑒𝑙 𝑖𝑠 𝑎𝑏𝑜𝑣𝑒 𝑡ℎ𝑒 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝟎, 𝑤ℎ𝑒𝑟𝑒 𝑝𝑖𝑥𝑒𝑙 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 < 𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 Define ED by 2 values 1: pixel is accepted, 0: pixel is rejected
- 15. NCNU EDGES DETECTION C a n n y d e t e c t o r N e a r F o c u s e d I m a g e F a r F o c u s e d I m a g e
- 16. NCNU COLOR DETECTION R G B d e t e c t o r 𝐶𝐷𝐸 = 𝒍𝒘𝒄𝒐𝒍𝒐𝒓(𝑰) 𝟎 ≤ 𝑰 𝟎 ≤ 𝒖𝒑𝒄𝒐𝒍𝒐𝒓 𝑰 𝟎 ∧ 𝒍𝒘𝒄𝒐𝒍𝒐𝒓(𝑰) 𝟏 ≤ 𝑰 𝟏 ≤ 𝒖𝒑𝒄𝒐𝒍𝒐𝒓 𝑰 𝟏 ∧ 𝒍𝒘𝒄𝒐𝒍𝒐𝒓(𝑰) 𝟐≤ 𝑰 𝟐 ≤ 𝒖𝒑𝒄𝒐𝒍𝒐𝒓 𝑰 𝟐 lwcolor: lower color values upcolor: upper color values Source Image (Near/Far focus) Split to 3 channels R channel G channel B channel
- 17. NCNU COLOR DETECTION R G B d e t e c t o r N e a r F o c u s e d I m a g e F a r F o c u s e d I m a g e
- 18. NCNU REGION DETECTIONN e a r a n d F a r r e g i o n d e t e c t o r 𝑫 = 𝟏, 𝑤ℎ𝑒𝑟𝑒 𝑝𝑖𝑥𝑒𝑙 𝑖𝑠 𝑖𝑛 𝑛𝑒𝑎𝑟 𝑓𝑜𝑐𝑢𝑠 𝑟𝑒𝑔𝑖𝑜𝑛 𝟎, 𝑤ℎ𝑒𝑟𝑒 𝑝𝑖𝑥𝑒𝑙 𝑖𝑠 𝑖𝑛 𝑓𝑎𝑟 𝑓𝑜𝑐𝑢𝑠 𝑟𝑒𝑔𝑖𝑜𝑛 𝟎. 𝟓, 𝑤ℎ𝑒𝑟𝑒 𝑝𝑖𝑥𝑒𝑙 𝑖𝑠 𝑖𝑛 𝑎𝑛𝑜𝑡ℎ𝑒𝑟 𝑟𝑒𝑔𝑖𝑜𝑛
- 19. NCNU REGION DETECTIONN e a r a n d F a r r e g i o n d e t e c t o r N e a r F o c u s e d I m a g e F a r F o c u s e d I m a g e
- 20. NCNU REGION DETECTIONN e a r a n d F a r r e g i o n d e t e c t o r N e a r F o c u s e d I m a g e F a r F o c u s e d I m a g e C o m b i n e I m a g e
- 21. NCNU 𝑰 𝑭 𝒙, 𝒚 = 𝑫 𝒙, 𝒚 𝑰 𝟏 𝒙, 𝒚 + (𝟏 − 𝑫 𝒙, 𝒚 )𝑰 𝟐(𝒙, 𝒚) COMBINE MULTI-IMAGE F u s i o n A l g o r i t h m
- 22. NCNU CONCLUSION • Solving the speed and performance of key point feature determination. • The proposed algorithm can avoid out of memory and help to better processing the multi-focus image obtained on the phone. FULTURE WORK • Apply Deep Learning to Object an Region detection and Object recognition.
- 23. NCNU CONCLUSION Thanks for your listening! Q&A
Editor's Notes
- #10: - Illumination: chiếu sáng, nói về contract khác nhau. Scale Rotation Full perspective: