Paper fingerprinting using alpha-masked image matching
1 like310 views
The document summarizes research from Canon Information Systems Research Australia on identifying paper fingerprints (PFP) for authentication purposes. It discusses using alpha-masked image matching and inpainting to make PFP matching more robust to changes in documents, such as printing. Experiments show alpha-masked correlation and normalized correlation are most effective at matching PFPs, even when a significant portion of the image is changed or masked. The researchers conclude PFP matching could be further improved by scanning documents at multiple orientations to separate diffuse and specular reflections.
![Australian R&D with global impact
Solution 1: alpha-masked correlation [Fitch et al]
Use weight α to mask out change areas during least-square matching:
E ( x0 , y0 ) = ∑ ( f1 ( x, y ) − f 2 ( x − x0 , y − y0 ) ) α1 ( x, y )α 2 ( x − x0 , y − y0 )
2
x, y
= α1 f12 ⊗ α 2 − 2α1 f1 ⊗ α 2 f 2 + α1 ⊗ α 2 f 22
If images f1 & f2 have zero mean, alpha-masked α f ⊗ α 2 f2
EN = −2 1 1
correlation can be simplified to normalized correlation: α1 ⊗ α 2
Paper 1 Cross-correlation
Peak strength = 12.44
Same
printed
texts
Alpha-masked correlation
Paper 2 Peak strength = 4.04
α
Copyright CISRA Slide 7 Printed 30 November 2009](https://image.slidesharecdn.com/dicta09slides-091218222956-phpapp02/85/Paper-fingerprinting-using-alpha-masked-image-matching-7-320.jpg)
![Australian R&D with global impact
Fill factor experiment: synthetic mask
Mask is successively eroded to reduce the fill-factor
50
100
alpha-masked correlation [8]
45
90 normalized correlation (section 2.3)
40 mean-filled correlation (section 3.3)
fill = 0.77 80
inpainting correlation (section 3.3)
PFP 1
35
70 correlation of non-matching pairs
match strength
30
60
fill = 0.58
25
50
PFP 2
20
40
fill = 0.45 15
30
…
10
20
Paper 1
after printing 5
10
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
fill = 0.05
mask fill factor
Copyright CISRA Slide 9 Printed 30 November 2009](https://image.slidesharecdn.com/dicta09slides-091218222956-phpapp02/85/Paper-fingerprinting-using-alpha-masked-image-matching-9-320.jpg)
![Australian R&D with global impact
Match strength vs. fill factor from a real document
The document is scanned twice, 256x256 image patches are matched
25
50
alpha-masked correlation [8]
40
20
fill = 0.94
match strength
30
15
fill = 0.86
20
10
fill = 0.73
…
10
5
0
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
fill = 0.16
mask fill factor
Copyright CISRA Slide 10 Printed 30 November 2009](https://image.slidesharecdn.com/dicta09slides-091218222956-phpapp02/85/Paper-fingerprinting-using-alpha-masked-image-matching-10-320.jpg)
Paper fingerprinting using alpha-masked image matching
- 1. Australian R&D with global impact Paper 55 Session3B 12:10PM-12:30PM on Thu 3 Dec Paper fingerprinting using alpha-masked image matching Tuan Pham, Stuart Perry, and Peter Fletcher Canon Information Systems Research Australia Copyright CISRA Slide 1 Printed 30 November 2009
- 2. Australian R&D with global impact Overview 1. Paper FingerPrint (PFP) Random structure of paper Uniqueness 2. Alpha-masked image matching Normalized correlation Image inpainting 3. PFP robustness Evaluation Possible improvements Copyright CISRA Slide 2 Printed 30 November 2009
- 3. Australian R&D with global impact The The original Paper fingerprint coupon fingerprint Intrinsic characteristic of a piece of paper that uniquely describes itself Document authentication application Paper Fingerprint System Match Strength Not Original Threshold Original Comparison Copyright CISRA Slide 3 Printed 30 November 2009
- 4. Australian R&D with global impact Paper FingerPrinting (PFP) using a scanner Office paper Copyright CISRA Slide 4 Printed 30 November 2009
- 5. Australian R&D with global impact PFP matching using cross-correlation Paper FingerPrint is a 256x256 8-bit grayscale scan at 600dpi PFP match is determined based on correlation peak strength: Peak strength > 10 → match Peak strength < 10 → non-match Using extreme value theory, the false alarm rate is about 1x10-50 ___ Fisher-Tippett distribution fit Non-Match PFP strengths Matching PFP strengths Copyright CISRA Slide 5 Printed 30 November 2009
- 6. Australian R&D with global impact Correlation is not robust against change We swap 7.5% of pixels around → match strength drops below 10 Printing also decreases match strength & increases false matches Some printed text Low correlation due to the creates a false negative influence of the printed areas Same sheet of paper Same Same printed text printed text High correlation due to creates a creates a false positive false positive the influence of the printed areas Different sheets of paper Copyright CISRA Slide 6 Printed 30 November 2009
- 7. Australian R&D with global impact Solution 1: alpha-masked correlation [Fitch et al] Use weight α to mask out change areas during least-square matching: E ( x0 , y0 ) = ∑ ( f1 ( x, y ) − f 2 ( x − x0 , y − y0 ) ) α1 ( x, y )α 2 ( x − x0 , y − y0 ) 2 x, y = α1 f12 ⊗ α 2 − 2α1 f1 ⊗ α 2 f 2 + α1 ⊗ α 2 f 22 If images f1 & f2 have zero mean, alpha-masked α f ⊗ α 2 f2 EN = −2 1 1 correlation can be simplified to normalized correlation: α1 ⊗ α 2 Paper 1 Cross-correlation Peak strength = 12.44 Same printed texts Alpha-masked correlation Paper 2 Peak strength = 4.04 α Copyright CISRA Slide 7 Printed 30 November 2009
- 8. Australian R&D with global impact Solution 2: inpainting followed by correlation Scanned paper with printed texts Filled-in with mean value Smooth inpainting How well do these different solutions compare to each other? Alpha-masked Normalized Mean-filled Inpainting Match 46.32 44.64 44.62 28.34 Non-match 8.16 5.84 5.88 5.36 Ratio 5.67 7.63 7.59 5.28 Normalized correlation is most discriminative, followed by mean-filled correlation Copyright CISRA Slide 8 Printed 30 November 2009
- 9. Australian R&D with global impact Fill factor experiment: synthetic mask Mask is successively eroded to reduce the fill-factor 50 100 alpha-masked correlation [8] 45 90 normalized correlation (section 2.3) 40 mean-filled correlation (section 3.3) fill = 0.77 80 inpainting correlation (section 3.3) PFP 1 35 70 correlation of non-matching pairs match strength 30 60 fill = 0.58 25 50 PFP 2 20 40 fill = 0.45 15 30 … 10 20 Paper 1 after printing 5 10 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 fill = 0.05 mask fill factor Copyright CISRA Slide 9 Printed 30 November 2009
- 10. Australian R&D with global impact Match strength vs. fill factor from a real document The document is scanned twice, 256x256 image patches are matched 25 50 alpha-masked correlation [8] 40 20 fill = 0.94 match strength 30 15 fill = 0.86 20 10 fill = 0.73 … 10 5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 fill = 0.16 mask fill factor Copyright CISRA Slide 10 Printed 30 November 2009
- 11. Australian R&D with global impact Improve PFP by multiple orientation scans Reflection from paper consists of diffuse and specular components These components can be separated by photometric stereography paper scanned at 0º paper scanned at 180º + _ + + diffuse reflectance specular reflectance Copyright CISRA Slide 11 Printed 30 November 2009
- 12. Australian R&D with global impact Improve PFP robustness by double-sided scan Verify the Paper FingerPrint on both sides. Front and back side must correlate The displacement between front and back side is fixed Front side Back side Correlation of scanned at 0° scanned at 0° front and back Copyright CISRA Slide 12 Printed 30 November 2009
- 13. Australian R&D with global impact Conclusions Conventional scanners capture internal structure of paper: Paper FingerPrint (PFP) PFPs are very unique and can be used for authentication PFPs can be matched even after printing PFPs can be made more robust using more than one scans Copyright CISRA Slide 13 Printed 30 November 2009
- 14. Australian R&D with global impact Thank you tuan.pham@cisra.canon.com.au Copyright CISRA Slide 14 Printed 30 November 2009