Graph Based Word Spotting Approach for Large Document Collections
1 like909 views
This document summarizes a graph-based word spotting approach for locating words in large collections of documents. It describes constructing a graph representation of words using graphemes based on convexities to be robust to handwriting deformations. An indexation method is used to retrieve target graphs based on the local structure attributes of nodes, represented as binary embeddings, to overcome the complexity of subgraph matching. The approach is evaluated on marriage records achieving a mean average precision of 48.64%, comparable to other methods while avoiding word segmentation. Future work will assess the stability of the graph representation in large multi-writer collections.
Graph Based Word Spotting Approach for Large Document Collections
- 1. Graph Based Word Spotting Approach for Large Document Collections Pau Riba, Josep Lladós and Alicia Fornés Third Graph-TA, 18 March 2015
- 2. Index Introduction Database Graph Construction Word spotting approach Graph Indexation Experiment Results Conclusions and Future Work
- 3. Introduction • Word Spotting: Locate a given query word in an image in terms of shape features • Most techniques use statistical representations. However, we propose a graph-based approach. • The proposed structural representation is suitable to be robust to the inherent deformations of handwriting. • To overcome the high computational complexity of subgraph matching through an Indexation formalism for graph retrieval.
- 4. The Barcelona Marriage Licenses • Old Marriage Licenses of the Cathedral of Barcelona (Spain) – Pope Benedict XIII established in 1408 a marriage fee (for building the Cathedral) – 244 books (15th -19th centuries) – Approx. 700.000 marriage licenses from 90 parish churches – The books include information on the couples, their parents, their jobs, and the tax paid depending on their social class NAME DATE JOB PLACE FEE NAME NAME
- 6. Word Spotting Approach • Graph Construction. • Graph Matching: – Bipartite Graph Matching: Suboptimal approximation of Graph Edit Distance. • Retrieval depending on the edition cost.
- 7. Graph Indexation • Binary embedding for each node, a vector of attributes representing their local structure. – The attributes count the length of a walk of order k originated in a vertex with label l. (Idea of the Morgan Index). – Binary-valued hash function to convert the vector to binary. – Graph retrieval in terms of finding target graphs whose nodes have a small Hamming distance from the query node. – Those nodes will vote into regions and afterwards the most voted ones are chosen as candidates to contain the query.
- 8. • Query: • Some retrieved words: • Typical errors Experiments: Qualitative Results
- 9. • 27 pages of marriage records • 6544 segmented words with 1751 transcriptions • 514 queries with 32 different word classes • Classic problems: Experiments: Quantitative Results Method mAP DTW 19,20 Graph-based 24,60 BoVW 30,00 Loci-based 40,06 nrHOG 56,06 Proposed 48,64 Binarization Lexical Variations Shared Letters
- 11. • 11 pages • 3609 words • 40 queries with • 8 different words Experiments: Quantitative Results Query Transcription Precision Recall mAP Eularia 0,0080 0,8462 0,7959 Hieronyma 0,0118 0,7875 0,9329 Jua$ 0,0149 0,5389 0,8490 defunct 0,0271 0,7886 0,6372 donsella 0,0420 0,8215 0,9454 pages 0,0590 0,9352 0,9463 rebere$ 0,0645 0,7676 0,9815 viudo 0,0133 0,6455 0,9231 Total 0,0301 0,7664 0,8764
- 12. Conclusions and Future Work • Graph-based representation is comparable to statistical approches in terms of performance and time requirements. • Graphemes based on convexities can be stable under the deformations of handwriting. • Graph indexing approach can deal with large collections and avoid the segmentation of words at the same time. • Future work will focus on the evaluation of the stability of graph- based representation in large multiwriter document collections.
- 13. Thank you!