Graph Regularised Hashing (ECIR'15 Talk)
2 likes865 views
The document presents an overview of Graph Regularised Hashing (GRH), detailing its two-step iterative model that includes graph regularization and data-space partitioning for effective hashcode learning. It evaluates GRH against previous hashing methods using standard datasets, showcasing its superior accuracy and efficiency in retrieval tasks with less necessary supervision. Future work aims to expand GRH applications to cross-modal hashing scenarios.
![Previous work
Data-independent: Locality Sensitive Hashing (LSH) [Indyk.
98]
Data-dependent (unsupervised): Anchor Graph Hashing
(AGH) [Liu et al. ’11], Spectral Hashing (SH) [Weiss ’08]
Data-dependent (supervised): Self Taught Hashing (STH)
[Zhang ’10], Supervised Hashing with Kernels (KSH) [Liu et
al. ’12], ITQ + CCA [Gong and Lazebnik ’11], Binary
Reconstructive Embedding (BRE) [Kulis and Darrell. ’09]](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-9-320.jpg)
![Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07][1]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)
[1] Diaz, F.: Regularizing query-based retrieval scores. In: IR
(2007)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-13-320.jpg)
![Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-14-320.jpg)
![Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-15-320.jpg)
![Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-16-320.jpg)
![Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-17-320.jpg)
![Datasets/Features
Standard evaluation datasets [Liu et al. ’12], [Gong and
Lazebnik ’11]:
CIFAR-10: 60K images, GIST descriptors, 10 classes1
MNIST: 70K images, grayscale pixels, 10 classes2
NUSWIDE: 270K images, GIST descriptors, 21 classes3
True NNs: images that share at least one class in common
[Liu et al. ’12]
1
http://www.cs.toronto.edu/~kriz/cifar.html
2
http://yann.lecun.com/exdb/mnist/
3
http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-34-320.jpg)
![Evaluation Metrics
Hamming ranking evaluation paradigm [Liu et al. ’12], [Gong
and Lazebnik ’11]
Standard evaluation metrics [Liu et al. ’12], [Gong and
Lazebnik ’11]:
Mean average precison (mAP)
Precision at Hamming radius 2 (P@R2)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-35-320.jpg)
![GRH Timing (CIFAR-10 @ 32 bits)
Timings (s)
Method Train Test Total
GRH 42.68 0.613 43.29
KSH [1] 81.17 0.103 82.27
BRE [2] 231.1 0.370 231.4
[1] Liu, W.: Supervised Hashing with Kernels. In: CVPR (2012)
[2] Kulis, B.: Binary Reconstructive Embedding. In: NIPS (2009)](https://image.slidesharecdn.com/ecirtalk-150329062642-conversion-gate01/85/Graph-Regularised-Hashing-ECIR-15-Talk-44-320.jpg)
Graph Regularised Hashing (ECIR'15 Talk)
- 1. Graph Regularised Hashing Sean Moran and Victor Lavrenko Institute of Language, Cognition and Computation School of Informatics University of Edinburgh ECIR’15 Vienna, March 2015
- 2. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
- 3. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
- 4. Locality Sensitive Hashing H DATABASE
- 6. Locality Sensitive Hashing 110101 010111 H H QUERY DATABASE HASH TABLE 010101 111101 .....
- 7. Locality Sensitive Hashing 110101 010111 H H COMPUTE SIMILARITY NEAREST NEIGHBOURS QUERY 010101 111101 ..... H DATABASE QUERY HASH TABLE
- 8. Locality Sensitive Hashing 110101 010111 111101 H H Content Based IR Image: Imense Ltd Image: Doersch et al. Image: Xu et al. Location Recognition Near duplicate detection 010101 111101 ..... H QUERY DATABASE QUERY NEAREST NEIGHBOURS HASH TABLE COMPUTE SIMILARITY
- 9. Previous work Data-independent: Locality Sensitive Hashing (LSH) [Indyk. 98] Data-dependent (unsupervised): Anchor Graph Hashing (AGH) [Liu et al. ’11], Spectral Hashing (SH) [Weiss ’08] Data-dependent (supervised): Self Taught Hashing (STH) [Zhang ’10], Supervised Hashing with Kernels (KSH) [Liu et al. ’12], ITQ + CCA [Gong and Lazebnik ’11], Binary Reconstructive Embedding (BRE) [Kulis and Darrell. ’09]
- 10. Previous work Method Data-Dependent Supervised Scalable Effectiveness LSH Low SH Low STH Medium BRE Medium ITQ+CCA Medium KSH High GRH High
- 11. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
- 12. Graph Regularised Hashing (GRH) Two step iterative hashing model: Step A: Graph Regularisation Lm ← sgn α SD−1 Lm−1 + (1−α)L0 Step B: Data-Space Partitioning for k = 1. . .K : min ||hk ||2 + C N i=1 ξik s.t. Lik (hk xi + bk ) ≥ 1 − ξik for i = 1. . .N Repeat for a set number of iterations (M)
- 13. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07][1] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1) [1] Diaz, F.: Regularizing query-based retrieval scores. In: IR (2007)
- 14. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
- 15. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
- 16. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
- 17. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
- 18. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 ba c 1 1 1 S a b c a 1 1 0 b 1 1 1 c 0 1 1 D−1 a b c a 0.5 0 0 b 0 0.33 0 c 0 0 0.5 L0 b1 b2 b3 a −1 −1 −1 b −1 1 1 c 1 1 1
- 19. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 ba c 1 1 1 L1 = sgn −1 0 0 −0.33 0.33 0.33 0 1 1
- 20. Graph Regularised Hashing (GRH) -1 1 1 -1 1 1 ba c 1 1 1 L1 = b1 b2 b3 a −1 1 1 b −1 1 1 c 1 1 1
- 21. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
- 22. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
- 23. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
- 24. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
- 25. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
- 26. Graph Regularised Hashing (GRH) ba c e f g h d
- 27. Graph Regularised Hashing (GRH) ba c e f g h d
- 28. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 1 -1 -1 1 -1 -1 1 1 1
- 29. Graph Regularised Hashing (GRH) 1 1 1 -1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 1 1 1 1 -1 -1 1 -1 -1
- 30. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 -1 1 1 1 1 1 First bit flipped 1 1 -1 Second bit flipped 1 -1 -1
- 31. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 h1 . x−b1=0 h1 Negative (-1) half space Positive (+1) half space 1 1 -1 1 -1 -1 -1 1 1
- 32. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -11 -1 -1 ba c e f g h 1 1 -1 d -1 1 1 -1 1 1 h2 Positive (+1) half space h2 . x−b2=0 Negative (-1) half space
- 34. Datasets/Features Standard evaluation datasets [Liu et al. ’12], [Gong and Lazebnik ’11]: CIFAR-10: 60K images, GIST descriptors, 10 classes1 MNIST: 70K images, grayscale pixels, 10 classes2 NUSWIDE: 270K images, GIST descriptors, 21 classes3 True NNs: images that share at least one class in common [Liu et al. ’12] 1 http://www.cs.toronto.edu/~kriz/cifar.html 2 http://yann.lecun.com/exdb/mnist/ 3 http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm
- 35. Evaluation Metrics Hamming ranking evaluation paradigm [Liu et al. ’12], [Gong and Lazebnik ’11] Standard evaluation metrics [Liu et al. ’12], [Gong and Lazebnik ’11]: Mean average precison (mAP) Precision at Hamming radius 2 (P@R2)
- 36. GRH vs Literature (CIFAR-10 @ 32 bits) LSH BRE STH KSH GRH (Linear) GRH (RBF) 0.10 0.15 0.20 0.25 0.30 0.35 mAP Linear GRH Non-linear GRH
- 37. GRH vs Literature (CIFAR-10 @ 32 bits) LSH BRE STH KSH GRH (Linear)GRH (RBF) 0.10 0.15 0.20 0.25 0.30 0.35 mAP GRH's straightforward objective outperforms more complex objectives
- 38. GRH vs Literature (CIFAR-10) 16 24 32 40 48 56 64 0.10 0.15 0.20 0.25 0.30 0.35 0.40 LSH BRE KSH GRH # Bits mAP
- 39. GRH vs Literature (CIFAR-10) Small amount of supervision required 16 24 32 40 48 56 64 0.10 0.15 0.20 0.25 0.30 0.35 0.40 LSH BRE KSH GRH # Bits mAP +25-30%
- 40. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits) GRH (Linear) GRH (RBF) 0.00 0.05 0.10 0.15 0.20 0.25 0.30 LSH ITQ+CCA mAP Linear GRH Non-Linear GRH
- 41. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits) GRH (Linear) GRH (RBF) 0.00 0.05 0.10 0.15 0.20 0.25 0.30 LSH ITQ+CCA mAP Eigendecomposition not necessary - saves O(d^3)
- 42. GRH vs # Supervisory Data-Points (CIFAR-10) Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 mAP Linear GRH Non-linear GRH Linear GRH Non-linear GRH
- 43. GRH vs # Supervisory Data-Points (CIFAR-10) Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 mAP Small amount of supervision required
- 44. GRH Timing (CIFAR-10 @ 32 bits) Timings (s) Method Train Test Total GRH 42.68 0.613 43.29 KSH [1] 81.17 0.103 82.27 BRE [2] 231.1 0.370 231.4 [1] Liu, W.: Supervised Hashing with Kernels. In: CVPR (2012) [2] Kulis, B.: Binary Reconstructive Embedding. In: NIPS (2009)
- 46. Conclusions and Future Work Supervised hashing model that is both accurate and easily scalable Take-home messages: Regularising bits over a graph is effective (and efficient) for hashcode learning An intermediate eigendecomposition step is not necessary Hyperplanes (linear hypersurfaces) can achieve a very good retrieval accuracy Future work: extend to the cross-modal hashing scenario (e.g. Image ↔ Text, English ↔ Spanish)
- 47. Thank you for your attention Sean Moran Code and datasets available at: sean.moran@ed.ac.uk www.seanjmoran.com