Graph Regularised Hashing
1 like456 views
The document presents Graph Regularised Hashing (GRH), a supervised hashing model for efficient nearest-neighbor search in large datasets. It details a two-step iterative process that leverages an adjacency matrix to improve the accuracy of hash-codes by updating them based on nearest neighbors and learning binary classifiers for data partitioning. GRH outperforms other methods in terms of mean average precision and training speed, making it both an accurate and scalable solution.
![GRAPH REGULARISED HASHING
SEAN MORAN†, VICTOR LAVRENKO
† SEAN.MORAN@ED.AC.UK
RESEARCH QUESTION
• Locality sensitive hashing (LSH) [1] fractures the input feature
space space with randomly placed hyperplanes.
• Can we do better by using supervision to adjust the hyperplanes?
INTRODUCTION
• Problem: Constant time nearest-neighbour search in large datasets.
• Hashing-based approximate nearest neighbour (NN) search:
– Index image/documents into the buckets of hashtable(s)
– Encourage collisions between similar images/documents
110101
010111
111101
H
H
Content Based IR
Image: Imense Ltd
Image: Doersch et al.
Image: Xu et al.
Location Recognition
Near duplicate detection
010101
111101
.....
H
QUERY
DATABASE
QUERY
NEAREST
NEIGHBOURS
HASH TABLE
COMPUTE
SIMILARITY
• Advantages:
– O(1) lookup per query rather than O(N) (brute-force)
– Memory/storage saving due to compact binary codes
GRAPH REGULARISED HASHING (GRH)
• We propose a two step iterative hashing model, Graph Regularised
Hashing (GRH) [5]. GRH uses supervision in the form of an adja-
cency matrix that specifies whether or not data-points are related.
– Step A: Graph Regularisation: the K-bit hashcode of a data-
point is set to the average of the data-points of its nearest
neighbours as specified by the adjacency graph:
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
∗ S: Affinity (adjacency) matrix
∗ D: Diagonal degree matrix
∗ L: Binary bits at iteration m
∗ α ∈ {0, 1}: Linear interpolation parameter
– Step A is a simple sparse-sparse matrix multiplication, and can
be implemented very efficiently. Any existing hash function
e.g. LSH [1] can be used to initialise the bits in L0
– Step B: Data-Space Partitioning: the hashcodes produced in
Step A are used as the labels to learn K binary classifiers. This
is the out-of-sample extension step, allowing the encoding of
data-points not seen before:
for k = 1. . .K : min ||wk||2
+ C
Ntrd
i=1 ξik
s.t. Lik(wk xi + tk) ≥ 1 − ξik for i = 1. . .Ntrd
∗ wk: Hyperplane k tk: bias k
∗ xi: data-point i Lik: bit k of data-point i
∗ ξik: slack variable K: # bits Ntrd: # data-points
• Steps A-B are repeated for a set number of iterations (M) (e.g. < 10).
The learnt hyperplanes wk can then be used to encode unseen data-
points (via a simple dot-product).
STEP A: GRAPH REGULARISATION
• Toy example: nodes are images with 3-bit LSH encoding. Arcs
indicate nearest neighbour relationships. We show two images
(c,e) having their hashcodes updated in Step A:
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
1 -1 -1
1 -1 -1
1 1 1 -1 1 1
1 1 -1
STEP B: DATA-SPACE PARTITIONING
• Here we show a hyperplane being learnt using the first bit as
(highlighted with bold box) as label. One hyperplane is learnt
per bit.
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
w1. x−t1=0
w1
Negative
(-1)
half space
Positive
(+1)
half space
1 1 -1
1 -1 -1
-1 1 1
QUANTITATIVE RESULTS (CIFAR-10) (MORE DATASETS IN PAPER)
• Mean average precision (mAP) image retrieval results using
GIST features on CIFAR-10 ( : is significant at p < 0.01):
CIFAR-10
16 bits 32 bits 48 bits 64 bits
ITQ+CCA [2] 0.2015 0.2130 0.2208 0.2237
STH [3] 0.2352 0.2072 0.2118 0.2000
KSH [4] 0.2496 0.2785 0.2849 0.2905
GRH [5] 0.2991 0.3122 0.3252 0.3350
• Timings (seconds) averaged over 10 runs. GRH is 1) faster to
train and 2) is faster to encode unseen data-points:
Training Testing Total
GRH [5] 8.01 0.03 8.04
KSH [4] 74.02 0.10 74.12
BRE [6] 227.84 0.37 228.21
SUMMARY OF KEY FINDINGS
• First both accurate and scalable supervised hashing model
• Future work will extend GRH to streaming data sources
• Code online: https://github.com/sjmoran/grh
References:
[1] P. Indyk, R. Motwani: Approximate nearest neighbors: Towards removing the curse of dimensionality. In: STOC (1998).
[2] Y. Gong, S. Lazebnik: Iterative Quantisation. In: CVPR (2011). , [3] D. Zhang et al. Self-Taught Hashing. In: SIGIR (2010)., [4] W. Liu et
al. Supervised Hashing with Kernels. In: CVPR (2012)., [5] S. Moran, V. Lavrenko. Graph Regularised Hashing. In: ECIR (2015).,
[6] B. Kulis, T. Darrell et al. Binary Reconstructive Embedding. In: NIPS (2009).](https://image.slidesharecdn.com/grhposter-150326150643-conversion-gate01/85/Graph-Regularised-Hashing-1-320.jpg)
Graph Regularised Hashing
- 1. GRAPH REGULARISED HASHING SEAN MORAN†, VICTOR LAVRENKO † SEAN.MORAN@ED.AC.UK RESEARCH QUESTION • Locality sensitive hashing (LSH) [1] fractures the input feature space space with randomly placed hyperplanes. • Can we do better by using supervision to adjust the hyperplanes? INTRODUCTION • Problem: Constant time nearest-neighbour search in large datasets. • Hashing-based approximate nearest neighbour (NN) search: – Index image/documents into the buckets of hashtable(s) – Encourage collisions between similar images/documents 110101 010111 111101 H H Content Based IR Image: Imense Ltd Image: Doersch et al. Image: Xu et al. Location Recognition Near duplicate detection 010101 111101 ..... H QUERY DATABASE QUERY NEAREST NEIGHBOURS HASH TABLE COMPUTE SIMILARITY • Advantages: – O(1) lookup per query rather than O(N) (brute-force) – Memory/storage saving due to compact binary codes GRAPH REGULARISED HASHING (GRH) • We propose a two step iterative hashing model, Graph Regularised Hashing (GRH) [5]. GRH uses supervision in the form of an adja- cency matrix that specifies whether or not data-points are related. – Step A: Graph Regularisation: the K-bit hashcode of a data- point is set to the average of the data-points of its nearest neighbours as specified by the adjacency graph: Lm ← sgn α SD−1 Lm−1 + (1−α)L0 ∗ S: Affinity (adjacency) matrix ∗ D: Diagonal degree matrix ∗ L: Binary bits at iteration m ∗ α ∈ {0, 1}: Linear interpolation parameter – Step A is a simple sparse-sparse matrix multiplication, and can be implemented very efficiently. Any existing hash function e.g. LSH [1] can be used to initialise the bits in L0 – Step B: Data-Space Partitioning: the hashcodes produced in Step A are used as the labels to learn K binary classifiers. This is the out-of-sample extension step, allowing the encoding of data-points not seen before: for k = 1. . .K : min ||wk||2 + C Ntrd i=1 ξik s.t. Lik(wk xi + tk) ≥ 1 − ξik for i = 1. . .Ntrd ∗ wk: Hyperplane k tk: bias k ∗ xi: data-point i Lik: bit k of data-point i ∗ ξik: slack variable K: # bits Ntrd: # data-points • Steps A-B are repeated for a set number of iterations (M) (e.g. < 10). The learnt hyperplanes wk can then be used to encode unseen data- points (via a simple dot-product). STEP A: GRAPH REGULARISATION • Toy example: nodes are images with 3-bit LSH encoding. Arcs indicate nearest neighbour relationships. We show two images (c,e) having their hashcodes updated in Step A: -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 1 -1 -1 1 -1 -1 1 1 1 -1 1 1 1 1 -1 STEP B: DATA-SPACE PARTITIONING • Here we show a hyperplane being learnt using the first bit as (highlighted with bold box) as label. One hyperplane is learnt per bit. -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 w1. x−t1=0 w1 Negative (-1) half space Positive (+1) half space 1 1 -1 1 -1 -1 -1 1 1 QUANTITATIVE RESULTS (CIFAR-10) (MORE DATASETS IN PAPER) • Mean average precision (mAP) image retrieval results using GIST features on CIFAR-10 ( : is significant at p < 0.01): CIFAR-10 16 bits 32 bits 48 bits 64 bits ITQ+CCA [2] 0.2015 0.2130 0.2208 0.2237 STH [3] 0.2352 0.2072 0.2118 0.2000 KSH [4] 0.2496 0.2785 0.2849 0.2905 GRH [5] 0.2991 0.3122 0.3252 0.3350 • Timings (seconds) averaged over 10 runs. GRH is 1) faster to train and 2) is faster to encode unseen data-points: Training Testing Total GRH [5] 8.01 0.03 8.04 KSH [4] 74.02 0.10 74.12 BRE [6] 227.84 0.37 228.21 SUMMARY OF KEY FINDINGS • First both accurate and scalable supervised hashing model • Future work will extend GRH to streaming data sources • Code online: https://github.com/sjmoran/grh References: [1] P. Indyk, R. Motwani: Approximate nearest neighbors: Towards removing the curse of dimensionality. In: STOC (1998). [2] Y. Gong, S. Lazebnik: Iterative Quantisation. In: CVPR (2011). , [3] D. Zhang et al. Self-Taught Hashing. In: SIGIR (2010)., [4] W. Liu et al. Supervised Hashing with Kernels. In: CVPR (2012)., [5] S. Moran, V. Lavrenko. Graph Regularised Hashing. In: ECIR (2015)., [6] B. Kulis, T. Darrell et al. Binary Reconstructive Embedding. In: NIPS (2009).