ICCV2013 reading: Learning to rank using privileged information
3 likes1,309 views
The paper discusses learning using privileged information (LUPI) in supervised binary classification, where additional data helps distinguish easy and hard examples for better prediction accuracy. It explores various methods like SVM and SVMrank, testing different types of privileged info across experimental setups, demonstrating that rank transfer often yields superior results. The findings suggest that while additional privileged information can enhance performance, its suitability is still uncertain and requires further investigation.
![Linear SVM
• Ordinary convergence rate = 𝑂𝑂(𝑁𝑁−1/2
)
• It improves to 𝑂𝑂(𝑁𝑁−1
)
– if we knew the optimal slack values 𝜉𝜉𝑖𝑖 in advance
(OracleSVM [Vapnik+ 2009])
min
𝑤𝑤∈ℝ𝑑𝑑,𝑏𝑏∈ℝ,𝜉𝜉𝑖𝑖∈ℝ](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-7-320.jpg){kind=icon}
![SVM+
• A 1st model for LUPI
– Use privileged data as a proxy to the oracle
– Parameterize 𝜉𝜉𝑖𝑖 = 𝑤𝑤∗, 𝑥𝑥𝑖𝑖
∗
+ 𝑏𝑏∗
[Vapnik+ NN2009, NIPS2010]](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-9-320.jpg){kind=icon}
![Why should SVM+ be improved?
• Cannot be solved by popular SVM packages
– Although good optimization algorithms were
derived [Pechyony+ 2011], they work only with the dual.](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-10-320.jpg){kind=icon}
![SVMrank
• Slack variables tell us which training example
pairs are easy / hard / impossible to separate
[Joachims KDD2002]](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-12-320.jpg){kind=icon}
![(1) Attributes as privileged info
• Animals with Attributes Dataset
– 10 species ( = classes), 85 properties ( = attributes)
• Features: 2000-dim SURF
• Privileged: 85-dim predicted attributes
[Lampert+ PAMI2014]
• Learn 1-vs-1 classifiers with 100 training
samples](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-17-320.jpg){kind=icon}
![(3) Texts as privileged info
• IsraelImages dataset [Bekkerman+ CVPR2007]
– 11 classes, 1800 images with a textual description
up to 18 words
• Features: 4096-dim Fisher vectors
• Privileged: BoWs from the texts
• Learn 1-vs-1 classifiers
Desert Trees](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-22-320.jpg){kind=icon}
![(4) Rationales as privileged info
• Hot or Not dataset [Donahue+ ICCV2011]
• Features: 500-dim densely sampled SIFT from
the whole image
• Privileged: 500-dim densely sampled SIFT
from the rationales](https://image.slidesharecdn.com/140327-140506224448-phpapp02/85/ICCV2013-reading-Learning-to-rank-using-privileged-information-24-320.jpg){kind=icon}
ICCV2013 reading: Learning to rank using privileged information
- 1. ICCV2013 reading 2014.3.28 Akisato Kimura (@_akisato)
- 2. Paper to read (Presented at ICCV2013)
- 3. Problem dealing with in this paper • Learning using privileged information (LUPI) – Training • Feature vectors : 𝑋𝑋 = 𝑥𝑥1, … , 𝑥𝑥𝑁𝑁 , 𝑥𝑥𝑖𝑖 ∈ ℝ𝑑𝑑 • Label annotation : 𝑌𝑌 = 𝑦𝑦1, … , 𝑦𝑦𝑁𝑁 , 𝑦𝑦𝑖𝑖 ∈ ℕ • Additional information : 𝑋𝑋∗ = 𝑥𝑥1 ∗ , … , 𝑥𝑥𝑁𝑁 ∗ , 𝑥𝑥𝑖𝑖 ∗ ∈ ℝ𝑑𝑑∗ – Testing • Prediction function : 𝑓𝑓: ℝ𝑑𝑑 → ℕ • No additional information required
- 4. Privileged information?? • Applicable to several scenarios in CV
- 5. Formulation • Generic supervised binary classification – Training • Feature vectors : 𝑋𝑋 = 𝑥𝑥1, … , 𝑥𝑥𝑁𝑁 , 𝑥𝑥𝑖𝑖 ∈ ℝ𝑑𝑑 • Label annotation : 𝑌𝑌 = 𝑦𝑦1, … , 𝑦𝑦𝑁𝑁 , 𝑦𝑦𝑖𝑖 ∈ {+1, −1} • Additional information : 𝑋𝑋∗ = 𝑥𝑥1 ∗ , … , 𝑥𝑥𝑁𝑁 ∗ , 𝑥𝑥𝑖𝑖 ∗ ∈ ℝ𝑑𝑑∗ – Testing • Prediction function : 𝑓𝑓: ℝ𝑑𝑑 → ℝ • No additional information required
- 6. Key idea • Privileged information allow us to distinguish between easy and hard examples – If the privileged data is easy to classify, then the original data would also be easy to classify. – … under the assumption that the privileged data is similarly informative about the problem at hand.
- 7. Linear SVM • Ordinary convergence rate = 𝑂𝑂(𝑁𝑁−1/2 ) • It improves to 𝑂𝑂(𝑁𝑁−1 ) – if we knew the optimal slack values 𝜉𝜉𝑖𝑖 in advance (OracleSVM [Vapnik+ 2009]) min 𝑤𝑤∈ℝ𝑑𝑑,𝑏𝑏∈ℝ,𝜉𝜉𝑖𝑖∈ℝ
- 8. Slack variables in SVM • Slack variables tell us which training examples are easy / hard to classify – 𝜉𝜉𝑖𝑖 = 0 → easy – 𝜉𝜉𝑖𝑖 ≫ 0 → hard min 𝑤𝑤∈ℝ𝑑𝑑,𝑏𝑏∈ℝ,𝜉𝜉𝑖𝑖∈ℝ
- 9. SVM+ • A 1st model for LUPI – Use privileged data as a proxy to the oracle – Parameterize 𝜉𝜉𝑖𝑖 = 𝑤𝑤∗, 𝑥𝑥𝑖𝑖 ∗ + 𝑏𝑏∗ [Vapnik+ NN2009, NIPS2010]
- 10. Why should SVM+ be improved? • Cannot be solved by popular SVM packages – Although good optimization algorithms were derived [Pechyony+ 2011], they work only with the dual.
- 11. Learning to rank setup instead • Underlying idea is the same • Using the privileged data to identify easy / hard-to-separate sample pairs – Instead of using it to identify easy / hard-to- classify samples
- 12. SVMrank • Slack variables tell us which training example pairs are easy / hard / impossible to separate [Joachims KDD2002]
- 13. Proposed method: Rank transfer • The strategy is similar to SVM+, but indirect. 1. SVMrank on 𝑋𝑋∗ (The ranking function 𝑓𝑓∗) 2. Margins 𝜌𝜌𝑖𝑖𝑖𝑖 = 𝑓𝑓∗ 𝑥𝑥𝑖𝑖 ∗ − 𝑓𝑓∗ (𝑥𝑥𝑗𝑗 ∗ ) ∀𝑖𝑖, 𝑗𝑗 𝑦𝑦𝑖𝑖 > 𝑦𝑦𝑗𝑗 • 𝜌𝜌𝑖𝑖𝑖𝑖 ≫ 0 : easy, 𝜌𝜌𝑖𝑖𝑖𝑖 ≈ 0 : hard, 𝜌𝜌𝑖𝑖𝑖𝑖 < 0 : impossible 3. SVMrank on 𝑋𝑋 with data-dependent margins
- 14. Intuition • If it was difficult to correctly rank a pair on 𝑋𝑋∗ , also it will also be difficult on 𝑋𝑋 1. Pairs (𝑖𝑖, 𝑗𝑗) with small margins 𝜌𝜌𝑖𝑖𝑖𝑖 have more limited influence on 𝑤𝑤 2. Incorrectly ranked pairs are ignored. 1. 2.
- 15. Why not Rank transfer? • We can use standard SVM packages! – For the SVMrank on 𝑋𝑋∗ this is clear. – For the SVMrank on 𝑋𝑋 we need variable transformations
- 16. Experiments • 4 different types of privileged information – All of those can be handled in a unified framework. • 4 different methods to be compared – SVM, SVMrank, SVM+, Rank transfer • Evaluation metric = Average Precision
- 17. (1) Attributes as privileged info • Animals with Attributes Dataset – 10 species ( = classes), 85 properties ( = attributes) • Features: 2000-dim SURF • Privileged: 85-dim predicted attributes [Lampert+ PAMI2014] • Learn 1-vs-1 classifiers with 100 training samples
- 18. (1) Results • Rank transfer is the best.
- 20. (2) Bounding box as privileged info • Fine-grained setup on ILSVRC2012 – 17 classes with variety of snakes • Features: 4096-dim Fisher vector from the whole images • Privileged: 4096-dim Fisher vector from the bounding box regions • Learn 1-vs-rest classifiers
- 21. (2) Results • SVM+ is the best, ranking strategies do not seem suitable for this setup.
- 22. (3) Texts as privileged info • IsraelImages dataset [Bekkerman+ CVPR2007] – 11 classes, 1800 images with a textual description up to 18 words • Features: 4096-dim Fisher vectors • Privileged: BoWs from the texts • Learn 1-vs-1 classifiers Desert Trees
- 23. (3) Results • Reference (privileged only) is the best • All the others produce almost the same. – Note that, high accuracy in the privileged space does not necessarily mean that the privileged information is helpful for the target task.
- 24. (4) Rationales as privileged info • Hot or Not dataset [Donahue+ ICCV2011] • Features: 500-dim densely sampled SIFT from the whole image • Privileged: 500-dim densely sampled SIFT from the rationales
- 25. (4) Results • Reference is the best. • Rank transfer performs better for male class. • Hard to draw a conclusion.
- 26. Appendix: Margin transfer • One possible alternative to Rank transfer
- 27. But not so good…
- 28. Last words • The idea is nice, easy to use. • More privileged information, better performance? --- needs discussions • Which types of privileged information are suitable? --- unknown