Image Aesthetic Assessment With Deep Learning
Download as PPTX, PDF3 likes1,367 views
The document discusses the evaluation of image aesthetics through machine learning techniques, highlighting the impact of factors like lighting, contrast, and composition on perceived quality. It critiques fixed input size constraints in deep learning networks that compromise image aesthetics and introduces spatial pyramid pooling as a solution for handling varying input sizes. The text emphasizes the importance of high-quality images in user-generated content platforms, particularly for enhancing engagement metrics such as click-through rates.
Image Aesthetic Assessment With Deep Learning
- 2. Who am I ? M a c h i n e L e a r n i n g E n g i n e e r A m i t K u s h w a h a
- 3. Image Aesthetics W H A T D E F I N E S A N I M A G E ? ★ Image aesthetic evaluation aims to classify photos into high quality or low quality from the perspective of human. ★ Many visual features have been explored under this formulation (handcrafted features), ranging from low-level image statistics, such as edge distributions and color histograms, to high-level photographic rules, such as the rule of thirds and golden ratio
- 4. Beautiful Capture “ B E A U T Y I S R E A L L Y I N T H E E Y E O F T H E B E H O L D E R ” ✦ While everyone has different tastes, there are universally accepted norms when it comes to beauty – things which everyone pretty much agrees are beautiful, like sunsets or sunrises over the mountains or the ocean. A S U B J E C T I V E C O N J E C T U R E
- 5. T E C H N I Q U E S O F I M A G E C A P T U R I N G A S U B J E C T I V E C O N J E C T U R E ✦ Image aesthetics can be affected by the different usages of lighting, contrast , and image composition Beautiful Capture
- 6. The Two Images F A C T O R S L I K E L I G H T I N G , C O N T R A S T , A N G L E D E T E R M I N E T H E A E S T H E T I C S
- 7. Significance of Aesthetic Assessment ✦ With the ever increasing user generated content, biggest challenge is to showcase high quality images to users. ✦ Zomato measures the performance of restaurant for ad sales with click through rate (CTR) as one of the parameter. Restaurants with High Quality Display Image have greater CTR. 01 02 04 06 WRITESOMETHING
- 8. Photo Gallery P H O T O S D O H A V E A S T O R Y
- 9. Let’s solve Image Aesthetics with the most disruptive technology over the last five years.
- 10. Deep Learning S T A T E O F T H E A R T I N M A N Y F I E L D S N O W
- 11. Fixed Size Input Constraint D E E P M O D E L I N P U T L A Y E R ✦ This constraint of fixed size input in Deep Network compromises the aesthetics of original image. input images need to be transformed via cropping, scaling, or padding, which often damages image composition, reduces image resolution, or causes image distortion, thus compromising the aesthetics of the original images. Input Layer of Network
- 12. A M A Z I N G I N T H E I R O R I G I N A L A S P E C T R A T I O Original Image
- 13. Transformed Image R E S I Z I N G L O S E S T H E O R I G I N A L A E S T H E T I C S O F I M A G E O R I G I N A L R E S I Z E D
- 14. Transformed Image O R I G I N A L R E S I Z E D R E S I Z I N G L O S E S T H E O R I G I N A L A E S T H E T I C S O F I M A G E
- 15. DEEP LEARNING Demystifying the Network ✦ The first few layers of networks are either convolution or pooling layer, followed by fully connected layers. ✦ The fixed length input is only the constraint of fully-connected layer. Be it convolution or pooling, both perform operations on input, with the spatial information.
- 16. Spatial Pyramid Pooling ✦ Network with another pooling strategy, ‘spatial pyramid pooling’ just before the fully connected layer, makes it capable of taking input of any size and generate fixed length representation.
- 17. Max Pooling O U T P U T I S P R O P O R T I O N A L T O I N P U T Kernel size is fixed
- 18. Spatial Pyramid Pooling Output dimension (Bin Size , Bin Size)
- 20. Fixed Bin Numbers Fixed Kernel Size SPP-Net CNN
- 21. Training a SPP-net Z O M A T O D A T A S E T U S E D T R A I N I N G L O S S T R A I N I N G A C C U R A C Y M A X - A C C U R A C Y - 7 7 . 2
- 24. It’s often neglected, but plays a major role in classification learning. S C A L E I N V A R I A N T L E A R N I N G Allows to explore CNN without the constraint of fixed size representation of input. F I X E D S I Z E I N P U T Take Aways S P A T I A L P Y R A M I D P O O L I N G N E T
- 25. R E F E R E N C E S ✦ Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition” ✦ Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks 2012 Composition-Preserving Deep Photo Aesthetics Assessment ✦ Long Mai, Hailin Jin, Feing Liu, Composition-Preserving Deep Photo Aesthetics Assessment
- 26. h t t p s : / / w w w . g i t h u b . c o m / y a r d s t i c k 1 7 / h t t p s : / / w w w . l i n k e d i n . c o m / i n / y a r d s t i c k 1 7 / a m i t _ k u s h w a h a @ o u t l o o k . c o m Thank You F i n d m e h e r e
Editor's Notes
- #8: To change the image behind the Mock up. Select the layer - > Right Click -> Send to Back -> Delete the image -> Drag & Drop your Own Picture -> Send to Back (again)
- #19: To change the image behind the Mock up. Select the layer - > Right Click -> Send to Back -> Delete the image -> Drag & Drop your Own Picture -> Send to Back (again)
- #20: To change the image behind the Mock up. Select the layer - > Right Click -> Send to Back -> Delete the image -> Drag & Drop your Own Picture -> Send to Back (again)
- #25: To change the image behind the Mock up. Select the layer - > Right Click -> Send to Back -> Delete the image -> Drag & Drop your Own Picture -> Send to Back (again)