Realism of image composits
1 like283 views
The document discusses improving the realism of image composites. It identifies key statistics like luminance, color temperature, and saturation that control realism. Human subject experiments show these statistics significantly affect judgments of realism. The document proposes an automatic algorithm that selects image zones and adjusts the mean values of relevant statistics to make composites appear more realistic. An evaluation finds the algorithm performs better than previous methods and comes close to manual editing in terms of rated realism. Limitations include not addressing relighting and needing stronger theory for zone selection.
![Alterna7ve
to
alpha
maAe:
seamlessly
blending
• Feathering
• Laplacian
pyramids
[Odgen
et
al.
1985]
• Gradient-‐domain
composi7ng
[Perez
et
al.
2003]](https://image.slidesharecdn.com/cg-130319110143-phpapp02/85/Realism-of-image-composits-11-320.jpg)
Realism of image composits
- 1. Understanding and Improving the Realism of Image Composites Aliya Ibragimova University of Fribourg
- 2. Agenda • Realis7c Image composi7ng • Iden7fying key sta7s7cs • Human subject experiments • Algorithm • Results
- 4. Composi7ng procedure 1. Foreground Alpha maAe 2. New background Composite
- 5. Color Transfer Technique (CTT) Reinhard et al. 2001 ‘Match color’ feature of Photoshop
- 6. Color transfer technique: limita7ons Cut-‐and-‐paste Match Color Conflates the effects of reflectance and illumina7on
- 7. Improvements of CTT (ColorComp) Lalonde and Effros 2007 • Analyze huge dataset of natural images : difference in distribu7on of realis7c and unrealis7c images • Recolor regions for realis7c composi7ng Limita7ons: requires a large dataset, depends on the presence of images that are similar to the target
- 8. # of pixels Professional compositors Shadows Midtones Highlights Brightness • Isolate highlights and match their colors and brightness • Balance mid-‐tones with gamma correc7on • Match the shadow regions
- 9. Color Harmony Cohen-‐Or et al. 2006 Harmonic colors are sets of colors that are aesthe7cally pleasing in terms of human visual percep7on.
- 10. Color Harmony Cohen-‐Or et al. 2006 Limita7ons: obtained images are not necessary realis7c, ignores luminance and contrast, the approach has not been quan7ta7vely evaluated
- 11. Alterna7ve to alpha maAe: seamlessly blending • Feathering • Laplacian pyramids [Odgen et al. 1985] • Gradient-‐domain composi7ng [Perez et al. 2003]
- 12. Alterna7ve to alpha maAe: seamlessly blending foreground background Cut-‐and-‐paste Gradient-‐domain Limita7ons: 2 source images should have similar colors and textures
- 13. Problem statement • Which sta7s7cs control realism? • How do these sta7s7cs affect human judgment of realism? • Automa7c algorithm to improve realism?
- 14. Good sta7s7cs • Highly correlated between foreground and background • Easy to adjust • Independent from each other
- 15. Categories of sta7s7cal measures • Luminance • Color temperature (CCT) • Satura7on • Local contrast • Hue (circular sta7s7cs)
- 16. Sta7s7cal measures Standard devia7on # of pixels mean Brightness • Mean • Standard devia7on • High • Low • Kurtosis • Entropy
- 17. Find correla7on • Pearson correla7on coefficient • Standard devia7on of offset δi = Mif – Mib M – measure f – foreground b – background i – sta7s7cs
- 18. Sta7s7cal experiment • Use large (4126 images) labeled dataset • Select the most correlated sta7s7cs
- 19. Luminance
- 21. Satura7on
- 23. Results: sta7s7cal experiment • luminance, color temperature, satura7on, local contrast are most correlated • mean of zones correlate more than other sta7s7cal measures • mean of high and low zones correlate more than mean of en7re histogram
- 24. Human subjects experiment Experiment with human subjects on Amazon Mechanical Turk (MTurk) • 20 natural images • 3 key sta7s7cs (luminance, color temperature, satura7on)
- 25. Human subject experiments Foreground luminance Background luminance
- 28. Automa7c composite adjustment • Zone selec7on using machine learning (random forest classifier) T = s * σg, s = 0.1 Three binary classifiers: • Pick smallest changes if mul7ple zones • Features for foreground and background and per sta7s7c: std, skew, kurt, entropy, p1, p2 … p20.
- 29. Pipeline Input: foreground and background image 1. Match H-‐zone of contrast using S-‐shape 2. Select zone and adjust mean of luminance 3. Select zone and adjust mean of CCT 4. Select zone and adjust satura7on Adjust algorithm is greedy, could iterate several 7mes if needed
- 30. Results and evalua7on Cut-‐and-‐paste Manual MatchColor ColorComp Ours
- 31. Results and evalua7on Cut-‐and-‐paste Manual MatchColor ColorComp Ours
- 33. Results and evalua7on Thurstone’s Law of Compara7ve Judgement
- 34. Results and evalua7on • BeAer than previous methods • Close to manual edi7ng Limita7ons: • Religh7ng • Theory for zone selec7on
- 35. Ques7ons?