Image understanding and artificial intelligence
0 likes995 views
The document discusses the evolution and significance of image understanding in artificial intelligence, defining it as the analysis and generation of images by computers. It covers various methodologies and models for interpreting images, emphasizing the role of contextual information and spatial relationships in understanding scenes. Additionally, it highlights ongoing research and collaborations in the field to develop effective algorithms and frameworks for image annotation and reasoning.
![CdR
LVR
ThR
PuR
GPR
ClR
V3
CC
icR
Knowledge
Graphs
[COLLIOT, DERUYVER, ...]
Stochastic grammars
[ZHU, MUMFORD,..]
Ontologies
[DAMERON,HU,...]
Formal
representation
Data
Reasoning
Decision
I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 11 / 22](https://image.slidesharecdn.com/8confimageai-190206093035/85/Image-understanding-and-artificial-intelligence-16-320.jpg)
Image understanding and artificial intelligence
- 1. Image Understanding and Artificial Intelligence Isabelle Bloch LTCI, T´el´ecom ParisTech, Universit´e Paris-Saclay isabelle.bloch@telecom-paristech.fr 2019 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 1 / 22
- 2. What is image understanding? From the 1960’s to today: Miller and Shaw (1968): survey of linguistic methods for picture processing, defined as analysis and generation of pictures by computers, with or without human interaction. Clowes (1971): linguistic approach for picture interpretation (pattern description language). Reiter (1989): interpretation = logical model of sets of axioms. Ralescu (1995): image understanding = verbal description of the image contents. Bateman (2010): needs for a semantic layer for spatial language. Xu et al. (2014): image interpretation = assigning labels or semantics representations to regions of a scene. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 2 / 22
- 3. What is image understanding? Here: Beyond individual object recognition. Objects in their context, spatial arrangement. Global scene interpretation. Semantics extraction. Providing verbal descriptions of image content. Dynamic scenes: recognition and description of actions, gestures, emotions.. Inference, higher level reasoning. Important role of Artificial Intelligence. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 2 / 22
- 4. A few examples A lot of work on image annotation: object → several objects → scene. Magritte, 1928 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
- 5. A few examples A lot of work on image annotation: object → several objects → scene. Millet et al., 2005 (rules, spatial reasoning...) Region without spatial relations with spatial relations 1 sky sky 2 grass tree 3 tree tree 4 building building I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
- 6. A few examples A lot of work on image annotation: object → several objects → scene. Venus? Thanks to H. Maˆıtre I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
- 7. A few examples A lot of work on image annotation: object → several objects → scene. “Show and tell”: Vinyals et al., 2015 (convolutional neural networks, deep learning) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
- 8. A few examples A lot of work on image annotation: object → several objects → scene. Kulkarni et al., 2013 I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 3 / 22
- 9. Skubic et al., 2003 (fuzzy modeling of spatial relations) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 4 / 22
- 10. Ogiela et al. 2002, Trzupek et al. 2010 (graphs and grammars) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 5 / 22
- 11. An abnormal structure is present in the brain. A peripheral non-enhanced tumor is present in the left hemisphere. Atif et al., 2014 (spatial reasoning, abduction) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 6 / 22
- 12. Morimitsu et al., 2015 (graphs, Bayesian tracking, hidden Markov models) I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 7 / 22
- 13. Data vs. knowledge Is everything in the data? Powerful methods and impressive results. Accessibility of data. Size and number of data. Cost of learning. Importance of knowledge. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 8 / 22
- 14. Imperfect information, multiple nature of information I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 9 / 22
- 15. Models for image understanding From models to interpretation Develop mathematical models to represent knowledge (context, expert, spatial organization...), information contained in images (geometry, statistics, shape, appearance...), and to combine them (fusion process), ⇒ operational and efficient algorithms for image understanding Semantic gap. Knowlegde representation and reasoning. Pathological or unexpected cases. Conversely: from images to models example: individual anatomical models, virtual patient. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 10 / 22
- 16. CdR LVR ThR PuR GPR ClR V3 CC icR Knowledge Graphs [COLLIOT, DERUYVER, ...] Stochastic grammars [ZHU, MUMFORD,..] Ontologies [DAMERON,HU,...] Formal representation Data Reasoning Decision I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 11 / 22
- 17. Examples, within various collaborations with hospitals and companies: I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 12 / 22
- 18. Symbolic and structural representations and reasoning Morphology Duality Mathematical Adjunction Uncertainty modeling Spatial relations Knowledge representation Reasoning (revision, fusion, abduction, spatial reasoning) Preference modeling Image understanding Math−Music ... Set, functions Images Fuzzy sets Graphs Hypergraphs FCA Logics Satisfaction systems CNN Image processing and analysis Structural representations (data and knowledge) Lattices Learning I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 13 / 22
- 19. Modeling fuzzy spatial relations Mathematical models: combining fuzzy sets and mathematical morphology. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 14 / 22
- 20. With Alessandro Delmonte and Pietro Gori I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 15 / 22
- 21. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 16 / 22
- 22. Graph-based reasoning and constraint satisfaction problem, with Jamal Atif, Geoffroy Fouquier and Olivier Nempont I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 17 / 22
- 23. Conceptual graphs and complex spatial relations, with Carolina Vanegas I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 18 / 22
- 24. Image interpretation as an abduction problem, with Jamal Atif and Yifan Yang K |= (γ → ϕ) Finding the “best” explanation to the observations taking into account expert knowledge. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 19 / 22
- 25. With Yongchao Xu, Thierry G´eraud, Hadrien Bertrand, Roberto Ardon, Matthieu Perrot Specialized Layers Fine feature maps Coarse feature maps Base network architecture n 0 n-1R nG n+1B N SegmentationInput Original 3D volume I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 20 / 22
- 26. Adult Child Ref. segmentation 3D U-Net Transfer learning I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 21 / 22
- 27. Research in Artificial Intelligence at LTCI - IMAGES Team Covers several topics in the general AI cartography (see #FranceIA): Machine learning. Pattern recognition. Interaction. Knowledge representation. Decision and uncertainty management. Reasoning. I. Bloch (LTCI, T´el´ecom ParisTech) Image Understanding and AI 2019 22 / 22