First part_B-Image Processing - 1 of 2).pdf

Content
A. Introduction to Computer Vision
B. Image Processing (1/2)
C. Image Recognition & Retrieval
D. Video Recognition & Retrieval
E. Mobile Visual Search
F. Deep Learning Based CV
G. Special Lectures on CV

4
Image Processing & Applications
Definition of Image Processing and Some applications
◆ Digital image processing is the use of a digital computer to process digital
images through an algorithm. (Wikipedia)
◆ Image processing is a method to perform some operations on an image, in
order to get an enhanced image or to extract some useful information from
it. (Dr. G. Anbarjafari, University of Tartu)

Trends of Machine Vision
Machine Vision Technology
Content
Brief Introduction to MV(Machine Vision)
Machine Vision Applications
I.C. Leadframe
Automatic Inspection System(1996)

7
1. Brief Introduction to Machine Vision
What is Machine Vision?
- Machine Vision : Computer Vision technology that is applied to
production lines of manufacturing(inspection, assembly, transportation etc.)
industries.
<Visual Inspection> <Visual Assembly>
<Visual Transportation>
<Applications>

8
- Extended Machine Vision
. A machine vision system (MVS) is a type of technology that enables a
computing device to inspect, evaluate and identify still or moving
images in various industries.
<Extended Applications>

9
Benefits of Machine Vision
Strategic Goal Machine Vision Applications
Higher Quality Inspection, measurement, gauging, and assembly verification
Increased Productivity
Repetitive tasks formerly done manually are now done by
MVS (Machine Vision System)
Production Flexibility Measurement and gauging / Robot guidance / Prior
operation verification by MVS
Less Machine Downtime and
Reduced Setup Time Changeovers programmed in advance
More Complete Information and
Tighter Process Control Manual tasks can now provide computer data feedback
Lower Capital Equipment Costs
Adding vision to a machine improves its performance, avoids
obsolescence (노후화)
Lower Production Costs One vision system vs. many people / Detection of flaws early
in the process
Scrap Rate Reduction (불량률 감소) Inspection, measurement, and gauging
Inventory Control (재고 관리) OCR and Identification
Reduced Floorspace (바닥 면적 감소) Vision System vs. Operator

10
2. Trends of Machine Vision
Brief History
(National Instrument)

11
Market Trends
- The global machine vision market size is expected to grow from USD 10.7
billion in 2020 to USD 14.7 billion by 2025, at a *CAGR of 6.5% during the
forecast period. (marketsandmarkets, 2019)
- The major driving factors in the market is the increasing need for quality
inspection and automation, growing demand for vision-guided robotic systems,
increasing adoption of vision-guided robotic systems, and many more.
* CGAR(compound annual growth rate, 연평균 성장률)

12
Component & Application Market Trends
- Based on component, the market comprises smart camera, embedded system,
frame grabber, lighting, and lenses..
- By application, the market is divided into positioning, identification, verification,
measurement, and flaw detection.
- Market by type comprises 1D, 2D, and 3D measurements, of which 3D
measurement is estimated to grow at the highest CAGR, owing to trend of 3D
imaging in the production line.

13
Market Trends
World
(Top 10 of thousands of companies)
Korea
(52 Companies)
- The key market players Cognex Corporation (US), Basler AG (Germany), Omron
Corporation (Japan), Keyence (Japan), National Instruments (US), Sony Corporation
(Japan), Teledyne Technologies (US), Texas Instruments (US), Intel Corporation (US),
ISRA Vision (Germany), Sick AG (Germany) and FLIR Systems (US).

14
6 Typical Technology Trends of Machine Vision for 2020
- The trend towards robotics and smart manufacturing makes MV(Machine Vision
technology) an indispensable tool for industrial automation.
- MV has gradually replaced quality inspection performed by humans.
- MV provides object recognition capabilities with varying degrees of accuracy and
reliability.
- Manufacturers employ machine vision to improve the traceability of their products
for enhanced consistency, productivity, and overall quality.
https://www.arcweb.com/blog/major-machine-vision-trends-2020
https://www.qualitymag.com/articles/95664-machine-vision-trends
https://www.visiononline.org/blog-article.cfm/Machine-Vision-Trends-to-Watch-in-2020/249
1. 3D Imaging
2. Deep Learning & AI
3. Embedded Vision
4. Robotics
5. Non Visible Imaging
6. Liquid Lenses and High-resolution Optics

15
1. 3D Imaging
- 3D imaging is a process to render a three-dimensional image on a
two-dimensional surface by creating the optical illusion of depth.
- Types of 3D imaging technologies - stereo vison, time of flight and
structured lights.
- Typical Applications – 3D measurement, Bin picking, Assembly (car)
3D measurement
Bin Picking
Assembly (car)

16
2. Deep Learning in Machine Vision
- Deep Learning in Machine Vision has enabled machine vision systems
to adapt autonomously to manufacturing variations.
- Roles of DL in MV - handles complex surface textures and variations in part
appearance and acclimatizes to new examples (without re-programming core networks).
- Typical Applications – part location, inspection, classification and OCR

17
3. Embedded Vision
- Embedded vision refers to the integration of a camera and processing
board that use computer vision algorithms.
- Advantages of Embedded vision - small size, light weight, low cost,
low energy consumption.
- Typical Applications – autonomous mobile robots, driver assistance
systems, industrial drones, biometrics, medical imaging, and space imaging.
(Microsemi Corporation)

18
4. Robotics in Machine Vision
- Machine vision systems are employed in a robot for recognizing
and viewing objects.
- Advantages of Robotics in Machine Vision - increase flexibility,
improve productivity, and employ human operators only in tasks
where they can add the most value.
- Applications
• 3D machine vision: providing guidance
and location for a wide range of
assembly and inspection applications.
• Powerful cameras, faster vision
processors and advanced software:
allowed robots to complete more than
one task without re-programming.

19
5. Non Visible Imaging – infrared wavelengths
- Infrared imaging is carried out in the same way as visible light imaging,
with cameras sensitive to the near infrared range of wavelengths.
- Advantages
• Ability to operate in low-light conditions: useful for security and
surveillance, as night time conditions.
• Return on investment (ROI): save an enormous amount through
avoided downtime, repair costs, and the costs of new equipment.
- Applications: Aerial Thermography, Thermography in Automotive
Industry/Chemical Industry/Medicine/for Plant Inspections & Security ….

20
6. Liquid lenses
- Devices that can change focus depending on an external signal
(usually a change in current or voltage) without requiring any
mechanical change in the lens as in manually focusable lenses.
- Advantages
• No moving parts, High speed(1/100th of second), Low power consumption(~1mW),
Silence, High shock resistance, A large focal length range
- Applications: Digital photography, Industrial data capture, Barcode reading(1D and 2D)

21
Types of Machine Vision Systems (H/W)
1) 1-D machine Vision
- 100% continuous web inspection and classification
- Uses line-scan cameras
- Materials inspected include:
• Metals
• Plastics
• Paper
* Utilized for ultra high precision measurement/visual inspection
3. Machine Vision Technology

22
2)-1 : 2-D Machine Vision – Line Scan
- Image is built line by line
- Movement is needed
- Requires encoder to track movement
- Short exposure times
2)–2 : 2D Machine Vision – Area Scan

23
3) 3-D Machine Vision: used for the precise three- dimensional inspection and
measurement of complex 3D free formed surfaces.
a) Laser triangulation - using the fixed angular offset of the camera and laser
positions, it is possible to derive the linear distance between the inspection
surface and the camera’s sensor.
b) Stereo vision - by comparing information about a scene from two vantage points,
3D information can be extracted by examining the relative positions of objects in
the two panels
c) Light stripe projection - projects a light plane into the object scene. The idea is
to intersect the projection ray of the examined image point with the light plane.
(Laser triangulation) (Stereo Vision) (Light stripe projection)

24
d. Shape from shading - process of computing the three dimensional shape of a
surface from one image of that surface.
e. White light interferometry - non-contact optical method for surface height
measurement on 3-D structures with surface profiles varying between tens of
nanometers and a few centimeters.
f. Time of flight - the measurement of the time taken by an object, particle or wave
(be it acoustic, electromagnetic, etc.) to travel a distance through a medium.
(Shape from shading)
(White light interferometry) (ToF: Time of flight)

25
Key parts of Machine Vision Systems (H/W)

26
Machine Vision System (S/W)
- General Machine Vision Algorithms
• Image transformation/geometric manipulation  Content Statistics
• Image enhancement/preprocessing  Connectivity
• Edge Detection  Correlation
• Geometric Search  OCR/OCV  Color processing

27
Target Positioning (Guidance)
4. Typical Machine Vision Tasks
- Determines part position (x, y, and angle)
• Automates handling of parts for machines:
➢ Alignment & Placement
➢ 2D & 3D Picking
➢ Eliminates need for factoring & improves robot
flexibility
- Vision tool alignment, fixturing
• Locate at least one feature on a part for the
purpose of calculating the (x, y) position and
rotation of the part to position other vision
tools precisely
* First step in every Machine Vision application is to find the Target precisely

28
- Problems of Target Positioning : geometrical pattern matching problems
Target Image

29
Visual Inspection
- Correct location
• Orientation
• Skew
- Quality
• Defect Detection
• Surface Inspection
• Contaminants
- Completeness
• Fill Level
• Feature Presence
• Counting
• Assembly Verification

30
Gauging
- Precise dimensioning
• Automated metrology and data recording
- Ensure tolerances
• Diameters, Gaps, Bushings, Threads, etc.
Bushing
Thread cutting

31
Identification
- Read codes
• Bar codes & 2-D Matrix
• Labels & direct part mark
- Read characters
• OCR / OCV
- Recognize objects
• Based on color, shape, or size

32
◆Project Summary
5. IC Lead Frame Inspection System
Project Name
(Period/Research Fund, M$)
Main Content
Role in
Project
Related Basic Theory &
Application System
a) Development of an
Automatic I.C Lead Frame
Inspection System
(‘91~’94/0.41)
- Model-based inspection Algorithm
- Development of an Automatic I.C Lead
Frame inspection S/W
- Implementation of real-time field system
Project
Director
- Basic Theory
: Image formation &
enhancement
: Image measurement
: Sub-pixel image
processing
- Application System
: Visual inspection
b) Commercialization of I.C
Lead Frame Inspection
System
(‘95~’96/0.13)
- Development of practical I.C lead Frame
automatic inspection system
- Development of real-time image
processing technology
Project
Director
c) Development of a Surface
Analysis S/W Using Sub-
pixel Image Processing
(‘98~’00/0.33)
- Development of high precision
measurement technology using sub-pixel
- Development of surface Inspection and
depth measurement function
- Improvement of feeding device(company)
Project
Director

33
IC Lead Frame
- Lead frames are the metal structures inside a chip package that carry
signals from the die to the outside
- Yield good production ratio 70% or less in 1996
- Sample inspection by all human eyes
Lead Frame
Die & wire bonding
molding & packaging
IC

34
The Need for IC Lead Frame Automatic Inspection System
- Reduction of Lead Frame production cost
- Stabilizing Lead Frame Quality
- Requires flexibility in inspecting various lead frames in one system
- Severe competition in semiconductor industry : localization of equipment
and import substitution is absolutely necessary

35
Requirements for IC Lead Frame Inspection System
a) Real time processing : at least faster than production process
b) Precision : must be more than 1/10 precise of the inspection process
requirements
c) Repeatability & Reproducibility : the result should always be the same if
the same object is measured in the same way but in other environments
a) b)
c)

36
Development procedure of IC Lead Frame Inspection System
Definition of Inspection Specification and Requirements
(a) Design of Real-time Computer system
(b) Development of CAD-based inspection algorithm
(c) Development of real lead frame based inspection algorithm
(d) Improvement of precision using sub-pixel
Development of Real-time field system and testing

37
(a) Real-time Computer system
(b) Lead Frame CAD Data
(c) Real Lead Frame Data (d) Sub-pixel Analysis (Tabatani’s)

38
<Concept of Subpixel edge detection based on linear interpolation>
(a) Ideal luminance profile, (b) luminance profile incident on the image
plane, (c) linear interpolation of the image samples (Federico Pedersini et al.)
◆ Sub Pixel Interpolation
- Often some of the image pixels are smaller than CCD pixels.
- Sub pixel Interpolation is to enhance the resolution of images via linear
interpolation of the image samples.
<Example of Sub pixel interpolation>
(https://optinav.pl/2016/08/08/image-processing-subpixel-edge-detection/)

39
Type of Inspection
(IC Lead Frame Inspection System, ETRI & Poongsan Microtech, SUNG WOO Techron)
- Lead spacing inspection : CAD Data
- Lead Frame geometry inspection & Taping inspection : morphological
processing using difference of two real images of Lead Frames
Spacing Inspection
Taping Inspection

40
100% success in
Localization &
Commercialization

42
Another System – BLU(Back Light Unit) Automatic Inspection System
BLU Automatic Inspection System
(T LED, 2002)
BLU
Luminance uniformity test

Background
State-of-the-art Technologies
Content
Restoration of Mural paintings
of Buseoksa Temple
Others : Koguryo tomb etc.
3D Restoration of Cheongryongsa
Daeungjeon at Anseong

46
1. Background
- Cultural assets are a valuable historical heritage that humanity must preserve
forever.
- However, the management, protection and preservation of these cultural
properties are difficult due to technological difficulties, and have been damaged by
natural disasters such as fire, rainy season and recent environmental pollution.
- Much research is underway to preserve and restoration of cultural properties, to
determine their academic value, and to collect enough information to be recorded
30‘s 60’s
Goguryeo (BC37 ~ AD668) Muyongchong Wall Paintings
Ruined Goguryeo Tomb

47
1. Background
- Computer restoration of cultural properties is one of these studies, 2 methods.
1) Restoration of disappearing or damaged cultural properties -
restoration of wall paintings
2) Computer simulation for restoration of cultural properties - 3D
restoration of ancient buildings and statues
1) La Cappella Sistina in Vaticano - Di Michelangelo
2) Mireuksaji Stone Pagoda

48
2. State-of-the-art Technologies
Restoration of Wall Painting
- Restoration of Wall paintings of the Yeongju Buseok-sa temple (National
treasure No. 46, Korea 1996)
- Tehera murals (Greece, 2013)
Infra red image After restoration(摹寫圖)
<Yeongju Buseok-sa temple, Korea> <Tehera murals, Greece>

49
- Mural paintings of ancient Roman villas (2008-2011, United Kingdom)
• Mural of an ancient Roman-era villa in Oplontis, a coastal city near Pompeii, Italy.
• AD79, buried underground by Vesuvius volcanic eruption, excavated since 1984
Current (side photo) Current (front photo) after restoration

50
3D restoration of ancient buildings and statues
- 3D Restoration of Seokguram (2008, Korea)
- Hwangnyongsa 9 story tower restoration (2016, Korea)
<3D Scanning> <3D Modelling>
<Restored Hwangnyongsa Temple to Computer Image>

51
- Sagalassos ruins (2004, Turkey)
• Remains of Ancient Rome(AD 160 ~ 180) in Sagalassos, Turkey
<before restoration> <after restoration>

52
- Rome Reborn(2012, Italy)
• Rome Reborn is an international initiative whose goal is the creation
of 3D digital models of ancient Rome
• First settlement in the late Bronze Age (ca. 1000 B.C.) to the
depopulation of the city in the early Middle Ages (ca. A.D. 550).

53
- CyArk 3D Heritage Archive 500 Project(2003 ~ , USA)
• CyArk is a non-profit entity whose mission is to digitally preserve cultural
heritage sites through collecting, archiving and providing open access to data
created by laser scanning, digital modeling, and other state-of-the-art
technologies.
• CyArk was founded in 2003 as a project of the Kacyra Family Foundation (KFF),
located in Orinda, California USA.

54
3. Restoration of Wall paintings of Buseoksa Temple
Story behind project
- August 1991 : Meeting of Two Ministers. Plan to promote domestic
cultural assets restoration projects such as the restoration of 5 story
pagoda in Japan (法隆寺, ほうりゅうじ)
- MST -> ETRI -> Dr. OH, Two-months night work
<Eo-Ryong Lee, Minister of Ministry of Culture>
<Jin-Hyun Kim, Minister of MST(Ministry of
Science and Technology)>

55
Wall Paintings of Buseoksa Temple
- The oldest Buddhist painting in Korea drawn on the inner wall at the
time of the construction in 1377.
- Consists of four statues of the 4 四天王像 and 2 菩薩像 and designated
as the national treasure No. 46 of the Republic of Korea in Dec. 20, 1962.
- Currently dismantled and kept in a separate building. A lot of damage
during dismantling necessitates a round recovery.
<Buseogsa josadang> <Wall paintings of Buseogsa josadang>

56
Wall Paintings of Buseoksa Temple
- Information of Wall Painting
• In fresco painting, wall painting is composed of sketch on the walls
and color with 4-5 kinds of minerals.
• fresco painting can be preserved for over 1,000 years
<Composition of Wall Painting>

57
Computational restoration of Buseoksa Josadang wall paintings
(a) Input of Wall Painting to Color and Infrared Images
- Color image for color analysis, Infrared Image for sketch analysis
- One Wall Painting divided into 48 parts for detail input
(b) Developed various image transformation / enhancement module
- Orthogonal transform, enhancement and smoothing (Histogram
transformation, noise removal, filtering, etc.)
- Edge and line detection (Differential, Template Matching, Huristic
search, etc.)
- Detection of various geometric features (area, length,
circumference, moment, compression / expansion, etc.)
(c) Extract image feature information
- Extraction of color and sketch information using image processing
S/W

58
Computational restoration of Buseoksa Josadang wall paintings
<Input Wall Painting>
<Sketch & Color Extraction>
<Restored Wall Painting>

59
4. Others
Restoration of mural paintings of Koguryo tombs
<5蓋墳 4號墓, Koguryo tombs, JiAn,
China>
<Many Wall Paintings>

61
- Super high-resolution, large-capacity rubbed copy of Monumental
character image
- Developed various image transformation/enhancement module
. black and white image, geometry reversal / contrast emphasis
. extraction and analysis of arbitrary feature quantity
. expansion, reduction, and removal of arbitrary magnification, etc.
Image processing and C.G. S/W Tool for reading Monumental character

62
- X-ray image input, digitization, image enhancement and edge detection
- Detection of various geometric features and comparison of geometrical
features between two images
S/W Tool for Ancient Sword Analysis

63
5. 3D Restoration of Cheongryongsa Daeungjeon
Anseong Cheongryongsa Daeungjeon (安城靑龍寺大雄殿)
<Anseong Cheongryongsa Daeungjeon (安城靑龍寺大雄殿)>
- National Treasure No. 824.
- Very sophisticated and beautiful building using various building members
- Three double-sided, four-sided double-glazed roof building.
- A granite stone column was laid on a foundation made of natural stone, and it was built
as a pillar on top of it, and a square (平枋) was laid on the top of the pillar with
Changbang.
- The square of the inner and outer exuding neck (內外三出目) raised the horror.
- It is very peculiar to write the raw wood as a pole, which is not processed at all.

64
- Development of next generation digital building cultural property using
building members
- Preserving traditional culture through exploration of experience contents
of architecture cultural assets: e-heritage content.

65
- 3D Digitization using building members

66
- Pipeline of Restoration of Building cultural assets

68
EBS Docuprime “천불천탑의 신비 미얀마” (2015.5.18~20)

First part_B-Image Processing - 1 of 2).pdf

More Related Content

Similar to First part_B-Image Processing - 1 of 2).pdf (20)

Recently uploaded (20)

First part_B-Image Processing - 1 of 2).pdf