CI image processing

Advance Techniques of
Computational Intelligence for
Biomedical Images Analysis
Dr. Meenakshi Sood
Associate Professor
NITTTR, Chandigarh, India
meenakshi@nitttrchd.ac.in

Computational intelligence
 Theory, design, application, and development of
biologically and linguistically motivated
computational paradigms According to Engelbrecht (2006)
9/19/20202 Dr MEENAKSHI S NITTTR CHD

The adaptive mechanisms include the following AI
paradigms that exhibit an ability to learn or adapt to
new environments:
 Swarm Intelligence (SI),
 Artificial Neural Networks (ANN),
 Evolutionary Computation (EC),
 Artificial Immune Systems (AIS), and
 Fuzzy Systems (FS).

Introduction
computationally intelligent system is
characterized with the capability of
 computational adaptation,
 fault tolerance, and
 high computation speed.

Adaptation
1: the act or process of adapting : the state of being adapted
2: adjustment to environmental conditions
Adapt: to make fit (as for a specific or new use or situation)
often by modification
Adaptation is any process whereby a
structure is progressively modified to give
better performance in its environment.
Holland 1992

Learning
Learning: knowledge or skill acquired by instruction or study
syn: knowledge
Learn: to gain knowledge or understanding of or skill in by
study, instruction or experience syn: discover
learning produces changes within an organism that, over time,
enables it to perform more effectively within its environment

Adaptation versus Learning
Adaptation in learning through making adjustments in
order to be more attuned to its environment.
 It involves a progressive modification of some
structure or structures, and uses a set of
operators acting on the structure(s) that
evolve over time.
 learning is more than just adaptation


 Learning is what an entire intelligent system
does.
 The ability to improve one’s performance over
time, is considered the main hallmark of
intelligence, and the greatest challenge of AI

Artificial Intelligence
Smart
erAdaptiv
e
Evolvable
Intelligent
Neural Networks
Fuzzy Systems
Evolutionary Computing

Difference between AI and CI
 Artificial Intelligence (AI) is the study of intelligent
behavior demonstrated by machines as opposed to
the natural intelligence in human beings
 Computational Intelligence (CI), is the study of
adaptive mechanisms to enable or facilitate
intelligent behavior in complex and changing
environments.

Computational Intelligence (CI)
 Collective system capable of accomplishing difficult
tasks in dynamic and varied environments without any
external guidance or control and with no central
coordination
 Achieving a collective performance which could not
normally be achieved by an individual acting alone

Computational Intelligence (CI)

Swarm Intelligence (SI)
 An artificial intelligence (AI) technique based on the
collective behavior in decentralized, self-organized
systems
 Generally made up of agents who interact with each
other and the environment
 No centralized control structures
 Based on group behavior found in nature

What is a Swarm?
 A loosely structured collection of interacting agents
 Agents:
 Individuals that belong to a group (but are not necessarily
identical)
 They contribute to and benefit from the group
 They can recognize, communicate, and/or interact with
each other
 The instinctive perception of swarms is a group of agents in
motion.

Examples of Swarms in Nature
 Classic Example: Swarm of Bees
 Can be extended to other similar systems:
 Ant colony
 Agents: ants
 Flock of birds
 Agents: birds
 Crowd
 Agents: humans
 Immune system
 Agents: cells and molecules

Bees
 Colony cooperation
 Regulate hive temperature
 Efficiency via Specialization: division of labour in the
colony
 Communication : Food sources are exploited according
to quality and distance from the hive

Wasps
 Pulp foragers, water foragers & builders
 Complex nests
 Horizontal columns
 Protective covering
 Central entrance hole

Ants
 Organizing highways to and from their foraging sites
by leaving pheromone trails
 Form chains from their own bodies to create a bridge
to pull and hold leafs together with silk
 Division of labour between major and minor ants

Why Insects?
 Insects have a few hundred brain cells
 However, organized insects have been known
for:
Architectural marvels
Complex communication systems
Resistance to hazards in nature

From Insects to Realistic
A.I. Algorithms

27
 The recent progress in machine learning and artificial
intelligence can be attributed to:
• Explosion of tremendous amount of data
• Cheap Computational cost due to the development of CPUs
and GPUs
• Improvement in learning algorithms
 Current excitement concerns a subfield called “Deep
Learning”.
9/19/2020Dr MEENAKSHI S NITTTR CHD

Why deeper?
9/19/2020Dr MEENAKSHI S NITTTR CHD28
• Deeper networks are able to use far fewer units per layer and far
fewer parameters, as well as frequently generalizing to the test
set.
• But harder to optimize!
• Choosing a deep model encodes a very general belief that the
function we want to learn involves composition of several
simpler functions.
Hidden layers (cascading tiers) of processing “Deep”
networks (3+ layers) versus “shallow” (1-2 layers)

Traditional and
Deep Learning networks

Curse of dimensionality
• The core idea in deep learning is that we assume that the
data was generated by the composition factors or features,
potentially at multiple levels in a hierarchy.
• This assumption allows an exponential gain in the
relationship between the number of examples and the
number of regions that can be distinguished.
• The exponential advantages conferred by the use of deep,
distributed representations counter the exponential challenges
posed by the curse of dimensionality.

Deep Neural Networks (DNN)
Deep Neural Network is a deep and wide Neural Network.
More number of hidden layers Many Input/ hidden nodes
Deep
Wide

Continued….
• Utilizes learning algorithms that
derive meaningful data using
hierarchy of multiple layers that
mimics the neural network of human
brain.
• If we provide the systems tons of
information, it begins to understand it
and respond in a useful way.
• Can learn increasingly complex
features and train complex networks.
• More specific and more general-
purpose than hand-engineered
features.

Universality Theorem
Reference for the reason:
http://neuralnetworksandde
eplearning.com/chap4.html
Any continuous function f
M
: RRf N

Can be realized by a network
with one hidden layer
(given enough hidden
neurons)
Why “Deep” neural network not “Fat” neural network?
Deeper is Better?

Fat + Short v.s. Thin + Tall
1x 2x …… Nx
Deep
1x 2x …… Nx
……
Shallow

Fat + Short v.s. Thin + Tall
Seide, Frank, Gang Li, and Dong Yu. "Conversational Speech Transcription Using
Context-Dependent Deep Neural Networks." Interspeech. 2011.
Layer X
Size
Word Error
Rate (%)
Layer X
Size
Word Error
Rate (%)
1 X 2k 24.2
2 X 2k 20.4
3 X 2k 18.4
4 X 2k 17.8
5 X 2k 17.2 1 X 3772 22.5
7 X 2k 17.1 1 X 4634 22.6
1 X 16k 22.1

Why Deep?
 Deep → Modularization
Image
Sharing by the
following classifiers
as module
Classifier
1
Classifier
2
Classifier
3
Classifier
4
Basic
Classifier

Why Deep?
 Deep → Modularization
1x
2x
……
Nx
……
……
……
……
……
……
→ Less training data?

Hand-crafted
kernel function
SVM
Source of image: http://www.gipsa-lab.grenoble-
inp.fr/transfert/seminaire/455_Kadri2013Gipsa-lab.pdf
Apply simple
classifier
Deep Learning
1x
2x
…
Nx
…
…
…
y1
y2
yM
…
……
……
……
simple
classifier
Learnable kernel

o Manually designed features are often over-specified, incomplete
and take a long time to design and validate
o Learned Features are easy to adapt, fast to learn
o Deep learning provides a very flexible, (almost?) universal,
learnable framework for representing world, visual and linguistic
information.
o Can learn both unsupervised and supervised
Why is DL useful?
In ~2010 DL started outperforming other
ML techniques
first in speech and vision, then NLP

Size of Data
Performance
Traditional ML algorithms
“Deep Learning doesn’t do different things,
it does things differently”

Technology
Deep learning is a fast-growing field, and new architectures,
variants appearing frequently.
1. Convolution Neural Network (CNN)
CNNs exploit spatially-local
correlation by enforcing a local
connectivity pattern between
neurons of adjacent layers.

Architecture
CNNs are multilayered neural networks that include input and
output layers as well as a number of hidden layers:
 Convolution layers – Responsible for filtering the input
image and extracting specific features such as edges, curves,
and colors.
 Pooling layers – Improve the detection of unusually placed
objects.
 Normalization layers – Improve network performance by
normalizing the inputs of the previous layer.
 Fully connected layers – In these layers, neurons have full
connections to all activations in the previous layer (similar to
regular neural networks).

Sample Architecture

Convolutional Neural Networks,
or CNN, ConvNET
a visual network,
 is a class of deep, feed-forward (not recurrent) artificial neural networks that
are applied to analyzing visual imagery.
 Input can have very high dimension. Using a fully-connected
neural network would need a large amount of parameters.
 CNNs are a special type of neural network whose hidden units are
only connected to local receptive field. The number of parameters
needed by CNNs is much smaller.

 In the first component, the CNN runs multiple
convolutions and pooling operations in order to detect
features it will then use for image classification.
 In the second component, using the extracted features, the
network algorithm attempts to predict what the object in
the image could be with a calculated probability.
 CNNs are widely used for implementing AI in image
processing and solving such problems as signal processing,
image classification, and image recognition.
CNN, ConvNET

 There are numerous types of CNN architectures such
as AlexNet, ZFNet, faster R-CNN,
and GoogLeNet/Inception.
 The choice of a specific CNN architecture depends on
the task at hand.
 For instance, GoogLeNet shows a higher accuracy for
leaf recognition than AlexNet or a basic CNN.
 At the same time, due to the higher number of layers,
GoogLeNet takes longer to run.

Pre-Trained Models
• URLs: https://github.com/BVLC/caffe/wiki/Model-Zoo
http://deeplearning4j.org/model-zoo

Summary of Conv Layer
• Accepts a volume of size W1×H1×D1
• Requires four hyperparameters:
– Number of filters K
– their spatial extent F
– the stride S
– the amount of zero padding P
• Produces a volume of size W2×H2×D2
– W2=(W1−F+2P)/S+1
– H2=(H1−F+2P)/S+1
– D2=K
• With parameter sharing, it introduces F⋅F⋅D1 weights per filter, for a
total
of (F⋅F⋅D1)⋅K weights and K biases.
• In the output volume, the d-th depth slice (of size W2×H2) is the
result of performing a valid convolution of the d-th filter over the
input volume with a stride of S, and then offset by d-th bias.

Cont..
2. Recurrent Neural Network (RNN)
RNNs are called recurrent because they perform the same
task for every element of a sequence, with the output being
depended on the previous computations.

RNN
1x 2x
2y1y
1a 2a
Memory can be considered
as another input.
The output of hidden layer
are stored in the memory.
copy

1
Output
Delay
Hidden
Units
Inputs
Inputs x(t) outputs y(t) hidden state s(t)
the memory of the network
A delay unit is introduced to hold
activation until they are processed at the
next step
The decision a recurrent net reached at
time step t-1 affects the decision it will
reach one moment later at time step t.
So recurrent networks have two
sources of input, the present and the
recent past, which combine to
determine how they respond to new
data

Cont…
3. Long-Short Term Memory
LSTM can learn "Very Deep Learning" tasks that require
memories of events that happened thousands or even
millions of discrete time steps ago.

RNN vs LSTM

Deep Learning Development and
Deployment Cycle

Image Processing
Analyzing and manipulating images with a computer.
1. Import an image with an optical scanner or directly
through digital photography.
2. Manipulate or analyze the image in some way. This
stage can include image enhancement and data
compression, or the image may be analyzed to find
patterns that aren't visible by the human eye.
3. Output the result The result might be the image altered in
some way or it might be a report based on analysis of the
image.

Main purposes of image processing:
 Visualization represents processed data in an
understandable way, giving visual form to objects that
aren’t visible, for instance.
 Image sharpening and restoration improves the quality
of processed images.
 Image retrieval helps with image search.
 Object measurement allows you to measure objects in
an image.
 Pattern recognition helps to distinguish and classify
objects in an image, identify their positions, and
understand the scene.

Image processing includes EIGHT
key phases

Image processing includes eight
key phases
 Image acquisition is the process of capturing an
image with a sensor and converting it into a
manageable entity.
 Image enhancement improves the quality of an input
image and extracts hidden details from it.
 Image restoration removes any possible corruptions
(blur, noise, or camera misfocus) from an image in
order to get a cleaner version. This process is based
mostly on probabilistic and mathematical models.
 Color image processing includes processing of
colored images and different color spaces. Depending
on the image type, we can talk
about pseudocolor processing (when colors are
assigned grayscale values) or RGB processing (for
images acquired with a full-color sensor).

Cont….
 Image compression and decompression allow for
changing the image size and resolution. Compression
is responsible for reducing these size and resolution
while decompression is used for restoring images to
the original.
 Morphological processing describes the shape and
structure of the objects in an image.
 Image recognition is the process of identifying
specific features of particular objects in an image.
Image recognition often uses such techniques
as object detection, object recognition, and
segmentation.
 Representation and description is the process of
visualizing processed data.It was originally published
on https://www.apriorit.com/

 It’s difficult to accomplish all these tasks manually,
especially when it comes to processing massive amounts
of data. Here’s where AI and machine learning
(ML) algorithms become very helpful.
 The use of AI and ML boosts both the speed of data
processing and the quality of the final result.
 For instance, with the help of AI platforms, we can
successfully accomplish such complex tasks as object
detection, face recognition, and text recognition.

Biomedical SIGNAL PROCESSING
9/19/202062
 Application of engineering principles and techniques to
the medical field to close the gap between engineering
and medicine.
 Guide the medicine to use innovative technical tools
such as humanistic models, realistic simulations, web-
based online resources, etc.
 It combines the design and problem solving skills of
engineering with medical and biological sciences to
improve healthcare diagnosis and treatment.

Benign Malignant
CT Images US Images
Microscopic Images of Blood
MRI Images
DNA sequence signal
 Non-invasive visualization of internal organs, tissue,
etc.
Medical Imaging

Biomedical Images
MAJOR MODALITIES
9/19/202064
Image – a 2D signal f(x,y) or 3D f(x,y,z)

Biomedical Image Processing
9/19/202065
 Biomedical imaging informatics is a dynamic field, recently
evolving from focusing purely on image processing to
broader informatics.
 Having images in digital format makes them amenable to
image processing methodologies for enhancement, analysis,
display, storage, and even enhanced interpretation.
 It is a field that combines the expertise of
engineering with medical needs for the progress
of health care.

Quantitative imaging applications
9/19/202066
 use quantifiable features extracted from medical images for
a variety of decision support applications, such as
 the assessment of an abnormality to suggest a diagnosis, or
 to evaluate the severity, degree of change, or
 status of a disease, injury, or chronic condition.
In general, the quantitative imaging computer reasoning
systems apply a mathematical model (e.g., a classifier) or other
machine learning methods to obtain a decision output based on
the imaging inputs.
Dr MEENAKSHI S NITTTR CHD

Methods, Techniques, and Tools for
Image Processing with AI

Techniques for Image Processing with AI
 Pixelation — turning printed pictures into the digitized
ones
 Linear filtering — processing input signals and
producing the output ones which are subject to the
constraint of linearity
 Edge detection — finding meaningful edges of the
image’s objects
 Anisotropic diffusion — reducing the image noise
without removing crucial parts of the picture
 Principal components analysis — extracting the
features of the image

Open-source libraries for AI-based
image processing
 Computer vision libraries contain common image
processing functions and algorithms.
 Currently, there are several open-source libraries that you
can use when developing image processing and computer
vision features:
 OpenCV
 VXL
 AForge.NET
 LTI-LibIt

Machine learning frameworks and
platforms for IP
To make the development process a bit faster and easier,
you can use special platforms and frameworks.
 TensorFlow
 Caffe
 MATLAB Image Processing Toolbox
 Computer Vision by Microsoft
 Google Cloud Vision
 Google Colaboratory (Colab)

Tensor Flow
 Google’s TensorFlow is a popular open-source framework
with support for machine learning and deep learning. Using
TensorFlow, you can create and train custom deep learning
models.
 The framework also includes a set of libraries, including ones
that can be used in image processing projects and computer
vision applications.
Caffe
 Convolutional Architecture for Fast Feature Embedding
(Caffe) is an open-source C++ framework with a Python
interface. In the context of image processing, Caffe works
best for solving image classification and image segmentation
tasks. The framework supports commonly used types of deep
learning architectures as well as CPU- and GPU-based
accelerated libraries. 9/19/202071 Dr MEENAKSHI S NITTTR CHD

MATLAB Image Processing Toolbox
 This platform provides an image processing toolbox (IPT),
which includes multiple algorithms and workflow
applications for image processing, visualization, analysis,
and algorithm development.
 This toolbox can be used for noise reduction, image
enhancement, image segmentation, 3D image processing,
and other tasks. Many of the IPT functions support C/C++
code generation, so they can be used for deploying
embedded vision systems and desktop prototyping.

 Computer Vision is a cloud-based service provided by
Microsoft that gives you access to advanced algorithms that can
be used for image processing and data extraction.
 It allows you to perform image processing tasks such as:
 Analyzing visual features and characteristics of the image
 Moderating image content
 Extracting text from images
Cloud Vision is part of the Google Cloud platform and offers a set
of image processing features. It provides an API for integrating
such features as image labeling and classification, object
localization, and object recognition.
 Cloud Vision allows you to use pre-trained machine learning
models and create and train custom machine learning models for
solving different image processing tasks.

 Google Colaboratory (Colab)
 Google Colaboratory, otherwise known as Colab, is a free
cloud service that can be used not only for improving your
coding skills but also for developing deep learning
applications from scratch.
 Google Colab makes it easier to use popular libraries such as
OpenCV, Keras, and TensorFlow when developing an AI-
based application.
 The service is based on Jupyter Notebooks, allowing AI
developers to share their knowledge and expertise in a
comfortable way. Plus, in contrast to similar services, Colab
provides free GPU resources.

Deep learning - opened new doors in
medical image analysis
 Applications of deep learning in healthcare covers a
broad range of problems ranging from cancer
screening and disease monitoring to personalized
treatment suggestions.
 Various sources of data today - radiological imaging
(X-Ray, CT and MRI scans), pathology imaging and
recently, genomic sequences have brought an
immense amount of data at the physicians disposal.

Medical Image Processing using DNN
 Diabetic Retinopathy
 Histological and Microscopical Elements Detection
 Gastrointestinal (GI) Diseases Detection
 Cardiac Imaging
 Tumor Detection
 Alzheimer’s and Parkinsons Diseases Detection
 Brain lesion segmentation
 Lungs cancer

Applications in biomedical image
engineering
Deep featurerepresentation
Detection of organs and body parts
Cell detection inhistopathologicalimages
Segmentation

Computational Intelligence Performance Metrics
Percent correct
Average sum-squared error
Evolutionary algorithm effectiveness measures
Mann-Whitney U Test
Receiver operating characteristic curves
Recall, precision, sensitivity, specificity, etc.
Confusion matrices, cost matrices
Chi-square test 9/19/202078 Dr MEENAKSHI S NITTTR CHD

Dataset Sources
https://archive.ics.uci.edu/ml/datasets.php
https://www.kaggle.com/datasets
https://catalog.data.gov/dataset
https://ieee-dataport.org/datasets

Type of CNN
 Fully Convolutional Network
 The concept of a Fully Convolutional Network (FCN) was first offered by a team of researchers from the
University of Berkeley. The main difference between a CNN and FCN is that the latter has a convolutional
layer instead of a regular fully connected layer. As a result, FCNs are able to manage different input sizes.
Also, FCNs use downsampling (striped convolution) and upsampling (transposed convolution) to make
convolution operations less computationally expensive.
 This type of neural network is the perfect fit for image segmentation tasks when the neural network divides
the processed image into multiple pixel groupings which are then labeled and classified. Some of the most
popular FCNs used for semantic segmentation are DeepLab, RefineNet, and Dilated Convolutions.
 Deconvolutional Neural Network
 Deconvolutional Neural Networks (DNNs) are neural networks performing inverse convolutional models
where the input data is first unpooled and only then convoluted.
 Basically, DNNs use the same tools and methods as convolutional networks but in a different way. This type
of neural network is a perfect example of using artificial intelligence for image recognition as well as for
analyzing processed images and generating new ones. And, in contrast to regular CNNs, deconvolutional
networks can be trained in an unsupervised fashion.
 Generative Adversarial Network
 Generative Adversarial Networks (GANs) are supposed to deal with one of the biggest challenges neural
networks face these days: adversarial images.
 Adversarial images are known for causing massive failures in neural networks. For instance, a neural
network can be fooled if you add a layer of visual noise called perturbation to the original image. And even
though the difference is nearly unnoticeable to the human brain, computer algorithms struggle to classify
adversarial images properly (see Figure 3).


CI image processing

More Related Content

What's hot (20)

Similar to CI image processing (20)

Recently uploaded (20)

CI image processing