Autoencoders Tutorial | Autoencoders In Deep Learning | Tensorflow Training | Edureka

AI & DEEP LEARNING with TENSORFLOW www.edureka.co/ai-deep-learning-with-tensorflow

Agenda
❖ Why do we need Autoencoders?
❖ What are Autoencoders?
❖ Properties of Autoencoders
❖ Autoencoders Training & Architecture
❖ Types of Autoencoders
❖ Applications of Autoencoders

Multidimensional Data
Data represented best
Slow performance, High Precision
Low Dimensional Data
Reduced Precision
High Performance

PCA vs Autoencoders
Non-linear Transformations
Non-linear activation function
and multiple layers
Convolutional Layers
An autoencoder doesn’t have to
learn dense layers
Higher Efficiency
More efficient to learn several
layers with an autoencoder
Multiple Transformations
It gives a representation as the
output of each layer

Why Autoencoders?
Original Image Autoencoder PCA

Introduction to Autoencoders
An autoencoder neural network is an
unsupervised Machine learning
algorithm that applies
backpropagation, setting the target
values to be equal to the inputs.

Autoencoders
Key Facts about Autoencoders
➢ It is an unsupervised ML algorithm similar to
PCA
➢ It minimizes the same objective function as
PCA
➢It is a neural network
➢The neural network’s target output is its input

Autoencoders
Code02
Encoderr01
Decoder03
Components of Autoencoders

Autoencoders
Encoderr01
This is the part of the network
that compresses the input into
a latent space representation.

Autoencoders
Code02
This is the part of the network
represents the compressed
input that is fed to the decoder

Autoencoders
Decoder03
This part aims to reconstruct
the input from the latent space
representation

Properties of Autoencoders
Data-specific
Autoencoders are only able to
meaningfully compress data similar to
what they have been trained on
02
01
03 Lossy
The output of the autoencoder will not
be exactly the same as the input, it will
be a close but degraded representation
Unsupervised
Autoencoders are considered an
unsupervised learning technique since
they don’t need explicit labels to train on

Training Autoencoders
Number of
Layers
Code Size
Loss Function
Number of
node per
layers

Training Autoencoders
Smaller size results in more compression
The autoencoder can have many layers
Mean squared error or binary cross entropy
Stacked autoencoders look like a sandwich
Code Size
Number of Layers
Loss Function
Number of node per layers
Number of
Layers
Code Size
Loss Function
Number of
node per
layers

Architecture of Autoencoders

Bottleneck approach is an approach to for deciding
which aspects of observed data are relevant
information and what aspects can be thrown away
➢Compactness of representation, measured as the
compressibility
➢Representation retains about some behaviourally
relevant variables

In the neural net world, the
encoder is a neural network
that outputs a
representation z of data x.
Encoder

Decoder
In deep learning, the decoder
is a neural net that learns to
reconstruct the data x given a
representation z.

Loss Function

Reconstruction Loss

Regularizer

AI & DEEP LEARNING with TENSORFLOW www.edureka.co/ai-deep-learning-with-tensorflow .
Types of Autoencoders

Convolution Autoencoders

Convolution Autoencoders use the convolution
operator to learn to encode the input in a set of simple
signals and then try to reconstruct the input from
them.

In the image domain where the signals are finite, this formula becomes:
In the 2D discrete space, the convolution operation is defined as:

AI & DEEP LEARNING with TENSORFLOW www.edureka.co/ai-deep-learning-with-tensorflowCopyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
3
Use Case of CAE:
Image Reconstruction
Image Colorization
Advanced Applications
learns to remove noise or reconstruct missing parts.
Noisy Version is converted to clean version
the network fills the gaps in the image.

1
2
3
Use Case of CAE:
Image Colorization
➢ maps circles and squares from an image to the same image but with Colors
➢ Purple is formed sometimes because of blend of colors, where network
hesitates between circle or square.

1
22
3
Use Case of CAE:
Image Colorization
Latent space clusteringFully image colorization
Generating higher resolution
images

Sparse Autoencoders
Sparse autoencoders offer us an alternative method
for introducing an information bottleneck
without requiring a reduction in the number of
nodes at our hidden layers

Sparse Autoencoders
L1 Regularization
KL Divergence

Deep Autoencoders
A deep autoencoder is composed of two,
symmetrical deep-belief networks-
• First four or five shallow layers
representing the encoding half of the net
• second set of four or five layers that
make up the decoding half.

Deep Autoencoders
The first layer of the Deep
Autoencoder learns first-
order features in the raw
input such as edges in an
image

Deep Autoencoders
The second layer learns
second-order features
corresponding to patterns
in the appearance of first-
order features

1
2
3
Use Case: Deep
Autoencoders
Image Search
Data Compression
Topic Modeling
& Information
Retrieval

1
22
3
Use Case: Deep
Autoencoders
Image Search
Data Compression
Topic Modeling
& Information
Retrieval Original Image

Contractive Autoencoders
A contractive autoencoder is an
unsupervised deep learning technique
that helps a neural network encode
unlabeled training data.

Contractive Autoencoders

AI & DEEP LEARNING with TENSORFLOW www.edureka.co/ai-deep-learning-with-tensorflow .
Applications of Autoencoder

DimensionalityReduction
Image ColoringDenoisingImages
WatermarkRemoval
FeatureVariation
ImageColoring

Feature VariationDenoisingImages
WatermarkRemoval
FeatureVariation
ImageColoring

Dimensionality ReductionDenoisingImages
WatermarkRemoval
FeatureVariation
ImageColoring

Denoising Images
DenoisingImages
WatermarkRemoval
FeatureVariation
ImageColoring

Watermark Removal
DenoisingImages
WatermarkRemoval
FeatureVariation
ImageColoring

Autoencoders Tutorial | Autoencoders In Deep Learning | Tensorflow Training | Edureka

More Related Content

What's hot (20)

Similar to Autoencoders Tutorial | Autoencoders In Deep Learning | Tensorflow Training | Edureka (20)

More from Edureka! (20)

Recently uploaded (20)

Autoencoders Tutorial | Autoencoders In Deep Learning | Tensorflow Training | Edureka