Tensor Explained

Tomomi Research Inc.
Tensor
: Data representation for deep learning
2018/09/21 (Fri)
Dr. Seong-Hun Choe

Agenda
9/22/2018 2©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Tensor : Dimension
2. Tensor : Shape
3. Real world data : Which tensor?

Tensor
WTF is a tensor?
Remember that it is a just container for data.
the data are almost numerical data.
So, Tensor is a container for numbers.
Various explanations,
but very hard to understand its concept
?
?

Keywords
1. Tensor is a container of numbers.
2. Tensor is a generalization of matrices to an arbitrary number of dimensions.
3. In tensor, dimension is often called axis.
4. number of dimension (=axis) is called ranks.

1. Scalar (0D tensor)
A tensor that contains only one number is called a scalar
• example : 12
dimension can be shown with ndim method.

2. Vector (1D tensor)
An array of numbers is called a vector, or 1D tensor
• example : [12,3,6,14]

3. Matrix (2D tensor)
An array of vectors is a matrix, or 2D tensor
• example : [[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]

4. 3D tensor
It is just nD tensor from 3D tensor.
• example :[[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]]

Agenda
2. Tensor : Shape

Key attributes
1. Number of dimension(=axes)
2. Shape : how many dimensions in the tensor has along each axis.
3. Data type : dtype in python, (float32, float64, unit8 and so on.)
Shape is very important in deep learning programming.

2.1. Scalar (0D tensor)
Scalar has empty shape.
• example : 12

2.2 Vector (1D tensor)
1D tensor has a shape with a single element, such as (4,)
• example : [12,3,6,14]

2.3. Matrix (2D tensor)
2D tensor has a shape such as (3,4). it is familiar with matrix
representation.
• example : [[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]

2.4. 3D tensor
3D tensor has a shape (3, 3, 4)
• example :[[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]]

2.5. MNIST example
Keras & Tensorflow
• loading the mnist dataset:
• dimension of train images
-> 3D tensor
• shape of train images
-> 3D tensor여서(m,n,p)의 형태
내용은、(28,28)の어레이가 60000개 있습니다. 라는
의미
• data type of train images
-> 8 bit integer

Summary (1) : Tensor
dimension 0 1 2 3 4
Name Scalar Vector Matrix 3D tensor 4D tensor
Another name 0D tensor 1D tensor 2D tensor 3D tensor 4D tensor
Example 12 [12,3,6,14] …
Shape
()
empty
(4,) (3,4) (3,3,4) (5,3,3,4)

Agenda
2. Tensor : Shape

3. Real-World examples of data as tensors
Name Tensor Shape
Vector data* 2D tensor (samples, feature)
Timeseries data or
sequence data
3D tensor (samples, timesteps, features)
Images 4D tensor (samples, height, width, channels)
Video 5D tensor (samples, frames, height, width, channels)
Vector data is different with vector. vector is 1D tensor.

3.1. Vector data
2D tensor
Samples Age ZIP code Income
1 12 123-324 10k
2 34 234-567 13k
3 12 349-874 20k
…
9,999 45 874-988 30k
10,000 56 888-234 12k
Example : Actual personal data,
3
10,000
Shape : (samples, feature)
= (10,000, 3)
• Numpy array
[[12, 123-324,10k],
[34,234-567,13k],
…
[56 ,888-234, 12k]]

time
(min.)
current price the highest
price
the lowest
price
0
1
2
…
390
time
(min.)
price
the lowest
price
0
1
2
…
390
time
(min.)
price
the lowest
price
0
1
2
…
390
3.2. Timeseries data or sequence data
3D tensor
time
(min.)
price
the lowest
price
0
1
2
…
390
Example: stock price dataset (1 year)
3
390
min.
250 days
Shape : (samples, timesteps, features)
= (250, 390, 3)

3.3 Image data
4D tensor
Example : A batch of 128 color images of size 256 * 256
Shape : (samples, height, width, channels)
= (128, 256, 256, 3)

3.3 Video data
5D tensor
Example : 60 second, 144X 156 Youtube video clip sample at 4 fps would be 240 frames.
A batch of 4 such video clips
Shape : (samples, frames, height, width, channels)
= (4, 240, 144, 156, 3)
total = 4 * 240 * 144 * 156 * 3 = 106,168,320
if dtype of the tensor is float32,
total memory will be 405 MB!

Summary (2) : Real data
Vector data
Timeseries data or
sequence data
Image data Video data
Tensor dimension 2D 3D 4D 5D
Example personal data annual stock data batch of color image batch of video frame
Example
Shape
(samples, feature)
= (10,000, 3)
(samples, timesteps, features)
= (250, 390, 3)
(samples, height, width,
channels)
= (128, 256, 256, 3)
(samples, frames, height, width,
channels)
=(4, 240, 144, 156, 3)

Tensor at Sony neural network console
(samples, channels, height, width)
= (128, 1, 28, 28)

Tensor at Tensorflow
주로Placeholder를 이용해 input data를 준비할때
(samples, timesteps, features)
= (?, 25, 1)
Sample수를 placeholder안에
지정하지 않을때는 、None으로 기입.

Tensor at Keras
(samples, time_setp, features)
= (176, 25,1)

Tensor Explained

More Related Content

What's hot (20)

Similar to Tensor Explained (13)

More from Seong-Hun Choe (15)

Recently uploaded (20)

Tensor Explained