![Bulletin of Electrical Engineering and Informatics
Vol. 10, No. 2, April 2021, pp. 716~723
ISSN: 2302-9285, DOI: 10.11591/eei.v10i2.2740 716
Journal homepage: http://beei.org
Experimental analysis of non-Gaussian noise resistance on
global method optical flow using bilateral in reverse confidential
Darun Kesrarat1
, Vorapoj Patanavijit2
1
Department of Information Technology, Vincent Mary School of Science and Technology, Assumption University,
Bangkok, Thailand
2
Department of Computer and Network Engineering, Vincent Mary School of Engineering, Assumption University,
Bangkok, Thailand
Article Info ABSTRACT
Article history:
Received Aug 14, 2020
Revised Dec 12, 2020
Accepted Feb 5, 2021
This paper presents the analytical of non-Gaussian noise resistance with the
aid of the use of bilateral in reverse confidential with the optical flow. In
particular, optical flow is the sample of the image’s motion from the
consecutive images caused by the object’s movement. It is a 2-D vector
where every vector is a displacement vector displaying the motion from the
first image to the second. When the noise interferes with the image flow, the
approximated performance on the vector in optical flow is poor. We ensure
greater appropriate noise resistance by applying bilateral in reverse
confidential in optical flow in the experiment by concerning the error vector
magnitude (EVM). Many noise resistance models of the global method
optical flow are using for comparison in our experiment. And many
sequenced image data sets where they are interfered with by several types of
non-Gaussian noise are used for experimental analysis.
Keywords:
Optical flow
Non-Gaussian noise
Error vector magnitude
Bilateral filter
Reverse confidential
This is an open access article under the CC BY-SA license.
Corresponding Author:
Darun Kesrarat
Department of Information Technology
Vincent Mary School of Science and Technology
Assumption University, Bangkok, Thailand
Email: darunksr@gmail.com
1. INTRODUCTION
During the last three-decades, optical flow [1, 2] is an approach that was applied in several fields
such as motion estimation to point the moving object motion vector and motion compression. In traditional
image compression [3], motion compensation [4], video encoding [5, 6], movement tracking [7, 8], and the
distinction among the movement frame is coded. Then, the higher resolution is constructed in super-
resolution [9, 10]. Consequently, optical flow is the computation approach for identifying the motion vector
(MV) on each pixel. Several approaches of optical flow were presented such local method [11], phase
correlation method [12], block-based method [13] but this paper concentrates on the proposed global method
optical flow (G-OF) [14]. Because the G-OF is certain concerning the differential gradient-based algorithms
where it has been applied for several area in advance because of its acceptable in performance for motion
estimation. Many research works focus on the computationally efficient [15, 16] but this paper focus on the
noise resistance approaches.
In the actual world, most regarding the sequences are interfered with by way of many prerequisites
so purpose noises upon the sequences. The main problem is noise interference. The performance of the
optical flow is sensitive to the noise that degrades the result in motion estimation. Many approaches focus on
pre-processing such a noise remover model [17, 18] to remove the noise on the image earlier. And many](https://image.slidesharecdn.com/192740-210813003753/85/Experimental-analysis-of-non-Gaussian-noise-resistance-on-global-method-optical-flow-using-bilateral-in-reverse-confidential-1-320.jpg)
![Bulletin of Electr Eng & Inf ISSN: 2302-9285
Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat)
717
approaches focus post-processing on a specific area such as many approaches in advance of optical flow have
been proposed during the last decade to improve the performance over the interfered noises such as the
bilateral function [19] and reverse confidential [20]. The concept of the reverse confidential in optical flow
and the bilateral function in optical flow was proved and presented an effective result in noise resistance.
This paper focuses on the use of the fusion approach among the bilateral filter and the reverse
confidential called the application of bilateral in reverse confidential with G-OF [21]. The objective of this
paper is to analyze the performance in the non-Gaussian noise resistance on global method optical flow using
bilateral in reverse confidential. The non-Gaussian noises are used in the experimental analysis such as salt &
pepper, speckle, and poisson. In the experimental, we analyze the performance in noise resistance with the
error vector magnitude (EVM) where the EVM concerns the degree of missed direction in the MV from the
optical flow’s outcome. And we also compare the performance in noise resistance against the other noise
resistance approaches to ensure greater appropriate noise resistance. The paper organization is arranged as
follows. Section 2 explains the research method in optical flow. Section 3 explains the experimental result.
Section 4 explains the conclusion.
2. RESEARCH METHODS IN GLOBAL METHOD OPTICAL FLOW AND NOISE
RESISTANCE APPROACHES
In this section, it is explained the global method optical flow and the noise resistance approaches on
the optical flow.
2.1. Global method optical flow
Global method optical flow (G-OF) [14] is an ordinary approach for pointing the 2-D vector motion
estimation via the use of “intensity of brightness level” with four-point mean differences [22, 23] and
“smoothness variation”. The iterative equations are calculated to obtain image velocity as in (1):
1
2 2 2
i i
x x y k
i i
x y
I I u I v I
u u
I I
+
+ +
= −
+ +
1
2 2 2
i i
y x y k
i i
x y
I I u I v I
v v
I I
+
+ +
= −
+ +
(1)
where
i
u and
i
v denote neighborhood average of horizontal and vertical. i is the number of iterative
calculations. The gradient intensity is:
(2)
where I(x,y,k) is the gradient intensity of an element (x,y) in the images at frame no. k.
2.2. Bilateral function in optical flow
Based on the effectiveness of noise removal from bilateral function, the use of a bilateral function
(BF) in correspondent with optical flow [19] found effectiveness in the noise tolerance on the result of MV.
The main process in corresponding the bilateral function in optical flow is to set the bilateral function [24-27]
over the result of the G-OF. In BF, the bilateral kernel is defined as:
(3)
In our work, we assigned 7× of v(x) for deviation in δa and deviation of intensity I(x) in δb based on
the traditional work [24]. Then, the kernel (ɸ) is occupied with the prepared function as:
(4)](https://image.slidesharecdn.com/192740-210813003753/85/Experimental-analysis-of-non-Gaussian-noise-resistance-on-global-method-optical-flow-using-bilateral-in-reverse-confidential-2-320.jpg)
![ ISSN: 2302-9285
Bulletin of Electr Eng & Inf, Vol. 10, No. 2, April 2020 : 716 – 723
718
In our work, we assigned ±7 concerning neighborhood for M, and K is the kernel normalization
based on the traditional work [24].
(5)
where m is a local range of neighborhood.
2.3. Reverse confidential optical flow
In the reverse confidential (RC) [20], the outcome of the MV from 2-ways was concerned in order
to ensure the stability of the optical flow approach. Then, the confidence function used the result of MV from
both directions to determine the confidence level and it was used to adjust the final result of MV.
In RC, the confidence function is defined as:
(6)
where vl and vl- are normal and reverse MV, and n is the number of neighbors MV. s is coordinate (x,y) in 2D
image and avoids the division by zero in the equation. The confidence levels are used to define the final
result MV in average of regional (N(s0)) by:-
(7)
where 0
( ( ))
n
l
v s is computed from the reliability of neighborhood N(s0) of the location s0=(x, y, k).
2.4. Bilateral in reverse confidential optical flow
The bilateral in reverse confidential (B-RC) [21] is the fusion approach that combines the traditional
bilateral function in correspondence with the reverse confidential approach. The process of B-RC is shown in
Figure 1. By using the 2-ways concept of RC, the bilateral function is applied to the MV of both direct and
the confidence is applied later. Then, the final outcome of MV is pointed from the confidence level.
Calculate the MV
with G-OF under
the 2-ways concept
of RC approach
Calculate Final MV
by confidence rate
based on RC
approach
Adopt Bilateral
function on 2-ways
MV
Get the set of
Image in
consecutive
Sequences
Consecutive
images
MV in
2 ways
Bilateral
vector
in 2-ways
Figure 1. The process of B-RC
3. EXPERIMENTAL AND ACHIEVEMENT
In this work, we used the global smoothness weight (α)=0.5 and use 4-point central differences
mask coefficient [22] for gradient estimation on G-OF. Four different well-known QCIF-176×144 sequences
(Akiyo, container, coastguard, and foreman) up to 50 frames on each were used on the test. Figure 2 presents
the sample of each sequence. Then, 5 kinds of non-Gaussian noise (poisson noise, salt & pepper noise at
density 0.005 and 0.025, and speckle noise at variance 0.01 and 0.05) were put in the sequence for testing.
Indefinitely, 20 types of the sequence were tested. And we evaluated the ordinary G-OF, BF, and RC in
comparison with B-RC.
Definitely, the achievement of noise tolerance was determined by the EVM. EVM is a measure of
how accurately in motion vector to quantify the performance where the lower in value of EVM means the
better in performance. In this work, we compare with the original ground truth MV and count only the no. of
non-zero movement vector with the root-mean-square by the average in the experimental. The Table 1
summarized the average EVM of each approach over 100 frames on the different types of non-Gaussian
noise. Figures 3-6 present the achievement of EVM in a frame by frame of the experiment sequences on the
different types of non-Gaussian noise by the graph.](https://image.slidesharecdn.com/192740-210813003753/85/Experimental-analysis-of-non-Gaussian-noise-resistance-on-global-method-optical-flow-using-bilateral-in-reverse-confidential-3-320.jpg)
![Bulletin of Electr Eng & Inf ISSN: 2302-9285
Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat)
719
(a) (b) (c) (d)
Figure 2. Experiment sequences, (a) Akiyo, (b) Coastguard, (c) Container, (d) Foreman
Table 1. Summarized the average EVM
Akiyo Coastguard Container Foreman
AVG
EVM
SD of
EVM
AVG
EVM
SD of
EVM
AVG
EVM
SD of
EVM
AVG
EVM
SD of
EVM
Poisson
G-OF [14] 3.653 0.305 3.608 0.148 4.252 0.113 3.776 0.543
RC [19] 3.199 0.282 3.097 0.145 3.761 0.118 3.304 0.496
BF [20] 3.218 0.272 3.064 0.146 3.746 0.106 3.702 0.424
*B-RC [21] 3.057 0.253 2.903 0.152 3.558 0.112 3.488 0.441
Salt (d=0.005)
G-OF [14] 3.658 0.889 2.975 0.291 4.846 0.770 3.675 1.103
RC [19] 3.583 0.846 2.667 0.246 4.780 0.744 3.494 1.041
BF [20] 3.385 0.392 2.761 0.164 4.181 0.340 5.215 0.670
*B-RC [21] 3.314 0.362 2.681 0.155 4.113 0.326 5.072 0.663
Salt (d=0.025)
G-OF [14] 6.424 1.533 4.005 0.361 9.107 1.572 5.321 0.861
RC [19] 6.162 1.464 3.566 0.315 8.761 1.500 4.989 0.803
BF [20] 5.109 1.090 3.458 0.287 8.262 1.322 6.063 0.555
*B-RC [21] 4.773 0.974 3.224 0.243 7.825 1.236 5.721 0.522
Speckle (v=0.01)
G-OF [14] 3.727 0.349 3.758 0.189 4.416 0.097 3.868 0.524
RC [19] 3.233 0.315 3.189 0.178 3.830 0.099 3.355 0.490
BF [20] 3.192 0.267 3.092 0.139 3.783 0.103 3.707 0.423
*B-RC [21] 3.024 0.260 2.916 0.148 3.573 0.103 3.484 0.441
Speckle (v=0.05)
G-OF [14] 4.034 0.314 4.080 0.175 4.683 0.132 4.258 0.439
RC [19] 3.417 0.296 3.403 0.159 3.993 0.119 3.618 0.453
BF [20] 3.449 0.298 3.323 0.154 3.982 0.124 3.716 0.420
*B-RC [21] 3.165 0.270 3.073 0.157 3.714 0.123 3.442 0.444
(a) (b) (c)
(d) (e)
Figure 3. The achievement of EVM in a frame by frame of Akiyo, (a) Poisson, (b) Salt & pepper (d=0.005),
(c) Salt & pepper (d=0.025), (d) Speckle (v=0.01), (e) Speckle (v=0.05)](https://image.slidesharecdn.com/192740-210813003753/85/Experimental-analysis-of-non-Gaussian-noise-resistance-on-global-method-optical-flow-using-bilateral-in-reverse-confidential-4-320.jpg)
![ ISSN: 2302-9285
Bulletin of Electr Eng & Inf, Vol. 10, No. 2, April 2020 : 716 – 723
722
REFERENCES
[1] A. Burton and J. Radford (editors), "Thinking in perspective: critical essays in the study of thought processes,"
Routledge, vol. 646, 1978.
[2] D. H. Warren and E. R. Strelow (editors), "Electronic spatial sensing for the blind: contributions from perception,"
Springer Science & Business Media, vol. 99, 1985.
[3] A. J. Qasim, R. Din, and F. Q. A. Alyousuf, "Review on techniques and file formats of image compression,"
Bulletin of Electrical Engineering and Informatics, vol. 9, no. 2, pp. 602-610, April 2020.
[4] Z. Chen, T. He, X. Jin, and F. Wu, "Learning for video compression," IEEE Transactions on Circuits and Systems
for Video Technology, vol. 30, no. 2, pp. 566-576, 2020.
[5] T. Wiegand, "Draft ITU-T Rec. and Final Draft International Standard of Joint Video Specification (ITU-T Rec.
H.264-ISO/IEC 14 496-10 AVC)," Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, document JVT-
G050r1.doc, JVT of ITU-T and ISO/IEC JTC1, 7th
Meeting, Pattaya, 2003.
[6] "Information Technology-Coding of Audio Visual Objects-Part 2: Visual, JTC1/SC29/WG11, ISO/IEC 14 469-2
(MPEG-4 Visual)," 2000.
[7] C. S. Royden and K. D. Moore, "Use of speed cues in the detection of moving objects by moving observers,"
Vision Research, vol. 59, pp. 17-24, 2012.
[8] S. A. Mahmoudi, M. Kierzynka, P. Manneback, and K. Kurowski, "Real-time motion tracking using optical flow
on multiple GPUs," Bull. Pol. Ac.:Tech., vol. 62, no. 1, pp. 139-150, 2014.
[9] C. Deng, J. Liu, W. Tian, S. Wang, H. Zhu, and S. Zhang, "Image super-resolution reconstruction based on L1/2
sparsity," Bulletin of Electrical Engineering and Informatics, vol. 3, no. 3, pp. 155-160, September 2014
[10] D. Kesrarat, K. Thakulsukanant, and V. Patanavijit, "A novel elementary spatial expanding scheme form on SISR
method with modifying Geman&McClure function," TELKOMNIKA (Telecommunication Computing Electronics
and Control), vol. 17, no. 5, pp. 2554-2560, Oct 2019.
[11] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," In
Proceeding of Defense Advanced Research Projects Agency (DARPA) Image Understanding Workshop, pp. 121-
130, 1981.
[12] D. J. Fleet and A. D. Jepson, "Computation of component image velocity from local phase information," Computer
Vision, vol. 5, no. 1, pp.77-104, 1990.
[13] T. R. Reed (editor), "Digital image sequence processing, compression, and analysis," Chemical Rubber Company
(CRC) Press, 2005.
[14] B. K. P. Horn and B.G. Schunck, "Determining optical flow," Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203,
1981.
[15] Y. Su, Y. Huang, and C-C. J. Kuo, "Efficient text classification using tree-structured multi-linear principal
component analysis," IEEE 24th
International Conference on Pattern Recognition (ICPR), pp. 585-590, 2018.
[16] Y. Su, R. Lin, and C. C. J. Kuo "Tree-structured multi-stage principal component analysis (TMPCA): Theory and
applications," Expert Systems with Applications, vol. 118, pp. 355-364, 2019.
[17] A. Khmag, S. Ghoul S. A. R. Al-Haddad, and N. Kamarudin, "Noise level estimation for digital images using local
statistics and its applications to noise removal," TELKOMNIKA (Telecommunication Computing Electronics and
Control), vol. 16, no. 2, pp. 915-924, April 2018.
[18] L. Abderrahim, M. Salama, and D. Abdelbaki, “Novel design of a fractional wavelet and its application to image
denoising,” Bulletin of Electrical Engineering and Informatics, vol. 9, no. 1, pp. 129-140, February 2020
[19] D. Kesrarat. and V. Patanavijit, "Experimental analysis of performance comparison on both linear filter and
bidirectional confidential technique for spatial domain optical flow algorithm," The Electrical
Engineering/Electronics, Computer, Communications and Information Technology Association (ECTI)
Transactions on Computer and Information Technology, vol. 7, no. 2, pp. 157-168, 2013.
[20] R. Li and S. Yu, "Confidence based optical flow algorithm for high reliability," In Proceedings of ICASSP, pp.785-
788, 2008.
[21] D. Kesrarat. and V. Patanavijit, "Robust global based spatial correlation optical flow in bidirectional confidential
technique with bilateral filter," In Proceedings of IEEE Explore of the International Conference on Intelligent
Informatics and BioMedical Sciences (ICIIBMS 2015), pp. 1-6, 2015.
[22] J. L. Barron, D. J. Fleet, and S. S. Beauchemin, "Performance of optical flow techniques," International Journal of
Computer Vision, vol. 12, no. 1, pp. 43-77, 1994.
[23] D. Fleet and Y. Weiss, "Optical flow estimation", Handbook of Mathematical Models in Computer Vision,
Springer, pp. 237-257, 2006.
[24] S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, "Bilateral filtering: Theory and applications," Found. Trends
Compt. Graph. Vis., vol. 4, no. 1, pp.1-73, 2008.
[25] M. G. Mozerrov, "Constrained optical flow estimation as a matching problem," IEEE Trans. Image Processing,
vol. 22, no. 5, pp. 2044-2055, 2013.
[26] J. Sun, Y. Li, S. B. Kang, and H. Y. Shum, "Symmetric stereo matching for occlusion handling," In proceeding of
IEEE Conf. Comput. Vis.Pattern Recognit., vol. 2, pp. 399-406, 2005.
[27] M. Zhang and B. Gunturk, "A new image denoising method based on the bilateral filter," IEEE International
Conference on Acoustics, Speech and Signal Processing, ICASSP 2008, pp. 929-932, 2008.](https://image.slidesharecdn.com/192740-210813003753/85/Experimental-analysis-of-non-Gaussian-noise-resistance-on-global-method-optical-flow-using-bilateral-in-reverse-confidential-7-320.jpg)
Experimental analysis of non-Gaussian noise resistance on global method optical flow using bilateral in reverse confidential
- 1. Bulletin of Electrical Engineering and Informatics Vol. 10, No. 2, April 2021, pp. 716~723 ISSN: 2302-9285, DOI: 10.11591/eei.v10i2.2740 716 Journal homepage: http://beei.org Experimental analysis of non-Gaussian noise resistance on global method optical flow using bilateral in reverse confidential Darun Kesrarat1 , Vorapoj Patanavijit2 1 Department of Information Technology, Vincent Mary School of Science and Technology, Assumption University, Bangkok, Thailand 2 Department of Computer and Network Engineering, Vincent Mary School of Engineering, Assumption University, Bangkok, Thailand Article Info ABSTRACT Article history: Received Aug 14, 2020 Revised Dec 12, 2020 Accepted Feb 5, 2021 This paper presents the analytical of non-Gaussian noise resistance with the aid of the use of bilateral in reverse confidential with the optical flow. In particular, optical flow is the sample of the image’s motion from the consecutive images caused by the object’s movement. It is a 2-D vector where every vector is a displacement vector displaying the motion from the first image to the second. When the noise interferes with the image flow, the approximated performance on the vector in optical flow is poor. We ensure greater appropriate noise resistance by applying bilateral in reverse confidential in optical flow in the experiment by concerning the error vector magnitude (EVM). Many noise resistance models of the global method optical flow are using for comparison in our experiment. And many sequenced image data sets where they are interfered with by several types of non-Gaussian noise are used for experimental analysis. Keywords: Optical flow Non-Gaussian noise Error vector magnitude Bilateral filter Reverse confidential This is an open access article under the CC BY-SA license. Corresponding Author: Darun Kesrarat Department of Information Technology Vincent Mary School of Science and Technology Assumption University, Bangkok, Thailand Email: darunksr@gmail.com 1. INTRODUCTION During the last three-decades, optical flow [1, 2] is an approach that was applied in several fields such as motion estimation to point the moving object motion vector and motion compression. In traditional image compression [3], motion compensation [4], video encoding [5, 6], movement tracking [7, 8], and the distinction among the movement frame is coded. Then, the higher resolution is constructed in super- resolution [9, 10]. Consequently, optical flow is the computation approach for identifying the motion vector (MV) on each pixel. Several approaches of optical flow were presented such local method [11], phase correlation method [12], block-based method [13] but this paper concentrates on the proposed global method optical flow (G-OF) [14]. Because the G-OF is certain concerning the differential gradient-based algorithms where it has been applied for several area in advance because of its acceptable in performance for motion estimation. Many research works focus on the computationally efficient [15, 16] but this paper focus on the noise resistance approaches. In the actual world, most regarding the sequences are interfered with by way of many prerequisites so purpose noises upon the sequences. The main problem is noise interference. The performance of the optical flow is sensitive to the noise that degrades the result in motion estimation. Many approaches focus on pre-processing such a noise remover model [17, 18] to remove the noise on the image earlier. And many
- 2. Bulletin of Electr Eng & Inf ISSN: 2302-9285 Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat) 717 approaches focus post-processing on a specific area such as many approaches in advance of optical flow have been proposed during the last decade to improve the performance over the interfered noises such as the bilateral function [19] and reverse confidential [20]. The concept of the reverse confidential in optical flow and the bilateral function in optical flow was proved and presented an effective result in noise resistance. This paper focuses on the use of the fusion approach among the bilateral filter and the reverse confidential called the application of bilateral in reverse confidential with G-OF [21]. The objective of this paper is to analyze the performance in the non-Gaussian noise resistance on global method optical flow using bilateral in reverse confidential. The non-Gaussian noises are used in the experimental analysis such as salt & pepper, speckle, and poisson. In the experimental, we analyze the performance in noise resistance with the error vector magnitude (EVM) where the EVM concerns the degree of missed direction in the MV from the optical flow’s outcome. And we also compare the performance in noise resistance against the other noise resistance approaches to ensure greater appropriate noise resistance. The paper organization is arranged as follows. Section 2 explains the research method in optical flow. Section 3 explains the experimental result. Section 4 explains the conclusion. 2. RESEARCH METHODS IN GLOBAL METHOD OPTICAL FLOW AND NOISE RESISTANCE APPROACHES In this section, it is explained the global method optical flow and the noise resistance approaches on the optical flow. 2.1. Global method optical flow Global method optical flow (G-OF) [14] is an ordinary approach for pointing the 2-D vector motion estimation via the use of “intensity of brightness level” with four-point mean differences [22, 23] and “smoothness variation”. The iterative equations are calculated to obtain image velocity as in (1): 1 2 2 2 i i x x y k i i x y I I u I v I u u I I + + + = − + + 1 2 2 2 i i y x y k i i x y I I u I v I v v I I + + + = − + + (1) where i u and i v denote neighborhood average of horizontal and vertical. i is the number of iterative calculations. The gradient intensity is: (2) where I(x,y,k) is the gradient intensity of an element (x,y) in the images at frame no. k. 2.2. Bilateral function in optical flow Based on the effectiveness of noise removal from bilateral function, the use of a bilateral function (BF) in correspondent with optical flow [19] found effectiveness in the noise tolerance on the result of MV. The main process in corresponding the bilateral function in optical flow is to set the bilateral function [24-27] over the result of the G-OF. In BF, the bilateral kernel is defined as: (3) In our work, we assigned 7× of v(x) for deviation in δa and deviation of intensity I(x) in δb based on the traditional work [24]. Then, the kernel (ɸ) is occupied with the prepared function as: (4)
- 3. ISSN: 2302-9285 Bulletin of Electr Eng & Inf, Vol. 10, No. 2, April 2020 : 716 – 723 718 In our work, we assigned ±7 concerning neighborhood for M, and K is the kernel normalization based on the traditional work [24]. (5) where m is a local range of neighborhood. 2.3. Reverse confidential optical flow In the reverse confidential (RC) [20], the outcome of the MV from 2-ways was concerned in order to ensure the stability of the optical flow approach. Then, the confidence function used the result of MV from both directions to determine the confidence level and it was used to adjust the final result of MV. In RC, the confidence function is defined as: (6) where vl and vl- are normal and reverse MV, and n is the number of neighbors MV. s is coordinate (x,y) in 2D image and avoids the division by zero in the equation. The confidence levels are used to define the final result MV in average of regional (N(s0)) by:- (7) where 0 ( ( )) n l v s is computed from the reliability of neighborhood N(s0) of the location s0=(x, y, k). 2.4. Bilateral in reverse confidential optical flow The bilateral in reverse confidential (B-RC) [21] is the fusion approach that combines the traditional bilateral function in correspondence with the reverse confidential approach. The process of B-RC is shown in Figure 1. By using the 2-ways concept of RC, the bilateral function is applied to the MV of both direct and the confidence is applied later. Then, the final outcome of MV is pointed from the confidence level. Calculate the MV with G-OF under the 2-ways concept of RC approach Calculate Final MV by confidence rate based on RC approach Adopt Bilateral function on 2-ways MV Get the set of Image in consecutive Sequences Consecutive images MV in 2 ways Bilateral vector in 2-ways Figure 1. The process of B-RC 3. EXPERIMENTAL AND ACHIEVEMENT In this work, we used the global smoothness weight (α)=0.5 and use 4-point central differences mask coefficient [22] for gradient estimation on G-OF. Four different well-known QCIF-176×144 sequences (Akiyo, container, coastguard, and foreman) up to 50 frames on each were used on the test. Figure 2 presents the sample of each sequence. Then, 5 kinds of non-Gaussian noise (poisson noise, salt & pepper noise at density 0.005 and 0.025, and speckle noise at variance 0.01 and 0.05) were put in the sequence for testing. Indefinitely, 20 types of the sequence were tested. And we evaluated the ordinary G-OF, BF, and RC in comparison with B-RC. Definitely, the achievement of noise tolerance was determined by the EVM. EVM is a measure of how accurately in motion vector to quantify the performance where the lower in value of EVM means the better in performance. In this work, we compare with the original ground truth MV and count only the no. of non-zero movement vector with the root-mean-square by the average in the experimental. The Table 1 summarized the average EVM of each approach over 100 frames on the different types of non-Gaussian noise. Figures 3-6 present the achievement of EVM in a frame by frame of the experiment sequences on the different types of non-Gaussian noise by the graph.
- 4. Bulletin of Electr Eng & Inf ISSN: 2302-9285 Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat) 719 (a) (b) (c) (d) Figure 2. Experiment sequences, (a) Akiyo, (b) Coastguard, (c) Container, (d) Foreman Table 1. Summarized the average EVM Akiyo Coastguard Container Foreman AVG EVM SD of EVM AVG EVM SD of EVM AVG EVM SD of EVM AVG EVM SD of EVM Poisson G-OF [14] 3.653 0.305 3.608 0.148 4.252 0.113 3.776 0.543 RC [19] 3.199 0.282 3.097 0.145 3.761 0.118 3.304 0.496 BF [20] 3.218 0.272 3.064 0.146 3.746 0.106 3.702 0.424 *B-RC [21] 3.057 0.253 2.903 0.152 3.558 0.112 3.488 0.441 Salt (d=0.005) G-OF [14] 3.658 0.889 2.975 0.291 4.846 0.770 3.675 1.103 RC [19] 3.583 0.846 2.667 0.246 4.780 0.744 3.494 1.041 BF [20] 3.385 0.392 2.761 0.164 4.181 0.340 5.215 0.670 *B-RC [21] 3.314 0.362 2.681 0.155 4.113 0.326 5.072 0.663 Salt (d=0.025) G-OF [14] 6.424 1.533 4.005 0.361 9.107 1.572 5.321 0.861 RC [19] 6.162 1.464 3.566 0.315 8.761 1.500 4.989 0.803 BF [20] 5.109 1.090 3.458 0.287 8.262 1.322 6.063 0.555 *B-RC [21] 4.773 0.974 3.224 0.243 7.825 1.236 5.721 0.522 Speckle (v=0.01) G-OF [14] 3.727 0.349 3.758 0.189 4.416 0.097 3.868 0.524 RC [19] 3.233 0.315 3.189 0.178 3.830 0.099 3.355 0.490 BF [20] 3.192 0.267 3.092 0.139 3.783 0.103 3.707 0.423 *B-RC [21] 3.024 0.260 2.916 0.148 3.573 0.103 3.484 0.441 Speckle (v=0.05) G-OF [14] 4.034 0.314 4.080 0.175 4.683 0.132 4.258 0.439 RC [19] 3.417 0.296 3.403 0.159 3.993 0.119 3.618 0.453 BF [20] 3.449 0.298 3.323 0.154 3.982 0.124 3.716 0.420 *B-RC [21] 3.165 0.270 3.073 0.157 3.714 0.123 3.442 0.444 (a) (b) (c) (d) (e) Figure 3. The achievement of EVM in a frame by frame of Akiyo, (a) Poisson, (b) Salt & pepper (d=0.005), (c) Salt & pepper (d=0.025), (d) Speckle (v=0.01), (e) Speckle (v=0.05)
- 5. ISSN: 2302-9285 Bulletin of Electr Eng & Inf, Vol. 10, No. 2, April 2020 : 716 – 723 720 (a) (b) (c) (d) (e) Figure 4. The achievement of EVM in a frame by frame of coastguard, (a) Poisson, (b) Salt & pepper (d=0.005), (c) Salt & pepper (d=0.025), (d) Speckle (v=0.01), (e) Speckle (v=0.05) (a) (b) (c) (d) (e) Figure 5. The achievement of EVM in a frame by frame of CONTAINER, (a) Poisson, (b) Salt & pepper (d=0.005), (c) Salt & pepper (d=0.025), (d) Speckle (v=0.01), (e) Speckle (v=0.05)
- 6. Bulletin of Electr Eng & Inf ISSN: 2302-9285 Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat) 721 (a) (b) (c) (d) (e) Figure 6. The achievement of EVM in a frame by frame of FOREMAN, (a) Poisson, (b) Salt & pepper (d=0.005), (c) Salt & pepper (d=0.025), (d) Speckle (v=0.01), (e) Speckle (v=0.05) 4. CONCLUSION In our inspection, we concluded the experiment into 2 groups. The first group was concluded by the type of non-Gaussian noise and the second group was concluded by the characteristic of the experiment sequences. Because we found that the performance in noise resistance varies base on the types. Regarding the type of non-Gaussian noise, we found that different types of noise return different results from each approach. From the experimental, we found that: (i) The salt & pepper noise was the most impacted noise that degraded the performance of the optical due to the highest value of EVM, (ii) The performance of the B- RC achieved the best noise resistance over poisson and speckle noise and presented the second-best in salt & pepper noise in coastguard and foreman sequence. Regarding the characteristics of the experimental sequences, we found that it impacted the performance of the optical flow. In this experiment, we separate the sequences into 2 subgroups for analysis. There is a group of fast movement sequences such as coastguard and foreman sequence and the group of slow movement sequences such as Akiyo and container. From the experimental, we found that: (i) The fast movement sequence impacted to degrade the performance of the optical than slow movement sequence due to the higher value in average EVM of the fast movement sequences, (ii) The performance of the B-RC achieved the best over the slow movement sequence, (iii) The performance of the RC achieved the best over the fast movement sequence only in the low level of salt & pepper and speckle noise. With the increase of noise level in salt & pepper and speckle noise, the B-RC became the best in overall. From the overall result in the experiment, the B-RC presented the greatest result in noise resistance over the non-Gaussian noise. Then, the application areas such as motion tracking in the noisy environment are suitable to adopt this method to gain accuracy in optical flow. There is no silver bullet method for achieving the best result. Then, the study on several kinds of noises in correspond with several noise resistance approaches is an issue for future research to explore. ACKNOWLEDGEMENTS This research project was funded by Assumption University.
- 7. ISSN: 2302-9285 Bulletin of Electr Eng & Inf, Vol. 10, No. 2, April 2020 : 716 – 723 722 REFERENCES [1] A. Burton and J. Radford (editors), "Thinking in perspective: critical essays in the study of thought processes," Routledge, vol. 646, 1978. [2] D. H. Warren and E. R. Strelow (editors), "Electronic spatial sensing for the blind: contributions from perception," Springer Science & Business Media, vol. 99, 1985. [3] A. J. Qasim, R. Din, and F. Q. A. Alyousuf, "Review on techniques and file formats of image compression," Bulletin of Electrical Engineering and Informatics, vol. 9, no. 2, pp. 602-610, April 2020. [4] Z. Chen, T. He, X. Jin, and F. Wu, "Learning for video compression," IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 2, pp. 566-576, 2020. [5] T. Wiegand, "Draft ITU-T Rec. and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264-ISO/IEC 14 496-10 AVC)," Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, document JVT- G050r1.doc, JVT of ITU-T and ISO/IEC JTC1, 7th Meeting, Pattaya, 2003. [6] "Information Technology-Coding of Audio Visual Objects-Part 2: Visual, JTC1/SC29/WG11, ISO/IEC 14 469-2 (MPEG-4 Visual)," 2000. [7] C. S. Royden and K. D. Moore, "Use of speed cues in the detection of moving objects by moving observers," Vision Research, vol. 59, pp. 17-24, 2012. [8] S. A. Mahmoudi, M. Kierzynka, P. Manneback, and K. Kurowski, "Real-time motion tracking using optical flow on multiple GPUs," Bull. Pol. Ac.:Tech., vol. 62, no. 1, pp. 139-150, 2014. [9] C. Deng, J. Liu, W. Tian, S. Wang, H. Zhu, and S. Zhang, "Image super-resolution reconstruction based on L1/2 sparsity," Bulletin of Electrical Engineering and Informatics, vol. 3, no. 3, pp. 155-160, September 2014 [10] D. Kesrarat, K. Thakulsukanant, and V. Patanavijit, "A novel elementary spatial expanding scheme form on SISR method with modifying Geman&McClure function," TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 17, no. 5, pp. 2554-2560, Oct 2019. [11] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," In Proceeding of Defense Advanced Research Projects Agency (DARPA) Image Understanding Workshop, pp. 121- 130, 1981. [12] D. J. Fleet and A. D. Jepson, "Computation of component image velocity from local phase information," Computer Vision, vol. 5, no. 1, pp.77-104, 1990. [13] T. R. Reed (editor), "Digital image sequence processing, compression, and analysis," Chemical Rubber Company (CRC) Press, 2005. [14] B. K. P. Horn and B.G. Schunck, "Determining optical flow," Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203, 1981. [15] Y. Su, Y. Huang, and C-C. J. Kuo, "Efficient text classification using tree-structured multi-linear principal component analysis," IEEE 24th International Conference on Pattern Recognition (ICPR), pp. 585-590, 2018. [16] Y. Su, R. Lin, and C. C. J. Kuo "Tree-structured multi-stage principal component analysis (TMPCA): Theory and applications," Expert Systems with Applications, vol. 118, pp. 355-364, 2019. [17] A. Khmag, S. Ghoul S. A. R. Al-Haddad, and N. Kamarudin, "Noise level estimation for digital images using local statistics and its applications to noise removal," TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 16, no. 2, pp. 915-924, April 2018. [18] L. Abderrahim, M. Salama, and D. Abdelbaki, “Novel design of a fractional wavelet and its application to image denoising,” Bulletin of Electrical Engineering and Informatics, vol. 9, no. 1, pp. 129-140, February 2020 [19] D. Kesrarat. and V. Patanavijit, "Experimental analysis of performance comparison on both linear filter and bidirectional confidential technique for spatial domain optical flow algorithm," The Electrical Engineering/Electronics, Computer, Communications and Information Technology Association (ECTI) Transactions on Computer and Information Technology, vol. 7, no. 2, pp. 157-168, 2013. [20] R. Li and S. Yu, "Confidence based optical flow algorithm for high reliability," In Proceedings of ICASSP, pp.785- 788, 2008. [21] D. Kesrarat. and V. Patanavijit, "Robust global based spatial correlation optical flow in bidirectional confidential technique with bilateral filter," In Proceedings of IEEE Explore of the International Conference on Intelligent Informatics and BioMedical Sciences (ICIIBMS 2015), pp. 1-6, 2015. [22] J. L. Barron, D. J. Fleet, and S. S. Beauchemin, "Performance of optical flow techniques," International Journal of Computer Vision, vol. 12, no. 1, pp. 43-77, 1994. [23] D. Fleet and Y. Weiss, "Optical flow estimation", Handbook of Mathematical Models in Computer Vision, Springer, pp. 237-257, 2006. [24] S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, "Bilateral filtering: Theory and applications," Found. Trends Compt. Graph. Vis., vol. 4, no. 1, pp.1-73, 2008. [25] M. G. Mozerrov, "Constrained optical flow estimation as a matching problem," IEEE Trans. Image Processing, vol. 22, no. 5, pp. 2044-2055, 2013. [26] J. Sun, Y. Li, S. B. Kang, and H. Y. Shum, "Symmetric stereo matching for occlusion handling," In proceeding of IEEE Conf. Comput. Vis.Pattern Recognit., vol. 2, pp. 399-406, 2005. [27] M. Zhang and B. Gunturk, "A new image denoising method based on the bilateral filter," IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2008, pp. 929-932, 2008.
- 8. Bulletin of Electr Eng & Inf ISSN: 2302-9285 Experimental analysis of non-Gaussian noise resistance on global method optical… (Darun Kesrarat) 723 BIOGRAPHIES OF AUTHORS Darun Kesrarat received the B.S., M.S., and Ph.D. from the Department of Information Technology at Assumption University, Bangkok, Thailand. He is currently an Assistance Professor. His research areas include signal processing on Image/Video Reconstruction, SRR (Super-Resolution Reconstruction), Motion Estimation, and Optical Flow Estimation. Vorapoj Patanavijit received the B.Eng., M. Eng, and Ph.D. degrees from the Department of Electrical Engineering at the Chulalongkorn University, Bangkok, Thailand, in 1994, 1997 and 2007 respectively. He is currently an Associate Professor. He works in the field of signal processing and multidimensional signal processing, specializing, in particular, on Image/Video Reconstruction, SRR (Super-Resolution Reconstruction), Compressive Sensing, Enhancement, Fusion, Digital Filtering, Denoising, Inverse Problems, Motion Estimation, Optical Flow Estimation and Registration.