Improved echocardiography segmentation using active shape model and optical flow

TELKOMNIKA, Vol.17, No.2, April 2019, pp.809~818
ISSN: 1693-6930, accredited First Grade by Kemenristekdikti, Decree No: 21/E/KPT/2018
DOI: 10.12928/TELKOMNIKA.v17i2.11821  809
Received July 10, 2018; Revised October 23, 2018; Accepted November 12, 2018
Improved echocardiography segmentation using active
shape model and optical flow
Riyanto Sigit*
1
, Calvin Alfa Roji
2
, Tri Harsono
3
, Son Kuswadi
4
Politeknik Elektronika Negeri Surabaya, Indonesia
*Corresponding author, e-mail: riyanto@pens.ac.id
1
, calvin@ce.student.pens.ac.id
2
, trison@pens.ac.id
3
,
sonk@pens.ac.id
4
Abstract
Heart disease is one of the most dangerous diseases that threaten human life. The doctor uses
echocardiography to analyze heart disease. The result of echocardiography test is a video that shows the
movement of the heart rate. The result of echocardiography test indicates whether the patient’s heart is
normal or not by identifying a heart cavity area. Commonly it is determined by a doctor based on his own
accuracy and experience. Therefore, many methods to do heart segmentation is appearing. But, the
methods are a bit slow and less precise. Thus, a system that can help the doctor to analyze it better is
needed. This research will develop a system that can analyze the heart rate-motion and automatically
measure heart cavity area better than the existing method. This paper proposes an improved system for
cardiac segmentation using median high boost filter to increase image quality, followed by the use of an
active shape model and optical flow. The segmentation of the heart rate-motion and auto measurement of
the heart cavity area is expected to help the doctor to analyze the condition of the patient with better
accuracy. Experimental result validated our approach.
Keywords: active shape model, echocardiography, median high boost filter, optical flow
Copyright © 2019 Universitas Ahmad Dahlan. All rights reserved.
1. Introduction
Based on data from the World Health Organization (WHO), heart disease is one of the
most dangerous diseases in the world. From 1990 to 2017 WHO placed heart disease as the
number one killer in the world this data from WHO in 2017. According to WHO records in 2017,
deaths from heart disease and blood vessels are expected to rise to 20 million and by 2030 will
rise again to 23.6 million people. Echocardiography video is used by doctors to analyze cardiac
performance of patients. Echocardiography uses techniques by emitting ultrasound waves to
determine the size, shape, function of the heart and the condition of the heart valves visualized
into the video image. The results of the examination on echocardiography tests can only be
read by the doctor so that the analysis of the state of the heart depends on the accuracy and
experience of the doctor so that will cause a different analysis of each doctor in determining the
true state of the heart.
In previous research [1] to determine the cardiac performance of echocardiographic
video results have been implemented using several methods. Among them is to use
segmentation of triangle equations. The triangle method has the fastest segmentation of 108
ms, Sigit [1]. Another research is tracking by using Optical Flow method, error generated wide
heart cavity between testers with system about 0-7%, Rochmawati [2]. And using optical flow
farneback yields an error value of 0.13-9.85% for the left ventricle, 2.22-10% for the aorta, and
1.98-20.79% for the left atrium Baiq [3]. The segmentation using Active Shape Model. Then in
this paper using Partial Monte Carlo to calculate the area. This system has a 4.309% errors,
Shaleh [4]. This methodology has present that optical flow can improvement ejection fraction
measurement which focus on reconstruct left ventricle’s semi-automatically, Arvina [5].
Algorithm of ASM can be maximized using dual active shape model in the Bookstein
coordinates, El-Rewaidy [6]. In this paper, the algorithm of ASM using to search and tackle
segmentation of tissues with nearby organs of similar intensities, Esfandiarkhani [7]. This paper
proposes a robust approach based on the Active Shape Model (ASM) that is able to solve. The
difficulty inherent to the LV segmentation in MRI is that the images contain outliers (i.e.,
observations not belonging to the LV border) due to the presence of other structures,

 ISSN: 1693-6930
TELKOMNIKA Vol. 17, No. 2, April 2019: 809-818
810
Santiago [8]. The purpose of this paper is a better algorithm of ASM because the process of
fitting on the ASM, the results obtained are still less well, Santiago [9].
In this paper proposes a combination of ASM and PDAF to generate a stronger
segmentation of the parameter approach, El-Rewaidy [10]. In another study segmentation of the
heart cavity using another method, the watershed and showed better results than the active
contour [11]. In this research of cardiac cavity in two-dimensional short-axis, echocardiography
image with the method uses boundary and triangle equation to detect and reconstruct the
border [12]. In this paper, a new formulation of a Robust Active Shape Model (RASM) is present
to 3D segmentation of the left ventricle (LV) in cardiac MRI [13]. In this paper uses combine a
random forest classifier with an active shape model (ASM) to present a model-based learning
segmentation algorithm to detect the left ventricle (LV) boundary of the heart from ultrasound
(US) images [14]. In this paper presents a semi-automatic approach in this study that is capable
of identifying the endocardial borders robustly from cine magnetic resonance images using
deformation flow and optical flow [15]. This paper reports a quantification of cardiac motion in
Short-Axis (SAX) MRI images using the optical flow method [16]. In this paper presents
estimates the motion of cardiac ultrasound imaging and compare with several classical optical
flow estimation algorithms [17, 18]. In this paper presents a mathematical model that simulates
the real cardiac motion during myocardial tagging for evaluating Left Ventricular contractility for
Cardiac Magnetic Resonance Imaging (MRI) sequence [19].
In this paper presents a framework using the Active Shape Model (ASM) and Optical
Flow for real-time facial landmarks extraction and tracking [20]. In this paper presents a new
registration method between ultrasound (US) image and Magnetic resonance imaging (MRI),
using optical flow model for large deformation between US image (floating image) and MRI
(reference image), and presents a novel brightness-based algorithm for computation of optical
flow [21, 22]. in this paper presents Automatic segmentation of cardiac cavity images algorithm
to detect and reconstruct the boundary of the cardiac cavity using Collinear and Triangle
Equation [23]. In this paper presents a problem of accurate segmentation of the endocardial
contours of both ventricles and the epicardium of the entire heart using Optimal Features Active
Shape Model (OF-ASM) [24]. In this paper presents a segmentation the human left ventricle
(LV) in ultrasonic images, using Active Shape Models (ASM) and Active Contour Models
(ACM) [25]. Therefore, the new solutions needed to analyze the heart's performance quickly and
accurately. In this study, we will create a system to process the heart image in
echocardiography video to detect the movement of the heart cavity and the extent of the heart
cavity with active shape model and optical flow from input video echocardiography parasternal
short axis.
2. Research Method
In this part, we explain about the procedure to develop segmentation of cardiac image
for heart disease. The procedure which involves Median High Boost Filter, Active Shape Model
and Optical Flow are graphically described in a system diagram is shown Figure 1.
Figure 1. System diagram
Input Video Echocardiography
Median High Boost Filter
Active Shape Model
Optical Flow
Cardiac Cavity Area

TELKOMNIKA ISSN: 1693-6930 
Improved echocardiography segmentation using active shape model... (Riyanto Sigit)
811
2.1. Input Video Echocardiography
First, the input in this research are Videos from the heart rate movement from
echocardiography test (ultrasound cardiac). In this research, we have used the short axis videos
to test out our proposed segmentation technique. The example of the image comprising in short
axis view from the left ventricular cardiac cavity from echocardiography that represented shown
in a picture Input Video Echocardiography is shown Figure 2.
Figure 2. Input video echocardiography
2.2. Median High Boost Filter
The second step is applying Median High Boost filter to eliminate noise. The new
method which is a combination between median filtering and High Boost Filter called a Median
High Boost Filter that represented shown in a Median High Boost Filter diagram is shown
Figure 3, Sigit [1].
Source Median Filter High Boost Filter Output
Figure 3. Median High Boost Filter
In the implementation of the high boost filter algorithm, Eva [2], which uses the spatial
mask that represented is shown Figure 4 Kernel used for the High Boost Filter. To improve the
high-frequency components without affecting low-frequency components using high boost filter.
Figure 4. Kernel used for the High Boost Filter
2.3. Active Shape Model
After Median High Boost Filter, the next step is doing segmentation. The segmentation
proses is used to analyzed the heart rate motion and helped the system to measure the heart
cavity area. Active Shape Model has two steps, these are training and fitting, Shaleh [4].

 ISSN: 1693-6930
812
2.3.1. Training
The data extraction phase will produce information from the input data provided in
accordance with the learning outcomes of training stages is the meaning of Training, Shaleh [4].
Training has some steps that represented shown in a diagram Step of Trainin Active Shape
Model is shown Figure 5.
Landmark Point
Alignment Shape
Mean
Standard
Deviation
Eigenvalue and
Eigenvector
Figure 5. Step of Training Active Shape Model
2.3.1.1. Landmark Point
This step used for representation the shape into point. That point is called a “landmark
point” and after we make a point we save the data in .txt file. The data is coordinate from the
frame videos.
𝑋̃ = (𝑥1, 𝑦1, … … , 𝑥 𝑛, 𝑦 𝑛) (1)
The result of landmark point from twenty-five point to make a shape in the next step that
represented shown in picture Landmark Point is shown Figure 6.
Figure 6. Landmark Point
2.3.1.2. Aligment Shape
Landmark point differences in positions too far so do the alignment to the first form by
doing rotating and scaling. The formula can be represented as (2):
(2)

813
from the formula. Can be represented the step for the alignment shape in a diagram Alignment
Shape is shown Figure 7.
Input vector data is broken down into x & y points
Input data from a.txt file
Rotate angle at a predertemined value with the first video as a reference
Changing the result of the rotation into the return vector data
Show the shape into the polyline
Figure 7. Alignment Shape
2.3.1.3. Mean
In this step, calculate the mean shape of all the alignment form data by using the
formula below (2):
(3)
2.3.1.4. Standart Deviation
Standard Deviation is getting from the reduction of the alignment shape data with the
mean value. The result of the standard deviation is used to an input of the covariance matrix.
The formula can be represented as (4):
𝑑𝑥𝑖 = 𝑥𝑖 − 𝑥̅ (4)
After calculating Standard Deviation the next step is Covariance. A measure of how
much two different sets of data called covariance. To determine the level of related variables
varies together or how they may use the covariance. The formula can be represented as (5):
(5)
2.3.1.5. Eigenvalue and Eigenvector
To protect the data which have the largest variation corresponds to a linear
transformation of the high-dimensional space to the low dimensional space using the
eigenvector. The section representatives of eigenvectors eigenvalues are. The formula can be
represented as (6):
𝑆𝑝 𝑘 = 𝜆 𝑘 𝑝 𝑘, 𝜆 𝑘 ≥ 𝜆 𝑘+1 (6)
2.3.2. Fitting
In this part, to fit the mean shape result according to the new image using the fittings.
The first t-mode generates a form in training sets can be approximated by using a weighted
summation means and forms of irregularities, Shaleh [4]. The Fitting formula can be
represented by (7):
𝑥 = 𝑥̅ + 𝑃𝑏 (7)
With P=(p1, p2, ..., pt) as the first matrix eigenvectors of t and b=(b1, b1, ..., bt) as a weight
vector to initialize the parameter form b (eigenvector) to zero (meaning form) [4].
The result of fitting that represented shown in picture Fitting is shown Figure 8.

 ISSN: 1693-6930
814
Figure 8. Fitting
2.4. Optical Flow
To predict the movement of the image based on its intensity in the image sequence is
the optical flow. This will represent how far pixel motion pictures between 2 frames if applied to
a 2D image. One of the methods in computer vision is used for estimating from the optical flow
is the Lucas-Kanade. By combining information from several nearby pixels, Lucas Kanade's
method can resolve the ambiguity, Arvina [5].
The highest layer of the pyramid is the first undertaken by a Pyramidal Lucas-Kanade.
A reference starting point is the result of the Lucas-Kanade who used to work at the lower layer
and continues until the lowest level. Pyramidal Lucas-Kanade algorithm can be described as
follows in accordance with the working paper presented by the Bouquet in 2000 and this is the
steps are taken to find the position of the point on the next frame.
a. Initialization pyramid into I & J
{𝐼 𝐿} 𝐿 = 0, … … , 𝐿𝑚 𝑑𝑎𝑛 {𝐽 𝐿} 𝐿 = 0, … … . , 𝐿𝑚
b. Initialization first value of pyramid
c. Define point of u into I
L
frame
d. Define derivative I
L
into x value
e. Define derivative I
L
into y value
f. Calculate the spatial gradient matrix

815
g. initialization iteration of L-K (Lucas Kanade)
𝑣
̅0̅
= [0 0]
h. Search value of the difference image
i. Search value of the image mismatch vector
j. Calculate the optical flow value
𝑛 𝑘̅̅̅ = = 𝐺−1
𝑏 𝑘
̅̅̅
k. initialization vector to the next step
𝑣 𝑘̅̅̅ = 𝑣 𝑘−1̅̅̅̅̅̅ + 𝑛 𝑘̅̅̅
l. Calculate the optical flow value to L level
𝑑𝐿 = 𝒗 𝒌̅̅̅
m. Initialization value of pyramid to next level
n. Calculate the end of optical flow value
𝑑 = 𝑔 𝑜 + 𝑑 𝑜
o. Search coordinate point to J
𝑣 = 𝑢 + 𝑑
2.5. Cardiac Cavity Area
The last step in this research is Monte Carlo to calculate the heart cavity area.
To calculate the area of the heart cavity using Monte Carlo with how to generate a random
sample and then calculate the ratio of the sample between samples in the area of segmented
(nd) with the total sample (ns). Monte Carlo formula can be represented as:
(8)
Partial Monte Carlo is part of Monte Carlo algorithm. Partial Monte Carlo algorithm aims
to improve calculations faster than the normal Monte Carlo by way of dividing borders into some
parts and the search box on the border area, Sigit [1].
3. Results and Analysis
This section describes the final result of the system. Echocardiography video in short
axis view is used as input. The table in this part representation all about the system with some
experiment. Based on Table 1, can see that landmark point result with labeling set make a point
as much 25 points to make a shape. After the training set we do the Alignment Shape to make
some data training into one reference shape.

 ISSN: 1693-6930
816
Table 1. Landmark Point
Frame
Input Video
Video 1 Video 2
0
1
2
3
Based on Table 2, that can be seen from process calculate mean, standard deviation
etc. We get the real shape from the heart cavity area and from this shape we do the optical flow
to view the heart rate motion in video output to help the doctor analyze the heart patient.
Table 2. Result of Segmentation
Video Result
1 (a) Frame 0 (b) Frame 1
(c) Frame 2 (d) Frame 3
2
(a)
Frame 0 (b) Frame 2
(c) Frame ke-3 (d) Frame ke-4

817
Based on Figure 9 the method of ASM and Optical Flow represented the maximum
results so that when done widespread calculations using Monte Carlo method generated graph
data above that is a very small error value.
Figure 9. Chart from calculate the cavity area from the heart
4. Conclusion
The solution method is proposed to help doctors analyze heart disease is
echocardiograph segmentation of videos with the combine any methods. The Median High
Boost Filter is the result of research that is able to reduce noise and used to increase the light
intensity value of the image. The segmentation using Active Shape Model and Optical Flow. The
experimental result is able to detect heart rate-motion and heart cavity area. Methods used in
this paper is effective because it is based on the results obtained is a comparison of the
diagnosis made by the doctor is not much different from the results obtained from this system.
Reference
[1] Sigit, R Barakbah, AR Sulistijono, IA. Improved segmentation of cardiac image using triangle and
partial Monte Carlo. 2016 International Conference on Knowledge Creation and Intelligent Computing
(KCIC) IEEE. 2016.
[2] Sigit R, Rochmawati E. Segmentation echocardiography video using B-Spline and optical flow. 2016
International Conference on Knowledge Creation and Intelligent Computing (KCIC) IEEE. 2016
[3] R Sigit, T Harsono, BHA.Noor. Heart video tracking system on long axis view. 2016 International
Electronics Symposium (IES) IEEE. 2016.
[4] R Sigit, Setiawardhana SN Shaleh. Improved segmentation of cardiac image using active shape
model. 2017 International Seminar on Application for Technology of Information and Communication
(iSemantic) IEEE. 2017.
[5] Arvina AP, Riyanto S, Dwi KB. Improved Ejection Fraction Measurement on Cardiac Image Using
Optical Flow. 2017 International Electronics Symposium on Knowledge Creation and Intelligent
Computing (IES-KCIC) IEEE. 2017.
[6] H El-Rewaidy, AS Fahmy. Segmentation of the Right Ventricle in MR images using dual active shape
model in the Bookstein coordinates. 2015 IEEE 12th International Symposium on Biomedical Imaging
(ISBI). 2015.
[7] M Esfandiarkhani, M Delavari, AH Foruzan, YW Chen. Improving active shape models performance
in low-contrast images using a KNN-based search algorithm - with applications in liver segmentation.
22nd Iranian Conference on Biomedical Engineering (ICBME) IEEE. 2015.
[8] C Santiago, JC Nascimento, JS Marques. Segmentation of the left ventricle in cardiac MRI using a
probabilistic data association active shape model. 2015 37th Annual International Conference of the
IEEE Engineering in Medicine and Biology Society (EMBC). 2015.
[9] C Santiago, JC Nascimento, JS Marques. A robust active shape model using an
expectation-maximization framework. 2014 IEEE International Conference on Image Processing
(ICIP). 2014.
[10] H El-Rewaidy, ES Ibrahim, AS Fahmy. Segmentation of the right ventricle in MRI images using a dual
active shape model. in IET Image Processing. 2016; 10(10): 717723.
0
5000
10000
1 2 3 4 5 6 7 8 9 10
Manual Monte Carlo

 ISSN: 1693-6930
818
[11] J Nonthaporn Nakphu, Dyah Ekashanti Octorina Dewi. Apical Four-Chamber Echocardiography
Segmentation using Marker-controlled Watershed Segmentation. IEEE Conference on Biomedical
Engineering and Sciences, Miri, Sarawak, Malaysia. 2014.
[12] Riyanto Sigit, Mohd. Marzuki Mustafa, Aini Hussain, Oteh Maskon, Ika Faizura Mohd. Noh. Automatic
Border Detection of Cardiac Cavity Images Using Boundary and Triangle Equation. TENCON IEEE.
2009.
[13] Carlos Santiago, Jacinto C. Nascimento, Jorge S. Marques. A New Robust Active Shape Model
Formulation for Cardiac MRI Segmentation. ICIP IEEE. 2016.
[14] Gregg Belous, Andrew Busch, David Rowlands. Segmentation of the Left Ventricle from Ultrasound
Using Random Forest with Active Shape Model. First International Conference on Artificial
Intelligence, Modelling & Simulation IEEE. 2013.
[15] Zhen Zhou Wang. Segmentation of the left ventricle in short-axis sequences by combining
deformation flow and optical flow. IET Image Process. 2017.
[16] R Medina, E Ibarra, V Morocho, P Vanegas. Cardiac Motion Estimation in Magnetic Resonance
Images Using Optical Flow. LATIN AMERICA TRANSACTIONS, IEEE. 2016.
[17] Emiro Ibarra, Rub´en Medina, Villie Morocho, and Pablo Vanegas. Optical Flow as a Tool for Cardiac
Motion Estimation. Asia-Pacific Conference on Computer Aided System Engineering, IEEE. 2015
[18] Qishan Luo, and Dong C. Liu. Optical Flow Computation Based Medical Motion Estimation of Cardiac
Ultrasound Imaging, IEEE. 2011.
[19] Azza S, Hassanein, Ayman M, Khalifa, Walid Al-Atabany, Mohamed T, El-Wakad. Performance of
Optical Flow Tracking Approaches for Cardiac Motion Analysis. Middle East Conference on
Biomedical Engineering (MECBME) IEEE. 2014.
[20] Byungtae Ahn, Yudeog Han, In So Kweon. Real-time facial landmarks tracking using active shape
model and LK optical flow. International Conference on Ubiquitous Robots and Ambient Intelligence
(URAI) Daejeon, Korea, IEEE. 2012.
[21] Mo Hou, Xiaolin Wang, Maoyun Yang, Xiaobin Li, Biyu Li, Yuying Han. Research on optical flow
model based large deformation ultrasound-MRI registration. International Conference on Intelligent
Human-Machine Systems and Cybernetics IEEE. 2016.
[22] Sven Loncaxic, Zoran Majcenic. Optical Flow Algorithm for Cardiac Motion Estimation. Annual EMBS
International Conference, IEEE. 2000.
[23] Riyanto Sigit, Mohd. Marzuki Mustafa, Aini Hussain, Oteh Maskon, Ika Faizura Mohd Noh. Automatic
Segmentation of Cardiac Cavity Images Using Collinear and Triangle Equation. Technological
Developments in Networking, Education and Automation. 2010.
[24] S Ordas, L Boisrobert, M Huguet, AF Frangi. Active Shape Models with Invariant Optimal Features
(IOF-ASM) Application to Cardiac MRI Segmentation. IEEE. 2003.
[25] G Hamameh, T Gustavsson. Combining Snakes and Active Shape Models for Segmenting the
Human Left Ventricle in Echocardiographic Images. Computer in Cardiology. 2000.

Improved echocardiography segmentation using active shape model and optical flow

More Related Content

What's hot (16)

Similar to Improved echocardiography segmentation using active shape model and optical flow (20)

More from TELKOMNIKA JOURNAL (20)

Recently uploaded (20)

Improved echocardiography segmentation using active shape model and optical flow