SlideShare a Scribd company logo
International Refereed Journal of Engineering and Science (IRJES)
ISSN (Online) 2319-183X, (Print) 2319-1821
Volume 6, Issue 7 (July 2017), PP.50-58
www.irjes.com 50 | Page
Evaluation of Thresholding Based Noncontact Respiration Rate
Monitoring using Thermal Imaging.
*
Abdulkadir Hamidu Alkali
Department Of Computer Engineering.University Of Maiduguri, Borno State, Nigeria.
Corresponding author: *Abdulkadir Hamidu Alkali
Abstract: - A noncontact method for respiration rate monitoring using thermal imaging was developed and
evaluated. Algorithms to capture images, detect the location of the face, locate the corners of the eyes from the
detected face and thereafter locate the tip of the nose in each image were developed. The amount of emitted
infrared radiation was then determined from the detected tip of the nose. Signal processing techniques were then
utilised to obtain the respiration rate in real-time. The method was evaluated on 6 enrolled subjects after
obtaining all ethical approvals. The evaluations were conducted against two existing contact based methods;
thoracic and abdominal bands. Results showed a correlation coefficient of 0.9974 to 0.9999 depending on the
location of the ROI relative to the detected tip of the nose. The main contributions of the work was the
successful development and evaluation of the facial features tracking algorithms in thermal imagining, the
evaluation of thermal imaging as a technology for respiration monitoring in a hospital environment against
existing respiration monitoring systems as well as the real time nature of the method where the frame processing
time was 40 ms from capture to respiration feature plotting.
Keywords: - Image processing, Noncontact, Real-time, Respiration monitoring, Thermal imaging,
Thresholding.
INTRODUCTION
Oxygen is a naturally occurring element in the atmosphere which the body uses as part of a process to
obtain energy. Air-rich oxygen enters the lungs through inspiration from where oxygen goes into the blood
stream via the capillaries and is transported all over the body. During expiration de-oxygenated air containing
the produced carbon dioxide is returned to the atmosphere.
The average number of breathes taken over a predefined period of time is respiration rate. Its value
depends on several factors that include the person‟s age, gender and medical condition. It ranges between 12 to
60 cycles per minute in healthy adults and children. During respiration, a number of physiological effects take
place that are the bases for respiration monitoring. These include variations in air flow as air is inhaled and
exhaled, chest/abdomen movements, blood oxygen saturation variations, temperature changes in inhaled and
exhaled air and variations in the amount of emitted infrared radiation from the skin surface centred on the nose
and mouth.
An uneven respiration rate is a good indicator of imbalance of the normal body condition [1] that
studies [2-5] have shown to be a predictor of potential serious clinical events. Early detection and proactive
intervention of the predicted condition can avert a potential serious harm [6,7]. Respiration rate therefore, needs
to be accurately monitored to establish correct physiological operations in order to take the necessary clinical
steps. Existing gold standard devices for monitoring respiratory rate make contact with the subject‟s body
during monitoring and are therefore, referred to as contact based. They mostly require the subject to be
constrained and thus disturb the sleep position which affects normal breathing. Since a natural sleep position
does not alter breathing, it is therefore, critical that respiration monitoring does not alter it as well. In noncontact
respiration monitoring, the device is not attached to the subject‟s body and therefore, a more reliable medical
assessment can be obtained. Studies have reported that noncontact based methods for monitoring respiratory rate
are better than contact based methods [8-11]. They attributed this to improved patient‟s comfort, improved
accuracy due to elimination of distress as well as non-possibility of disruption of monitoring as a result of
dislodgement of the sensing device.
In this study, technique to facilitate noncontact respiration rate monitoring has been developed and its
effectiveness was evaluated. The technique was based on measuring detecting the amount of emitted infrared
radiation from the skin surface centred on the nose (thermal imaging method). This is based on a fact that during
respiration, exhaled air is warmer than inhaled air. The air temperature variation during respiration results in
temperature variation along the air flow path especially the area around the tip of the nose and upper lip regions
thereby resulting in corresponding variation in the amount of emitted infrared radiation from those areas. These
continuous variations in infrared emissions around the affected regions can be observed and processed by
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 51 | Page
thermal imaging (thermography). A respiration signal can then be produced from the resulting images.
Respiration rate is then obtained by processing the resulting signals.
Methodology
1.1 Camera Setup
The emissivity of the thermal camera was set to 0.98 as required for skin temperature measurement.
The computer ran on Windows 7 operating system with a memory of 2.88 GB having an Intel Core processor
that ran at 2.6 GHz. The camera was controlled using software that was developed using the National
Instrument's LabVIEW environment. The camera's field of view (FOV) lens was 24 degrees and it was setup
such that the thermal image 𝑓(𝑥, 𝑦) acquired from it consisted mostly of the subject's head and shoulders.
Ethical approvals were obtained at Sheffield Hallam University, the Sheffield Children's Hospital and the
National Health Service (NHS), United Kingdom.
1.2 Experimental Setup
The camera was mounted on a tripod below the subject's upper body, at about 1 m from the subject
who sat comfortably on a chair in a research laboratory. The camera was inclined at about 20o
to focus upward
towards the subject's face such that the subject's upper body was contained in the camera's field of view. This
was to ensure that a good profile of the area most affected by respiration was adequately obtained. A typical
setup is shown in Error! Reference source not found..
Thoracic and abdominal piezoelectric bands were used to record respiratory signal from the subjects.
The mean temperature and relative humidity of the room were 24.4°C (standard deviation of 0.35) and 45%
(standard deviation of 0.71) respectively.
The frame capture rate per second was 20 in which allowed real time processing of the data and
provided a sufficient sample rate. Each subject was monitored for at least 5 minutes but typically more than 10
minutes. The camera was controlled using software that was developed using the National Instrument's
LabVIEW environment. The flow chart of the algorithm involved in this method is shown in Figure 0-1.
Acquire Image
Filter & Segment
Create ROI & extract
Respiration feature
Start
End
Reduce Image Resolution
Monitoring
terminated?
YES
NO
Locate tip of nose
Iterative
Elimination
Algorithm
Obtain Horizontal & Vertical
profiles of image
Detect face indicate tracking
Obtain maximum pixel
value on the face Pm
Threshold image
using (Pm-10) as
lower threshold
Iterative
Elimination
Algorithm
Locate eyes’ corners
Figure 0-1: Flow chart of feature detection based on thresholding algorithm
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 52 | Page
1.3 Face Detection
A facial region affected by respiration needed to be identified such that respiration induced temperature
changes could be reliably detected. The best feature that suited this purpose was the nose but due to varying
temperature of the skin area of the nose as a result of respiration posed consistency issues for the nose to be
directly located with accuracy in thermal imaging. In order avoid this inconsistency and detect the nose in
thermal image, a secondary feature needed to be located that would act as a reference to the nose. This would
also eliminate user interaction at any point during the recording and therefore eliminate possible errors that
could be introduced as a result. The two corners of the eyes met the requirement of being a reference in
detecting the nose for two reasons. Firstly, the corners of the eyes have the highest temperature on the face and
secondly the horn of the nose was always between the two corners of the eyes. The acquired image was
therefore segmented to remove objects with lower temperatures to that of the skin. Unwanted objects with
similar temperatures to that of the skin that could have been missed were then eliminated using an iterative
elimination algorithm developed and described in [12, 13]. Thereafter the face was detected and enclosed in a
rectangle to indicate tracking also described in [12]. The coordinates of the rectangle enclosing the face were
saved as:
rightBottomyx
leftBottomyx
rightTopyx
leftTopyx
br
bl
tr
tl




,
,
,
,
(0-1)
1.4 Eyes' Corners Detection
In [14] we detected the eyes‟ corners in a slightly different manner which came with some errors right
from detecting the face as well as with the eyes‟ corners detection. These observations were improved upon in
this evaluation by addition of equations 2-4 and 2-5 after taking the anthropometry of the face into consideration,
as will be explained shortly. Since the corners of the eyes had the highest temperature over the face, they also
had the highest pixel values by translation. To detect the corners of the eyes, the search for the maximum pixel
values was carried out within the bounding rectangle. For the first acquisition, the subject was required to face
the camera so that both eyes' corners could be detected. The reason for this would be explained later in the text.
Starting from 𝑥𝑙, 𝑦𝑡 the pixel value (i.e. the first pixel) was recorded as the maximum pixel. Its value and
coordinates were stored as 𝑃𝑚𝑎𝑥 and 𝑥 𝑝, 𝑦𝑝 respectively. The next pixel 𝑥𝑙+1, 𝑦𝑡 is looked at and compared with
the stored maximum, if it is higher, 𝑃𝑚𝑎𝑥 and 𝑥 𝑝, 𝑦𝑝 are replaced. This is repeated for each pixel value within the
bounded face. Upon reaching 𝑥 𝑟, 𝑦 𝑏 (i.e. the last bottom most pixel value within the bounded facial image), the
maximum pixel value 𝑃𝑚𝑎𝑥 and its corresponding coordinates 𝑥 𝑝, 𝑦𝑝 gives an indication of the corner of one eye.
One corner is always located first as the two eye corners do not have the same temperature, thus, the
highest pixel value on the other eye corner would be different from that already detected. However, there is the
likelihood that the next lower pixel value than 𝑃𝑚𝑎𝑥 was located next to 𝑃𝑚𝑎𝑥 and therefore still belong to the
detected eye corner. If that was the case, then, locating two maximum values would not yield a pointer to the
location of the two eyes‟ corners. To deal with these problems while avoiding complex and computational
intensive algorithms, the face in image was segmented using the maximum pixel value 𝑃𝑚𝑎𝑥 . The segmentation
was such that the eye corners would be returned as blobs with a maximum length (radius) of 10 pixels.
Thresholding was used to achieve the segmentation using:
 


 
 LTHgif1
Otherwiseyxg 0, (0-2)
Where 10max  PLTH
This was a valid assumption as the corner of the second eye cannot be 10 pixel values less than the
detected corner of the first eye. After the segmentation, all regions with pixel values greater than 𝑇𝐻𝐿constituted
the new image. These included regions associated with both eyes' corners as well as the side on the nose and in
some cases the mouth region. When only the two regions associated with the corners of the eyes were present,
the process of identifying the two corners was straight forward. It was however noted that some of the anomalies
above were present, between or besides the location of the eyes' corners.
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 53 | Page
Only the upper half of the facial image was considered in the search for the corners of the eyes. This
was because the eyes are located between the vertex and the upper half of the face. This decision was not only
valid but also reduced the computation time by about half the normal time if the whole face was considered.
The segmented image was then dilated to connect regions that were close enough to the blobs. Holes
were then filled and then followed by eroding the image to return it to the original size before the dilation. The
numbers of particles in the image were counted and the maximum horizontal and vertical coordinates occupied
by each blob was returned. The coordinate of each particle contained in a blob was used to obtain the
approximate coordinates of the centre of gravity of each blob using:








Ni
i iy
N
gy
Ni
i ix
N
gx
1
1
1
1
(0-3)
Where 𝑥 𝑔, 𝑦𝑔 represent the average positions of the central points of the horizontal and vertical sections
in a blob and 𝑥𝑖, 𝑦𝑖 are the horizontal and vertical coordinate of a pixel in reference.
The blob containing the maximum pixel value 𝑃𝑚𝑎𝑥 was identified in reference to the distances
between its coordinates 𝑥 𝑝, 𝑦𝑝 and the centre of mass of each blob. The blob that had the least distance to 𝑃𝑚𝑎𝑥
was identified as the first eye corner blob and was used to locate the second eye corner. The blob representing
the second eye corner was identified using the following steps:
A blob was considered and its maximum vertical pixel length (considered as the diameter) was
obtained and then divided by 2 to get the radius, r.
The y-coordinate 𝑦𝑔 of the centre of gravity of the detected first eye 𝑒1 corner was then used to check
whether the blob in reference (𝑒 𝑥) met the criteria:
       regyxegyregy  11 (0-4)
If the blob met the criteria, then it was identified as the corner of the second eye and the coordinates of
its centre of gravity 𝑥 𝑔, 𝑦𝑔 𝑒2 were stored, otherwise the next blob (if any) was considered. The anthropometry
of the face gave an indication of the possible separation between the inner corners of the two eyes (i.e. the
intercanthal width) B as one fifth of the head breadth H [15, 16]. Therefore the horizontal separation between
the x-coordinate 𝑥 𝑔 of the centre of gravity of the detected eye corner 𝑒1 and the blob in reference (𝑒 𝑥) must
also meet the criteria:
56
lxrx
B
lxrx 


(0-5)
The limit in equation 2-5 was introduced as a lower boundary to avoid detecting false eye corner.
When there was about 45 degree sideways head movement in either direction from when the face was looking
straight into the camera, the two eyes' corners could still be detected. When there was a considerable sideways
head movement however, only one eye corner was visible due to occlusion. Obtaining just one corner of an eye
was not enough to identify where the nose was located using this method. To deal with this situation where one
corner was not visible, the pixel distance between the two eyes' corners were calculated during the first iteration.
This was the reason for the requirement of the subject to face the camera at the beginning of acquisition, as
mentioned. The coordinates of both eyes' corners in each image were also saved and were replaced as new
locations were obtained. The distance between the two eyes' corners for each image was then compared to the
present distance and the x-coordinates 𝑥 𝑔(𝑒1) and 𝑥 𝑔(𝑒2) of both eyes' corners to those obtained (and saved)
during the previous iteration. A difference less than zero indicated that there was a movement to the left by the
subject (right in the image) and for a difference greater than zero, there was a movement to the right by the
subject (left in the image). This was so that the direction of head movement was known in order to identify
which eye corner was not in the camera's FOV.
Identifying the direction of eyes' corners movement was the reason why the subject was required to
face the camera for the first image acquisition. Obstruction of one eye corner could arise when the subject
moved their head in a sideways direction. Knowing the eye corner that was obstructed from the camera's view
allowed obtaining a coordinate location that would serve in detecting the location of the nose, just as the other
eye corner would have were it not obstructed. To obtain the appropriate coordinate when one eye corner was not
visible (due to obstruction) when the direction of the head movement was known as well as the maximum pixel
distance between the centre of gravity of the two eyes' corners, the rectangle that contained the face was used.
When there was a movement towards the left of the image that resulted in obstruction of the second eye corner
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 54 | Page
(i.e. the left eye corner was obstructed), then the coordinate of 𝑥𝑙 and 𝑦𝑔 𝑒1 were used as the appropriate
coordinate in detecting the nose. If on the other hand, the right corner of the eye were obstructed, then 𝑥 𝑟 and
𝑦𝑔 𝑒1 were used. In either case, the appropriate coordinates were labelled as 𝑒2 while those for the visible eye
corner detected labelled as 𝑒1. This was valid since only either the right or left sides of the nose was visible
when the left or right eye corner was obstructed respectively. There was no facial image beyond the left or the
right side of the rectangle bounding the face at those moments respectively.
1.5 Nose Detection
Looking at the face, the horn of the nose which is a part of the nose was always located between the
corners of the eyes. The nostril being the opening through which inhalation and exhalation take place during
nasal breathing was the lower end of the nose. With the location of the two eyes' corners detected, all that was
required was to identify the location of the nose. To do that, the image bounded by the coordinates of the eyes'
corners (𝑥 𝑔, 𝑦𝑔(𝑒1) and 𝑥 𝑔, 𝑦𝑔(𝑒2)) and that by the y-coordinates of the lower rectangle bounding the face in
reference to the x-coordinates of the eyes' corners (𝑥 𝑔(𝑒1), 𝑦 𝑏and 𝑥 𝑔(𝑒2), 𝑦 𝑏) was considered. This area of the
face contained the nose and mouth regions. For each acquired frame, we considered the highest and lowest pixel
values 𝑁𝑝𝑚𝑎𝑥 and 𝑁𝑝𝑚𝑖𝑛 respectively. The image within the area specified was then segmented so as to remove
all areas that were not associated with the nose. To do this, the image was segmented using:
   UTHgLTHif1
Otherwise0yx,g

 (0-6)
Where  
min
minmax
2
pL
pp
U NTHand
NN
TH 


Considering the maximum and minimum pixel values for each acquired frame was in order to consider
the pixel values that were affected by respiration, i.e. during expiration, the infrared radiated around the nose
increases and so does the pixel values and the reverse occurs during inspiration. This was also noted by [17]. If
constant values were considered, then the nose could not have been segmented and subsequently could not be
detected in some frames. After the segmentation, only the area representing the nose remained in the image and
a rectangle was created to enclose it to indicate detection. The tip of the nose 𝑡 𝑁 in frontal (or near frontal) view
is the mid-point between the left and right bottom sides of the region enclosing the nose. For non-frontal view,
the respective side of the facial boundary was used as the tip of the nose.
1.6 Region of Interest (ROI)
The region between the tip of the nostrils and the vertex of the lip was always affected by respiration.
The region was close to the tip of the nose. Four regions of interest (ROI) were created in order to get different
respiration signals from different locations that were affected by respiration. The regions were:
1. ROI1: The average pixel value of the area within the rectangle bounding the detected nose area.
2. ROI2: A circle with radius of 5 pixels located beneath the left nostril. Its centre was located on the bottom
line of the rectangle bounding the detected nose and at a quarter of the width of the nose area from the left.
3. ROI3: A circle with radius of 5 pixels located beneath the right nostril. Its centre was located on the
bottom line of the rectangle bounding the detected nose and at a quarter of the width of the nose area from
the right.
4. ROI4: The non-zero pixel values within obtained in equation 3-4.
5.
Results And Discussion
After locating the corner of an eye and thresholding the image as described, the resulting image
contained areas associated with only the two corners of the eyes in most cases. Figure 0-1(b) shows the resulting
image after thresholding and Figure 0-1(c) shows the two eyes' corners after considering the upper half objects
in the facial image.
Figure 0-1: (a) Original image (b) the results of thresholding based on the temperature profile of first
eye's corner and (c) two objects (only) representing the two corners of the eyes.
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 55 | Page
In the instance indicated in Figure 0-1, locating the two corners of the eyes was by obtaining the
coordinates of the two corners. In some cases however, areas associated with the sides of the nose, the mouth
region as well as some irregular areas associated with skin marks (e.g. scars) also appeared in the image due to
the temperature similarities of the regions as shown in Figure 0-2.
Figure 0-2: (a) Original image (b) the results of thresholding based on the temperature profile of first
eye's corner and (c) more than two objects.
In these cases, locating the second eye corner was not as straight forward. This was because, there was
no exact prior knowledge indicating when more than two objects would be present as against two, the second
eye's algorithm had to be used. Objects appearing below the center of mass of the face were rejected as
explained earlier. The performance of the second eye's algorithm in detecting the eyes-corner pair is shown in
Figure 0-3, where even when more than two objects appeared in the image.
„
Figure 0-3: (a) Detection of eyes' corners in presence of more than two objects and (b) eye corners
mapped onto image.
By detecting one corner of an eye, the other eye's corner was detected by the second eye's algorithm
even when there were objects not related to the eye. The nose was then detected after obtaining the position of
the two eyes‟ corners. The effectiveness of the nose detection is shown in Figure 0-4.
Figure 0-4: Detection of the nose in different images.
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 56 | Page
The respiration signals were obtained from the four ROIs as previously described. The locations of the
ROIs are shown in Figure 0-5 for a subject.
Figure 0-5: (a) ROI1, (b) ROI2, (c) ROI3 and (d) ROI4.
ROI1 averages all the pixel values which includes parts of the corners of the eyes and is therefore
expected to have low pixel average due to the larger area and the temperature differences between the corners of
the eyes‟ and area around the nose contained within the ROI. ROI2 and ROI3 are circles (in red) of 5 pixels
radius beneath the left and right nostrils respectively and should have high amplitudes as the temperature
distribution within the ROIs are almost uniform.
The respiration signal simultaneously obtained from the 4 ROIs in one of the recordings is shown in
Figure 0-6.
Figure 0-6: Respiration signals from the four ROIs
All the respiration signals were normalized around zero in order to enable overlay comparison. ROI2
and ROI3 have similar respiration waveform characteristics which are attributed to the location of the ROIs.
ROI1 have higher amplitude that ROI2 and ROI3 as it had a larger range between its minimum and maximum
average pixel values which was derived due to inspiration-expiration temperature difference. ROI4 had the
highest amplitude as it had the largest range that was attributed to its location (i.e. the nose). During inspiration,
the nose cools especially around the post inspiration pause period and during expiration it is warmed. The area
occupied by the nose in ROI4 is considerably reduced during inspiration and the pixel values are generally small
due to temperature drop. ROI4 is also inverted as due to the effect of averaging the pixels. During inspiration the
numbers of pixels are less and the temperature range is less giving a higher average than during expiration
0 10 20 30 40 50 60
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t, sec.
Amplitude
ROI1
ROI2
ROI3
ROI4
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 57 | Page
where the temperature range is large and the numbers of pixels are also more thus giving a lower average which
the normalization ranges around the zero axes.
Figure 0-7 and Figure 0-8 show a minute-respiration signal of a subject comparing the threshold based
method obtained from ROI1 and ROI2 respectively against two ground truths (thoracic and abdominal bands)
derived.
Figure 0-7: The ROI1 derived against the ground truths respiration signal
The threshold derived respiration signal always lags behind the ground truths during inspiration while
during expiration it mostly lags and leads in few instances.
Figure 0-8: The ROI2 derived against the ground truths respiration signal
Table 0-1 provides a summary of the respiration rates from the 6 subjects included in the study
alongside that from the ground truths (i.e. the thoracic and abdominal bands). The mean age of the enrolled
subjects was 38.7 years with a standard deviation of 3.38 years.
Table 0-1: Measured respiration rates.
Subject Gender Age
Thermal Imaging Based Thoracic
Band
Based
Abdominal
Band
Based
ROI1 ROI2 ROI3 ROI4
FT1 Male 41 23.6 23.6 25.0 23.6 25.0 25.0
FT2 Male 41 18.9 18.9 18.9 18.9 19.0 19.0
FT3 Female 33 8.0 8.0 8.0 7.5 8.0 8.0
FT4 Male 36 9.6 9.6 9.6 9.6 9.7 9.7
FT5 Female 40 9.2 9.2 9.2 9.2 9.2 9.2
0 10 20 30 40 50 60
-0.4
-0.2
0
0.2
0.4
0.6
t, sec.
Amplitude
Abdominal Band Based
Thoracic Band Based
Thermal Imaging Based
0 10 20 30 40 50 60
-0.4
-0.2
0
0.2
0.4
0.6
t, sec.
Amplitude
Abdominal Band Based
Thoracic Band Based
Thermal Imaging Based
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 58 | Page
FT6 Male 41 16.9 16.9 16.9 16.9 16.9 16.9
For FT1, the ground truth RR was 25 cycles per minute in both, however, in thermal imaging threshold
based method the RR was 23.6 cycles per minute in ROI1, ROI2 and ROI4 derived respirations. This is a 1.4
cycles per minute less than the ground truth. Although ROI3 derived was the same as the ground truth, three of
the regions had less RR. FT2 and FT4 had RRs that were 0.1 cycles less that the corresponding ground truth
RRs in all the ROIs. FT5 and FT6 had the same RRs in all the ROIs as the ground truth. FT3 had the same RR
in all the ROIs as the ground truth except ROI4.
CONCLUSION
Thermal imaging based respiration rate monitoring based on threshold was developed. The method
detected the face of the subject then the two corners of the eyes and thereafter located the nose in thermal
images. Respiration features were extracted within created ROIs using a statistical measure. Respiration signals
were then obtained from succeeding features before obtaining the respiratory rate over a period of time. The
method was evaluated on 6 adult volunteers in a hospital setting as presented in the results and discussions
section.Six subjects participated in the evaluation of the respiration rate monitor based on thresholding. Four
ROIs were created around and on the detected nose region and respiration signals obtained simultaneously from
each. Correlation of the respiratory rates with the corresponding ground truths was at least 0.9977 in all the four
ROIs; ROI4 had 0.9977, ROI1 and ROI2 had 0.9984 while ROI3 had 0.9999.
ACKNOWLEDGEMENT
The author wishes to acknowledge the subjects that volunteered in the research and most especially his
research supervisory team and the sleep unit staff of the Sheffield Children Hospital.
REFERENCES
[1]. M. A. Cretikos, R. Bellomo, K. Hillman, J. Chen, S. Finfer and A. Flabouris, “Respiratory rate: the
neglected vital sign,” The Medical Journal of Australia, 188(11), 2008, pp. 657 - 659.
[2]. C. P. Subbe, R. G. Davies, E. Williams, P. Rutherford and L. Gemmell, “Effect of Introducing the
Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care
utilisation in acute medical admission.,” Anaesthesia, 58, 2003, 797-802.
[3]. Y. Nam, J. Lee and K. H. Chon, “Respiratory rate estimation from the built-in cameras of smartphones
and tablets.,” Annals of Biomedical Engineering, 42(4), 2014, 885-898.
[4]. F. Q. Al-Khalidi, “Facial tracking method for noncontact respiration rate monitoring,” Proc. 7th
International Symposium on Communication Systems Networks and Digital Signal Processing,
Newcastle upon Tyne, United Kingdom, 2010, 751-754
[5]. M. Cullinane, G. Findlay, C. Hargraves and S. Lucas, “"An acute problem?" National Confidential
Enquiry into Patient Outcome and Death.,” London, 2005.
[6]. D. Konrad, G. Jäderling, M. Bell, F. Granath, A. Ekbom and C. Martling, “Reducing in-hospital cardiac
arrests and hospital mortality by introducing a medical emergency team.,” Intensive Care Medicine ,
36(1), 2010, 100-106.
[7]. J. Hosking, J. Considine and N. Sands, “Recognising clinical deterioration in emergency department
patients.,” Australasian Emergency Nursing Journal, 17(5), 2014, 59-67.
[8]. R. Murthy, I. Pavlidis and P. Tsiamyrtzis, “Touchless monitoring of breathing function,” Proc. 26th
Annual International Conference of the IEEE Engineering in Medicine and Biology Society, California,
United States of America, 2004, 1195-1199.
[9]. F. Q. Al-Khalidi, R. Saatchi, D. Burke and H. Elphick, “Respiration rate monitoring: A review,” Pediatric
Pulmonol., 46, 2011, 523-529.
[10]. M. Bartula, T. Tigges and J. Muehlsteff, “Camera-based system for contactless monitoring of respiration,”
Proc. 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society,
Osaka, Japan, 2672-2675.
[11]. I. Smith, D. Krucheck, J. Mackay and N. Fahrid, “Respiratory rate measurement: A comparison of
methods.,” British Journal of Healthcare Assistants , 5(1), 2011, 18-23.
[12]. A. H. Alkali, R. Saatchi, H. Elphick, D. Burke, “Facial tracking in thermal images for real-time
noncontact respiration rate monitoring,” Proc. 2013 European Modelling Symposium, Manchester,
United Kingdom, 2013, 265-270.
[13]. A. H. Alkali, “Iterative Elimination Algorithm for Thermal Image Processing,” Arid Zone Journal of
Engineering, Technology and Environment, 10, 2014, 85-93.
[14]. A. H. Alkali, R. Saatchi, H. Elphick and D. Burke, “Eyes' corners detection in infrared images for real-
time noncontact respiration rate monitoring,” Proc.2014 World Congress on Computer Applications and
Information Systems, Hammamet, Tunisia, 2014, 1-5
Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.
www.irjes.com 59 | Page
[15]. J. W. Young, “Head and Face Anthropometry of Adult U.S. Citizens,” Federal Aviation Administration,
Wasington, DC, United States of America, 1993.
[16]. L. G. Farkas, M. J. Katic and C. R. Forrest, “International Anthropometric Study of Facial Morphology in
Various Ethnic Groups/Races.,” Journal of Craniofacial Surgery, 16, 2015, 615-646.
[17]. J. Fei and I. Pavlidis, “Thermistor at a Distance: Unobtrusive Measurement of Breathing.,” IEEE
Transactions on Biomedical Engineering, vol. 57, pp. 988 - 998, 2010.
*Abdulkadir Hamidu Alkali. "Evaluation of Thresholding Based Noncontact Respiration Rate
Monitoring using Thermal Imaging." International Refereed Journal of Engineering and Science
(IRJES) 6.7 (2017): 50-58.

More Related Content

PDF
A review on different technical specifications of respiratory rate monitors
PPTX
Respiratory Rate Measurement
PDF
An algorithm for obtaining the frequency and the times of respiratory phases...
PDF
Contactless Real-Time Vital Signs Monitoring Using a Webcam
PDF
Research article-2
PDF
Ambulatory Devices Measuring Cardiorespiratory Activity with Motion
PDF
Viral inspection (1)
A review on different technical specifications of respiratory rate monitors
Respiratory Rate Measurement
An algorithm for obtaining the frequency and the times of respiratory phases...
Contactless Real-Time Vital Signs Monitoring Using a Webcam
Research article-2
Ambulatory Devices Measuring Cardiorespiratory Activity with Motion
Viral inspection (1)

Similar to Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. (20)

PDF
Arduino uno based obstructive sleep apnea detection using respiratory signal
PPTX
4.Intensive care monitoring system .pptx
PPT
Noninvasive Tests for Asthma Diagnosis
PPTX
Ergonomics in anaesthesia
PDF
Waldmann A. et al.: Position-dependent distribution of lung Ventilation – A f...
PDF
Computed tomography scans image processing for nasal symptoms severity predi...
PDF
4. fm jdst published article 11 12
PPTX
A study on “impact of artificial intelligence in covid19 diagnosis”
PPTX
CM Physiological Patient Monitor pp.pptx
PDF
Comparison of Tympanic and Rectal temperature in febrile pediatric patients a...
PPTX
Document from Nithish Shekar.pptx
PDF
IRJET - Sleep Apnea Detection using Physiological Signals
PDF
Table of Contents - June 2021, Volume 12, Number 3
PDF
Development of Diagnostic Tools for Pathologies of Respiratory System using P...
PDF
Sleep Apnea Identification using HRV Features of ECG Signals
PPTX
Pupilscan Seminar at King's College London 20/11/19
PDF
Investigation of deformation of the cornea during tonometry using FEM
PDF
Modeling and optimization of nebulizers' performance in non-invasive ventilat...
PDF
Development of a Respiration Rate Meter –A Low-Cost Design Approach
PPTX
basic of polysomnography presentation.pptx
Arduino uno based obstructive sleep apnea detection using respiratory signal
4.Intensive care monitoring system .pptx
Noninvasive Tests for Asthma Diagnosis
Ergonomics in anaesthesia
Waldmann A. et al.: Position-dependent distribution of lung Ventilation – A f...
Computed tomography scans image processing for nasal symptoms severity predi...
4. fm jdst published article 11 12
A study on “impact of artificial intelligence in covid19 diagnosis”
CM Physiological Patient Monitor pp.pptx
Comparison of Tympanic and Rectal temperature in febrile pediatric patients a...
Document from Nithish Shekar.pptx
IRJET - Sleep Apnea Detection using Physiological Signals
Table of Contents - June 2021, Volume 12, Number 3
Development of Diagnostic Tools for Pathologies of Respiratory System using P...
Sleep Apnea Identification using HRV Features of ECG Signals
Pupilscan Seminar at King's College London 20/11/19
Investigation of deformation of the cornea during tonometry using FEM
Modeling and optimization of nebulizers' performance in non-invasive ventilat...
Development of a Respiration Rate Meter –A Low-Cost Design Approach
basic of polysomnography presentation.pptx
Ad

More from IRJESJOURNAL (20)

PDF
Drying of agricultural products using forced convection indirect solar dryer
PDF
The Problems of Constructing Optimal Onboard Colored RGB Depicting UAV Systems
PDF
Flexible Design Processes to Reduce the Early Obsolescence of Buildings
PDF
Study on Performance Enhancement of Solar Ejector Cooling System
PDF
Flight Safety Case Study: Adi Sucipto Airport Jogjakarta - Indonesia
PDF
A Review of Severe Plastic Deformation
PDF
Annealing Response of Aluminum Alloy AA6014 Processed By Severe Plastic Defor...
PDF
Correlation of True Boiling Point of Crude Oil
PDF
Combined Geophysical And Geotechnical Techniques For Assessment Of Foundation...
PDF
ICT governance in Higher Education
PDF
Gobernanzade las TIC en la Educacion Superior
PDF
The Analysis and Perspective on Development of Chinese Automotive Heavy-duty ...
PDF
Research on The Bottom Software of Electronic Control System In Automobile El...
PDF
Evaluation of Specialized Virtual Health Libraries in Scholar Education Evalu...
PDF
Linking Ab Initio-Calphad for the Assessment of the AluminiumLutetium System
PDF
Thermodynamic Assessment (Suggestions) Of the Gold-Rubidium System
PDF
Elisa Test for Determination of Grapevine Viral Infection in Rahovec, Kosovo
PDF
Modelling of Sealing Elements Wearing
PDF
Determining Loss of Liquid from Different Types of Mud by Various Addictives ...
PDF
Analysis of IFT (Interfacial Tension) and Viscosity of Various Polymer Based ...
Drying of agricultural products using forced convection indirect solar dryer
The Problems of Constructing Optimal Onboard Colored RGB Depicting UAV Systems
Flexible Design Processes to Reduce the Early Obsolescence of Buildings
Study on Performance Enhancement of Solar Ejector Cooling System
Flight Safety Case Study: Adi Sucipto Airport Jogjakarta - Indonesia
A Review of Severe Plastic Deformation
Annealing Response of Aluminum Alloy AA6014 Processed By Severe Plastic Defor...
Correlation of True Boiling Point of Crude Oil
Combined Geophysical And Geotechnical Techniques For Assessment Of Foundation...
ICT governance in Higher Education
Gobernanzade las TIC en la Educacion Superior
The Analysis and Perspective on Development of Chinese Automotive Heavy-duty ...
Research on The Bottom Software of Electronic Control System In Automobile El...
Evaluation of Specialized Virtual Health Libraries in Scholar Education Evalu...
Linking Ab Initio-Calphad for the Assessment of the AluminiumLutetium System
Thermodynamic Assessment (Suggestions) Of the Gold-Rubidium System
Elisa Test for Determination of Grapevine Viral Infection in Rahovec, Kosovo
Modelling of Sealing Elements Wearing
Determining Loss of Liquid from Different Types of Mud by Various Addictives ...
Analysis of IFT (Interfacial Tension) and Viscosity of Various Polymer Based ...
Ad

Recently uploaded (20)

PDF
Digital Logic Computer Design lecture notes
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PDF
composite construction of structures.pdf
PDF
PPT on Performance Review to get promotions
PDF
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPT
Mechanical Engineering MATERIALS Selection
PPTX
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
PDF
Model Code of Practice - Construction Work - 21102022 .pdf
PDF
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
PPTX
bas. eng. economics group 4 presentation 1.pptx
PDF
R24 SURVEYING LAB MANUAL for civil enggi
PPTX
additive manufacturing of ss316l using mig welding
PPTX
web development for engineering and engineering
PPTX
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
DOCX
573137875-Attendance-Management-System-original
Digital Logic Computer Design lecture notes
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
composite construction of structures.pdf
PPT on Performance Review to get promotions
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
Embodied AI: Ushering in the Next Era of Intelligent Systems
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
Operating System & Kernel Study Guide-1 - converted.pdf
Automation-in-Manufacturing-Chapter-Introduction.pdf
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
Mechanical Engineering MATERIALS Selection
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
Model Code of Practice - Construction Work - 21102022 .pdf
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
bas. eng. economics group 4 presentation 1.pptx
R24 SURVEYING LAB MANUAL for civil enggi
additive manufacturing of ss316l using mig welding
web development for engineering and engineering
Recipes for Real Time Voice AI WebRTC, SLMs and Open Source Software.pptx
573137875-Attendance-Management-System-original

Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging.

  • 1. International Refereed Journal of Engineering and Science (IRJES) ISSN (Online) 2319-183X, (Print) 2319-1821 Volume 6, Issue 7 (July 2017), PP.50-58 www.irjes.com 50 | Page Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. * Abdulkadir Hamidu Alkali Department Of Computer Engineering.University Of Maiduguri, Borno State, Nigeria. Corresponding author: *Abdulkadir Hamidu Alkali Abstract: - A noncontact method for respiration rate monitoring using thermal imaging was developed and evaluated. Algorithms to capture images, detect the location of the face, locate the corners of the eyes from the detected face and thereafter locate the tip of the nose in each image were developed. The amount of emitted infrared radiation was then determined from the detected tip of the nose. Signal processing techniques were then utilised to obtain the respiration rate in real-time. The method was evaluated on 6 enrolled subjects after obtaining all ethical approvals. The evaluations were conducted against two existing contact based methods; thoracic and abdominal bands. Results showed a correlation coefficient of 0.9974 to 0.9999 depending on the location of the ROI relative to the detected tip of the nose. The main contributions of the work was the successful development and evaluation of the facial features tracking algorithms in thermal imagining, the evaluation of thermal imaging as a technology for respiration monitoring in a hospital environment against existing respiration monitoring systems as well as the real time nature of the method where the frame processing time was 40 ms from capture to respiration feature plotting. Keywords: - Image processing, Noncontact, Real-time, Respiration monitoring, Thermal imaging, Thresholding. INTRODUCTION Oxygen is a naturally occurring element in the atmosphere which the body uses as part of a process to obtain energy. Air-rich oxygen enters the lungs through inspiration from where oxygen goes into the blood stream via the capillaries and is transported all over the body. During expiration de-oxygenated air containing the produced carbon dioxide is returned to the atmosphere. The average number of breathes taken over a predefined period of time is respiration rate. Its value depends on several factors that include the person‟s age, gender and medical condition. It ranges between 12 to 60 cycles per minute in healthy adults and children. During respiration, a number of physiological effects take place that are the bases for respiration monitoring. These include variations in air flow as air is inhaled and exhaled, chest/abdomen movements, blood oxygen saturation variations, temperature changes in inhaled and exhaled air and variations in the amount of emitted infrared radiation from the skin surface centred on the nose and mouth. An uneven respiration rate is a good indicator of imbalance of the normal body condition [1] that studies [2-5] have shown to be a predictor of potential serious clinical events. Early detection and proactive intervention of the predicted condition can avert a potential serious harm [6,7]. Respiration rate therefore, needs to be accurately monitored to establish correct physiological operations in order to take the necessary clinical steps. Existing gold standard devices for monitoring respiratory rate make contact with the subject‟s body during monitoring and are therefore, referred to as contact based. They mostly require the subject to be constrained and thus disturb the sleep position which affects normal breathing. Since a natural sleep position does not alter breathing, it is therefore, critical that respiration monitoring does not alter it as well. In noncontact respiration monitoring, the device is not attached to the subject‟s body and therefore, a more reliable medical assessment can be obtained. Studies have reported that noncontact based methods for monitoring respiratory rate are better than contact based methods [8-11]. They attributed this to improved patient‟s comfort, improved accuracy due to elimination of distress as well as non-possibility of disruption of monitoring as a result of dislodgement of the sensing device. In this study, technique to facilitate noncontact respiration rate monitoring has been developed and its effectiveness was evaluated. The technique was based on measuring detecting the amount of emitted infrared radiation from the skin surface centred on the nose (thermal imaging method). This is based on a fact that during respiration, exhaled air is warmer than inhaled air. The air temperature variation during respiration results in temperature variation along the air flow path especially the area around the tip of the nose and upper lip regions thereby resulting in corresponding variation in the amount of emitted infrared radiation from those areas. These continuous variations in infrared emissions around the affected regions can be observed and processed by
  • 2. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 51 | Page thermal imaging (thermography). A respiration signal can then be produced from the resulting images. Respiration rate is then obtained by processing the resulting signals. Methodology 1.1 Camera Setup The emissivity of the thermal camera was set to 0.98 as required for skin temperature measurement. The computer ran on Windows 7 operating system with a memory of 2.88 GB having an Intel Core processor that ran at 2.6 GHz. The camera was controlled using software that was developed using the National Instrument's LabVIEW environment. The camera's field of view (FOV) lens was 24 degrees and it was setup such that the thermal image 𝑓(𝑥, 𝑦) acquired from it consisted mostly of the subject's head and shoulders. Ethical approvals were obtained at Sheffield Hallam University, the Sheffield Children's Hospital and the National Health Service (NHS), United Kingdom. 1.2 Experimental Setup The camera was mounted on a tripod below the subject's upper body, at about 1 m from the subject who sat comfortably on a chair in a research laboratory. The camera was inclined at about 20o to focus upward towards the subject's face such that the subject's upper body was contained in the camera's field of view. This was to ensure that a good profile of the area most affected by respiration was adequately obtained. A typical setup is shown in Error! Reference source not found.. Thoracic and abdominal piezoelectric bands were used to record respiratory signal from the subjects. The mean temperature and relative humidity of the room were 24.4°C (standard deviation of 0.35) and 45% (standard deviation of 0.71) respectively. The frame capture rate per second was 20 in which allowed real time processing of the data and provided a sufficient sample rate. Each subject was monitored for at least 5 minutes but typically more than 10 minutes. The camera was controlled using software that was developed using the National Instrument's LabVIEW environment. The flow chart of the algorithm involved in this method is shown in Figure 0-1. Acquire Image Filter & Segment Create ROI & extract Respiration feature Start End Reduce Image Resolution Monitoring terminated? YES NO Locate tip of nose Iterative Elimination Algorithm Obtain Horizontal & Vertical profiles of image Detect face indicate tracking Obtain maximum pixel value on the face Pm Threshold image using (Pm-10) as lower threshold Iterative Elimination Algorithm Locate eyes’ corners Figure 0-1: Flow chart of feature detection based on thresholding algorithm
  • 3. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 52 | Page 1.3 Face Detection A facial region affected by respiration needed to be identified such that respiration induced temperature changes could be reliably detected. The best feature that suited this purpose was the nose but due to varying temperature of the skin area of the nose as a result of respiration posed consistency issues for the nose to be directly located with accuracy in thermal imaging. In order avoid this inconsistency and detect the nose in thermal image, a secondary feature needed to be located that would act as a reference to the nose. This would also eliminate user interaction at any point during the recording and therefore eliminate possible errors that could be introduced as a result. The two corners of the eyes met the requirement of being a reference in detecting the nose for two reasons. Firstly, the corners of the eyes have the highest temperature on the face and secondly the horn of the nose was always between the two corners of the eyes. The acquired image was therefore segmented to remove objects with lower temperatures to that of the skin. Unwanted objects with similar temperatures to that of the skin that could have been missed were then eliminated using an iterative elimination algorithm developed and described in [12, 13]. Thereafter the face was detected and enclosed in a rectangle to indicate tracking also described in [12]. The coordinates of the rectangle enclosing the face were saved as: rightBottomyx leftBottomyx rightTopyx leftTopyx br bl tr tl     , , , , (0-1) 1.4 Eyes' Corners Detection In [14] we detected the eyes‟ corners in a slightly different manner which came with some errors right from detecting the face as well as with the eyes‟ corners detection. These observations were improved upon in this evaluation by addition of equations 2-4 and 2-5 after taking the anthropometry of the face into consideration, as will be explained shortly. Since the corners of the eyes had the highest temperature over the face, they also had the highest pixel values by translation. To detect the corners of the eyes, the search for the maximum pixel values was carried out within the bounding rectangle. For the first acquisition, the subject was required to face the camera so that both eyes' corners could be detected. The reason for this would be explained later in the text. Starting from 𝑥𝑙, 𝑦𝑡 the pixel value (i.e. the first pixel) was recorded as the maximum pixel. Its value and coordinates were stored as 𝑃𝑚𝑎𝑥 and 𝑥 𝑝, 𝑦𝑝 respectively. The next pixel 𝑥𝑙+1, 𝑦𝑡 is looked at and compared with the stored maximum, if it is higher, 𝑃𝑚𝑎𝑥 and 𝑥 𝑝, 𝑦𝑝 are replaced. This is repeated for each pixel value within the bounded face. Upon reaching 𝑥 𝑟, 𝑦 𝑏 (i.e. the last bottom most pixel value within the bounded facial image), the maximum pixel value 𝑃𝑚𝑎𝑥 and its corresponding coordinates 𝑥 𝑝, 𝑦𝑝 gives an indication of the corner of one eye. One corner is always located first as the two eye corners do not have the same temperature, thus, the highest pixel value on the other eye corner would be different from that already detected. However, there is the likelihood that the next lower pixel value than 𝑃𝑚𝑎𝑥 was located next to 𝑃𝑚𝑎𝑥 and therefore still belong to the detected eye corner. If that was the case, then, locating two maximum values would not yield a pointer to the location of the two eyes‟ corners. To deal with these problems while avoiding complex and computational intensive algorithms, the face in image was segmented using the maximum pixel value 𝑃𝑚𝑎𝑥 . The segmentation was such that the eye corners would be returned as blobs with a maximum length (radius) of 10 pixels. Thresholding was used to achieve the segmentation using:        LTHgif1 Otherwiseyxg 0, (0-2) Where 10max  PLTH This was a valid assumption as the corner of the second eye cannot be 10 pixel values less than the detected corner of the first eye. After the segmentation, all regions with pixel values greater than 𝑇𝐻𝐿constituted the new image. These included regions associated with both eyes' corners as well as the side on the nose and in some cases the mouth region. When only the two regions associated with the corners of the eyes were present, the process of identifying the two corners was straight forward. It was however noted that some of the anomalies above were present, between or besides the location of the eyes' corners.
  • 4. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 53 | Page Only the upper half of the facial image was considered in the search for the corners of the eyes. This was because the eyes are located between the vertex and the upper half of the face. This decision was not only valid but also reduced the computation time by about half the normal time if the whole face was considered. The segmented image was then dilated to connect regions that were close enough to the blobs. Holes were then filled and then followed by eroding the image to return it to the original size before the dilation. The numbers of particles in the image were counted and the maximum horizontal and vertical coordinates occupied by each blob was returned. The coordinate of each particle contained in a blob was used to obtain the approximate coordinates of the centre of gravity of each blob using:         Ni i iy N gy Ni i ix N gx 1 1 1 1 (0-3) Where 𝑥 𝑔, 𝑦𝑔 represent the average positions of the central points of the horizontal and vertical sections in a blob and 𝑥𝑖, 𝑦𝑖 are the horizontal and vertical coordinate of a pixel in reference. The blob containing the maximum pixel value 𝑃𝑚𝑎𝑥 was identified in reference to the distances between its coordinates 𝑥 𝑝, 𝑦𝑝 and the centre of mass of each blob. The blob that had the least distance to 𝑃𝑚𝑎𝑥 was identified as the first eye corner blob and was used to locate the second eye corner. The blob representing the second eye corner was identified using the following steps: A blob was considered and its maximum vertical pixel length (considered as the diameter) was obtained and then divided by 2 to get the radius, r. The y-coordinate 𝑦𝑔 of the centre of gravity of the detected first eye 𝑒1 corner was then used to check whether the blob in reference (𝑒 𝑥) met the criteria:        regyxegyregy  11 (0-4) If the blob met the criteria, then it was identified as the corner of the second eye and the coordinates of its centre of gravity 𝑥 𝑔, 𝑦𝑔 𝑒2 were stored, otherwise the next blob (if any) was considered. The anthropometry of the face gave an indication of the possible separation between the inner corners of the two eyes (i.e. the intercanthal width) B as one fifth of the head breadth H [15, 16]. Therefore the horizontal separation between the x-coordinate 𝑥 𝑔 of the centre of gravity of the detected eye corner 𝑒1 and the blob in reference (𝑒 𝑥) must also meet the criteria: 56 lxrx B lxrx    (0-5) The limit in equation 2-5 was introduced as a lower boundary to avoid detecting false eye corner. When there was about 45 degree sideways head movement in either direction from when the face was looking straight into the camera, the two eyes' corners could still be detected. When there was a considerable sideways head movement however, only one eye corner was visible due to occlusion. Obtaining just one corner of an eye was not enough to identify where the nose was located using this method. To deal with this situation where one corner was not visible, the pixel distance between the two eyes' corners were calculated during the first iteration. This was the reason for the requirement of the subject to face the camera at the beginning of acquisition, as mentioned. The coordinates of both eyes' corners in each image were also saved and were replaced as new locations were obtained. The distance between the two eyes' corners for each image was then compared to the present distance and the x-coordinates 𝑥 𝑔(𝑒1) and 𝑥 𝑔(𝑒2) of both eyes' corners to those obtained (and saved) during the previous iteration. A difference less than zero indicated that there was a movement to the left by the subject (right in the image) and for a difference greater than zero, there was a movement to the right by the subject (left in the image). This was so that the direction of head movement was known in order to identify which eye corner was not in the camera's FOV. Identifying the direction of eyes' corners movement was the reason why the subject was required to face the camera for the first image acquisition. Obstruction of one eye corner could arise when the subject moved their head in a sideways direction. Knowing the eye corner that was obstructed from the camera's view allowed obtaining a coordinate location that would serve in detecting the location of the nose, just as the other eye corner would have were it not obstructed. To obtain the appropriate coordinate when one eye corner was not visible (due to obstruction) when the direction of the head movement was known as well as the maximum pixel distance between the centre of gravity of the two eyes' corners, the rectangle that contained the face was used. When there was a movement towards the left of the image that resulted in obstruction of the second eye corner
  • 5. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 54 | Page (i.e. the left eye corner was obstructed), then the coordinate of 𝑥𝑙 and 𝑦𝑔 𝑒1 were used as the appropriate coordinate in detecting the nose. If on the other hand, the right corner of the eye were obstructed, then 𝑥 𝑟 and 𝑦𝑔 𝑒1 were used. In either case, the appropriate coordinates were labelled as 𝑒2 while those for the visible eye corner detected labelled as 𝑒1. This was valid since only either the right or left sides of the nose was visible when the left or right eye corner was obstructed respectively. There was no facial image beyond the left or the right side of the rectangle bounding the face at those moments respectively. 1.5 Nose Detection Looking at the face, the horn of the nose which is a part of the nose was always located between the corners of the eyes. The nostril being the opening through which inhalation and exhalation take place during nasal breathing was the lower end of the nose. With the location of the two eyes' corners detected, all that was required was to identify the location of the nose. To do that, the image bounded by the coordinates of the eyes' corners (𝑥 𝑔, 𝑦𝑔(𝑒1) and 𝑥 𝑔, 𝑦𝑔(𝑒2)) and that by the y-coordinates of the lower rectangle bounding the face in reference to the x-coordinates of the eyes' corners (𝑥 𝑔(𝑒1), 𝑦 𝑏and 𝑥 𝑔(𝑒2), 𝑦 𝑏) was considered. This area of the face contained the nose and mouth regions. For each acquired frame, we considered the highest and lowest pixel values 𝑁𝑝𝑚𝑎𝑥 and 𝑁𝑝𝑚𝑖𝑛 respectively. The image within the area specified was then segmented so as to remove all areas that were not associated with the nose. To do this, the image was segmented using:    UTHgLTHif1 Otherwise0yx,g   (0-6) Where   min minmax 2 pL pp U NTHand NN TH    Considering the maximum and minimum pixel values for each acquired frame was in order to consider the pixel values that were affected by respiration, i.e. during expiration, the infrared radiated around the nose increases and so does the pixel values and the reverse occurs during inspiration. This was also noted by [17]. If constant values were considered, then the nose could not have been segmented and subsequently could not be detected in some frames. After the segmentation, only the area representing the nose remained in the image and a rectangle was created to enclose it to indicate detection. The tip of the nose 𝑡 𝑁 in frontal (or near frontal) view is the mid-point between the left and right bottom sides of the region enclosing the nose. For non-frontal view, the respective side of the facial boundary was used as the tip of the nose. 1.6 Region of Interest (ROI) The region between the tip of the nostrils and the vertex of the lip was always affected by respiration. The region was close to the tip of the nose. Four regions of interest (ROI) were created in order to get different respiration signals from different locations that were affected by respiration. The regions were: 1. ROI1: The average pixel value of the area within the rectangle bounding the detected nose area. 2. ROI2: A circle with radius of 5 pixels located beneath the left nostril. Its centre was located on the bottom line of the rectangle bounding the detected nose and at a quarter of the width of the nose area from the left. 3. ROI3: A circle with radius of 5 pixels located beneath the right nostril. Its centre was located on the bottom line of the rectangle bounding the detected nose and at a quarter of the width of the nose area from the right. 4. ROI4: The non-zero pixel values within obtained in equation 3-4. 5. Results And Discussion After locating the corner of an eye and thresholding the image as described, the resulting image contained areas associated with only the two corners of the eyes in most cases. Figure 0-1(b) shows the resulting image after thresholding and Figure 0-1(c) shows the two eyes' corners after considering the upper half objects in the facial image. Figure 0-1: (a) Original image (b) the results of thresholding based on the temperature profile of first eye's corner and (c) two objects (only) representing the two corners of the eyes.
  • 6. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 55 | Page In the instance indicated in Figure 0-1, locating the two corners of the eyes was by obtaining the coordinates of the two corners. In some cases however, areas associated with the sides of the nose, the mouth region as well as some irregular areas associated with skin marks (e.g. scars) also appeared in the image due to the temperature similarities of the regions as shown in Figure 0-2. Figure 0-2: (a) Original image (b) the results of thresholding based on the temperature profile of first eye's corner and (c) more than two objects. In these cases, locating the second eye corner was not as straight forward. This was because, there was no exact prior knowledge indicating when more than two objects would be present as against two, the second eye's algorithm had to be used. Objects appearing below the center of mass of the face were rejected as explained earlier. The performance of the second eye's algorithm in detecting the eyes-corner pair is shown in Figure 0-3, where even when more than two objects appeared in the image. „ Figure 0-3: (a) Detection of eyes' corners in presence of more than two objects and (b) eye corners mapped onto image. By detecting one corner of an eye, the other eye's corner was detected by the second eye's algorithm even when there were objects not related to the eye. The nose was then detected after obtaining the position of the two eyes‟ corners. The effectiveness of the nose detection is shown in Figure 0-4. Figure 0-4: Detection of the nose in different images.
  • 7. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 56 | Page The respiration signals were obtained from the four ROIs as previously described. The locations of the ROIs are shown in Figure 0-5 for a subject. Figure 0-5: (a) ROI1, (b) ROI2, (c) ROI3 and (d) ROI4. ROI1 averages all the pixel values which includes parts of the corners of the eyes and is therefore expected to have low pixel average due to the larger area and the temperature differences between the corners of the eyes‟ and area around the nose contained within the ROI. ROI2 and ROI3 are circles (in red) of 5 pixels radius beneath the left and right nostrils respectively and should have high amplitudes as the temperature distribution within the ROIs are almost uniform. The respiration signal simultaneously obtained from the 4 ROIs in one of the recordings is shown in Figure 0-6. Figure 0-6: Respiration signals from the four ROIs All the respiration signals were normalized around zero in order to enable overlay comparison. ROI2 and ROI3 have similar respiration waveform characteristics which are attributed to the location of the ROIs. ROI1 have higher amplitude that ROI2 and ROI3 as it had a larger range between its minimum and maximum average pixel values which was derived due to inspiration-expiration temperature difference. ROI4 had the highest amplitude as it had the largest range that was attributed to its location (i.e. the nose). During inspiration, the nose cools especially around the post inspiration pause period and during expiration it is warmed. The area occupied by the nose in ROI4 is considerably reduced during inspiration and the pixel values are generally small due to temperature drop. ROI4 is also inverted as due to the effect of averaging the pixels. During inspiration the numbers of pixels are less and the temperature range is less giving a higher average than during expiration 0 10 20 30 40 50 60 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 t, sec. Amplitude ROI1 ROI2 ROI3 ROI4
  • 8. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 57 | Page where the temperature range is large and the numbers of pixels are also more thus giving a lower average which the normalization ranges around the zero axes. Figure 0-7 and Figure 0-8 show a minute-respiration signal of a subject comparing the threshold based method obtained from ROI1 and ROI2 respectively against two ground truths (thoracic and abdominal bands) derived. Figure 0-7: The ROI1 derived against the ground truths respiration signal The threshold derived respiration signal always lags behind the ground truths during inspiration while during expiration it mostly lags and leads in few instances. Figure 0-8: The ROI2 derived against the ground truths respiration signal Table 0-1 provides a summary of the respiration rates from the 6 subjects included in the study alongside that from the ground truths (i.e. the thoracic and abdominal bands). The mean age of the enrolled subjects was 38.7 years with a standard deviation of 3.38 years. Table 0-1: Measured respiration rates. Subject Gender Age Thermal Imaging Based Thoracic Band Based Abdominal Band Based ROI1 ROI2 ROI3 ROI4 FT1 Male 41 23.6 23.6 25.0 23.6 25.0 25.0 FT2 Male 41 18.9 18.9 18.9 18.9 19.0 19.0 FT3 Female 33 8.0 8.0 8.0 7.5 8.0 8.0 FT4 Male 36 9.6 9.6 9.6 9.6 9.7 9.7 FT5 Female 40 9.2 9.2 9.2 9.2 9.2 9.2 0 10 20 30 40 50 60 -0.4 -0.2 0 0.2 0.4 0.6 t, sec. Amplitude Abdominal Band Based Thoracic Band Based Thermal Imaging Based 0 10 20 30 40 50 60 -0.4 -0.2 0 0.2 0.4 0.6 t, sec. Amplitude Abdominal Band Based Thoracic Band Based Thermal Imaging Based
  • 9. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 58 | Page FT6 Male 41 16.9 16.9 16.9 16.9 16.9 16.9 For FT1, the ground truth RR was 25 cycles per minute in both, however, in thermal imaging threshold based method the RR was 23.6 cycles per minute in ROI1, ROI2 and ROI4 derived respirations. This is a 1.4 cycles per minute less than the ground truth. Although ROI3 derived was the same as the ground truth, three of the regions had less RR. FT2 and FT4 had RRs that were 0.1 cycles less that the corresponding ground truth RRs in all the ROIs. FT5 and FT6 had the same RRs in all the ROIs as the ground truth. FT3 had the same RR in all the ROIs as the ground truth except ROI4. CONCLUSION Thermal imaging based respiration rate monitoring based on threshold was developed. The method detected the face of the subject then the two corners of the eyes and thereafter located the nose in thermal images. Respiration features were extracted within created ROIs using a statistical measure. Respiration signals were then obtained from succeeding features before obtaining the respiratory rate over a period of time. The method was evaluated on 6 adult volunteers in a hospital setting as presented in the results and discussions section.Six subjects participated in the evaluation of the respiration rate monitor based on thresholding. Four ROIs were created around and on the detected nose region and respiration signals obtained simultaneously from each. Correlation of the respiratory rates with the corresponding ground truths was at least 0.9977 in all the four ROIs; ROI4 had 0.9977, ROI1 and ROI2 had 0.9984 while ROI3 had 0.9999. ACKNOWLEDGEMENT The author wishes to acknowledge the subjects that volunteered in the research and most especially his research supervisory team and the sleep unit staff of the Sheffield Children Hospital. REFERENCES [1]. M. A. Cretikos, R. Bellomo, K. Hillman, J. Chen, S. Finfer and A. Flabouris, “Respiratory rate: the neglected vital sign,” The Medical Journal of Australia, 188(11), 2008, pp. 657 - 659. [2]. C. P. Subbe, R. G. Davies, E. Williams, P. Rutherford and L. Gemmell, “Effect of Introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admission.,” Anaesthesia, 58, 2003, 797-802. [3]. Y. Nam, J. Lee and K. H. Chon, “Respiratory rate estimation from the built-in cameras of smartphones and tablets.,” Annals of Biomedical Engineering, 42(4), 2014, 885-898. [4]. F. Q. Al-Khalidi, “Facial tracking method for noncontact respiration rate monitoring,” Proc. 7th International Symposium on Communication Systems Networks and Digital Signal Processing, Newcastle upon Tyne, United Kingdom, 2010, 751-754 [5]. M. Cullinane, G. Findlay, C. Hargraves and S. Lucas, “"An acute problem?" National Confidential Enquiry into Patient Outcome and Death.,” London, 2005. [6]. D. Konrad, G. Jäderling, M. Bell, F. Granath, A. Ekbom and C. Martling, “Reducing in-hospital cardiac arrests and hospital mortality by introducing a medical emergency team.,” Intensive Care Medicine , 36(1), 2010, 100-106. [7]. J. Hosking, J. Considine and N. Sands, “Recognising clinical deterioration in emergency department patients.,” Australasian Emergency Nursing Journal, 17(5), 2014, 59-67. [8]. R. Murthy, I. Pavlidis and P. Tsiamyrtzis, “Touchless monitoring of breathing function,” Proc. 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, California, United States of America, 2004, 1195-1199. [9]. F. Q. Al-Khalidi, R. Saatchi, D. Burke and H. Elphick, “Respiration rate monitoring: A review,” Pediatric Pulmonol., 46, 2011, 523-529. [10]. M. Bartula, T. Tigges and J. Muehlsteff, “Camera-based system for contactless monitoring of respiration,” Proc. 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 2672-2675. [11]. I. Smith, D. Krucheck, J. Mackay and N. Fahrid, “Respiratory rate measurement: A comparison of methods.,” British Journal of Healthcare Assistants , 5(1), 2011, 18-23. [12]. A. H. Alkali, R. Saatchi, H. Elphick, D. Burke, “Facial tracking in thermal images for real-time noncontact respiration rate monitoring,” Proc. 2013 European Modelling Symposium, Manchester, United Kingdom, 2013, 265-270. [13]. A. H. Alkali, “Iterative Elimination Algorithm for Thermal Image Processing,” Arid Zone Journal of Engineering, Technology and Environment, 10, 2014, 85-93. [14]. A. H. Alkali, R. Saatchi, H. Elphick and D. Burke, “Eyes' corners detection in infrared images for real- time noncontact respiration rate monitoring,” Proc.2014 World Congress on Computer Applications and Information Systems, Hammamet, Tunisia, 2014, 1-5
  • 10. Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging. www.irjes.com 59 | Page [15]. J. W. Young, “Head and Face Anthropometry of Adult U.S. Citizens,” Federal Aviation Administration, Wasington, DC, United States of America, 1993. [16]. L. G. Farkas, M. J. Katic and C. R. Forrest, “International Anthropometric Study of Facial Morphology in Various Ethnic Groups/Races.,” Journal of Craniofacial Surgery, 16, 2015, 615-646. [17]. J. Fei and I. Pavlidis, “Thermistor at a Distance: Unobtrusive Measurement of Breathing.,” IEEE Transactions on Biomedical Engineering, vol. 57, pp. 988 - 998, 2010. *Abdulkadir Hamidu Alkali. "Evaluation of Thresholding Based Noncontact Respiration Rate Monitoring using Thermal Imaging." International Refereed Journal of Engineering and Science (IRJES) 6.7 (2017): 50-58.