2. recent report, in 2015, nearly 5 lacs of women died due to
brain tumors [2]. In addition, the World Health Organiza-
tion (WHO) depicts that nearly 1.5 million women might
lose their life due to brain tumors [2, 3]. The US as one of
the most advanced nations with the best healthcare infra-
structure too witnessed nearly 2.5 lacs of brain tumor
patients and forty thousand deaths in 2017 [2].
According to the definition of brain tumor, it is the mul-
tiplication with the exponential rate of masses or dead cells
within or across the brain. The rapid growth of affected cell
deaths influences new cells as well as destroys them more
resulting in the formation of a cancerous patch inside or
on the brain. However, some effortlessly noticeable indica-
tions of brain tumors are skin-dimpling, reddish and dry
skin, swollen lymph nodes, pain, and swelling on certain
area(s). In the main part of current diagnosis paradigms,
radiologists or clinicians utilize conducting a manual analy-
sis of distinct healthcare modalities like OCT, hematoxylin
and eosin (H&E) staining, histopathological images, ultra-
sound image, mammogram, and magnetic resonant imaging
(MRI). Several methodologies categorize that every target
sample like allied tissues, histopathological images, and
biopsy images is categorized as malignant and benign.
Benign tissues generally include abnormal epithelial cells
that are not usually cancerous, as majorly epithelial cells do
not convert as brain tumors. Conversely, malignant tumor
cells represent those that kill the normal cells and spread
abnormally, leading to cancer [4]. Therefore, analyzing these
complicated characteristics even with traditional image pro-
cessing approaches can develop computational fatigue as
well as can further result in misdiagnosis [5]. The process
needs training over large and substantial microfeatures of
both the malignant and benign tissues to attain a reliable
diagnosis [6], although such traditional methods are error-
prone and time-consuming that can upsurge the mortality
and morbidity rate [7]. It shows the requirement of early
detection from a great trustworthy and robust computer-
aided diagnosis (CAD) solution. Vision-based innovations
towards CAD solutions involving machine learning and
image processing techniques show promising efficacy.
Manipulating the visual therapeutic consequences can sup-
port understanding the tumor cell metastasis that can
improve the survival rate of the patient [8].
Clinicians can decide the optimal diagnosis decision by
understanding the phase and traits of brain tumors. How-
ever, brain tumors by manual identification utilizing micro-
scopic biopsy images are generally reliant on the expertise
and skills of doctors as it changes from person to person.
Conversely, insufficient certain quantitative measures
restrict the diagnosis optimality of manual cancer. Machine
learning and vision-based approaches to alleviate it are esti-
mated as more effective towards the clinical prognosis and
diagnosis [9]. In recent times, approaches involve diverse
sophisticated processes available that apply brain biopsy
images, brain ultrasound, and histopathological data to
categorize it as malignant and benign [10], though the
optimal possible mammogram feature learning ability for
medical images is a must to conduct the brain tumor diag-
nosis. It requires that the CAD systems be equipped with
superior feature learning as well as classification
approaches [11]. Traditional medical imaging models
implement feature extraction to do brain tumor diagnosis
and estimate the region of interest (ROI) or nucleus seg-
mentation, although it suffers from false prediction and
is computationally exhaustive. Several machine learning
algorithm-based methods have been suggested for brain
tumor recognition [12].
Several machine learning approaches are the Support
Vector Machine (SVM), Decision Tree (DT), Naïve Bayes
(NB), Artificial Neural Network (ANN), and ensemble tech-
niques. Nevertheless, several current machine-learning
approaches that apply traditional feature extraction, ROI
segmentation, and preprocessing phases [13] that develop
an entire system are computationally limited as well as
exhaustive. Conversely, several researchers claim different
performance by considering similar methodologies resulting
in indefinite consistency. For example, the SVM-based tech-
nique shows the accuracy of (%), though the identical
research provided lesser accuracy. Unquestionably, such
methods experience false negative/positive performance
making their appropriateness for application in the real-
world doubtful. Several current methods depict less speci-
ficity and sensitivity [14]. We discover in detail that a
substantial amount of studies that has been obtained are
contradictory, illustrating the diverse performance with
identical machine learning algorithms. Several approaches
have been created as a classifier-sensitive solution despite
the characteristics of the information playing a crucial role
towards maximizing the cancer diagnosis. Such approaches
use the deep learning technique as the most viable
approach. Brain tumor diagnosis requires the understand-
ing of more depth features.
A convolutional neural network (CNN) and its variants
have been utilized amongst the major deep learning (DL)
models to assess the diverse medical images [15–18]. Deep
learning can perform the histopathological image classifica-
tion as a (deep) feature extractor towards a diagnostic deci-
sion. Unquestionably, DL approaches such as ResNet [15]
and AlexNet [16] have achieved superior results concerning
the major current machine learning methodologies. The
effectiveness turns out to be more refined because of the
independence in the direction of further feature extractors
[18]. Nevertheless, the classification efficiency of such meth-
odologies is mainly reliant on the extraction of features as
well as consequent utilization [18]. Majorly, DL approaches
have been framed with a substantially huge dataset to
increase its learning capability, though, in the practical
world, it needs a definite pretrained model to conduct a pre-
cise classification. It creates a classification reliant on the
pretrained characteristics. These cases need the application
of the specific secondary pretrained framework in concur-
rence with patients’ primary (or own) set of (incomplete)
images which may provide a false negative/positive outcome.
It can make the entire system untrustworthy and can have a
contrary influence on diagnosis conclusion. Furthermore,
just applying features that are too deep can lead to informa-
tion redundancy as well as overfitting that ultimately lessen
the entire performance.
2 BioMed Research International
3. Research studies expose that deep feature extraction as
well as characterizing it by utilizing a definite effective
machine learning model that can make superior perfor-
mance, particularly with lesser data size, i.e., practical for
the application in the real world. Because of previously
described main interpretations, this research explains a
robust deep hybrid included ML model for the classification
of tissues of a brain tumor using the ResNet50 architecture.
As the name specifies, this suggested model implements two
well-recognized DL approaches, the modified ResNet50 with
the Enhanced Watershed Segmentation (EWS), to extract
the features for optimal set retrieval to obtain best features
for classification. Implementation of the ResNet50 model
utilizes 10-fold cross-validation-based classification that
can attain the accuracy of 92, precision of 0.92, sensitivity
of 1.0, specificity of 0.92, F-measure of 0.95, and AUC of
0.94. The suggested approach is created by utilizing the
MATLAB 2019b platform where the DDSM dataset revealed
the simulation outcomes to outperform the main at-hand
solutions for the tissue classification of brain tumors.
According to Akkus et al. [19], the CNN model was used
for the tumor genomic prediction with 87.70% accuracy.
Zhao and Jia [20] worked on the BRATS dataset 2013 for a
patch-wise CNN model and found 81% accuracy for tumor
segmentation. Pereira et al. [21] used another CNN model
for brain tumor segmentation and got the accuracy of 88%.
Brosch et al. [22] used 3D CNN with segmentation and
got 84% accuracy.
1.1. Contribution. The proposed model uses the modified
ResNet50 model, watershed technique, and transfer learning
approach which makes the final model more reliable with a
higher accuracy. This paper uses the EWS algorithm that
consists of three steps: first is filtering for morphological res-
toration of the object surface as the initial and the final
action; second is the markup extraction which marks the
least worthy feature on the image and fixes it; last is the
mark-based transformation which shows the accurate image
using the concept of transformation. The proposed model
uses the modified ResNet50 model by including five convo-
lutional layers and three fully connected layers to the exist-
ing ResNet50 model. The proposed method can retain the
optimal computational efficiency with high-dimensional
deep features. Due to this, the proposed model is able to
achieve the exact location of boundary pixels and is useful
to define the exact location of the tumor.
The rest of the paper is organized as follows: Section 2
describes the proposed methodology; Section 3 presents
results and discussion; finally, Section 4 draws the
conclusion.
2. Methodology
The proposed work is presented by the data flow diagram
with the step-by-step methodology in Figure 1. Firstly, data
preprocessing is performed; then, the output images go
through the Enhanced Watershed Segmentation (EWS)
algorithm technique and find out the contour points of the
image. Then, the image augmentation technique is applied
to all images and loaded into the modified ResNet50 model
(modification done by transfer learning concept), and
then, the results are obtained in the form of a ROC graph,
model loss, accuracy, precision, specification, and sensitiv-
ity of the model.
2.1. Dataset Description. In this paper, we have used the
dataset from the online platform Kaggle. In this dataset, a
total of 253 images exist with different imaging modalities,
but we have used only MRI images which are categorized
by the proposed method. After that, the dataset is split into
three ratios of training, testing, and validation, i.e., (75, 20,
5), (80, 10, 10), and (75, 15, 10). Required software details
for the experimental work are a GPU-based system with
min 4 Gb RAM and Anaconda environment software setup.
2.2. Data Preprocessing. Preprocessing is the preparation of
datasets with valuable information and removing unwanted
ones. Preprocessing is critical in computer vision, especially
in medical image analysis, where inaccurate input can
degrade the performance of a very effective classifier. This
research involves the usage of three preprocessing stages,
MRI image resizing, augmentation, and global pixel normal-
ization, before allocating them to the segmentation stage.
The database utilized to investigate with the DL approach
is extremely heterogeneous, so effective segmentation
requires consisting to resize it with a similar height as well
as width. The first step involves preprocessing where the
images have been resized and cropped according to the sim-
ilar width/height consisting of the RGB image as three-
channel depth. The entire images are resized as 256 × 256
× 3. It has 256 widths and height and 3 depths in RGB.
The second stage applies global pixels normalizing to the
cropped images. Because each image’s pixel varies from 0
to 255, it is recommended to normalize it between 0 and 1
for better deep learning training. Normalization is the pro-
cess of converting image pixels to a common scale across
all images in a dataset [23]. Global pixel normalization
(GPN) is the process to scale pixel data on a given range,
often 0-1. While the images inside the database fluctuate
from 0 to 255, this is required to increase data from 0 to 1.
Using GPN converges the gradient descent faster than ignor-
ing normalization. GPN has also been included for this
research as
y = ðy − yminÞ/ðymax − yminÞ, where “y” is the fea-
ture vector and ymin and ymax are the minimum and maxi-
mum values of the feature vector, respectively.
2.2.1. Feature Engineering Process. In the feature engineering
process, data preparation is performed for feature selection.
It includes following steps:
Step 1: select data/integrate data: in this step, the dataset
selection and denormalization are performed which will be
further used in the preprocessing step.
Step 2: preprocess data: in this step, data cleaning and
sampling is done. This step includes scaling, handling the
missing values, balancing the data, standardization, and nor-
malization as well.
Step 3: transform data: in this step, data is converted into
the transformed form.
3
BioMed Research International
4. 2.3. Watershed Technique. On a gray-scale image, a water-
shed is a transition. Image morphology is used to segment
regions in watershed segmentation. To segment regions,
the watershed uses regional minima as seeds. It is a hybrid
method that combines boundary and region-based growing
techniques. (This transformation could be considered as a
topographic region growing method.) At least one seed
point must be chosen inside of each object in the image,
including the backdrop. The markers are chosen manually
or automatically depending on the application-specific
information included in the objects. After the objects have
been marked, morphological watershed transformation can
be used to grow them.
The watershed, on the other hand, is a standard segmen-
tation technique for separating objects in a picture. Pixel
data is treated as a local topography by the watershed
method (elevation). The watershed segmentation methods
treat an image as a topographic relief, with the value of each
image element indicating the image’s height at that location.
In the study, the term element is utilized to merge the
notions of pixel and voxel. Rather than the original image,
watershed segmentation is frequently applied to the result
of the image’s distance transform.
2.3.1. Enhanced Watershed Segmentation (EWS) Algorithm
(1) Filter for Morphological Restoration. The standard pre-
smoothing filter reduces noise and irregular features well,
but it loses contour edge information, causing a change in
the region contour [24]. The edge shape data of the objective
can be very much protected when filtering and denoising the
MRI picture of the chest tumor. It also does not produce a
shift in the contour of the rebuilt image. Morphological res-
toration is defined as
Mn+1 = Mn ⊕ StEl
ð Þ
x, ð1Þ
where: Mrs
StElðy, xÞ shows a morphological restoration
image of mask y reformed by the x (marker image), where
StEl is the structural element, x is the real image, Mn is the
last iteration resultant image, and M0 is the initial iteration
of y.
Equation (1) is iterated as far as possible when Mn+1 =
Mn.Since morphological restoration might eliminate surface
components and brilliant commotion more modest than
underlying components, morphological shut rebuilding can
do likewise and recuperate the objective edge. Notwithstand-
ing, using just morphological or shut reclamation can just
dispense with one commotion or detail from the picture,
causing a change in the objective form’s position. The sur-
face subtleties and concealing clamor can be eliminated
simultaneously when the crossbreed introductory and last
rebuilding activities are utilized. The picture morphology
rebuilding is utilized to expand the limit data while dimin-
ishing the quantity of pseudoleast qualities when the half,
half starting, and last reclamation activities are utilized.
The morphological initial and final restoration action
based on the initial and final actions is defined as
Hrs
StEl = Irs
StEl Frs
StEl x
ð Þ, x
½ , ð2Þ
where Irs
StElðxÞ is the initial action restoration and Frs
StElðxÞ
is the final action restoration.
(2) Markup Extraction. High ratio nonsensical segmentation
findings can be adopted by marker extraction after morpho-
logical restoration filtering of breast tumor MRI images. Fix
Enhanced watershed
segmentation
Data set Resize
Filter for morphological
restoration
Markup extraction
Markup based
transformation
∙
∙
∙
Find
extreme
point
Augmentation
Spilt
the dataset
Crop the
image
Normalization
Find
contour
Threshold
Data preprocessing
Denoising
Modified
ResNet50
model
Classification
output
Figure 1: Data flow diagram of brain tumor analysis by using Enhanced Watershed Segmentation (EWS) algorithm technique with
modified ResNet50 model.
4 BioMed Research International
5. it. Before applying the EWS algorithm on the objective
image, mark the least worthy of the objective locale in the
slope picture and mask the superfluous values. To avoid
oversegmentation issues, only the objective area’s minimum
value is permitted to be preserved.
The important challenge with this strategy is selecting
the Th (threshold), extracting the minimum value whose
depth is less than Th. The Th-minima approach has the ben-
efit of giving the threshold directly. However, Th is fixed and
the adaptation is single. The adaptive acquisition approach
can be used to select the threshold Th to prevent artificial
setting variables. When the Th-minima transform is used
to extract the marker of the rebuilt gradient image, Th is
automatically obtained using the maximum intergroup var-
iance approach, thereby identifying the objective region
where the breast tumor occurs.
Step 1: assuming Th is a threshold, the objective picture
is separated into two groups by Th: the objective group G0
comprises pixels in the greyscale of f0, 1, 0:Thg, and the
objective group G0 includes pixels in the greyscale of Th+1,
Th+2, 0, and L-1.
Step 2: determine the objective group G0 intragroup var-
iation and the background group G0 intergroup varian-
ce.The following is the variation within the group:
Var2
P
ð Þ = P0Var2
0
ð Þ + P1Var2
1
ð Þ = 〠
Th
a=0
a − S0
2
pa + 〠
L−1
a=Th+1
a − S1
2
pa,
ð3Þ
where probability of occurrence of objective group G0 is
P0 and probability of occurrence of background group G1 is
P1.
The mean value of objective group G0 is S0 and back-
ground group G1 is S1.
The variance of the objective group G0 is Var2
ð0Þ and
the background group G1 is Var2
ð1Þ.
The variance between classes is
Var2
n
ð Þ = P0 S1 − ST
2
+ P1 S1 − ST
2
= P0P1 S1 − S0
2
:
ð4Þ
The overall variance is
Var2
T
ð Þ = Var2
n
ð Þ + Var2
P
ð Þ: ð5Þ
Step 3: determine the ideal threshold. When Var2
ðPÞ
finds its lowest or Var2
ðnÞ finds its maximum using the sort-
ing search method, the threshold is the ideal threshold.
Th = arg min
0≤ThL
Var2
P
ð Þ
, ð6Þ
or
Th = arg max
0≤ThL
Var2
n
ð Þ
: ð7Þ
(3) Mark-Based Transformation of a Watershed. The
approach is used to generate theTh, and afterward, the
gradient-recreated image Hrs
StEl is separated utilizing the
drawn-out least change strategy to compel the marker to
show up at any rate esteem.
After catching the neighborhood’s least worth marker
picture related to the objective locale, utilizing the minim
burden approach, adjust the gradient picture such that other
pixel values are consistent as needed.
The watershed segmentation operation is done on the
gradient picture Hmark
following the minimum value forced
minimum procedure.
HEWS = EWS Hmark
, ð8Þ
where the EWS ð Þ algorithm shows transformation and
the value is HEWS.
2.4. Data Augmentation. For efficient implementation and
learning, train deep learning models with a large amount
of high-quality data [17]. The DL model should be trained
on large datasets to maximize learning and avoid overfitting.
Furthermore, DL model performance and learning accuracy
improve with high-quality and ample training data. Data
augmentation (DA) is a technique for enhancing or chang-
ing a dataset. It is also a technique for making many copies
of an original image utilizing a variety of approaches. One
way of data augmentation is to increase the quantity of the
training data to keep the labels [18, 25, 26]. The DA with
oversampling type generates artificial illustrations as well
as associates them with the current dataset. This includes
the data fraternization, GANs (generative adversarial net-
works), and feature space DA [11, 27]. To produce twelve
clones of each shot, the effort uses twelve different augmen-
tation types with the same number of parameters. Deep
learning necessitates a big label dataset because it involves
billions of variables, many of which are unavailable, particu-
larly in medical data [23]. Produce excellent labeled datasets
by combining deep augmentation techniques with easy
manipulation approaches to create artificial datasets (TFs).
The number of basic TFs that can be applied is unlimited.
By mixing enhanced and original datasets, small and original
datasets can become large.
2.4.1. Image Augmentation Step. The proposed brain tumor
detection through a machine learning algorithm involves
four basic techniques such as data processing, transfer-
learning technique as the ResNet50 model, feature extrac-
tion, and segmentation. The designed novel also is shown
in Algorithm 1. The two types of augmentation techniques
used in the proposed work are as follows:
(i) Geometric transformations: in this technique, ran-
domly flip, crop, rotate, or translate images, and that
is just the tip of the iceberg as the given value.
(ii) Color space transformations: change RGB color
channels and intensify any color.
5
BioMed Research International
6. In the proposed algorithm, firstly, load and import the
dataset and then split it into the three-part training set, test
set, and validation data. After that, select any one original
image and apply the biggest contour to find out the extreme
point. Then, crop the image and save it. Apply this function
to all images. Further, resize all images on the same scale.
After that, apply augmentation techniques on the training
dataset to increase the number of images in the dataset for
model training. Figure 2 shows the image set of the brain
tumor (input). Figure 3 shows the resized image. Figure 4
shows data augmentation on the original image. Figure 5
shows the augmented images.
We have used two types of augmentation techniques in
the proposed work:
(i) Geometric transformations: in this technique, the
image is randomly flipped, cropped, rotated, or
translated.
(ii) Color space transformations: in this technique, any
color is intensified and the HSV channel is changed
to an RGB color channel.
After getting the resized image, augmentation is per-
formed whose parameters are set as given in Table 1.
2.5. Feature Extraction Based on Transfer Learning with
Pretrained Networks. Transfer learning (TL) becomes the
motivation for the prevailing remote learning models as well
as utilizing the enlightening knowledge acquired for a single
assignment to sort out the identical ones. This work involves
the usage of data provided by a pretrained system to under-
stand the new models given by new information. Using TL
to standardize a pretrained system is typically faster and eas-
ier. Using deep learning algorithms, we can quickly learn
new skills. The TL is seen by many data analysts and
researchers as a strong tool for artificial intelligence advance-
ment. It involves a general difference among the traditional
method utilized to provide the training to ML system models
and using a method to follow TL standards. Traditional
learning is detached as well as occurs just for definite data-
sets/assignments and to train separate models with them. It
involves no preservation of information that could be
shifted from one operation in hand to the next task. In
the TL scenario, the user can use the data (features,
weights, etc.) from a previously trained model to overcome
problems such as having insufficient knowledge for the
current task. The fundamental concept of TL is depicted
in Figure 6. Deep learning models/frameworks consist of
the layered structures that can acquire the understanding
of several features at many levels.
These levels are finally combined to produce a single
layer that is entirely connected. It enables the user to use a
pretrained system (such as GoogleNet or AlexNet) without
having to use the final layer as a feature extractor for several
applications [28]. For example, removing the last layer (clas-
sification layer) from pretrained networks like GoogleNet or
AlexNet could help turn a new domain image into a multi-
dimensional vector based on hidden states, allowing the user
to extract features from the new domain using pretrained
domain data. Such strategies are widely utilized for con-
ducting transfer learning by using deep learning techniques.
Visually distinct features can be extracted by using fine-
tuning for each pretrained network (PTN). Transfer learning
can increase the efficiency and competence of a convolu-
tional neural network by replacing the last layers of the net-
work. Rather than retraining the entire classifier structure,
CNN weights are added to the top of the PTN in this case.
The PTN weights are moved from the source dataset to the
target dataset in this condition. The major goal, which is
the recommended work, is to replace the PTN layer with a
new softmax layer. For this study, we employed the previ-
ously mentioned CNN designs and kept the goal neuron
datasets from the last fully connected layer. Figure 7 shows
the transfer learning approach for the proposed model
where “head” means top of the network. Models and net-
works are not repeated in transfer learning. According to
the notion, to create a target model, a few requirement-
based adjustments are required in the source model.
1 Ld (n) # load dataset
2 Split dataset (SpðnÞ) into three parts (test (nt), training (ntr) and validation (nv)), SpðnÞ = Sp (nt, ntr, nv)
3 Get original image (SpðnoÞ) and into threshold value(SpðnoThÞ), find biggest contour (Sp[BcðnoThÞ]), extreme point
(Sp½EpfBcðnoThÞg) and crop the image (SpðCiÞ)
SpðnoÞ = = threshold
then
SpðnoThÞ = = biggest contour; Sp½BcðnoThÞ
then
Sp½BcðnoThÞ = = extreme point; Sp½EpfBcðnoThÞg
then
Sp½EpfBcðnoThÞg = = SpðCiÞ
4 Apply step (iii) for all images.
SpðCiÞ = Spðno, no+1, no++Þ; #loop for all
5 SpðCnÞ == Szð224,224Þ #resize the all images
6 SzðCnÞ == AgfSzðCnÞg #apply augmentation for all images
Algorithm 1: Algorithm for image augmentation.
6 BioMed Research International
7. 2.6. Modified ResNet50 Model. Figure 8 shows the modified
ResNet50 model for brain tumor detection.
In the ResNet50, we have deleted and added few layers
which are named as the modified ResNet50 model. We have
added the dropout layer, flatten layer, dropout layer, and
dense layer to the last layer in ResNet50 and used a sigmoid
function to transform the model’s output into a probability
score. Table 2 shows the modified model architecture of
the layers used.
2.7. Model Training. CNN requires the augmentation of arti-
ficial data as a general procedure in the case of small data-
sets. However, MRI images involve abundant samples
including tumorous as well as healthy tissue or cells. Due
to this fact, most recent studies used the deep learning
approach to segment the brain without any exploration of
data augmentation [29] and used the augmentation tech-
nique in their work. However, this technique is inefficient
in this developed model. Generally, data augmentation is
of two types. This work is based on the influence of data aug-
mentation by enhancing the sample numbers utilizing rota-
tions. Such studies mainly consist of two variants as the
multiples of 90°
(such as 180°
, 270°
, and 360°
) corresponding
to the respective suggested methodology for rotations. On
the other hand, the second variant represents the three rota-
tion angles as uniform distribution in the form of an array
with equally spaced angles. Such a kind of rotation strategies
can enhance performance covering all the regions of pro-
vided image datasets. However, it reduces the sensitivity of
both variants in the challenge dataset. This provides mean
gain involving all types of rotations. The brain tumor detec-
tion technique needs such a kind of augmentation tech-
niques to estimate the information even from a blurred or
incomplete set of images. Figure 5 shows that the augmented
image of an original image changes according to the
required parameter. Table 3 shows the results obtained from
the training, test, and validation ratio of 75%, 20%, and 5%,
respectively, before augmentation [30]. Table 4 shows the
results obtained from the training, test, and validation ratio
of 75%, 20%, and 5%, respectively, after augmentation.
3. Results and Discussion
In this section, the results are shown in different forms in
matrix form which show true and false way images with
Step 1.
get the original image
Step 2.
find the biggest contour
Step 3.
find the extreme points
Step 4.
crop the image
Figure 2: Image setting of brain tumor.
(a) A tumor (no)
(b) A tumor (yes)
Figure 3: Resizing of image.
Original image
Figure 4: Data augmentation on the original image.
7
BioMed Research International
8. positive and negative after calculating the result for a param-
eter like accuracy, precision, sensitivity, and specification. In
the next section, the ROC graph means a ROC curve
(receiver operating characteristic curve) is a graph showing
the performance of a classification model at all classification
thresholds. Therefore, the ROC curve shows the proposed
model and work loss and accuracy in the curve form. Then,
in the tabular form, the result shows the parameter calcula-
tion. Moreover, according to that, the split best ratio for
the methodology is 75%-20%-5%.
3.1. Confusion Matrix. As the considered dataset is not large
enough, the next step is to perform the augmentation on the
complete images. This work involves the application of sev-
eral different augmentation types such as brightness, verti-
cal/horizontal flip, rotation, the random crop of size 128,
and many more described in detail in the earlier section.
The last stage involves the suggested deep learning model
involving the training of the modified ResNet50 with
Augemented images
Figure 5: Original image to augmented image.
Table 1: Parameter used in augmentation step.
S.
no.
Parameter
Value
taken
Remark
1.
Rotation_
range
15 By specifying the rotation_range, image is randomly rotated.
2.
Width_shift_
range
0.05
The width_shift_range is a floating point number; the image randomly shifts, either towards the left or the
right by a given value.
3. Rescale 1/255 Scales the image as per the given value
4. Shear_range 0.05
Shear transformation slants the shape of the image. This is different from rotation in the sense that in
shear transformation, we fix one axis and stretch the image at a certain angle known as the shear angle.
5.
Height_shift_
range
0.05 The image is shifted vertically instead of horizontally.
6.
Brightness_
range
[0.1, 1.5] The brightness_range specifies the range for randomly picking a brightness shift value from.
7.
Horizontal_
flip
True On a random basis, the image is horizontally flipped.
8. Vertical_flip True On a random basis, the image is vertically flipped.
Source data Target data
Source model
Source labels
Transfer
learned
knowledge
Target labels
Target model
Figure 6: Transfer learning approach.
8 BioMed Research International
9. Enhanced Watershed Segmentation (EWS) algorithm tech-
nique on the segmented images.
Hyperparameters have a vital role in model learning
due to the direct control of training behavior in the deep
learning model as well as have an impactful effect on the
performance of training. The appropriate hyperparameters
are critical for a successful deep learning training architec-
ture, while the erroneous ones can result in poor learning
accuracy. If the model’s learning rate is high, the frame-
work may collide, but if it is low, it may miss the required
data pattern [23]. A large number of eligible hyperpara-
meters makes it easier to find and organize trials with
large datasets. Figure 9 depicts the confusion matrix with
92% and 90% accuracy.
The optimal hyperparameters are chosen by observing
the clues available during training by monitoring valida-
tion/test loss early in the training. In this paper, signs of
underfitting or overfitting of the test or validation loss were
observed early in the training process to tune the hyperpara-
meters. This helps to tune the architecture and hyperpara-
meters with short runs of a few epochs.
Data1
Data2
Model1
Model1
Task 1
Task 2
Transfer learning
Knowledge transfer
Head
NewHead
Predictions1
Predictions2
Figure 7: Transfer learning approach for the proposed model.
Input
image
Sigmoid
ADAM
optimizer
Classification
output
+ +
Remove
last
layer
Dropout
layer
Hidden layers
ResNet50 model Added into ResNet50 model
Modified ResNet50 model
Flatten
layer
Dense
layer
Dropout
layer
Figure 8: Modified ResNet50 model for brain tumor detection.
Table 2: Modified model architecture.
Layer (type) Output shape Parameter #
ResNet50 (model) (None, 7, 7, 2048) 23,587,712
Dropout_3 (dropout) (None, 7, 7, 2048) 0
Flatten_2 (flatten) (None, 100,352) 0
Dropout_4 (dropout) (None, 100,352) 0
Dense_2 (dense) (None, 1) 100,353
Total parameters: 23,688,065
Trainable parameters: 100,353
Nontrainable parameters: 23,587,712
9
BioMed Research International
10. 1 import ResNet50
#Load the model
2 ResNet50 = = DLT {ResNet50(Ll)}; Ed (ResNet50)
#Change the required according to concept
3 Ed{ResNet50 + L(Do + F + Do + D) + apply act.(sigmoid) + apply Adam optimizer}
#Edit into model last layer (transfer learning concept (add 04 layers (dropout layer (Do), flatten layer (F), dropout layer (Do), dense
layer(D)), activate the sigmoid function, apply adam optimizer with values))
4 Generate Mt SET (Ep = 150, S/ Ep = 50, S/V = 25)
#Generate the model training [Mt] and set the model epochs, set per epochs (S), and set per validation (V)
5 Generate graphs of MAcc MLoss
#Plot the model accuracy (MAcc) and model loss (MLoss)
6 Confusion_mtx = confusion_mtx(prediction)
#Find out the accuracy of the model
Algorithm 2: Algorithm for computation of model loss and accuracy using ResNet50.
Table 3: Numerical results for different ratios of training, test, and validation sets before augmentation.
Case no.
Training set Test set Validation set
Numerical results
Accuracy Precision Sensitivity Specificity
% Image count % Image count % Image count Test Val. Test Val. Test Val. Test Val.
Case 1 75 193 20 50 5 10 0.67 0.85 0.85 0.63 0.91 0.77 0.93 0.73
Case 2 80 203 10 25 10 25 0.60 0.78 0.68 0.58 0.84 0.70 0.86 0.65
Case 3 75 193 15 35 10 25 0.83 0.81 0.91 0.74 0.69 0.91 0.79 0.91
Table 4: Numerical results for different ratios of training, test, and validation sets after augmentation.
Case no.
Training set Test set Validation set
Numerical results
Accuracy Precision Sensitivity Specificity
% Image count % Image count % Image count Test Val. Test Val. Test Val. Test Val.
Case 1 75 193 20 50 5 10 0.90 0.92 0.84 1 0.94 83 0.96 80
Case 2 80 203 10 25 10 25 0.66 0.84 0.74 0.62 0.90 0.76 0.92 0.71
Case 3 75 193 15 35 10 25 0.89 0.87 0.97 0.80 0.75 0.97 0.85 0.97
Confusion matrix
True
label
(0, ‘NO’)
(1, ‘YES’)
True
label
(0, ‘NO’)
(1, ‘YES’)
Predicted label
(0, ‘NO’) (1, ‘YES’) (0, ‘NO’) (1, ‘YES’)
Predicted label
0
1
2
3
4
5
Confusion matrix
30
16
5 0
1 4
1
3
30
25
20
15
10
5
Figure 9: Confusion matrix with 92% and 90% accuracy.
10 BioMed Research International
11. 3.2. ROC Graph. With regard to Study I and Study II ROC
curves at various edge settings, the TPR is compared to the
FPR. Affectability, survey, and chance of acknowledgment
are all terms used to describe the true positive rate [24, 31,
32]. The likelihood of feigned caution is frequently phrased
as 1 distinction. It can in like manner be considered as a plot
of the power as a component of the sort I slip-up of the deci-
sion guideline (when the show is resolved from just an illus-
tration of the general population, it will, in general, be
considered as assessors of these sums). Figure 10 shows the
suggested model’s DICE score training and validation accu-
racy curve. Our suggested model (modified ResNet50 with
Enhanced Watershed Segmentation (EWS) algorithm
approach) achieves an exceptional DICE score of 92% when
training on validation data and above 90 percent when train-
ing on training data.
Generally, the ROC curve summarizes the performance
by combining confusion matrices at all threshold values.
Confusion matrix goes deeper than classification accuracy
by showing the correct and incorrect (i.e. true or false) pre-
dictions on each class. The proposed model shows same
thing in confusion matrix as false and true values (as a true
label and predicted label in Figure 9).
3.3. Comparison Results. Table 5 shows the results in the
form of parameters like precision, sensitivity, specificity,
ROC curve
True
positive
rate
1.0
0.8
1.0
0.8
0.6
0.6
False positive rate
0.4
0.4
0.2
0.2
0.0
0.0
ROC curve
True
positive
rate
1.0
0.8
1.0
0.8
0.6
0.6
False positive rate
0.4
0.4
0.2
0.2
0.0
0.0
Figure 10: Plotting of ROC to present TPR and FPR.
Table 5: Resultant of accuracy and other parameters.
Metrics/methods TP TN FP FN
Precision
[TP/ TP + FP
ð Þ]
Sensitivity
[TP/ TP + FN
ð Þ]
Specificity
[TN/ TN + FN
ð Þ]
Accuracy
[TP + TN/ TP + TN + FP + FN
ð Þ]
Study I
(test set)
16 30 03 01 84% 94% 96% 90%
Study II
(validation set)
5 4 0 1 100% 83% 80% 92%
Table 6: Comparison between watershed-based modified ResNet50 model and Enhanced Watershed Segmentation (EWS) algorithm-based
modified ResNet50 model.
Parameter
Watershed using modified ResNet50
Enhanced Watershed Segmentation (EWS) algorithm
with modified ResNet50 model
Precision Sensitivity Specificity Accuracy Precision Sensitivity Specificity Accuracy
Test set 76% 86% 88% 82% 84% 94% 96% 90%
Validation set 92% 75% 72% 86% 100% 83% 80% 92%
Table 7: Comparison with existing model and techniques.
Model/author’s name Accuracy
Akkus et al. [19] 87.70%
Zhao and Jia [20] 81%
Pereira et al. [21] 88%
Brosch et al. [22] 84%
Proposed work 92%
11
BioMed Research International
12. and accuracy. So according to that, the highest accuracy of
the Enhanced Watershed Segmentation (EWS) algorithm
in the modified ResNet50 model is 92% in the validation
set and 90% in the test set. Table 6 shows the comparison
between the watershed algorithm-based modified ResNet50
model and the Enhanced Watershed Segmentation (EWS)
algorithm-based modified ResNet50 model. It is observed
that the best result is obtained from the Enhanced Water-
shed Segmentation (EWS) algorithm-based modified
ResNet50 model. Table 7 presents comparison results with
existing models and techniques.
3.4. Model Accuracy and Loss Diagram. In general, model
loss and model accuracy depend on the epoch size. We have
chosen an epoch size of 150, and the step per epoch is 06.
You should set the number of epochs as high as possible
and terminate training based on the error rates. The training
(epoch) is organized with batches of data. The optimization
of a single batch can decrease the accuracy of the other part
of the dataset and decrease the overall accuracy. If the test
accuracy starts to decrease by increasing epochs, then it
might be an indication of overfitting. If the validation error
starts increasing by increasing epochs, then it might be an
Accuracy
0.90
0.85
140
0.80
Epochs
Model accuracy
0.75
0.70
0 20 40 60 80 100 120
Train set
Val set
Figure 11: Graph between model accuracy and epochs.
Loss
1.6
140
Epochs
Model loss
0
1.4
1.2
1.0
0.8
0.6
0.4
0.2
20 40 60 80 100 120
Train set
Val set
Figure 12: Graph between model loss and epochs.
12 BioMed Research International
13. indication of overfitting. As long as the validation and train-
ing errors keep dropping, training should continue.
From the model accuracy (Figure 11), the accuracy of the
model is increased by an increase in the accuracy of the
period. It means that precision completely depends on the
model’s iteration or training size.
The cross-entropy function regulates the loss of a model
and evaluates the model loss. From the model loss
(Figure 12), it is seen that the model loss lowers with an
increase in time.
4. Conclusion
The goal of this work is to develop a novel hybrid deep
feature-based machine-learning model that uses the modi-
fied ResNet50 and the Enhanced Watershed Segmentation
(EWS) algorithm approach to detect brain tumors. Unlike
old methods, this proposed model improves performance
by advanced techniques such as deep learning and machine
learning algorithms. To extract deep features, the proposed
method combines two well-known and verified deep learn-
ing models (the modified ResNet50 and the Enhanced
Watershed Segmentation (EWS) algorithm). Deep features
combined with strategic integration, according to this study,
could improve brain tumor tissue pattern learning and clas-
sification. ResNet50, which comprises five convolutional
layers and three fully connected layers, can be used in this
fashion to extract different features for the study. The latter,
on the other hand, was a modified model. For calculating
brain tumor segment detection DICE score, postprocessing
stages have also been suggested. The suggested hybrid
CNN model has a 92% accuracy when compared to state-
of-the-art approaches. The deep learning CNN system mod-
ified ResNet50 with Enhanced Watershed Segmentation
(EWS) algorithm technique is recommended for segmenting
and automatically detecting brain tumors from MRI images.
In the future, the proposed model can be trained for
multimodal images. The time complexity is one of the
drawbacks of the proposed method which can be reduced
in the future. The modified ResNet50 model heavily
depends on batch normalization layers; this can be imple-
mented in the future.
Data Availability
The authors confirm that all relevant data are included in the
article and/or its supplementary information files. Addition-
ally, the derived data supporting the findings of this study
are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Major authors contributed in (i) the concept and design of
the work; (ii) the contribution of knowledge for the pro-
posed algorithm and model; (iii) the acquisition, analysis,
and interpretation of the data; (iv) the drafting of the article;
and (v) the revision of the article.
Acknowledgments
This research was supported by Taif University Researchers
Supporting Project Number TURSP-2020/306, Taif Univer-
sity, Taif, Saudi Arabia.
References
[1] I. Ali, W. A. Wani, and K. Saleem, “Cancer scenario in India
with future perspectives,” Cancer Therapy, vol. 8, no. 1,
pp. 56–70, 2011.
[2] World Health Organization, WHO Position Paper on Mam-
mography Screening, World Health Organization, Geneva,
2014.
[3] R. L. Siegel, K. D. Miller, and A. Jemal, “Cancer statistics,
2017,” American Cancer Society, vol. 67, no. 1, pp. 7–30, 2017.
[4] K. Roy, D. Banik, D. Bhattacharjee, and M. Nasipuri, “Patch-
based system for classification of breast histology images using
deep learning,” Computerized Medical Imaging and Graphics,
vol. 71, pp. 90–103, 2019.
[5] S. Sh and H. Tiwari, “A review paper on medical image pro-
cessing,” International Journal of Research Granthaalayah,
vol. 5, no. 4RACSIT, pp. 21–29, 2017.
[6] L. He, L. R. Long, S. Antani, and G. R. Thoma, “Histology
image analysis for carcinoma detection and grading,” Com-
puter Methods and Programs in Biomedicine, vol. 107, no. 3,
pp. 538–556, 2012.
[7] M. Veta, J. P. Pluim, P. J. Van Diest, and M. A. Viergever,
“Breast cancer histopathology image analysis: a review,” IEEE
Transactions on Biomedical Engineering, vol. 61, no. 5,
pp. 1400–1411, 2014.
[8] J. Ferlay, M. Colombet, I. Soerjomataram et al., “Cancer inci-
dence and mortality patterns in Europe: estimates for 40 coun-
tries and 25 major cancers in 2018,” European Journal of
Cancer, vol. 103, pp. 356–387, 2018.
[9] W. Zhi, H. W. F. Yueng, Z. Chen, S. M. Zandavi, Z. Lu, and
Y. Y. Chung, “Using transfer learning with convolutional neu-
ral networks to diagnose breast cancer from histopathological
images,” in International Conference on Neural Information
Processing, no. article 10637, 2017Springer Cham, 2017.
[10] S. SH, H. Tiwari, and D. Verma, “Deep learning technique to
improve breast cancer detection on screening mammogra-
phy,” Journal of Critical Reviews, vol. 7, no. 20, 2019.
[11] Y. M. George, H. H. Zayed, M. I. Roushdy, and B. M.
Elbagoury, “Remote computer-aided breast cancer detection
and diagnosis system based on cytological images,” IEEE Sys-
tems Journal, vol. 8, no. 3, pp. 949–964, 2014.
[12] A. Chan and J. A. Tuszynski, “Automatic prediction of tumour
malignancy in breast cancer with fractal dimension,” Royal
Society Open Science, vol. 3, no. 12, 2016.
[13] C. D. Lehman, R. D. Wellman, D. S. Buist, K. Kerlikowske,
A. N. Tosteson, and D. L. Miglioretti, “Diagnostic accuracy
of digital screening mammography with and without
computer-aided detection,” JAMA Internal Medicine,
vol. 175, no. 11, pp. 1828–1837, 2015.
[14] J. J. Fenton, S. H. Taplin, P. A. Carney et al., “Influence of
computer-aided detection on performance of screening
13
BioMed Research International
14. mammography,” New England Journal of Medicine, vol. 356,
no. 14, pp. 1399–1409, 2007.
[15] S. SH and H. Tiwari, “A determined way for detection of brain
tumor using morphological operations and PSVM classifier,”
IJSER, vol. 10, no. 11, 2019.
[16] E. K. Kim, H. E. Kim, K. Han et al., “Applying data-driven
imaging biomarker in mammography for breast cancer
screening: preliminary study,” Scientific Reports, vol. 8, no. 1,
p. 2762, 2018.
[17] A. Hamidinekoo, E. Denton, A. Rampun, K. Honnor, and
R. Zwiggelaar, “Deep learning in mammography and breast
histology, an overview and future trends,” Medical Image
Analysis, vol. 47, pp. 45–67, 2018.
[18] R. J. Burt, N. Torosdagli, N. Khosravan et al., “Deep learning
beyond cats and dogs: recent advances in diagnosing breast
cancer with deep neural networks,” The British Journal of
Radiology, vol. 91, no. 1089, p. 20170545, 2018.
[19] Z. Akkus, I. Ali, J. Sedlář et al., “Predicting deletion of Chro-
mosomal Arms 1p/19q in low-grade gliomas from MR images
using Machine Intelligence,” Journal of Digital Imaging,
vol. 30, no. 4, pp. 469–476, 2017.
[20] L. Zhao and K. Jia, “Deep feature learning with discrimination
mechanism for brain tumor segmentation and diagnosis,” in
International Conference on Intelligent Information Hiding
and Multimedia Signal Processing (IIH-MSP), pp. 306–309,
Adelaide, SA, Australia, 2015.
[21] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumor
segmentation using convolutional neural networks in MRI
images,” IEEE Transactions on Medical Imaging, vol. 35,
no. 5, pp. 1240–1251, 2016.
[22] T. Brosch, L. Y. W. Tang, Y. Yoo, D. K. B. Li, A. Traboulsee,
and R. Tam, “Deep 3D convolutional encoder networks with
shortcuts for multiscale feature integration applied to multiple
sclerosis lesion segmentation,” IEEE Transactions on Medical
Imaging, vol. 35, no. 5, pp. 1229–1239, 2016.
[23] H. M. Rai, K. Chatterjee, and S. Dashkevich, “Automatic and
accurate abnormality detection from brain MR images using
a novel hybrid UnetResNext-50 deep CNN model,” Biomedi-
cal Signal Processing and Control, vol. 66, article 102477, 2021.
[24] N. Bhargava, A. K. Sharma, A. Kumar, and P. S. Rathoe, “An
adaptive method for edge-preserving denoising,” in 2017 2nd
International Conference on Communication and Electronics
Systems (ICCES), pp. 600–604, Coimbatore, India, 2017.
[25] S. Cascianelli, R. Bello-Cerezo, F. Bianconi et al., “Dimension-
ality reduction strategies for CNN-based classification of histo-
pathological images,” in International Conference on
Intelligent Interactive Multimedia Systems and Services,
pp. 21–30, Springer, 2018.
[26] P. W. Huang and Y. H. Lai, “Effective segmentation and clas-
sification for HCC biopsy images,” Pattern Recognition,
vol. 43, no. 4, pp. 1550–1563, 2010.
[27] N. Sinha and A. G. Ramkrishan, “Automation of differential
blood count,” in Proceedings of the Conference on Convergent
Technologies for Asia-Pacific Region (TINCON ‘03), pp. 547–
551, Bangalore, India, 2003.
[28] C. Shih, L. Youchen, C. H. Chen, and W. C. Chu, “An early
warning system for hemodialysis complications utilizing
transfer learning from HD IoT dataset,” in 2020 IEEE 44th
Annual Computers, Software, and Applications Conference
(COMPSAC), Madrid, Spain, 2020.
[29] A. K. Sharma, A. Nandal, L. Zhou, A. Dhaka, and T. Wu,
“Brain tumor classification using modified VGG model-
based transfer learning approach,” in IOS Press, SOMET-
2021, Mexico, 2021.
[30] N. Y. Ilyasova, R. A. Paringer, A. S. Shirokanev, and N. S.
Demin, An Approach to Semantic Segmentation of Retinal
Images Using Deep Neural Networks for Mapping Laser Expo-
sure Zones for the Treatment of Diabetic Macular Edema,
Springer Science and Business Media LLC, 2022.
[31] A. K. Sharma, A. Nandal, A. Dhaka, and R. Dixit, “A survey on
machine learning-based brain retrieval algorithms in medical
image analysis,” Health and Technology, vol. 10, pp. 1359–
1373, 2020.
[32] A. K. Sharma, A. Nandal, A. Dhaka, and R. Dixit, “Medical
image classification techniques and analysis using deep learn-
ing networks: a review,” in Health Informatics: A Computa-
tional Perspective in Healthcare, pp. 233–258, Springer,
Singapore, 2021.
14 BioMed Research International