Comparative Analysis of ML and DL Models for Gender Prediction in Devanagari Script

Publication
doi
Preprint
doi
*correspondence: email@institution.edu
Comparative Analysis of ML and DL Models for Gender
Prediction in Devanagari Script
Preprint, compiled January 5, 2024
Nischal Lal Shrestha
Abstract
This study investigates the feasibility of identifying gender via personal names written in the Devanagari
script. While extensive research exists for languages like English, Russian, and Brazilian, little attention
has been given to low-resource language like Nepali. This study addresses this gap, aiming to enhance
gender classification in Nepali names. We compare traditional machine learning algorithms along
with advanced contextualized transformer architectures like variants of BERT (Bidirectional Encoder
Representations from Transformers) specifically fine-tuned for detecting gender among individuals with
Nepali names. Our experiments aim at exploring how effectively these techniques capture linguistic
nuances unique to the Nepali language and assess their overall performance in terms of accuracy,
precision, recall, and F1 score metrics. Both experiments reveal superior performance by the BERT
variants DeBERTa and DistilBERT, demonstrating the merits of advanced NLP techniques for gender
classification in Nepali. The study also show that the additional encoding of name endings (last
character of a name) improve the performance of traditional ML algorithms by 10%. Through this
work, we contribute to gender classification methodologies tailored specifically for the linguistic nuances
of the Nepali language.
1 Introduction
Gender detection has been a captivating research area
within AI. Detecting gender is a multifaceted process, often
involving innovative approaches using Natural Language
Processing (NLP) and Computer Vision (CV). Accurate
gender identification in domains like newspapers, social
sites, articles, etc provides valuable insights into existing
biases, representation, and diversity [1, 2]. Gender classifi-
cation is a complex task that requires careful consideration
of cultural, social and individual factors. Accurate gen-
der detection helps create targeted advertisements, gives
users personalized content, makes customer support faster,
and assists market researchers in understanding audience
segments. Inclusive and unbiased AI services require con-
tinuous improvements in reliable gender detection.
Many research has been conducted utilizing NLP [3, 4, 5],
CV [6, 7] for gender detection task. There are commercial
applications too which can predict gender based on the
name [8, 9, 10]. These commercial applications primarily
rely on dictionaries to predict the gender and are not freely
accessible for the community. Moreover, these commercial
applications do not have much information for low resources
language.
Although there has been many approach to detect the
gender using the name for languages like English, Russian,
Brazilian [3, 4, 5], very little or no has been research done
for low resource language like Nepali Name in Nepali script.
The unique linguistic characteristics and cultural nuances
of low-resource languages like Nepali remain largely unex-
plored in the context of gender prediction. In this paper,
we applied a range of machine learning and deep learning
algorithms to analyze 502,468 Nepali names written in the
Nepali script for gender detection. For the gender classi-
fication task of Nepali names, we trained traditional ML
algorithms, including Random Forest, K Neighbours Clas-
sifier, Decision Trees, Naive Bayes, Logistic Regression,
LDA, QDA, SVM, gradient boosting, and BERT-based
classifiers like DistilBERT and DeBERTa, on a dataset
comprising Nepali names written in the Nepali script, split
into an 80:20 train-test ratio.
2 Related Works
Computer Vision applications mainly use facial features
[11, 12, 13] to detect the gender of a person. The use
of Convolutional Neural Networks (CNNs) and other ad-
vanced algorithms allows for robust gender classification
based on facial attributes and patterns [14].
Identifying the gender of a person based on their name
is a common NLP task and can be accomplished using
techniques like gender pronoun usage, language models
and machine learning, gender specific name etc.
In [15] Goswami et al. conducted a gender detection exper-
iment on 9,660 gender-labeled blog posts from blogger.com.
Their model, incorporating features such as slang word
statistics and sentence length, achieved an impressive ac-
curacy of 89.3%.
Vashisth and Meehan [16] delved into gender inference using
NLP techniques, including bag of words, word embedding,
logistic regression, SVM, and Naive Bayes, using Twitter
data.
In the paper [4] the authors used a method utilizing three
types of features (word endings, character n-grams and
dictionary of names) combined within a linear supervised
model to detect the gender by full name for Russian lan-
guage. The paper shows that the proposed strategy is
highly successful as accuracy upto 96% is achieved.

Preprint – Comparative Analysis of ML and DL Models for Gender Prediction in Devanagari Script
2
Similarly, [5] implemented several ML algorithms to detect
gender of a name for Brazillian language. According to
[5] some models accurately predict gender in more than
95% of the cases. The paper also show that recurrent
models overcome the feedforward models in the binary
classification of gender.
In [3], Yifan Hu et. al. proposed a high accuracy character
level algorithm to detect the gender of a person using name.
The paper also show that using last name in addition to
the first name improve the performance of the model.
3 Dataset and Preprocessing
3.1 Nepali Names
Nepali names, unlike English names, present distinctive
challenges for machine learning algorithms due to their
unique linguistic and cultural attributes. For example,
when transliterating the English name "Albert Einstein"
into Nepali, it becomes एल्बर्ट आइन्स्टाइन. The Devanagari
script introduces different characters, illustrating the com-
plexities that machine learning algorithms must navigate
when handling Nepali names.
Furthermore, Nepali names often exhibit gender-specific
associations based on certain characters at the end. For
instance, names ending with "ा" (aa), "ी" (ii), "◌ू" (uu)
or "या" (yaa) are commonly linked to feminine gender.
Examples include "सीता" (Sita), "मन्जु" (Manju) and "रजनी"
(Rajni).
The gender association can be emphasized through the
presence or absence of specific characters. For instance,
"Ishani" (ईशानी) is feminine, while "Ishan" (ईशान) is mascu-
line, showcasing how the addition or removal of characters
conveys gender-specific information in Nepali names.
3.2 Dataset
The experimental dataset comprises a total of 502,468
Nepali names, with 235,870 marked as male and 266,598
as female. The dataset was acquired through web scraping
from the official website of Election Commission, Nepal[17].
The preprocessing of the scraped data involved several
steps to ensure its quality and reliability. First, faulty
names were systematically eliminated from the dataset.
Entries containing gibberish elements such as hyphens
(-), periods (.), or random numbers were identified and
removed. Names with obvious mistakes that were easily
observable were excluded from the dataset. It’s essential
to note that the original dataset remained unaltered for
reasons unrelated to the aforementioned issues.
To address the possibility of a single name being associated
with both genders, a frequency-based approach was imple-
mented to assign labels. In instances where a name had
entries for both male and female genders, the gender label
was determined based on the higher count. This approach
aimed to enhance the accuracy and consistency of gender
labeling within the dataset, acknowledging the potential
ambiguity of certain names across genders.
Table 1 presents a sample data of the overall dataset. The
table includes the first names in Nepali script along with
the corresponding counts of females and males, as well as
the gender classification.
Table 1: Sample Data
First Name #Female #Male Gender
रितेन्द्रक
ु मार 0 1 1
नित्रकला 1 0 0
बागुर 0 3 1
बेलसपुरा 26 0 0
पुलहोवा 0 1 1
The figure 1 visually represents the gender distribution
within the dataset, where ’0’ corresponds to female names
and ’1’ corresponds to male names. Notably, the bar graph
reveals that the number of occurrences for female names
(266,598) exceeds that of male names (235,870).
Figure 1: Gender Distribution of Names
Figure 2 depicts a histogram illustrating the frequency
distribution of names based on their respective lengths.
The histogram exhibits a right-skewed distribution. As
observed in the histogram , a significant proportion of
names predominantly falls within the length range of 3 to
11 characters. Moreover, there are very few names with
the name length greater than 14.

3
Figure 2: Distribution of Name Lengths
3.3 Tokenization
Tokenization in Natural Language Processing (NLP) is the
process of breaking down a text into smaller units known
as tokens. Typically, in NLP tasks, texts are broken down
into words. However, for this paper, names are tokenized
by breaking them into individual characters, as names are
often comprised of single words.
After tokenization, the CountVectorizer is employed for
feature extraction. The CountVectorizer from scikit-learn
is utilized to convert the tokenized names into a numerical
format suitable for machine learning models. This trans-
formation allows us to represent each name as a numerical
vector, capturing the relationships between individual char-
acters and providing a foundation for gender classification.
3.4 Class Weights
As the dataset is imbalanced, we utilize class weights when
finetuning pretrained models. Class weights enable adjust-
ing the loss computation to penalize false predictions for
minority classes more heavily compared to majority class.
Weight for the majority female name is 0.4691 and weight
for the minorty male name is 0.5309. This configuration
encourages the model to maintain a delicate equilibrium
balancing the learning rates for each class, ultimately bene-
fiting the final performance on the challenging imbalanced
dataset.
4 Models for Gender Detection
Traditional ML algorithms like like Random Forest , K
Neighbours Classifier, Decision Trees, Naive Bayes, Logistic
Regression, LDA, QDA, SVM, gradient boosting along with
BERT based classifier like DistilBERT and DeBERTa were
trained and evaluated for the task. There were 2 types
of method based on input of the traditional ML models.
These two types of method were trained separately based
on inputs and there result is compared.
• Experiment 1: Only first name was given as
input to model.
• Experiment 2: First name along with additional
encoding for name endings was given as input to
model. For instance, if the first name is एलिशा,
then additional encoding for name endings i.e "ा"
(ii) is also added to first name as input. So, the
input for this experiment becomes एलिशा + "ा".
Figure 3 illustrates the comprehensive flow of Experiment
1 involving traditional ML algorithms. The process begins
with tokenizing the input at the character level, followed
by vectorization. Subsequently, the model is trained based
on the chosen algorithm, leading to the classification of the
input name.
Figure 3: Experiment 1 Flow Diagram
Figure 4 illustrates flow of Experiment 2. It is similar
to the Experiment 1 except additional encoding is added
for the name endings. The input is is tokenized at the
character level, followed by vectorization. Then, the input
is forwarded to the choosen ML model, leading to the
classification of the input name.

4
Figure 4: Experiment 2 Flow Diagram
4.1 Decision Tree
Decision tree, a widely used supervised learning method
for tackling classification and regression problems, oper-
ates as a structured classifier. In this framework, internal
nodes correspond to features of a dataset, branches repre-
sent the decision-making process, and each leaf node indi-
cates the classification result. This approach involves split-
ting datasets into tree-like structures to facilitate decision-
making [18]. For our problem, we employed a Decision Tree
as a classifier. Our decision tree model, designed to pre-
dict the gender of a given name, utilizes the Gini impurity
function. The entire training set serves as the root, and
the character-level vector is recursively distributed based
on gender.
4.2 Random Forest
Random forests are a group of methods that involve creat-
ing an ensemble (or forest) of decision trees. These trees are
developed using a randomized version of the tree induction
algorithm. We implemented a Random Forest classifier
with the parameter n_estimators set to 100, specifying the
number of trees in the forest. Furthermore, we employed
the Gini impurity function to evaluate the quality of splits
within each decision tree.
4.3 LDA and QDA
Linear Discriminant Analysis (LDA) and Quadratic Dis-
criminant Analysis (QDA) are two fundamental classifica-
tion methods in statistical and probabilistic learning [19].
LDA assumes that features are normally distributed and
have a common covariance matrix for each class. It seeks
a linear combination of features to maximize the separa-
tion between class means while minimizing within-class
spread. On the other hand, QDA relaxes the assumption
of a common covariance matrix, allowing each class to have
its covariance structure.
4.4 Naive Bayes
The Naive Bayes Classifier relies on the Bayesian theo-
rem and is well-suited for scenarios with high-dimensional
inputs. Despite its simplicity, Naive Bayes has demon-
strated effectiveness, often outperforming more intricate
classification methods [20].
4.5 K Neighbors Classifier
It involves categorizing new examples based on the pre-
dominant category assignment of their closest counterparts
within a reference dataset. We used K-nearest neighbors
for binary classification. The classifier had 5 neighbors and
used a simple weight function.
4.6 Support Vector Machine
Support Vector Machine (SVM) stands as a widely adopted
Supervised Learning algorithm applied to both classifica-
tion and regression tasks, with a primary focus on Classifi-
cation problems in Machine Learning. The objective of the
SVM algorithm is to identify the optimal line or decision
boundary, effectively classifying n-dimensional space to
assign new data points to their respective categories. The
concept of a hyperplane is central to achieving the best
decision [18].
4.7 Logistic Regression
Logistic regression is a classification techniques which is
used to identify the most sutable model to establish a rela-
tionship between the dependent and independent variables.
In this study, the logistic regression classifier was imple-
mented with a stopping criteria tolerance of 0.0001 and
norm-L2 in the penalization.
4.8 Gradient Boosting
Gradient Boosting Classifier is a machine learning algo-
rithm used for classification tasks. It is a part of the
ensemble learning methods and is particularly powerful
for building robust and accurate predictive models. The
algorithm builds an ensemble of weak learners, typically
decision trees, and combines them to create a strong pre-
dictive model.
4.9 DeBERTa and DistilBERT
4.9.1 DeBERTa
DeBERTa (Decoding-enhanced BERT with Disentangled
Attention) is a Transformer-based neural language model
that aims to improve the BERT and RoBERTa models. It
introduces two novel techniques Disentangled Attention
which enhances the model’s ability to capture intricate
relationships between tokens by reducing attention entan-
glement, and Enhanced Mask Decoder which refines the

5
decoding mechanism for improved language understanding.
These mechanisms collectively contribute to DeBERTa’s en-
hanced performance in various natural language processing
tasks [21].
4.9.2 DistilBERT
DistilBERT is a compressed version of BERT (Bidirectional
Encoder Representations from Transformers), designed for
improved computational efficiency while maintaining con-
siderable performance. Knowledge distillation is performed
during the pre-training phase to reduce the size of a larger
BERT model by 40%. Developed by Hugging Face, Distil-
BERT uses knowledge distillation, learning from the larger
BERT model as its teacher. With a reduced number of
layers and parameters, DistilBERT achieves a streamlined
architecture, making it suitable for resource-constrained
environments [22, 23].
5 Performance Metrics
To evaluate the performance of the trained model we are
using 4 performance metrics namely Accuracy, Recall, Pre-
cision and F1 Score.
5.1 Accuracy
Accuracy is a number between 0 and 1. It measures the
overall correctness of a model. It is calculated as the ratio
of correctly predicted instances (sum of correct value) to
the total samples measured.
Accuracy = TP + TN
Total Predictions × 100%
5.2 Recall
Recall also known as sensitivity is the percentage of posi-
tivie samples that were correctly labelled.
Recall = TP
TP + FN × 100%
5.3 Precision
Precision also known as positive predictive value is the
percentage of samples that were correctly labelled as "pos-
itive" compared to all the samples that were labelled as
"positive".
Precision = TP
TP + FP × 100%
5.4 F1 Score
F1 scores is the harmonic mean of precision and recall. It
considers both precision and recall in a single value.
F1 Score = 2×Precision×Recall
Precision+Recall
In the above formulas,
• True Positives (TP) represent the instances that
are correctly labeled as positives by the model.
• False Negatives (FN) represent the instances
that are incorrectly labeled as negatives by the
model.
• True Negatives (TN) represent the instances
that are correctly labeled as negatives by the
model.
• Lastly, False Positives (FP) represent the in-
stances that are incorrectly labeled as positive by
the model.
6 Experimental Results
As we considered gender classification as a binary classi-
fication task, the input to the model was the first name
and output was gender. All the models were trained on
Google colab. For traditional ML models PyCaret was
used and for training DeBERTa and DistilBERT PyTorch
was used. For ML models default parameters as provided
by the PyCaret implentatioin was used expect for cross
validation where 5 fold cross validation is used. For Deep
Learning models following parameters were used.
• Train Size = 64, Valid Size = 32
• Epochs = 5, Learning Rate = 0.00001
• Loss = BCELoss, Optimizer = Adam
• Max length of input = 15
• weight decay=0.01
6.1 Reuslts for Experiment 1
Table 2 displays the results obtained from Experiment 1,
in which solely the given name served as input. Among
the BERT variants examined, DeBERTa and DistilBERT
surpassed conventional machine learning algorithms in per-
formance. Notably, DeBERTa achieved the greatest accu-
racy, reaching 0.8720, whereas DistilBERT trailed closely
behind. Regarding classical techniques, Random Forest
led the pack with an accuracy measure of 0.7682, whilst
Quadratic Discriminant Analysis (QDA) and Naive Bayes
delivered inferior results comparable to merely making a
random selection.
Table 2: Performance comparison of the algorithms on
Experiment 1
Model Name Accuracy Recall Precision F1
DeBERTa 0.8720 0.8710 0.8719 0.8714
DistilBERT 0.8699 0.8695 0.8694 0.8695
Random Forest 0.7682 0.7414 0.7588 0.7500
K Neighbors 0.7380 0.6860 0.7371 0.7107
Gradient Boosting 0.7276 0.6967 0.7153 0.7058
Decision Tree 0.7231 0.6754 0.7176 0.6958
Logistic Regression 0.7126 0.6830 0.6978 0.6903
SVM 0.7126 0.6961 0.6935 0.6942
LDA 0.7123 0.6843 0.6969 0.6906
Naive Bayes 0.5368 0.6200 0.5628 0.4784
QDA 0.5277 0.4981 0.6025 0.3943

6
6.2 Results for Experiment 2
Table 3 displays the results obtained from Experiment 2,
in which first name along with additional encoding for
name endings was provided as input. Adding additional
encoding result in better performance for traditonal ML
algorithm as around 10% in accuracy is seen for traditional
ML algorithm in experiment 2. Similarly to experiment
1, Random Forest performed better in the traditional ML
group with accuracy of 0.8525. For BERT variants, we did
not see an improvement in performance when additional
encoding is used compared to experiment 1 as the accuracy
reached to 0.8705 for DeBERTa and 0.8715 for DistilBERT.
This shows that the additional encoding of name endings
did not help in the performance of BERT variants.
Table 3: Performance comparison of the algorithms on
Experiment 1
Model Name Accuracy Recall Precision F1
DistilBERT 0.8714 0.8702 0.8715 0.8707
DeBERTa 0.8705 0.8701 0.8701 0.8701
Random Forest 0.8525 0.8516 0.8369 0.8442
K Neighbors 0.8437 0.8193 0.8431 0.8310
Logistic Regression 0.8283 0.8405 0.8027 0.8211
Gradient Boosting 0.8283 0.8596 0.7921 0.8245
LDA 0.8263 0.8526 0.7928 0.8216
SVM 0.8250 0.8589 0.7873 0.8216
Decision Tree 0.8112 0.7916 0.8030 0.7973
Naive Bayes 0.6724 0.3726 0.8404 0.5100
QDA 0.6156 0.3988 0.7898 0.4487
7 Limitation and Future Work
While conducting the study, several opportunities for ex-
tending and deepening the exploration emerged. However,
certain limitations were identified, which can be investi-
gated further:
• Like name endings, middle name play important
role in gender classification for Nepali name. For
example name with middle name like क
ु मार, राज,
लाल, प्रसाद, बहादुर, etc is conisdered masculine
whereas name with middle name like देवी, क
ु मारी,
माया, etc is considered feminine names. To improve
the performance of gender classification model fur-
ther investigation on the importance of middle
name could be considered.
• N-grams of the name endings could also be benifi-
cial along with the name endings. As in our study
we only considered the last character and not the
n-gram of endings like bigram or trigram of name
endings, we see a potential of improvement if we
consider n-grams too.
• A dictionary based approach can be considered
for common Nepali names for fast inference and
accurate results.
8 Conclusion
In conclusion, this study addressed the significant gap in
gender classification research for low-resource language
Nepali, specifically focusing on names written in the De-
vanagari Scrip. Moreover, the research focus on comparing
the performance of traditional machine learning algorithms
and advanced contextualized transformer architectures,
specifically fine-tuned variants of BERT, for gender classi-
fication in Nepali names. Utilizing an imbalanced dataset
with class weights assigned to handle the disparity between
the majority and minority classes, the study discovered
that DeBERTa outperformed other models in the task,
followed closely by DistilBERT. Among traditional algo-
rithms, Random Forest proved to be the most successful.
Although these findings offer meaningful contributions to
gender classification in the Nepali linguistic landscape, fur-
ther research remains necessary to explore the role of mid-
dle names, n-grams of name endings, and dictionary-based
approaches for fast and efficient inference. Continuous
advancements in reliable gender detection will undoubtedly
promote more inclusive and unbiased AI services, ensuring
fairness and equality in diverse applications.
References
[1] The Guardian. How we analysed 70m comments on
the guardian website. https://www.theguardian.
com/technology/2016/apr/12/how-we-analysed-
70m-comments-guardian-website, Year.
[2] Toptal. Is open source open to women.
https://www.toptal.com/open-source/is-open-
source-open-to-women, Year.
[3] Yifan Hu, Changwei Hu, Thanh Tran, Tejaswi Kasturi,
Elizabeth Joseph, and Matt Gillingham. What’s in a
name? – gender classification of names with character
based machine learning models, 2021.
[4] Alexander Panchenko and Andrey Teterin. Detecting
gender by full name: Experiments with the russian
language. volume 436, pages 169–182, 04 2014. ISBN
978-3-319-12579-4. doi: 10.1007/978-3-319-12580-0_
17.
[5] Rosana C. B. Rego, Verônica M. L. Silva, and Victor M.
Fernandes. Predicting gender by first name using
character-level machine learning, 2021.
[6] Vikas Sheoran, Shreyansh Joshi, and Tanisha R.
Bhayani. Age and Gender Prediction Using Deep
CNNs and Transfer Learning, page 293304. Springer
Singapore, 2021. ISBN 9789811610929. doi: 10.1007/
978-981-16-1092-9_25. URL http://dx.doi.org/10.
1007/978-981-16-1092-9_25.
[7] Majid Farzaneh. Arcface knows the gender, too!, 2021.
[8] Genderapi. https://genderapi.io/, .
[9] Genderize. https://genderize.io/, .
[10] Gender-api. https://gender-api.com/, .
[11] Ke Zhang, Ce Gao, Liru Guo, Miao Sun, Xingfang
Yuan, Tony X. Han, Zhenbing Zhao, and Baogang

7
Li. Age group and gender estimation in the wild with
deep ror architecture, 2017.
[12] Anand Venugopal, Yadukrishnan V, and Remya
Nair T. A svm based gender classification from
children facial images using local binary and non-
binary descriptors. pages 631–634, 03 2020. doi:
10.1109/ICCMC48092.2020.ICCMC-000117.
[13] Olarik Surinta and Thananchai Khamket. Gender
recognition from facial images using local gradient
feature descriptors. pages 1–6, 10 2019. doi: 10.1109/
iSAI-NLP48611.2019.9045689.
[14] Ahmad B. Hassanat, Abeer Albustanji, Ahmad S.
Tarawneh, Malek Alrashidi, Hani Alharbi, Mohammed
Alanazi, Mansoor Alghamdi, Ibrahim S Alkhazi, and
V. B. Surya Prasath. Deep learning for identifica-
tion and face, gender, expression recognition under
constraints, 2021.
[15] Sumit Goswami, Sudeshna Sarkar, and Mayur Rustagi.
Stylometric analysis of bloggers’ age and gender. 01
2009.
[16] Pradeep Vashisth and Kevin Meehan. Gender classi-
fication using twitter text data. pages 1–6, 06 2020.
doi: 10.1109/ISSC49989.2020.9180161.
[17] Voter list database - election commission of
nepal. https://election.gov.np/np/page/voter-
list-db.
[18] D. Kamelesun, R. Saranya, and P. Kathiravan. A
benchmark study by using various machine learning
models for predicting covid-19 trends, 2023.
[19] Benyamin Ghojogh and Mark Crowley. Linear and
quadratic discriminant analysis: Tutorial, 2019.
[20] Vikramkumar, Vijaykumar B, and Trilochan. Bayes
and naive bayes classifier, 2014.
[21] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and
Weizhu Chen. Deberta: Decoding-enhanced bert with
disentangled attention, 2021.
[22] Victor Sanh, Lysandre Debut, Julien Chaumond, and
Thomas Wolf. Distilbert, a distilled version of bert:
smaller, faster, cheaper and lighter, 2020.
[23] Hugging Face. Distilbert documentation, 2023. URL
https://huggingface.co/docs/transformers/
model_doc/distilbert.

Comparative Analysis of ML and DL Models for Gender Prediction in Devanagari Script

More Related Content

More from Nischal Lal Shrestha (6)

Recently uploaded (20)

Comparative Analysis of ML and DL Models for Gender Prediction in Devanagari Script