Working memory enhancement during early childhood based on the utilization of interactive gesture game-based learning technique

Indonesian Journal of Electrical Engineering and Computer Science
Vol. 21, No. 2, February 2021, pp. 768~775
ISSN: 2502-4752, DOI: 10.11591/ijeecs.v21.i2.pp768-775  768
Journal homepage: http://ijeecs.iaescore.com
Working memory enhancement during early childhood based
on the utilization of interactive gesture game-based learning
technique
D. Lakshmi1
, R. Ponnusamy2
1
Department of Computer Science and Engineering, Sathyabama Institute of Science and Technology, Tamilnadu, India
2
Department of Computer Science and Engineering, CVR College of Engineering, Telangana, India
Article Info ABSTRACT
Article history:
Received Feb 12, 2020
Revised Apr 5, 2020
Accepted Apr 18, 2020
The use of human computer interaction is considered to be the most
culturally and socially meritorious for the learning and playing activities of
children. In this paper, a human interaction recognition system (HIRS) that
includes gesture game-based learning is investigated for identifying its
suitability and applicability in stimulation of working memory and primitive
mathematical skills among the children in the early childhood period that
ranges from 5 and 8 years. In the proposed human interaction recognition
system, the hand gestures are facilitated by the user for the objective of
controlling the computer system based on the information extracted from the
user gestures. This proposed research was implemented in three phases using
a quasi-experimental design that in turn incorporates pre-test and post-test for
investigating the behavior of experimental and control group considered from
the respondents. In the first phase, the initial evaluation of the learner’s skill
is achieved. The second phase used the developed technology in order to
identify diversified parameters in different dimensions that contribute
towards the assessment of working memory and primitive mathematical
capabilities. Finally, the third phase is responsible for actual evaluation. In
the phases of evaluation, four working memory tests such as forward corsi
blocking-tapping test, backward corsi blocking-tapping test, forward digit
span test and backwardDigit Span test was conducted. In addition, the
evaluation was also conducted for assessing primitive mathematical skill of
children using TEDI-MATH. The results confirmed that gesture interactive
game-based learning (GIGL) used by the children exhibited a predominant
improvement in the working memory and primitive mathematical skills on
par with their usual school activities.
Keywords:
Corsi blocking-tapping test
Digit span test
Gesture interactive game-based
learning (GIGL)
Primitive mathematical skills
Working memory
This is an open access article under the CC BY-SA license.
Corresponding Author:
D. Lakshmi
Department of Computer Science and Engineering
Sathyabama Institute of Science and Technology
OMR, Chennai, Tamilnadu, India
Email: vlakshmidevan105@gmail.com
1. INTRODUCTION
From the recent past, Human Compute Interaction is considered as the potential field of research
which concentrates on the association between human individuals and technologies [1]. Human Computer
Interaction attempts at defining novel interacting paradigms with various strategies to understand the
methodologies that could be used by the humans for reliable communication with the computers [2]. These
new Human Computer Interaction models are suitable in a wide range of applications that applies to the field

Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 
Working memory enhancement during early childhood based on the utilization of interactive .... (D. Lakshmi)
769
of industry, research, education and entertainment [3]. The novel technologies and modern devices emerging
in the games industry in the recent years facilitate maximum level interactive experiences with maximized
natural interaction [4]. The capabilities facilitated by the new human computer models are comparatively far
more predominant when compared to the simple mouse, keyboard and game pad [5].
In particular, the new human computer interaction devices such as Wiimote and Kinect are
identified to play a key role in tracking the hands and body of the game players by recognizing appropriate
gestures and movements. These devices are also highly capable of ensuring a reliable and excellent natural
interactive experience [6]. They are responsible for providing a suitable environment in the development of
novel somato-sensory applications which enhance the degree of user motivation and immersion. They are
also responsible for enabling a high degree of fun games [7]. A human-computer interaction methodology
termed as Gesture Interactive Game-based Learning (GIGL) was developed in the recent past based on the
integration of core ideas and concept derived from the advent of novel interacting strategies, somato-sensory
devices and games [8]. This GIGL completely concentrates on the enhancement of the learning performance
based on the employment of interactive games. It wide opens diversified opportunities for acquiring and
learning complex contents through the utilization of novel and innovative paradigms identified in the recent
years [9].
For instance, GIGL facilitates a primitive foundation for the newly emerging models based on the
movements derived from the hand and body movements of the players [10]. GIGL techniques are particularly
concerned in the primary and secondary education for a diversified number of reasons. Initially, maximum
degree of natural interfaces is constructed through GIGL techniques [11]. Then, learners can use their body
as a tool in order to enhance their motivation and simultaneously reduce the degree of physical passivity. It
aids the teachers to supervise their children by solving specifically designed tasks and improving them based
on the feedback derived from the system in a real time scenario [12]. Most of study reports related to GIGL
confirmed the feasibility of enhancing the motor skills and learning capability with different age ranges.
However, the overall education of education facilitated through GIGL technologies not only need to
stimulate other types of capabilities rather than ensuring motor competencies and simple learning [13].
In this context, Executive functions refer to brain-based cognitive feature perceiving potential that
enables necessary self-regulation and thought in the children. These executive functions are the perception
capability generated completely based on the pre-frontal cortex region of the brain and aids in decision
making and goal achievement. This executive functions are categorized into three functional units based on
its region of origination from the brain. The first region is present in the pre-frontal cortex region and it plays
a vital role in activities verification and programming control. The second unit is present in the cortical
region for the purpose of post-control. It is responsible for receiving, storing and processing of stimulus or
information received in the brain. The third functional unit is present in the limbic part and it plays an anchor
role in the motivation and alert. Among the three functional units that generates executive functions of the
brain, the region present in the pre-frontal cortex region is considered to be the important unit based on the
study of Alexander Luria [14]. Further, the executive functions are classified into three types that correspond
to cognitive flexibility, inhibitory control and working memory. In particular, working memory is the system
of cognition that permits human beings to compute and recall a restricted degree of retained information
chunks that are present in the brain for a brief period. A plethora of studies have proved that working
memory is the central point of vitality in the process of knowledge acquisition. It confirmed its need for a
diversified number of complex cognitive abilities and tasks. In addition, working memory is considered as
better predicting parameter to quantify the success in academics rather than intelligence [15].
In this paper, Human Computer System based on GIGL is developed for recognizing the working
memory and mathematical skills of the children based on enforcement of working memory tests such as
forward Corsi Blocking-Tapping test, backward Corsi Blocking-Tapping test, Forward Digit Span test and
backwardDigit Span test. The hypothesis that quantifies the degree to which children acquire working
memory and mathematical skills through computer game play is verified experimentally through standard
tests. It focuses on the evaluation to assess fundamental mathematical skill of children using TEDI-MATH.
The experimental investigations are also conducted using pre-test and post-test for identifying the change in
working memory and mathematical skills of children after playing the hand gesture computer games.
Initially, Bull et al. [16] performed a longitudinal study that focused on the process of predicting the
association between executive functions during the early education stage of the childhood. It explored the
options that led to the superior performance of primary education involved in mathematics and reading
capabilities. This longitudinal study confirmed that an increase in the performance of executive functions
increases the performance of the children in reading and mathematics. Then, Geary and his colleagues
evaluated the association between the children’s performances in terms of mathematical tests and their
significances in attaining the tasks of processing speed and working memory [17]. They identified that
children with moderate performance in mathematics were rapid and more precise, when responding to

 ISSN: 2502-4752
Indonesian J Elec Eng & Comp Sci, Vol. 21, No. 2, February 2021 : 768 - 775
770
activities that highlights the needs for estimating retention, recovery and numerical sets determination with
respect to counting potential, linear estimation, and numerical information. They also proved that the
potential in realizing a numerical set was associated with the performance in the working visual-spatial
working mathematically and working memory skills.
Further, a longitudinal research was conducted to estimate the working memory, inhibitory control,
cognitive flexibility and Effiective Functions, since inhibitory control can play an anchor role in the
predictors of mathematical performance [18]. This longitudinal research made it evident that the working
memory of EF was better than the forecasting potentialities of Mathematics. The highlighted research is sub-
section of a group, who has evaluated and explored the vitalities of three EFs. Moreover, It is confirmed that
the working memory with respect to educational games improved the impact over the degree of working
memory with the predominant weight that aided in forecasting the failure or success of mathematical
perception in the near future.
2. RESEARCH METHOD
In this Hand Gesture Game-based Learning process, the occurrence of users and computers-based
interaction is completely through the natural movements as in the daily life. The hand gesture movements are
tracked by the utilization of somatosensory devices which possess the potentiality in scanning and compute
object localization, user’s body part or user’s body in the space under real time. This technology involved in
this learning process, completely concentrates on tracking the human movements with equal importance in
recognizing the head, hands, arms, face movements and the body as gestures. This technology plays a vital
role for enhancing the children to understand materials related to academics very easily. It facilitates the
experience of continuous development for the purpose of ensuring various channels for learning children. It
is derived from the innovation of novel low cost devices introduced by the Wiimote, Asus XtionPro and
Microsoft Kinect into the video game industry [19].
In particular, Wiimote is considered as the one-hand controller design which is similar to the
classical gamepad that incorporates a collection of infrared sensors and accelerometers. This Wiimote works
synergistically for permitting the user to manipulate and interact with entities based on pointing and gesture
recognition on the screen. It is used for diversified research tasks within the wide scope of children ages
ranging from senior children for the objective of training hand motor skills. It is used for helping children
with hyperactivity disorder, since they are deficit of attention. It is also used for demonstrating the
mechanism through which visual perception, motor control and hand-eye coordination could be simulated
based on the inclusion of interactive elements. Asus Xtiopro, Kinetic V1 and Kinetic V2 are very similar to
the Wiimote. Their capabilities of tracking are purely based on the range camera time-off light [20]. They
play an appropriate role in rebuilding the image depth associated with the capture volume for permitting the
system to track the joints of body in real time processing derived from more than single user. In specific,
Microsoft Kinect V2 comprises of Visual Gesture Builder and Kinect Studio V2 software packages for
satisfying the objective of natural gesture recognition. Kinect Studio V2 permits the option of information
recording attained by the sensor and the Visual Gesture Builder aids in defining new gestures in a visual,
agile and simple way [21]. The aforementioned software packages utilizes the merits of artificial intelligence
algorithms for perceiving and learning the properties of a gesture from a collection of samples which are
recorded and defined by the user previously [22]. The core benefit of the Kinect Studio V2 over the Wiimote
mainly lies in the absence of utilizing any key elements to hold [23].
In this gesture game, the hardware utilized comprises of a PC interconnected to Kinect and a TV
screen in which the applications are exhibited. This Kinect acts a somatotype device. Moreover, the learners
need to stand up in front of the screen from where they need to interact with applications as presented in
Figure 1.
The GIGL procedure consists of nine applications and each applications portrays a software
architecture specified in Figure 2. The main core of the software architecture is the Unity 3D application.
This Unity 3D application is considered as the most significant and potent graphics engines. It is run in a
cross platform environment that comprises of consoles, PC and mobile devices, since it is vitally used for the
development of video games. In this proposed project, two important characteristics are highlighted in order
to create an interactive 3D application for better convenience. The proposed project is capable in supporting
interactive communication based on the use of the Kinect. The system logic was potentially designed for
each individual action and is integrated with the graphics engine based on C# script programming language.
The MYSQL database used in the project stores different data derived from different experiences that
includes profile results and playing experience. The kind of interaction facilitated in this proposed
architecture is made very simple, since the system is developed for the utilization of very young children
ranging in the age group of 5-6 years. In this system, a small set of gestures such as select, drop and drag is

771
used. Moreover, the gestures in general are applied to each and every device and applications that could be
realized with and without issues. In addition, the particular characteristics of children phenotype that includes
the gesture recognizer size, fine motor skills, hand size, arms length and height are re-tuned. The complete
training process is incorporated based on the following steps such as i) the learners play several number of
times in front of the system based on the recorded series of examples, ii) it used Microsoft Kinect Studio V2,
iii) it utilized two sequences required for each specific individual gesture and iv) the effectiveness of the
recognition results are verified based on the training system.
Figure 1. The utilized interactive gesture game
The proposed system architecture is characterized based on the human expert which infers to the
system about the gesture moment through the complete timeline that tags the Visual Gesture construction
process that permits the system to be encoded and trained during the gesture in a file. This learning process is
also trained through the Adaboost algorithm, which aids the sensor to visualize any kind of generating
gesture.
The participants of this learning experiment were considered from four different classes in a primary
school of Puducherry, India. This research was approved by the specific institution in which the information
is collected and ensured for the participant safety after interacting with the instructors for the application of
inclusion and exclusion conditions. This research was conducted using 100 children, which included 6 and 7
years old level 2 preschoolers. In this research, two groups such as experimental (male-15, female-15) and
control group (male-14, female-16) is used for conducting research. This test was carried out from Febraury
2019 to September 2019. The activity experiment comprises of two blocks, in which the first block refers to
the initial demonstration where working methodology, previous game and technology presentation for
facilitating the technology adaptation. The block is required, since new technology and activities could
present complexities at these periods, since the children are considered in the step of incipient enhancement
of precise motor potentialities for the objective of attaining initial training.
The experimental individual, groups of five or pairs and experimental groups conducted for four
weekly sessions each comprising of forty five minutes in a particular area of the classroom. The complete
duration of the program was eight months, hence each student had attended thirty two sessions in a particular
class at the same school. In this gesture game period, the children finished all the ten games. Moreover, each
student completed a game in each and every session. The learning process was grouped into nine games and
activities with each individual activities were formulated for integrating the characteristics of learning games
and IPO models proposed in [24, 25]. In addition, the content used for instruction is linked to the
characteristics of game throughout the input stage for integrating the game and teaching material for learning
process.

 ISSN: 2502-4752
772
Figure 2. Proposed system architecture
3. RESULTS AND ANALYSIS
The statistical analysis of the proposed HIRS used for evaluating the parameters of working memory
and primitive mathematical skill is conducted using the version 23 of Statistical Package for Social Sciences
(SPSS). As aforementioned, initially two homogeneous groups were developed based on the investigated
variables of working memory and primitive mathematical skill is developed. The descriptive investigation
conducted with respect to working memory demonstrates that the mean and standard deviation corresponding
to the experimental group is 2.728 and 0.993 (Mean=2.728 and SD=0.993) respectively. Similarly, the mean
deviation and standard deviation related to the control group is 2.732 and 0.892 (Mean=2.732 and
SD=0.892). This descriptive analysis proved that the mean deviation and standard deviation estimated with
experimental group and control group are very close to one another. In addition, the Kruskal-Wallis test also
determined that are no significant differences between the experimental group and the control group under
working memory study variable( 524
.
0
2

 and 78
.
0

p ). The results made it evident that the groups
developed for experimenting the proposed HIRS with GIGL technology are comparable with the working
memory study variable as presented in Table 1 and 2.
Table 1. Differences in pre-test results conducted using the working memory study variable
Study variable Group Mean SD Rank 2

Working
Memory
Experimental Group 2.728 0.993 42.652
0.472 (non significant)
Control Group 2.732 0.892 18.231
Table 2. Differences in post results conducted using the working memory study variable

Working
Memory
Control Group 2.812 0.992 18.231
During the pre-test investigation, all the groups are determined to be enhanced with respect to
working memory skills. In the experimental group, the post-test scores (Mean=2.728 and SD=0.993) are
determined to be superior over the scores of the pre-test scores (Mean=2.728 and SD=0.993) generating
significant statistical minimization (Z = −4.78, p = 0.001) as portrayed in Table 3 and 4. In the control group,
the post-test scores (Mean=2.728 and SD=0.993) are similarly determined to be superior over the scores of
the pre-test scores (Mean=2.728 and SD=0.993) generating significant statistical minimization (Z = −4.78,
p = 0.001). This existence of statistical significant deviation between the experimental group is mainly
estimated due to the curriculum content, such that the participants gain enhancement in the prescribed

773
variable during the period of training. The presence of statistical significant deviation between the control
group is mainly estimated due to the educational intervention program used in the period of training. In this
context, the Kruskal-Wallis test was used for determining the differences between the experimental and
control groups in the post-test, which depicts that there is a statistically significant differencebetween the
experimental and control group in the working memory variable (χ 2 = 44.84, p = 0.001) and in the basic
mathematical skills variable (χ 2 = 41.65, p = 0.001).
Table 3. Differences in pre-test results conducted using mathematical ability as the study variable

Working
Memory
Control Group 2.813 0.912 19.124
Table 4. Differences in post results conducted using mathematical ability as the study variable

Working
Memory
Control Group 2.812 0.992 18.231
primitive mathematical skills. In the experimental group, the post-test scores (Mean=2.728 and SD=0.993)
are determined to be superior over the scores of the pre-test scores (Mean=2.728 and SD=0.993) generating
control groups in the post-test, which depicts that there is a statistically significant differences between the
Table 5. The pre-test results evaluated using U values of mann whitney test conducted using control and
experimental groups
Variable of study
Pre-tests
Control Group
Experimental
group
Working Memory
62.38 72.38
Table 6. Differences in post results evaluated using U values of mann whitney test conducted using control
and experimental groups
Variable of study
Pre-tests
Control Group
Experimental
group
Working Memory
69.46 74.96
primitive mathematical skills. In the experimental group, the post-test scores (Mean=2.728 and SD=0.993)
are determined to be superior over the scores of the pre-test scores (Mean=2.728 and SD=0.993) generating

 ISSN: 2502-4752
774
control groups in the post-test, which depicts that there is a statistically significant differences between the
4. CONCLUSION
This paper has developed and presented a innovative virtual interactive learning environment using
a gesture game for inducing working memory and primitive mathematical skills among the children in the
early childhood period. This interactive learning approach inherited a significant potential through which it
can be implemented individually either in small groups and pairs. It is thereby determined to be innovative,
since the majority of the programs implemented for educational technology are implemented individually.
This implementation of the learning program minimizes the required material resources and human
resources. This proposed approach also constructed a physical task session using IPO for the objective of
motivating the learning process, thereby the instructor can provide appropriate feedback through the behavior
of the individual participant. This motivation facilitated by the instructor acts as a tool for encouraging their
potential and creating interest in the learning content. The statistical investigation conducted using the data
derived through forward Corsi Blocking-Tapping test, backward Corsi Blocking-Tapping test, Forward Digit
Span test and backward Digit Span test exhibited a predominant improvement over theworking memory and
primitive mathematical skills among the children in the early childhood period. This statistical investigation
also proved that there exists a significant deviation between the mathematical skills and working memory
between the control and experimental control group.
REFERENCES
[1] B. C. Nelson et al., “Global channels of evidence for learning and assessment in complex game environments,”
British Journal of Educational Technology, vol. 42, no. 1, pp. 88-100, 2011.
[2] I. Ghergulescu and C. H. Muntean, “Motivation Monitoring and Assessment Extension for Input-Process-Outcome
Game Model,” International Journal of Game-Based Learning, vol. 4, no. 2, pp. 15-35, 2014.
[3] R. Bull and K. Lee, “Executive Functioning and Mathematics Achievement,” Child Development Perspectives,
vol. 8, no. 1, pp. 36-41, 2014.
[4] S. E. Gathercole et al., “Working memory in children with reading disabilities,” Journal of Experimental Child
Psychology, vol. 93, no. 3, pp. 265-281, 2006.
[5] I. C. Mammarella et al., “Math anxiety and developmental dyscalculia: A study on working memory processes,”
Journal of Clinical and Experimental Neuropsychology, vol. 37, no. 8, pp. 878-887, 2015.
[6] D. C. Geary, “Early Foundations for Mathematics Learning and Their Relations to Learning Disabilities,” Current
Directions in Psychological Science, vol. 22, no. 1, pp. 23-27, 2013.
[7] C. Wiklund-Hörnqvist et al., “Untangling the Contribution of the Subcomponents of Working Memory to
Mathematical Proficiency as Measured by the National Tests: A Study among Swedish Third Graders,” Frontiers
in Psychology, vol. 07, no. 1, pp. 22-36, 2016.
[8] S. Toll et al., “Executive Functions as Predictors of Math Learning Disabilities,” Journal of Learning Disabilities,
vol. 44, no. 6, pp. 521-532, 2010.
[9] F. F. Ng et al., “Inhibitory control in preschool predicts early math skills in first grade,” International Journal of
Behavioral Development, vol. 39, no. 2, pp. 139-149, 2014.
[10] J. Kray and N. K. Ferdinand, “How to Improve Cognitive Control in Development During Childhood: Potentials
and Limits of Cognitive Interventions,” Child Development Perspectives, vol. 7, no. 2, pp. 121-125, 2013.
[11] H. Hsiao and J. Chen, “Using a gesture interactive game-based learning approach to improve preschool children's
learning performance and motor skills,” Computers & Education, vol. 95, no. 1, pp. 151-162, 2016.
[12] C. Cheng and C. Su, “Game-based learning system for improving student's learning effectiveness in system
analysis course,” Procedia - Social and Behavioral Sciences, vol. 31, no. 1, pp. 669-675, 2012.
[13] H. Liang et al., “Hand gesture-based interactive puppetry system to assist storytelling for children,” The Visual
Computer, vol. 33, no. 4, pp. 517-531, 2016.
[14] A. R. Luria, “Disturbances of Higher Cortical Functions with Lesions of the Frontal Region,” Higher Cortical
Functions in Man, vol. 1, no. 1, pp. 246-365, 1980.
[15] E. Biddiss and J. Irwin, “Active Video Games to Promote Physical Activity in Children and Youth,” Archives of
Pediatrics & Adolescent Medicine, vol. 164, no. 7, 2007.
[16] R. Bull et al., “Short-Term Memory, Working Memory, and Executive Functioning in Preschoolers: Longitudinal
Predictors of Mathematical Achievement at Age 7 Years,” Developmental Neuropsychology, vol. 33, no. 3,
pp. 205-228, 2008.

775
[17] K. P. Raghubar et al., “Working memory and mathematics: A review of developmental, individual difference, and
cognitive approaches,” Learning and Individual Differences, vol. 20, no. 2, pp. 110-122, 2010.
[18] S. Toll et al., “Executive Functions as Predictors of Math Learning Disabilities,” Journal of Learning Disabilities,
vol. 44, no. 6, pp. 521-532, 2010.
[19] I. Friso-van den Bos et al., “Working memory and mathematics in primary school children: A meta-analysis,”
Educational Research Review, vol. 10, no. 2, pp. 29-44, 2013.
[20] D. Charles et al., “Game-based feedback for educational multi-user virtual environments,” British Journal of
Educational Technology, vol. 42, no. 4, pp. 638-654, 2011.
[21] R. Dallos and R. ie, “Becoming invisible: The effect of triangulation on children’s well-being,” Clinical Child
Psychology and Psychiatry, vol. 21, no. 3, pp. 461-476, 2015.
[22] B. C. Nelson et al., “Global channels of evidence for learning and assessment in complex game environments,”
British Journal of Educational Technology, vol. 42, no. 1, pp. 88-100, 2010.
[23] I. C. Mammarella et al., “Math anxiety and developmental dyscalculia: A study on working memory processes,”
Journal of Clinical and Experimental Neuropsychology, vol. 37, no. 8, pp. 878-887, 2015.
[24] R. Garris et al., “Games, Motivation, and Learning: A Research and Practice Model,” Simulation & Gaming,
vol. 33, no. 4, pp. 441-467, 2002.
[25] M. Prensky, “Digital game-based learning,” Computers in Entertainment, vol. 1, no. 1, p. 21, 2003.

Working memory enhancement during early childhood based on the utilization of interactive gesture game-based learning technique

More Related Content

Similar to Working memory enhancement during early childhood based on the utilization of interactive gesture game-based learning technique (20)

More from nooriasukmaningtyas (20)

Recently uploaded (20)

Working memory enhancement during early childhood based on the utilization of interactive gesture game-based learning technique