Using Gamification to Improve the Quality of Educating Children with Autism

Uso de la gamificación para mejorar la calidad educativa de los niños con autismo

  • Azadeh Afrasiabi Navan Department of computer engineering Imam Khomeini International University Qazvin
  • Ali Khaleghi Department of computer engineering, Imam Khomeini International University Qazvin, Iran
Palabras clave: autism spectrum disorders, recognising emotional states, game, gamification (en_US)
Palabras clave: trastornos del espectro autista, reconocer los estados emocionales, juego, gamificación (es_ES)

Resumen (en_US)

This study was conducted with a pre-test–post-test design with a control group in which participants were divided into two experimental (ten persons) and control (ten persons) groups. The Smile 1 intervention took place in 16 sessions of 30 min, which consisted of eight sessions of 1 h owing to the time spent by the parents of the participants in the study. Additionally, a Child Behaviour Check List (CBCL) was used to measure the effectiveness of this interventional game on reducing the behavioural problems of children that participated. The questionnaire consists of 113 questions in which 13 different behavioural factors are identified in the children. For this purpose, pre-test and post-test stages were performed for both groups. In fact, due to time constraints for holding intervention sessions, it was expected from the outset that significant changes in participant behaviour during the two months of interventions would not be observed.

Resumen (es_ES)

Este estudio se realizó con un diseño pre-test-post test con un grupo de control en el que los participantes se dividieron en dos grupos experimentales (diez personas) y de control (diez personas). La intervención de Smile 1 se llevó a cabo en 16 sesiones de 30 minutos, que consistieron en ocho sesiones de 1 hora debido al tiempo que pasaron los padres de los participantes en el estudio. Además, se utilizó una Lista de control de la conducta infantil (CBCL) para medir la eficacia de este juego intervencionista en la reducción de los problemas de conducta de los niños que participaron. El cuestionario consta de 113 preguntas en las que se identifican 13 factores conductuales diferentes en los niños. Para ello, se realizaron etapas de pre-test y post-test para ambos grupos. De hecho, debido a las limitaciones de tiempo para realizar las sesiones de intervención, se esperaba desde el principio que no se observaran cambios significativos en el comportamiento de los participantes durante los dos meses de las intervenciones.

Descargas

La descarga de datos todavía no está disponible.

Referencias

Alves, S., Marques, A., Queirós, C., Orvalho, V. (2013). LIFEisGAME Prototype: a serious game about emotions for children with autism spectrum disorders. PsychNology Journal, 11(3).

Ashayerih, H., Esteki, M. (2017). The Effect of Computer Games on Improving Student’s Critical Thinking. Advances in Cognitive Science, 19(1), 14-24.

Bhatt, S. K., De Leon, N. I., & Al-Jumaily, A. (2014). Augmented reality game therapy for children with autism spectrum disorder. International Journal on Smart Sensing & Intelligent Systems, 7(2). https://doi.org/10.21307/ijssis-2017-668

Bölte, S., Feineis-Matthews, S., Leber, S., Dierks, T., Hubl, D., Poustka, F. (2002). The development and evaluation of a computer-based program to test and to teach the recognition of facial affect. International Journal of Circumpolar Health, 61(sup2), 61-68. https://doi.org/10.3402/ijch.v61i0.17503

Christinaki, E., Vidakis, N., Triantafyllidis, G. A. (2014). A novel educational game for teaching emotion identification skills to preschoolers with autism diagnosis. Comput. Sci. Inf. Syst., 11(2), 723-743. https://doi.org/10.2298/CSIS140215039C

Deterding, S., Dixon, D., Khaled, R., Nacke, L. (2011b). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (pp. 9-15). ACM. https://doi.org/10.1145/2181037.2181040

Deterding, S., Sicart, M., Nacke, L., O'Hara, K., Dixon, D. (2011a). Gamification. using game-design elements in non-gaming contexts. In CHI'11 extended abstracts on human factors in computing systems (pp. 2425-2428). ACM. https://doi.org/10.1145/1979742.1979575

Levy, S., Mandell, D., Schultz, R. (2009). Autism. Lancet, 374(9701), 1627-1638. https://doi.org/10.1016/S0140-6736(09)61376-3

Mantziou, O., Vrellis, I., Mikropoulos, T. A. (2015). Do children in the spectrum of autism interact with real-time emotionally expressive human controlled avatars? Procedia Computer Science, 67, 241-251. https://doi.org/10.1016/j.procs.2015.09.268

Mera-Paz, J. A. (1). Gamificación una estrategia de fortalecimiento en el aprendizaje de la ingeniería de sistemas, experiencia significativa en la Universidad Cooperativa de Colombia sede Popayán. Revista Científica, 3(26), 3-11. Recuperado a partir de https://revistas.udistrital.edu.co/index.php/revcie/article/view/11085

Miranda, J. C., Fernandes, T., Sousa, A. A., Orvalho, V. (2011). Interactive technology: teaching people with autism to recognize facial emotions. Autism Spectrum Disorders-From Genes to Environment, 299-312.

Qin, S., Nagai, Y., Kumagaya, S., Ayaya, S., Asada, M. (2014). Autism simulator employing augmented reality: A prototype. In 4th international conference on development and learning and on epigenetic robotics (pp. 155-156). IEEE. https://doi.org/10.1109/DEVLRN.2014.6982972

Rovai, A., Ponton, M., Wighting, M., Baker, J. (2007). A comparative analysis of student motivation in traditional classroom and e-learning courses. International Journal on E-learning, 6(3), 413-432.

Silver, M., Oakes, P. (2001). Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism, 5(3), 299-316. https://doi.org/10.1177/1362361301005003007

Smith, S. D., Grigorenko, E., Willcutt, E., Pennington, B. F., Olson, R. K., DeFries, J. C. (2010). Etiologies and molecular mechanisms of communication disorders. Journal of developmental and behavioral pediatrics: JDBP, 31(7), 555. https://doi.org/10.1097/DBP.0b013e3181ee3d9e

Tanaka, J. W., Wolf, J. M., Klaiman, C., Koenig, K., Cockburn, J., Herlihy, L., Kaiser, M. D. (2012). The perception and identification of facial emotions in individuals with autism spectrum disorders using the Let’s Face It! Emotion Skills Battery. Journal of Child Psychology and Psychiatry, 53(12), 1259-1267. https://doi.org/10.1111/j.1469-7610.2012.02571.x

Willis, C. (2010). Teaching young children with autism spectrum disorder. ReadHowYouWant. com.

Cómo citar
Afrasiabi Navan, A., & Khaleghi, A. (2020). Uso de la gamificación para mejorar la calidad educativa de los niños con autismo. Revista Científica, 37(1), 90-106. https://doi.org/10.14483/23448350.15431
Publicado: 2020-01-01
Sección
Educación Científica

Introduction

The use of technology and new teaching methods is increasing globally to help improve the learning process and computer games and the concept of gamification play an important role in this field. Gamification specifically refers to the use of game elements and thinking in areas that do not have a game-like nature to enhance the user experience and user motivation. (Deterding, et al. 2011a)

Today, people with special and exceptional needs are of considerable concern. One of the most common neurodegenerative disorders is autism spectrum disability, which occurs in the first three years of life, and it has been reported in at least 60 out of 10,000 cases in developed countries. According to Autistic Spectrum Australia, three main areas are affected by autism: 1) disability in communicating, 2) disruptions in social interactions and 3) repetitive interests, activities and behaviours that involve a disability in communication as one of the most important behaviours (Qin, et al. 2014).

This disorder has a range of known types that are of the lowest severity, including Asperger's syndrome/high-functional autism, the classic type of autism, Pervasive Developmental Disorder Not Otherwise Specified (PDDNOS), Rett Syndrome, which is the most acute type of autism and is more common in girls than in boys, and Childhood Disintegrative Disorder.

According to studies, people with Asperger have high-performance, good verbal skills and moderate to high IQs and their only basic problem is social interactions and the correct use of language and theology. Accordingly, the basic training on this spectrum will lead to significant advances in most cases. In the classical type of autism, the patient may have a very high to very low performance. In the third type of this spectrum, there are people who have some and not all the features of classical autism (Levy, Mandell, and Schultz, 2009). It is not so severe that the person is in the classical class of autism but not as mild as the Asperger group. For the last two types, people with moderate to very low levels of activity have the most severe types of autism. Therefore, training of this group of autistic individuals has a very low impact and, in most cases, is unworkable. (Deterding, et al. 2011b)

Therefore, although a specific set of symptoms is noted for autistic people, each patient has a specific combination of symptoms with varying severity, and no two patients are exactly the same. From this perspective, when teaching children with autism spectrum disorders, implementation of an educational method adapted to the cognitive abilities of each child is extremely important.

However, research indicates that regular and traditional training is becoming increasingly informal, social, dynamic and involves more creative learning environments with the use of game elements and mechanics. As a result, the level of success, motivation, excitement, concentration and attention will increase in students and ultimately independent children will have improved learning.

The three main autism disturbances previously expressed include problems with emotional processing and the use of several non-verbal behaviours simultaneously, such as eye to eye contact, facial and emotional expressions and body movements and gestures during social interactions. These factors reflect the disturbances in the production of emotional states and the perception of the rules of those states, which are the two main elements in the processing of emotions. Faces are the most important element for expressing feelings and plays a significant role in verbal and non-verbal communications. However, communication deficits cause unwanted and frequent behavioural problems in these children. According to the theory of mind, many of the problems with autism in detecting emotions are due to their lack of understanding of the others views and the perception of the fact that people have different perspectives. From this perspective, paying attention to this disorder and trying to improve it in children with autism is very important as technology has recently played a significant role in teaching facial expressions and understanding the emotions of children with autism.

Thus, the aim of this study was to evaluate the effectiveness of gamification for improving the quality of education in autistic children; for a more accurate examination and because of the high importance of this disorder, the main focus of the research was on emotional states recognition. Thus far, many research studies and activities have been conducted both using common and traditional teaching methods and using modern methods and new technologies, which will be addressed in the coming sections.

Most of the studies performed in this area have weaknesses in that they either diminish or fade from comprehension. Indeed, while existing interventions can have a considerable impact on teaching specific behaviours to those with autism, the problem is that in many cases, such activities cannot be used comprehensively and generally and should be tailored to each behaviour.

All of these shortcomings have led us to take a rather small step to educate correct, timely and easy methods for those with autism to recognise emotional states, which is among the most fundamental problems in this group of people.

Related work on recognising emotional states

As previously mentioned, there are many studies and activities on identifying the states of emotion in people with autism, and we will discuss some of them in this section.

One of the computer-based programmes used to evaluate and educate different facial emotions to children with autism was designed by Bölte et al. (2002) In their research, the development and evaluation of a computer program for training and examining the ability to detect facial expressions was addressed. In this programme, approximately 1000 pictures of the faces of adult males and females were collected that had different emotional states and ethnicities (Tanaka, et al. 2012).

Additionally, according to previous studies, only the area of the eyes was isolated in each of the images to examine two different levels of facial recognition. In the second step, all images were entered into the computer program, and the seven main senses defined by Ekman and his colleagues (happiness, sadness, anger, hatred, fear, surprise and neutral state) were taken into consideration. These two steps are shown in Figure 1.

A computer-based programme for measuring facial recognition with focus on the eye area and then on the entire face based on the seven emotions defined by Ekman.

Figure 1: A computer-based programme for measuring facial recognition with focus on the eye area and then on the entire face based on the seven emotions defined by Ekman.

To evaluate the effectiveness of the programme, behavioural scales were studied before and after the programme, and the results of this programme indicated that besides homogeneity and stability concerns, facial and eye examinations are reliable tools for assessing facial recognition.

LIFE is GAME is a modular interactive learning system. This project is a serious game for learning facial emotions based on the interaction between humans and 3D avatars, and it uses real-time synthesis and automatic analysis of facial expressions. With its play structure, the system uses various activities to understand emotions, their causes and consequences and facial expressions tailored to any sense. This project has a Lego structure. (Mantziou, Vrellis, and Mikropoulos, 2015); (Christinaki, Vidakis, and G. Triantafyllidis, 2014).

This game has the following training modes: 1) Face Detection (Figure 2-a), 2) Creating a Face (Figure 2-b) where the child must act and create the desired emotional state, 3) Be your avatars! (Figure 2-c) in which the child, in addition to recognising the emotional state, must create the desired state of the programme in his / her own face, and 4) Experience a story! (Figure 2-d) in which the child must learn emotional states in real social situations.

LIFE is GAME 4 phases.

Figure 2: LIFE is GAME 4 phases.

Emotion Trainer is a multimedia computer program run in five different sections. The purpose of this study was to evaluate the effect of the Emotion Trainer’s computer intervention on the ability of high-functional autistics or Asperger patients to recognise emotional states. Each page of the programme displays a digital image of a face, a scene or an object and a short text that asks for that section with two or four response buttons. (Silver and Oakes, 2001). The user needs to answer 20 items to proceed to the next level. As an example of the programme, some pages is shown in Figure 3.

A page from the Emotion Trainer game.

Figure 3: A page from the Emotion Trainer game.

When choosing the correct answer to the questions, the child receives the ‘Bravo’ message, whereas the wrong answer immediately results in the message ‘try again’ and the programme helps the child with a small point of reference. When choosing the second wrong answer, the child will receive another try again message, and a more accurate guide will be given. This will continue until the child finds the correct answer.

Based on studies, in a wide range of activities, static images have been used in the field of emotional and facial recognition training for patients with autism, which in many cases due to the fact that a child with autism (of course on severe levels) has no vision or background of these images, the training is not performed properly. Additionally, most of the work done to understand the diagnosis of emotional states for children with autism is only designed for a specific group or age of children.

Additionally, a majority of the research and activities performed are focused on games that are designed to educate the emotional state of children with autism. Although these games have been successful in some cases, in many cases, they are only suitable for use with patients with high-functional autism owing to the complexity of the tasks and the game stages. However, the problem of detecting emotional states in moderate and severe autistics is more important. As we have observed, in many games designed in this field, various elements of the game have been used to encourage, provide feedback, and increase motivation in the player, which has been welcomed in some cases. However, in most cases, some of these elements in the games were provided in ways that the child with autism did not understand them and thus had no effect on the children.

In fact, the designs of most of these games are done without close attention to the children needs and their limitations and capabilities, while the requirements, limitation and abilities of people with autism are very different from those with normal growth and should be give special attention.

Finally, in Iran, according to our knowledge, the research and activities carried out in this field in practice or via interventions are very limited and in most cases irrefutable for various reasons, and this issue show the important need for educating autistics using appropriate training methods.

Research model

Investigating autism spectrum disorders, the specific needs of people with autism and their capabilities, and most importantly, the careful examination of past studies led us to determine that educating this spectrum of people, especially at an earlier age, and considering the specific needs of this group can have a positive effect on their growth and development. Thus, if we make small steps in the field of effective education for autistic people, it will be useful for future generations to provide more complete activities (Willis, 2010).

According to reviews, as presented in the previous section, teaching autistic individuals via visual means has had more positive outcomes than other methods. As a result, using computers, games and computer programs is a good choice for educating and motivating these individuals. Thus, the use of computer games and the concept of gamification for teaching children with autism has had a positive impact based on studies performed in the past.

The main purpose of this study was to evaluate the effectiveness of gamification on educating children with autism. Evaluating the effectiveness of gamification when training people with autism requires an educational game that can be used in the context of gamification. Because the range of disturbances and problems is very wide and different in the field of autism, to examine facial expression recognition more precisely, we focused on this area in this study. Thus, the educational game teaches different emotional states to children with autism using gamification concepts.

Meanwhile, the concept of ‘effectiveness’ in this study means that at the outset, does the child have an atypical ability to communicate, as determined by the concepts of gamification, computer games and training emotional states? Based on research, this connection exists in many cases. In the study lowing, does the designed game have a low but positive impact on the ability of autistic children to detect emotional states or not?

Owing to all these limitations and studies conducted in this field, the result was that the game should be tailored specifically to the needs of autistic children, and using the concept of gamification, for training emotional states to children with autism.

This educational game should consider the emotional state of children with autism in all three mild, moderate and severe cases. As a result, to maintain the comprehensiveness of the game for all spectra, the design of this game should be in accordance with the needs, limitations and abilities of children with autism. In this way, trainings should not be so complex and difficult that the child cannot understand it, and it should not be so simple that it does not require any effort. In the past, training has been performed in the context of serious games, while in this study, the goal was to design an educational game rather than a game in which one has to play out the elements of the game, such as a positive and negative feedback system, scoring, etc. Obviously, the two concepts of serious games and gamification are not separate, but the concept of gamification is comprehensive in a particular type of game, for example, serious games. However, unlike many games designed in this field, the use of elements in our developing game will be completely understandable for children with autism and is expected to have a positive effect on child performance.

Thus, in summary, this research has a general purpose and some sub-goals. As the next priority, the secondary goals are improving the ability of recognising emotional states of others in autistic patients, designing an educational game using the concept of gamification specifically for autistic children with particular attention to their needs and abilities.

How we achieve these goals will be described in the next section.

Methodology

Smile1’ game structure

To better measure the issue of diagnosing emotional states in children with autism and according to time constraints, it was decided to study four basic emotions of ‘happiness, sadness, anger and fear’ in this game. Initially, the four senses are taught to the child in the form of social stories, and subsequently, the child is asked to determine the facial expressions related to each part of the story in the form of questions.

The game ‘Smile 1’ was created and manufactured using the Construct 2 game engine. Construct 02 is a great engine for building HTML and two-dimensional games, which was developed by Scirra.

From the very beginning of designing this game, very important points have been taken into consideration, and in fact, considering these points, we made this game specifically suitable for children with autism. One of the points is an attractive, yet simple interface design for children with autism. For this reason, based on research on the characteristics of autistic children, to design a fun and entertaining environment for children, there are no harsh or nasty sounds in this game. This is because autistic children are sometimes more sensitive to hearing and sight than normal people, and intense and nasty sounds, light and colours can harass them. Additionally, because of the focus and attention deficits of these children, the gaming environment should be designed in a relaxed and non-intrusive way. This is especially important during the early stages of the game. For this reason, in the design of the game ‘Smile 1’, we observed this during different stages and sections of the game.

In the game ‘Smile 1’, two cartoon characters of a girl and boy were used. The reason for this choice is to provide familiarity of the children with male and female genders throughout the game. For this reason, with the guidance of autistic experts, these cartoon characters were designed in exactly the same shape and format so that the child with autism would not be confused when identifying their characteristics from the very beginning. The ‘girl’ character in all sections has a scarf because, according to experts, children with autism only identify women as wearing scarves or tents (in Iran). On the basis of equal figures, four senses were designed on the faces in a way that is understandable for these children. You can observe these four modes on the first page of the game in Figure 4:

Four emotional states in the game ‘Smile 1’ on the first page of the game.

Figure 4: Four emotional states in the game ‘Smile 1’ on the first page of the game.

In the game entry section, we attempted to create familiarity with the faces by showing these characters to the child. It also includes childish music that does not hurt the autistic children’s hearing and is interesting and exciting. In figure 5, you can see the login screen:

Log into the game ‘Smile 1’.

Figure 5: Log into the game ‘Smile 1’.

For example, if the child chooses a ‘happy’ feeling, as shown in Figure 6, the following pictures are shown from left to right, which, by showing pictures, the social story of that stage is read to the child.

Learning the ‘Happiness’ feeling in ‘Smile 1’.

Figure 6: Learning the ‘Happiness’ feeling in ‘Smile 1’.

Of course, we attempted to create simple animations to attract the child's attention to various parts of the story.

For example, in the first two images, we showed show both happiness by a smile and a laugh to the child, and when he heard the sound of a laugh and the appearance of teeth while laughing, the child became more familiar with the ‘happy’ sense. Because, according to studies, children with autism recognise happiness and laughter most of the time by seeing teeth. In the third and fourth images, using simple stories and their related images by using the concepts of ‘mom and dad’ in stories and words, such as birthday cake, balloon, canvas, and painting, which children with autism are most often familiar with, the ‘happy’ feeling is taught to the children with autism. Because children with autism do not have correct understanding of many of the abstract concepts around them, in all the stories, elements are used in which the child has an objective conception of, and, in fact, they noticed these concepts in their surroundings.

All elements used in the game ‘Smile 1’ stories were chosen by field experts in this field.

After hearing the ‘happy’ stories, questions related to this sense are as it shows in figure 7.

Questions about the sense of ‘happiness’ in the game ‘Smile 1’.

Figure 7: Questions about the sense of ‘happiness’ in the game ‘Smile 1’.

One each slide, after the story was read to the child, she is now asked: ‘Now tell me what you look like when you're happy?’ Although the stories are first-person, according to studies that will be referred to in the following chapters, children with autism cannot understand first-person stories. Thus, this method of asking a question was used to make the child feel closer to the stories and identify their own sensations.

For these stories, the child's answers to questions and options from easy to difficult are used; thus, in the first question, only two options are available for the child, while in the third question, these choices are divided into four modes that contain all senses. Additionally, if the answer to the question is correct, a star will light up for the child to encourage the child.

As shown on the game pages, all the stories and questions can be repeated or paused. The child can also go back to the previous stages of the game and begin the process from a certain point.

Additionally, in this section, to help the participant, if the child does not choose the correct answer after 25 s from the end of the story, the correct option is displayed in a large size so that the child's attention is drawn to that option. At the stage of answering questions, if the child chooses the wrong option, sentences that indicate the wrong choice are read to the child.

However, to better measure the response rate and progress of the child at each stage, each question is scored as follows. 0: In this sense, the child did not have any cooperation when answering the questions. 1: In the sense, the child ultimately answers the question but has given an incorrect answer and did not reach the correct answer. 2: In this sense, the child after one or more incorrect answers finally found the correct answer. 3: In this sense, the child chooses the correct answer at the very beginning.

Measuring the effectiveness of ‘Smile1’

Before the experiments, we first reviewed the authoritative sources mentioned in previous sections, considering that teaching different emotional states to autistic patients generally reduces their behavioural problems, increases communication skills, improves conversational skills, and so on, we decided to investigate the effectiveness of ‘Smile 1’ on the behavioural problems of these children. To achieve this goal, after the studies, the Children Behavioural Check List (CBCL) was selected for comprehensive and complete analysis.

The CBCL is a common method for identifying behavioural problems in children. This checklist is one of the parts of the ASEBA system and was designed by Thomas M. Achenbach in 1966. This test is designed for three different age ranges of 1-5 years, 6-14 years and over 14 years of age. By performing this test, a total of 17 child-based behavioural syndrome scales can be identified in a child: 1) Anxiety / Depression, 2) Corruption / Depression, 3) Physical complaints, 4) Social problems, 5) Thinking problems, 6) Attention problems, 7) Behavioural law, 8) Aggressive behaviour, 9) Emotional problems, 10) Anxiety problems, 11) Physical problems, 12) ADHD problems, 13) Behavioural problems, Confrontation, 14) Conduct problems, 15) Internalisation, 16) Extrapolation and 17) General problems.

In the present study, a second type of questionnaire was used for children ages 6 to 14 years. This type of questionnaire includes 113 questions. After the analysis, all 17 factors mentioned for each participant were deduced from this test.

Based on the results obtained, according to relevant documentation, the percentage rating of the scales for children with normal growth is 93. Upon analysing the tests completed by the parents of the participants in the study, in all the children, most behavioural factors are rated above 93, which confirms the existence of multiple behavioural problems in these autistic children.

Participants

The experimental group included children tested using the designed game, whereas the control group included children who did not participate in the game intervention sessions and who only have previous training according to the previous routine.

Then, to test the game, a random sample of people with autism should be selected. Hence, in most similar research studies and activities that aim to determine the effectiveness of an intervention (in this study, Smile1 game) on a specific group that include a control group, 10 to 15 people are examined in each ‘Experimental ‘and’ Control’ group.

Additionally, to confirm this, many studies, such as the research by Ashayerih, and Esteki, (2017); Alves et al. (2013); (Miranda, et al. 2011); Silver and Oake (2001); and many others, have used the same number as a sample.

All of these children are students trained in the Autism Charity Foundation, and their level of autism has been determined by specialists from the very beginning. Therefore, in this study, based on the recognition of specialists in this field, we did not perform a separate test for autism disorder diagnosis of the sample group.

Because pre-test-post-test and control experimental groups were included in this study, after reviewing the resources related to the number of samples required in similar research, ten children with autism in the mild, moderate and severe categories of autism who were both male and female in the age group of 6 to 14 years were included in the experimental group, and ten children with autism matched the experimental group in terms of age, gender and severity of autism were included as the control group.

Ten children were selected in each group, including seven boys and three girls, and five of them had high-functional autism, four had moderate autism and one had severe autism of the classical type.

A. Intervention sessions

After selecting the sample of the test case, the game sessions of ‘Smile 1’ began. The number of sessions required to play the intended game for the experimental group was determined by the opinion of the project adviser. Additionally, to confirm the correct selection, in similar studies, such as in research by Alves et al. (2013); Silver and Oake (2001); Bölte et al. (2002) and many others, the number of sessions of intervention was similarly selected.

Thus, it was decided that the number of sessions for teaching children would be a maximum of 16 sessions of 30 min, or, if the child collaborated, ten sessions of 45 min per child. By starting the intervention and observing the extent of the child's cooperation, it was decided to test the game in 16 sessions of 30 min for each child. Accordingly, if the child learned all four senses earlier and answered the questions correctly, the number of those sessions was also reduced. However, if the child did not cooperate during the sessions, the training effort lasted up to 16 sessions.

In this study, due to time limits of the parents of the participants, these sessions were held at eight 1-hour sessions for each participant, and the first meeting for all participants was to provide the participant with familiarity with the game environment and related explanations. As a result, in the following sections, the seven main intervention sessions that were used for the game are examined.

For each session, depending on the child’s cooperation, stories were taught that focused on each sense, and then questions were asked of the child, and only when the game changed from one session to another was the child partially paying attention to the stories and questions related to the previous section. All of the observations were recorded throughout each session, and the child’s general demeanour during each session was also asked of his parents to justify the process during each session. Below are pictures of the children when performing the game:

Participants of the ‘Smile 1’ game.
Figure 8: Participants of the ‘Smile 1’ game.

Results

Results of the ‘Smile 1’ game intervention sessions

During each session, the children’s responses to questions were identified and recorded. According to the description of the game after the last section, four feelings of ‘happiness, sadness, anger and fear’ were considered in this game. Each session tried to train all four senses and evaluate questions for each child. For each sense, three questions were asked in which, as previously stated, one of four (0, 1, 2, or 3) scores was assigned to each question.

In this way, the results recorded during the intervention sessions for one of the children are provided in Table 1 as an example.

Table 1: Scores earned by ‘Amir Ali’ for each question in the ‘Smile 1’ game.

Amir Ali, an 8-year-old boy with moderate autism, started his training 1.5 years ago. A lack of eye contact, difficulty with speaking and meaningful conversation, repetitive and stereotypical movements, and low focus and attention are his most important problems. The most troublesome problem of Amir Ali, as described above, has been detecting ‘angry’ feelings, although there has been difficulty with other senses.

Participant performances and feedback about the game and gamification elements

Additionally, during the sessions, the participant's performance and feedback related to the gamification elements that were considered in the game were recorded, as shown in Table 2.

Table 2: Participants' response table regarding the gamification elements.

In this table, the factors included: 1) the child's attention to the game page, 2) the child's attention to game music, 3) the child's attention to the positive feedback of the game, 4) the effectiveness of negative feedback, 5) child's attention to social stories, 6) child's understanding of the stories, 7) identifying the names of emotions (happiness, sadness, anger and fear), 8) the child's interest in the game's rating system (stars), 9) the effectiveness of guidance to the correct answer after the child’s choice, 10) the accompanying tutor in the response and 11) the child's perception of the first personal stories. These were ranked 0 or 1 for each child.

If the child did not show any interest in the factor or did not have any positive performance related to it, it was ranked as 0; it was ranked as 1 if the child was tangibly responsive to the factor involved and if it influenced him. In the next section, this table will be further examined.

Comparison of the CBCL test during pre-test and post-test stages

As discussed in the previous section, this research was performed using pre-tests and post-tests. As the first step, before starting the game intervention sessions, the parents of the children in both groups were asked to complete the CBCL test for their child.

In figure 9, the CBCL analysis of a participant in the pre-test phase is shown.

In the figure 9 below, two diagrams are shown; one is related to CBCL-based scales, and the second one refers to the overall score and child behavioural profiling based on DSM as prepared by ASEBA. As shown, the red line in both graphs shows the normal state line, which is equal to 93 for all scales. The blue line in this graph is related to the condition of the child tested at different scales, which is in most cases higher than 93 for this child with moderate autism.

CBCL analysis of ‘Amir Ali’ - Pre-test phase.

Figure 9: CBCL analysis of ‘Amir Ali’ - Pre-test phase.

After the end of the intervention sessions, the parents of the children in both two groups were again asked to complete the Child Behavioural Checklist questionnaire to compare the results of these tests during the pre-test and post-test stages to determine the effectiveness of intervention on reducing the behavioural issues of the participants.

As an example of the analysed questionnaires in the post-test, figure 10, shows the behavioural diagrams related to the child ‘Amir Ali’.

CBCL analysis of ‘Amir Ali’ - Post-test stage

Figure 10: CBCL analysis of ‘Amir Ali’ - Post-test stage

Upon comparing the results of the pre-test and post-test for Amir Ali, there were very few changes.

Supplemental intervention sessions to measure training durability

After the end of intervention sessions, with a two-month interval, the game was performed on three selected children during four additional 1-h sessions to evaluate its effectiveness and durability. For this purpose, ‘Khojaste’, ‘Amir Ali’, and ‘Danial’ were selected. The reason for selecting these three children was their cooperation during the intervention sessions and the appropriate level of their autism and their development during the main game sessions. In the tables below, you can see the results in these sessions separately.

According to Table 3, ‘Khojasteh’ and ‘Danial’, who had high-functional autism and showed good performance during the main intervention sessions also completed the stages well in these sessions. Although in the first two sessions, due to the weakness of short-term memory in autistic patients, in some cases, incorrect answers were received from these two. For Amir Ali, because he has a moderate level of autism and had a good but slow progression during the main intervention sessions, during these four sessions during a 2-month interval, he initially responded to some questions. It was difficult though owing to his familiarity with the game environment, and we observed much more cooperation and faster progress.

Table 3: List of scores for the three selected participants in additional intervention sessions over a 2-month interval.

However, instead of understanding the concepts, because patients with autism mostly (except those with high-functional autism who are similar to ordinary children) only remember things that they have heard, the effectiveness of short-term interventions may last like they should. In fact, according to an autism therapist, Mrs. ‘Motie’, teaching one subject to an autistic child requires at least 4 to 5 months. It is important to with repeat the topic again and again to complete the learning process so that the autistic person does not forget it.

Consequently, because the present study was conducted over a period of 2 months due to time constraints and the difficulty coordinating time with the children's parents, it could not be expected that the learning process would be prolonged or significant, although the main intervention sessions had very positive and constructive effects considering the children’s backgrounds.

Discussion

Each participant's progress divided by senses for all sessions

In the previous section, the results and scores of the Smile 1 game sessions were shown as an example for one of the participants in the experimental group. The diagram below shows the progress of this participant during each intervention session for the four ‘happy, sad, angry and afraid’ senses.

‘Amir Ali’ showed slow but noticeable progress during the intervention sessions. From the first session, he found it difficult to identify any of the four senses in the study. Of course, one of the reasons he had inappropriate performance during the first intervention session was his lack of cooperation due to the unfamiliar environment and the lack of anonymity of the coach, which led to a sharp decrease in his focus and increasing behavioural problems during the first session. However, as shown in the diagram, Amir Ali advanced well by the final intervention (seventh), and ultimately, he had the ability to recognise all four senses taught in the game.

As discussed in the previous section, ‘Amir Ali’ is one of three individuals who participated in four additional intervention sessions after a 2-month break to assess the durability of training. In the figure above, these four sessions (8, 9, 10 and 11) are shown. As shown in the figure 11, Amir Ali, due to his serious problems with focus and attention and his weak memory, which exists in most people with autism, his score decreases in the eighth session, although this drop was much better than during his first main intervention session, and certainly, the seven previous sessions created a good background for him. Of course, from the very beginning, ‘Amir Ali’ had trouble recognising the ‘anger’ feeling, which was also observed in four additional sessions. However, according to the figure 11, his progress in these four final sessions was very fast, and we ultimately achieved the desired results.

Separate senses before all meetings for ‘Amir Ali’.

Figure 11: Separate senses before all meetings for ‘Amir Ali’.

Overall progress of each participant in all intervention sessions

Below show the diagrams for the general development of one child as an example for all the sessions; however, the factors of game and gamification and the child's attention to these factors are taken into account, and its relevance to the child's performance was examined.

Although at the end of the seven main intervention sessions Amir Ali had trained all four senses and answered the questions about them, based on observations at each session and as shown in the above table, although he was paying attention to the related stories, he had no understanding of these stories and did not know the names of some feelings because of the inability to speak.

This is also why Amir Ali, in an intervention session after a two-month hiatus, initially, we observed a slight weakness in his performance regarding remembering names of the senses and related stories and a weakness in short-term memory. However, this weakness was quickly recouped at subsequent sessions. However, Amir Ali was one of the few participants in the study that had negative feedback of the game and more or less considered it when responding to questions, and he was interested in the game's scoring system.

Comparing the overall progress of all participants during the main intervention sessions

When the status and progress of each child during the game intervention sessions was identified, we were able to make comparisons between the performances of all participants in the experimental group during the seven main intervention sessions based on the average responses to the four desired senses, as shown in the figure 13.

Overall progress of ‘Amir Ali’ during all intervention sessions.

Figure 12: Overall progress of ‘Amir Ali’ during all intervention sessions.

Session Development Averages of all senses per each child.

Figure 13: Session Development Averages of all senses per each child.

As shown in figure 13, ‘Mohammad Ali’ and ‘Amir Reza’, who answered all the questions correctly, have a linear number of 3. However, ‘Shervin’, ‘Danial’, ‘Parsa’ and ‘Khojasteh’, who showed good performance during the intervention sessions, have numbers between 2.5 and 3 in the figure 13 above, which indicated a good performance. Their progress was good during the intervention sessions. The green line represents ‘Amir Ali’, who, despite moderate performance during the first intervention session, was very good at subsequent sessions. Additionally, there are two lines with very high volatility that represent ‘Amir Hossein’ and ‘Setare’ (crescent and orange lines), which, despite recessions during some of the intervention sessions, made good progress during the final sessions. In the lower part of the figure 13, the brown line represents ‘Yeganeh’, who has severe autism and did not cooperate properly during the intervention sessions.

Comparison of participants' progress during the sessions according to their level of autism

As shown in figure 14, a comparison was made between the participants' progress during the main intervention sessions by distinguishing their level of autism.

Comparison of progress during the main intervention sessions according to the autism levels of the participants.

Figure 14: Comparison of progress during the main intervention sessions according to the autism levels of the participants.

Based on the figure 14 above, participants with high-functional autism showed less but steady progress than the others. Overall, however, participants with moderate autism progressed more rapidly, although this was more unpredictable. According to previous studies, the main cause of these fluctuations is the disturbance of focus and attention as well as the behavioural and communication problems of these participants, which caused their lack of cooperation during some intervention sessions. In this study, there was no progression or hope for a participant with severe autism (Bhatt, et al. 2014); (Smith, et al. 2010). although this result cannot be generalised to all patients with severe autism. Those who suffer from severe autism have very low levels of education that make learning difficult, time-consuming, very slow and sometimes impossible.

Conclusions

The results of this study indicate that using games and gamification for educating autistic children, especially those with mild to moderate autism, can have positive effects in the development and promotion of their abilities. Of course, this only considered a small sample of autistic children and only related to the teaching of a particular subject, namely, the recognition of facial expressions by children with autism, using the ‘Smile 1’ game, which was specifically designed according to the needs of these children. Based on the results, we found that Smile 1 has the ability to teach facial expressions to autistic people because at the end of the main intervention sessions, the majority of participants and even those suffering from moderate autism and mild autism showed the ability to detect the four emotional states taught by the game.

Additionally, the overall aim of this study was to investigate the effectiveness of gamification for reducing the behavioural problems of autistic patients, and the results of the analysis of the Child Behavioural Check List during the pre-test and post-test stages and their comparison did not show a significant change in the behavioural problems of the children participating in this study. Of course, this result is understandable owing to the time limit required for the children to adjust to the sessions, which took place in the short time of 2 months via 8 sessions of 1 h. According to experts, behavioural changes in children with autism require continuous and longer-term training of 6 months and in most cases at least 1 year.

However, this study hopefully provides some small steps toward helping the development of children with autism in our beloved Iran.

References

  1. Alves, S., Marques, A., Queirós, C., Orvalho, V. (2013). LIFEisGAME Prototype: a serious game about emotions for children with autism spectrum disorders.PsychNology Journal,11(3).
  2. Ashayerih, H., Esteki, M. (2017). The Effect of Computer Games on Improving Student’s Critical Thinking.Advances in Cognitive Science,19(1), 14-24.
  3. Bhatt, S. K., De Leon, N. I., & Al-Jumaily, A. (2014). Augmented reality game therapy for children with autism spectrum disorder.International Journal on Smart Sensing & Intelligent Systems, 7(2). https://doi.org/10.21307/ijssis-2017-668 [Link]
  4. Bölte, S., Feineis-Matthews, S., Leber, S., Dierks, T., Hubl, D., Poustka, F. (2002). The development and evaluation of a computer-based program to test and to teach the recognition of facial affect.International Journal of Circumpolar Health,61(sup2), 61-68. https://doi.org/10.3402/ijch.v61i0.17503 [Link]
  5. Christinaki, E., Vidakis, N., Triantafyllidis, G. A. (2014). A novel educational game for teaching emotion identification skills to preschoolers with autism diagnosis.Comput. Sci. Inf. Syst.,11(2), 723-743. https://doi.org/10.2298/CSIS140215039C [Link]
  6. Deterding, S., Dixon, D., Khaled, R., Nacke, L. (2011b). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (pp. 9-15). ACM. https://doi.org/10.1145/2181037.2181040 [Link]
  7. Deterding, S., Sicart, M., Nacke, L., O'Hara, K., Dixon, D. (2011a). Gamification. using game-design elements in non-gaming contexts. In CHI'11 extended abstracts on human factors in computing systems (pp. 2425-2428). ACM.
  8. Levy, S., Mandell, D., Schultz, R. (2009). Autism. Lancet, 374(9701), 1627-1638. https://doi.org/10.1016/S0140-6736(09)61376-3 [Link]
  9. Mantziou, O., Vrellis, I., Mikropoulos, T. A. (2015). Do children in the spectrum of autism interact with real-time emotionally expressive human controlled avatars?Procedia Computer Science,67, 241-251. https://doi.org/10.1016/j.procs.2015.09.268 [Link]
  10. Miranda, J. C., Fernandes, T., Sousa, A. A., Orvalho, V. (2011). Interactive technology: teaching people with autism to recognize facial emotions.Autism Spectrum Disorders-From Genes to Environment, 299-312.
  11. Qin, S., Nagai, Y., Kumagaya, S., Ayaya, S., Asada, M. (2014). Autism simulator employing augmented reality: A prototype. In 4th international conference on development and learning and on epigenetic robotics (pp. 155-156). IEEE. https://doi.org/10.1109/DEVLRN.2014.6982972 [Link]
  12. Rovai , A., Ponton, M., Wighting, M., Baker, J. (2007). A comparative analysis of student motivation in traditional classroom and e-learning courses.International Journal on E-learning, 6(3), 413-432.
  13. Silver, M., Oakes, P. (2001). Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others.Autism, 5(3), 299-316. https://doi.org/10.1177/1362361301005003007 [Link]
  14. Smith, S. D., Grigorenko, E., Willcutt, E., Pennington, B. F., Olson, R. K., DeFries, J. C. (2010). Etiologies and molecular mechanisms of communication disorders.Journal of developmental and behavioral pediatrics: JDBP,31(7), 555. https://doi.org/10.1097/DBP.0b013e3181ee3d9e [Link]
  15. Tanaka, J. W., Wolf, J. M., Klaiman, C., Koenig, K., Cockburn, J., Herlihy, L., Kaiser, M. D. (2012). The perception and identification of facial emotions in individuals with autism spectrum disorders using the Let’s Face It! Emotion Skills Battery.Journal of Child Psychology and Psychiatry,53(12), 1259-1267. https://doi.org/10.1111/j.1469-7610.2012.02571.x [Link]
  16. Willis, C. (2010).Teaching young children with autism spectrum disorder. ReadHowYouWant.com.
Navan, A., Khaleghi, A. (2020). Using Gamification to Improve the Education Quality of Children with Autism. Revista Científica, 37(1), 90-106