Immersive Virtual Reality Enabled Interventions for Autism

Immersive Virtual Reality Enabled Interventions for Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

Chen Li (1,*,†), Meike Belter (2), Jing Liu (3,*), and Heide Lukosch (2)

  1. Department of Applied Social Science and Department of Computing, The Hong Kong Polytechnic
    University, Hong Kong SAR, China
  2. HIT Lab NZ, University of Canterbury, Christchurch 8041, New Zealand;
    meike.belter@pg.canterbury.ac.nz (M.B.); heide.lukosch@canterbury.ac.nz (H.L.)
  3. Department of Accountancy, Economics and Finance, Hong Kong Baptist University, Hong Kong SAR, China

* Correspondence: richard-chen.li@polyu.edu.hk (C.L.); jingliu@hkbu.edu.hk (J.L.)
† Current address: Faculty of Health and Social Sciences and Faculty of Engineering, The Hong Kong
Polytechnic University, 11 Yuk Choi Road, Hung Hom, Kowloon, Hong Kong SAR, China.

Abstract

Autism spectrum disorder (ASD) is characterized by persistent deficits in social commu-
nication and interaction, which can have significant impacts on daily life, education, and work.
Limited performance in learning and working, as well as exclusion from social activities, are com-
mon challenges faced by individuals with ASD. Virtual reality (VR) technology has emerged as a
promising medium for delivering interventions for ASD. To address five major research questions
and understand the latest trends and challenges in this area, a systematic review of 21 journal articles
published between 1 January 2010 and 31 December 2022 was conducted using the PRISMA approach.
A meta-analysis of 15 articles was further conducted to assess interventional effectiveness. The results
showed that most studies focused on social and affective skill training and relied on existing theories
and practices with limited adaptations for VR. Furthermore, the enabling technologies’ affordances
for the interventional needs of individuals with ASD were not thoroughly investigated. We suggest
that future studies should propose and design interventions with solid theoretical foundations, ex-
plore more interventional areas besides social and affective skill training, and employ more rigorous
experimental designs to investigate the effectiveness of VR-enabled ASD interventions.

1. Introduction

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by one’s persistent deficits in social communication and interaction across multiple contexts as well as restricted and repetitive patterns of behavior that could cause significant impairments in social and occupational functioning [1]. According to recent estimates from the Autism and Developmental Disabilities Monitoring (ADDM) network of Centers for Disease Control and Prevention (CDC) in the United States, about 1 in 54 children aged 8 years has been identified with ASD [2]. Epidemiological studies suggested that the prevalence of ASD does not vary significantly in different geographic regions or among different ethnic groups [2,3]. Theory of Mind (ToM) deficit, meaning the limited or absent ability of perspective-taking and understanding other people’s beliefs, was originally proposed to explain deficits of the ASD population in social communication and interaction [4,5,6]. Later, weak central coherence and executive dysfunction were identified and accounted for other ASD symptoms such as perceptual abnormalities and repetitive patterns of behaviour [7,8,9,10]. Currently, there is no clinical evidence that fully supports a particular treatment for ASD. However, a great number of evidence-based interventions have been developed and delivered to help children with ASD at a young age. The most widely delivered evidence-based interventions include Applied Behavior Analysis (ABA) [11], Picture Exchange Communication System (PECS) [12], Treatment and Education of Autistic and Related Communication Handicapped Children (TEACCH) [13], and Social-Communication, Emotional Regulation and Transactional Support (SCERTS) [14]. Besides evidence-based interventions, recent studies suggested that virtual reality (VR) technology could be a promising medium to deliver interventions for ASD [15,16].

VR technology is a combination of software and hardware technology, which aims to create an interactive and vivid computer-generated virtual environment, where people can experience “the sense of presence” [17,18]. VR can provide a safe and controllable environment, showing characteristics that are important when delivering ASD interventions. Features of VR enable the transfer of knowledge and skills gained in the environment to real-life scenes when some resemblance is realized [19,20,21]. This transfer used to be challenging for the ASD population in conventional interventions due to their rigid thinking and weakness in generalization [22,23]. Moreover, prior studies suggest that the ASD population primarily relies on visual thinking [24]. Computer-generated visual stimulations during VR exposure can not only attract attention with their visual cues but also help individuals with ASD think structurally [25,26,27].

VR comes in many forms, from virtual environments displayed on a desktop computer to immersive VR that one has to engage with using head-mounted displays (HMDs) (also known as, VR headsets). HMDs can enable highly immersive VR experiences but have to be worn directly on the user’s head, which can be a challenging procedure for children and young adolescents with ASD. The exploration of the use and effects of immersive VR on individuals with ASD started in the early 1990s. Early studies focused on the acceptance of VR devices and the comprehension of computer-generated visual stimulations by the ASD population. Strickland et al. reported the use of HMDs on two school-aged children with ASD, a 7.5-year-old girl and a 9-year-old boy [28]. Despite the poor display quality and the far-from-ideal ergonomics of the HMDs used for this study, the two children were found to be able to accept the HMDs with preliminary efforts. Moreover, there was strong evidence showing that both of them were able to comprehend and respond to computer-generated visual stimulations. Recent studies that utilized more user-friendly off-the-shelf HMDs demonstrated similar results—individuals on the spectrum, in general, can accept wearing HMDs with minimal preliminary efforts and are able to initiate meaningful interactions with the virtual objects as well as navigation in the simulated scenes [29,30]. Besides HMDs, surround-screen projection, such as the Cave Automatic Virtual Environment (CAVE) [31,32], can also provide a highly immersive VR experience. Although costly to build and maintain, the surround-screen projection that was originally designed to be a one-to-many presentation installation has its own advantages over HMDs when being used for delivering ASD interventions [27,30]. For example, during interventions, it is much easier for psychologists, psychiatrists, and other professionals to provide facilitation in this particular setting than for someone whose vision is completely blocked by HMDs. Compared to conventional desktop VR environments, both types of immersive VR technologies can deliver a higher sense of presence in the computer-simulated scenes [18,33], which is believed to be critical for easing the generalization process of individuals with ASD [19,20,21]. In this paper, we only include studies that utilize immersive VR to deliver ASD interventions, meaning that we do not look into work utilizing desktop VR environments.

In the past decade, the enabling software and hardware for VR have become more capable. Related studies show the potential of immersive VR-enabled ASD intervention, with a number of empirical studies in this period covering topic areas such as social skills [27,34,35,36], emotion recognition [27,34], daily living skills [21], vocational skills [37], and phobias [38,39]. In regard to the fast growth of applying VR for ASD intervention, a few review articles have been published, covering various aspects of this interdisciplinary research area. For example, the systematic review published by Bozgeyikli et al. mainly focused on the design considerations of the VR-enabled interventions for the ASD population [40]. It covered the task design, information presentation, and the VR system, but no findings were reported regarding the effectiveness of the interventions reviewed. Bradley and Newbutt also conducted a systematic review covering the area of application and the participant characteristics but limiting the enabling technology to HMDs [41]. However, more recent publications in this area of research reporting more rigorous experiment designs (e.g., randomized controlled trials) with a higher number of citations, such as [27,39], were not included in the most recent reviews (e.g., [42,43]). In addition, although previous reviews have highly praised VR as a promising medium for ASD intervention, the remaining challenges, current research, and practical gaps have yet to be clearly identified with the support of data. Hence, the present study aims to further contribute to this area by providing a systematic review and a meta-analysis of high-quality empirical studies on using immersive VR for ASD intervention. Five research questions to be addressed are listed as follows: (1) what are the areas of intervention in previous empirical studies? (2) What are the affordances of immersive VR in ASD intervention? (3) What are the theories applied to the design of the VR content? (4) How effective are these immersive VR-enabled interventions in various topic areas? (5) What are the latest research trends and the challenges that need to be further addressed in future studies? These questions have been chosen as they address the main issues around using immersive VR for ASD interventions. We believe that the answers to these questions will allow us to assess the value of immersive VR for individuals with ASD, as well as identify research gaps and directions for future research and development. These answers will contribute to the further development of this line of research, supporting the well-being of individuals on the spectrum.
 
 

2. Methods

The Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines were applied to guide the review process of this study [44]. PRISMA provides a standard peer-accepted methodology that uses a guideline checklist to contribute to the transparency, quality, and replicability of systematic reviews and meta-analyses. The detailed protocols are as follows.
 

2.1. Search Procedure and Criteria

A systematic literature search was conducted from four online databases, which are PubMed (https://pubmed.ncbi.nlm.nih.gov/), IEEEXplore (https://ieeexplore.ieee.org/), Education Resources Information Center (ERIC) (https://eric.ed.gov/), and Web of Science (https://www.webofknowledge.com/), in order to cover literature published under different disciplines. PubMed mainly comprises literature related to medical sciences. IEEEXplore was searched for technology-related publications. ERIC focuses on education and educational technology. Web of Science is believed to be an interdisciplinary database. The search was limited to journal articles published in English between 1 January 2010 and 31 December 2022. The search query included “autism” or “autism spectrum disorder” or “Asperger”, “virtual reality”, and “intervention” or “training” or “education” or “learning” or “rehabilitation” or “intervention”, but “survey” or “review” was explicitly excluded. The decision of having 1 January 2010 as the cut-off year of inclusion is mainly due to the lack of studies on this research topic before 2010. Although this research area can be traced all the way back to 1996 when Strickland et al. studied the use of HMDs on two children with ASD [28], there is a lack of follow-up studies from 1996 to 2010 [42]. The decision of including only journal articles is mainly because most of the conference papers had a clear focus on the technology and less emphasis on the psychological, psychiatric, or psychoeducational aspects of the studies. For example, many of these conference papers mentioned the use of VR as stimuli for capturing participants’ electroencephalogram or eye gaze data during VR exposure but did not provide any interventional procedures or did not mention the studies’ implications for ASD intervention. As mentioned above, this study aims to cover a broader spectrum of concerns in this research area; if such conference papers were included in the analysis, the interdisciplinary nature of this area might be overlooked. Although they are not included in the systematic review or the meta-analysis, those conference papers will contribute to this review and will be cited individually, especially when discussing the technical aspects of the research as well as the latest trends and challenges.
 

2.2. Selection Procedure

Each of the articles found during the literature search was reviewed by two independent evaluators (i.e., the first and second authors of this article) to determine whether it should be included or excluded by following pre-defined inclusion criteria. The inclusion criteria were (1) the interventions were delivered through immersive VR devices, (2) the participants of the studies were formally diagnosed with ASD according to DSM-V [1] or ICD-10 [45], (3) the interventions were assessed through repeated measures, and (4) the study should not be a single-case study. If the two evaluators could not agree on whether an article should be included, a third evaluator (i.e., the fourth author of this article) would make the decision.
 
 

3. Results

3.1. Search Results

The initial search yielded 108 articles from PubMed, 21 from IEEEXplore, 40 from ERIC, and 191 from Web of Science. After removing the duplicate records and retracting publications and manually identifying records from prior systematic reviews, 252 records were included in the initial screening (see Figure 1). The initial screening was performed using the titles, abstracts, and metadata of the records. During the initial screening, a total of 190 records were excluded. Specifically, two conference papers and four book chapters were first removed to ensure only rigorously peer-reviewed journal articles were taken into account for the analysis; 45 articles were excluded due to the fact that the technologies applied in the studies were not related to VR; 21 articles were excluded because they were either reviews or otherwise conceptual commentaries with no empirical information; 40 articles were removed because the participants of those studies were not formally diagnosed with ASD as formulated in our selection criteria or the participants had conditions other than ASD; in addition, three records were removed because the studies reported were neither related to VR nor ASD intervention. Next, the remaining 62 articles were further assessed for eligibility by full-text screening. In this step, the evaluators agreed to further remove 22 articles from the synthesis, because the VR technologies employed in those articles were desktop VR and could not be considered immersive VR; one article was excluded since the VR device described in the article was incomplete and thus unidentifiable; nine articles were removed due to the employment of the single-case design; two articles were excluded because the participants were not confirmed to have an ASD diagnosis; four articles were excluded due to evaluations that were either not aligned with the intervention objectives or no evaluation results were reported. In addition, three articles were removed due to the lack of interventions delivered to the participants. In the end, 21 articles were included in the following analysis (see Table 1). We have conducted both descriptive analyses of the articles, as well as quantitative meta-analyses. Below, we present the results of the two analytical steps.
 

Figure 1. The PRISMA flow diagram of the review.

Table 1. Characteristics of the interventions covered by included publications.

3.2. Descriptive Analysis

In the descriptive analysis of the articles, we present the vital topics that are discussed within the included studies, including the areas of intervention, the characteristics of the participants, the technology used, the theoretical foundation that the included studies build upon, as well as the evaluation approach and measures.
 

3.2.1. Areas of Intervention

The interventional focus of the included studies was on skills individuals diagnosed with ASD might lack, such as social skills, attention deficits, and communication skills. In detail, 11 articles focused on social and/or affective skill training, of which one article also covered executive functioning training [34] and one article also covered physical coordination [56]. It is observed that different articles may have different definitions of social skills. Defining social skills is considered to be challenging in either theoretical or practical terms because of the variety of behaviors subsumed under the rubric [63]. Besides social and/or affective skills, two articles focused on the intervention of phobias [38,39], three articles focused on daily life skill training, such as driving [48,58] and using public transportation [51], one article focused on social attention training [49], one article focused on joint attention training [54], one article focused on conversation skill training [53], one article focused on the training of motor skills [62], and one article focused on enhancing the participants’ cognitive functions [57].
 

3.2.2. Characteristics of Participants

A total number of 613 participants with ASD were included in the studies reported in the selected articles, covering an age range of 6 to 62 years. If the areas of intervention are specifically taken into consideration when analyzing the ages of participants, it is observed that studies focusing on social and/or affective skill training covered a narrower age range of 7 to 23 years old excluding participants of four studies where details of the participants’ ages were not reported [34,47,52,56]. Male participants represent a larger group than female participants. Specifically, 485 of the 584 participants (83.05%) were male, excluding 29 participants from [53,55,56] where participants’ gender was not reported in detail. This generally reflects the prevalence and characteristics of ASD; the estimated prevalence is around four to five times higher among males than among females [64]. However, this also introduces concerns over the gender imbalance and whether the approaches should be further evaluated among females with ASD.
 

3.2.3. Enabling Technology

Surround-screen projection is the most widely adopted enabling technology among the included studies. CAVE and one of its variants called the “Blue Room” [38,39] were used in five of the analyzed studies. L-shaped projection installations or display installations consisting of multiple screens with external sensors for tracking and data collection were used in three studies [34,47,57]. Ip et al. discussed the benefits of using CAVE for ASD intervention [27]; while being fully immersed in the VR experience, the surround-screen projection enables easier interaction between the participants and their trainers in the physical space, resulting in more effective and prompter facilitation if needed. However, this shortcoming of HMDs in the context of ASD intervention had been partially addressed by introducing program-controlled virtual agents with pre-scripted dialogues in the authors’ latest work [60]. Besides surround-screen projections, HMDs were also widely used in the included studies, and we found that HMDs were employed more in the latest studies published after 218. For example, Amaral et al. and Simões et al. both used the second generation of the Oculus Rift Development Kit HMDs [49,51], the devices that later evolved into the Oculus Rift and Oculus Rift S HMDs used in [60,61], while Ravindran et al. reported the use of Google Cardboard [54], a more primitive but much cheaper type of HMDs, in their study. However, there is generally a lack of comparison of different HMDs given the ASD population as the end users. Moreover, the acceptance of HMDs among the ASD population, especially children and young adolescents, was not thoroughly discussed in the nine studies that employed HMDs as the enabling technology.
 

3.2.4. Theoretical Foundations

The design of the VR scenarios, interventional protocols, and corresponding assessments all require a solid theoretical foundation. Surprisingly, 10 of the 21 analyzed articles did not explicitly report the theoretical foundations for the design of the reported immersive VR-enabled interventions. By examining the descriptions of the methods, especially the design of the VR scenarios, we identified that all of these 10 studies to some degree borrowed the idea of simulating real-life situations using VR, which generally matches the characteristics of the situated approach [65]. For the rest of the 11 analyzed articles in which the theoretical foundations for design were explicitly reported, one article reported the adoption of the combination of cognitive behavior therapy (CBT) [66] and mindfulness-acceptance based approach [46], one article reported the combination of CBT with the learning from simulation approach [58], one article reported the adoption of a combination of CBT and the graded exposure therapy approach [39]. One article reported the adoption of the graded exposure therapy approach only [38]; one article reported the adoption of gamification [51]; one article report the adoption of experiential learning theory [60]; and the remaining three articles all adopted one or more learning theories and/or approaches, such as task-based learning, adaptive learning, situated learning, and learning from simulation [27,34,48,55,61].
 

3.2.5. Evaluation Approach and Measures

Among the 21 included articles, 14 articles (66.67%) reported the use of standardized tests in their studies to measure the effectiveness of the immersive VR-enabled intervention. The use of computerized assessments and the assessment results were reported in nine (42.86%) included articles.
The use of computerized assessments seems to be straightforward since the enabling technology of the intervention, immersive VR systems, is computerized. For example, multiple sensors were used in [34,47] to capture interaction data during the interventions, allowing performance monitoring and analysis as the intervention was delivered. Similarly, the dialogue choices made by the participants when interacting with the virtual agents during interventions were captured for analysis in [56]. Baker-Ericzén et al. used a customized driving simulator to deliver the cognitive behavioral intervention for driving for teenagers and adults with ASD [58]. As the participants drove in the simulator, their performance was automatically recorded and quantified as the number of off-road crashes, vehicle collisions, pedestrian collisions, tickets, speeding, and centerline crossing by the computerized system. Hocking et al. used the Kinect sensor, a low-cost motion tracking device originally designed for playing motion-sensing games, to monitor the participants’ motor proficiency as the immersive VR-enabled exercise intervention progressed [62].
Physiological data seemed to be another good source for evaluating the effectiveness of immersive VR-enabled interventions. For example, Ravindran et al. used video recording for analyzing the participants’ joint attention during VR exposure [54]; the video clips were then analyzed manually by the researchers. On the other hand, eye tracking techniques were used in [49] for measuring joint attention automatically. Cox et al. also reported the use of eye-tracking techniques in their study; the driving performance was evaluated using the data captured directly in the virtual cockpit [48]. This approach greatly reduced the risk of assessment in this particular use case scenario. Simões et al. captured the electrodermal activity (EDA) for measuring the participants’ anxiety level during the VR exposure [51]. The use of electroencephalography-based brain–computer interfaces (BCI) was seen in [46] for measuring the attention level of the participants, but the results were not reported in the article. Electroencephalography data were also captured by De Luca et al. to study the potential effects of their immersive VR-enabled intervention on the participants’ brain functional connectivity [57].
 

3.3. Meta-Analysis on the Effectiveness

Among the 21 included articles, effect sizes and 95% confidence intervals (CI) were calculated for a total of 76 outcome measures reported in 15 articles, of which 20 outcome measures pertain to controlled trials reported in 5 articles [27,46,56,59,60]. Five articles were excluded from the analysis due to the reported data being either incomplete or insufficient for calculating the effects sizes [34,47,48,51,55]. Trial results were excluded if they were not directly for measuring the effectiveness of the intervention. Examples of such trial results include the measuring of system usability and cybersickness in [62]. In addition, the results of [50] were also excluded from the analysis since the raw data reported was highly similar to [27]; new data analysis tools were used in [50] so the focus of the article was solely on data analysis and presenting the data from a new perspective. Hence, to prevent misestimating the overall effectiveness, we decided to exclude these four articles from the analysis. Five articles reported the measurements of both the immediate and the retained interventional effects [38,39,49,57,61]. Since the relapse and retention of the interventional effects are beyond the scope of this work and the waiting time for the follow-up evaluations varies greatly from study to study, we only included the immediate effects in the meta-analysis. The methodological characteristics of the 15 included studies are summarized in Table 2 and Table 3.
 

Table 2. Methodological characteristics of studies employed uncontrolled trials and included them in the meta-analysis

Table 3. Methodological characteristics of studies employed controlled trials and included them in the meta-analysis.

4. Discussion

4.1. Area of Intervention

With respect to research question (1), areas of intervention of the studies analyzed most often involve the improvement of social and/or emotional skills. This result is expected because deficits in social and emotional skills are the two most important diagnostic criteria of ASD [1] and the deficits can greatly hinder one’s participation in various social functions. The use of immersive VR can help create a safe, controllable, and embarrassment-free environment, in which social and emotional skills can be practiced repeatedly across a variety of social contexts. Participants of such interventions are expected to gain the ability to better judge a given context and respond to it in a socially appropriate way. However, due to the broadness of both terminologies, there seems to be a lack of a common definition of the term social skills. For example, Politis et al. investigated conversation skill training using VR [53]. Seemingly, social conversation requires one to understand the social context as well as the emotion of other parties involved in the conversation—the ability of social perception [70]. This also requires emotional skills, such as recognition of others’ facial expressions and comprehension of emotions and the implicit reciprocal, which were not discussed explicitly in the article. Further, the studies analyzed dealt with a variety of interventions including the improvement of daily life skills, social and joint attention training, and facing general or specific phobias. All these areas of intervention could, following the articles looked into, significantly improve the life quality of individuals with ASD and help them better engage in social life.
 

4.2. Affordances of Immersive VR

With regard to the research question (2), the affordances of immersive VR in ASD intervention design have been partially discussed, but have yet to be fully investigated. The affordance of immersive VR that was most commonly mentioned in the articles included in our analysis was simulation fidelity, allowing participants to practice the skills and generalize to real-life situations. Unfortunately, generalization is difficult to measure and there was little evidence reported in the included articles. Another affordance that was not covered to a large extent but could potentially change the landscape of ASD intervention is the ability to deliver intervention remotely, especially when considering the current Coronavirus disease pandemic that is shaping the methods of intervention delivery. Moreover, measures of effectiveness can greatly benefit from immersive VR-enabled approaches. The computerized nature of VR allows for the collection of an enormous amount of data that can be analyzed to measure the interventional effectiveness as the intervention happens. The collection of driving performance data in the simulated cockpit in [48] is a great example of utilizing this affordance. Lorenzo et al. used computer vision algorithms to automatically recognize the participants’ facial expressions during the intervention [47], which aimed to assess the participants’ performance by examining the number of inadequate behaviors and/or facial expressions performed given the social situation. Physiological data, such as EDA [51] and electroencephalogram [46], was also captured in some of the studies, providing another way of measuring the participants’ responses to the interventions. It is expected to see such approaches being further developed to provide performance-based personalized intervention programs, given the great flexibility and customizability of VR scenarios.
 

4.3. Theoretical Foundation

Regarding research question (3) and the theoretical foundations of the studies investigated, clinical approaches such as graduated exposure and CBT have been adapted for guiding the design of the immersive VR-enabled interventions reported in [38,39]. CBT has shown to be effective in treating anxiety disorder, phobia, and depression among individuals with ASD by identifying unrealistic feelings or unhelpful thoughts, and thus changing their way of thinking and behaving [71]. In the case of applying CBT in an immersive VR environment, the computer-generated content can help the participants to better relax and break down the challenges into thoughts, feelings, and actions under the guidance of a therapist. Besides CBT, there are other clinical approaches, which may require a method of relaxing or the presentation of certain stimuli, and that can benefit from the utilization of immersive VR. Although not explicitly mentioned in many other included studies, the design of the ASD interventions clearly borrowed the idea from conventional learning and education. For example, Ip et al. created various scenarios that a primary school student will encounter on a school day [27], in which challenging situations would be introduced as the intervention progresses. The participants need to interpret the social contexts and choose a strategy to respond to these situations. The VR scenarios, as well as the humanoid virtual agents, were programmed to respond to the participants’ actions, providing instant feedback, which prompts the participants to reflect and rethink a new strategy if their actions were not socially appropriate in the contexts. These approaches of using immersive VR to allow free practicing and strategic reflection are very similar to Kolb’s experiential learning model (KELM) [72]. However, this might also bring questions regarding such approaches when being applied to individuals with ASD. Specifically, the ASD population may need more facilitation and scaffolding in the process, especially when they face a challenging situation. Adopting KELM requires learners to self-initiate the rethinking and reflecting process, which might not happen among individuals with ASD if there is insufficient facilitation. Ip et al. tried to address this issue by emphasizing the role of the trainers in their study [27]. Cox et al. adopted a similar approach; the participants were always accompanied by a driving coach in the virtual cockpit [48]. However, in some cases, facilitation from trainers might be limited by the VR environment. If HMDs were used as the enabling device, the participant would be totally immersed in the VR scenario and isolated from the surrounding. Whether the facilitation from trainers can still be effective in this case is yet to be fully investigated. Ip et al. tried to address the issue by placing program-controlled virtual agents with pre-scripted dialogues at the locations where the participants might need facilitation [60]. With the recent advancements of large language models (LLMs) and the development of ChatGPT (https://openai.com/blog/chatgpt (accessed on 26 May 2023))-like services, the solution can be further refined and is worth further investigation.
 

4.4. The Effectiveness of Intervention

With regard to research question (4), we can summarize that the effectiveness of immersive VR-enabled interventions for individuals with ASD needs to be further examined and investigated in future studies. The limitations of the included articles are obvious; most of the studies reported involved a rather low number of participants and the experimental designs were not rigorous enough to draw a solid conclusion regarding the interventional effectiveness. Among the 15 articles included for the effect size calculation, five articles reported controlled trials [27,46,56,59,60]. The combined effect size (𝑑=0.285

, 95% CI =[0.132,0.438]

) of all controlled trials suggested a small-to-medium combined effect of VR-enabled interventions over the controlled conditions. Because the interventional areas covered in all controlled trials had a focus on social and/or affective skills, it is safe to conclude that VR-enabled interventions are generally effective on social and/or affective skills. However, there is a lack of evidence to conclude whether the VR-enabled approaches are effective in other areas of ASD interventions. Furthermore, the long-term effectiveness and the ability to generalize seem to be the two major issues that need to be further addressed in future studies as well. Among all fifteen included articles, only three articles reported results from follow-up assessments regarding the lasting of the interventional effects [38,39,49]. As relapse is often observed in conventional ASD intervention, it would be interesting to see whether the interventional effects of an immersive VR-enabled program can last. Moreover, the technology’s ability to simulate real-life situations with great fidelity could potentially be extremely helpful for individuals with ASD, who are widely considered visual thinkers. Unfortunately, there is no in-depth research reported in the 15 included articles on the generalization of virtual scenarios to real-life situations.
 

4.5. Research Trends and Challenges

In examining the results presented, conclusions can be drawn concerning research question (5), identifying the latest research trends and challenges that need to be addressed in the area of immersive VR-enabled interventions for individuals with ASD.
With respect to research trends, it is evident that recent publications show a gradual increase in sample sizes and the implementation of more rigorous experimental designs. This trend suggests that the research area is maturing, particularly in the realm of social and affective skill training for individuals with ASD. Furthermore, the advancements in and accessibility of enabling hardware and software have facilitated the delivery of interventions in various settings and the development of interventional materials at reduced costs. For instance, Ip et al. initially developed a social and affective skill training program in a CAVE environment [27], but as HMDs became more accessible and ergonomic, later publications from the authors reported delivering interventional sessions in school settings [60]. In fact, a significant increase in the use of off-the-shelf HMDs was observed in publications after 2018, with nearly all VR content and experiences developed using readily available game engines such as Unity and Unreal Engine. It is anticipated that there will be continued growth in this research area as VR and Metaverse technologies become even more accessible.
Regarding the challenges that remain, the most pressing issue is the integration and adaptation of existing theories and practices to suit both VR and the specific interventional needs of individuals with ASD. As demonstrated in Table 1, 10 out of the 21 publications included in our systematic review did not clearly report the theoretical foundations of their studies. Furthermore, even among those publications that did report the utilization of existing theories, the manner in which these theories were adapted for VR-enabled ASD interventions remains unclear. In addition, the affordances of enabling technologies for the interventional needs of individuals with ASD have not been thoroughly investigated.
In conclusion, while the field of VR interventions for individuals with ASD is experiencing growth and maturation, there are still challenges to overcome. Further research must focus on the integration and adaptation of existing theories and practices for VR-enabled ASD interventions, as well as investigating the affordances of enabling technologies for the specific needs of individuals with ASD.
 

5. Limitations

This study has several limitations that should be acknowledged. First, despite using the Web of Science, an interdisciplinary and comprehensive database, in combination with PubMed, IEEEXplore, and ERIC, as well as manually searching previous systematic reviews to identify additional records, it is possible that some important articles were not included in the analyses. Second, the selection criteria included immersion as an objective capability of the enabling technologies [33], which led to the exclusion of some studies that employed less capable technologies. However, it is possible that these excluded studies still provided a high sense of presence, which is a key and desirable feature of the VR experience [18]. Unfortunately, few studies reported on the evaluation of a sense of presence, and to ensure objectivity and reproducibility, immersion had to be used as a selection criterion.
 

6. Conclusions

While our quantitative meta-analysis did not provide a clear indication of the effectiveness of immersive VR technology for ASD interventions other than social and/or affective skill training, there is promising growth and maturation in this research field. The systematic review results suggest a gradual increase in sample sizes and more rigorous experimental designs being employed in the latest publications. Furthermore, there is a greater focus on social and/or affective skill training for individuals with ASD. The advancements in and accessibility of enabling hardware and software have also facilitated the delivery of interventions in various settings and the development of interventional materials at reduced costs. Therefore, it is anticipated that there will be continued growth in this area of research and practice as VR and Metaverse technologies become even more accessible.
However, challenges remain in the integration and adaptation of existing theories and practices for VR-enabled ASD interventions, as well as investigating the affordances of enabling technologies for the specific needs of individuals with ASD. To address these challenges, future studies should focus on better utilizing the affordances of VR for the interventional needs of individuals with ASD, reporting the design of intervention procedures clearly, and implementing more solid theoretical foundations. With the increasing accessibility of off-the-shelf HMDs and other enabling hardware and software technologies, we expect to see more studies that utilize the enabling technologies in the near future. Ideally, these studies should connect the maturity level of technology to theoretical and methodological considerations, leading to valid research results.
In conclusion, while there is still much work to be conducted in the field of immersive VR-enabled interventions for individuals with ASD, the promising growth and maturation of the field suggest that more effective interventions can be developed for the specific needs of individuals with ASD by addressing the challenges that remain. Ultimately, the goal is to improve the quality of life for individuals with ASD by utilizing the enabling technologies of immersive VR to provide more accessible and effective interventions in various settings.
 

Author Contributions

Conceptualization, C.L., M.B. and H.L.; methodology, C.L., M.B., J.L. and H.L.; validation, C.L., M.B. and J.L.; formal analysis, C.L. and J.L.; data curation, C.L. and J.L.; writing—original draft preparation, C.L. and M.B.; writing—review and editing, C.L., M.B., J.L. and H.L.; visualization, C.L. and J.L.; supervision, H.L. All authors have read and agreed to the published version of the manuscript.
 

Funding

This research received no external funding.

References

  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®); American Psychiatric Association: Washington, DC, USA, 2013. [Google Scholar]
  2. Baio, J.; Wiggins, L.; Christensen, D.L.; Maenner, M.J.; Daniels, J.; Warren, Z.; Kurzius-Spencer, M.; Zahorodny, W.; Rosenberg, C.R.; Dowling, N.F. Prevalence of autism spectrum disorder among children aged 8 years-autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morb. Mortal. Wkly. Rep. Surveill. Summ. 2014, 63, 1–23. [Google Scholar]
  3. Elsabbagh, M.; Divan, G.; Koh, Y.J.; Kim, Y.S.; Kauchali, S.; Marcín, C.; Montiel-Nava, C.; Patel, V.; Paula, C.S.; Wang, C.; et al. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012, 5, 160–179. [Google Scholar] [CrossRef] [PubMed]
  4. Baron-Cohen, S.; Leslie, A.M.; Frith, U. Does the autistic child have a “theory of mind”? Cognition 1985, 21, 37–46. [Google Scholar] [CrossRef]
  5. Baron-Cohen, S. Mindblindness: An Essay on Autism and Theory of Mind; MIT Press: Cambridge, MA, USA, 1997. [Google Scholar]
  6. Baron-Cohen, S. Theory of mind and autism: A fifteen year review. In Understanding Other Minds: Perspectives from Developmental Cognitive Neuroscience; Baron-Cohen, S., Tager-Flusberg, H., Cohen, D.J., Eds.; Oxford University Press: Oxford, UK, 2000; Volume 2, pp. 3–20. [Google Scholar]
  7. Frith, U. Autism: Explaining the Enigma; Blackwell Publishing: Hoboken, NJ, USA, 2003. [Google Scholar]
  8. Frith, U.; Happé, F. Autism: Beyond “theory of mind”. Cognition 1994, 50, 115–132. [Google Scholar] [CrossRef] [PubMed]
  9. Ozonoff, S.; Pennington, B.F.; Rogers, S.J. Executive function deficits in high-functioning autistic individuals: Relationship to theory of mind. J. Child Psychol. Psychiatry 1991, 32, 1081–1105. [Google Scholar] [CrossRef]
  10. Ozonoff, S. Executive functions in autism. In Learning and Cognition in Autism; Springer: Berlin/Heidelberg, Germany, 1995; pp. 199–219. [Google Scholar]
  11. Foxx, R.M. Applied behavior analysis treatment of autism: The state of the art. Child Adolesc. Psychiatr. Clin. N. Am. 2008, 17, 821–834. [Google Scholar] [CrossRef]
  12. Bondy, A.S.; Frost, L.A. The picture exchange communication system. Focus Autistic Behav. 1994, 9, 1–19. [Google Scholar] [CrossRef]
  13. Mesibov, G.B.; Shea, V.; Schopler, E. The TEACCH Approach to Autism Spectrum Disorders; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2005. [Google Scholar]
  14. Rubin, E.; Prizant, B.M.; Laurent, A.C.; Wetherby, A.M. Social communication, emotional regulation, and transactional support (SCERTS). In Interventions for Autism Spectrum Disorders; Springer: Berlin/Heidelberg, Germany, 2013; pp. 107–127. [Google Scholar]
  15. Grynszpan, O.; Weiss, P.L.; Perez-Diaz, F.; Gal, E. Innovative technology-based interventions for autism spectrum disorders: A meta-analysis. Autism 2014, 18, 346–361. [Google Scholar] [CrossRef]
  16. Ip, H.H.; Li, C. Virtual reality-based learning environments: Recent developments and ongoing challenges. In Proceedings of the International Conference on Hybrid Learning and Continuing Education; Springer: Berlin/Heidelberg, Germany, 2015; pp. 3–14. [Google Scholar]
  17. Rheingold, H. Virtual Reality: Exploring the Brave New Technologies; Simon & Schuster Adult Publishing Group: London, UK, 1991. [Google Scholar]
  18. Steuer, J. Defining virtual reality: Dimensions determining telepresence. J. Commun. 1992, 42, 73–93. [Google Scholar] [CrossRef]
  19. Blascovich, J.; Loomis, J.; Beall, A.C.; Swinth, K.R.; Hoyt, C.L.; Bailenson, J.N. Immersive virtual environment technology as a methodological tool for social psychology. Psychol. Inq. 2002, 13, 103–124. [Google Scholar] [CrossRef]
  20. Wallace, S.; Parsons, S.; Westbury, A.; White, K.; White, K.; Bailey, A. Sense of presence and atypical social judgments in immersive virtual environments: Responses of adolescents with Autism Spectrum Disorders. Autism 2010, 14, 199–213. [Google Scholar] [CrossRef] [PubMed]
  21. Adjorlu, A.; Høeg, E.R.; Mangano, L.; Serafin, S. Daily living skills training in virtual reality to help children with autism spectrum disorder in a real shopping scenario. In Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Baltimore, MD, USA, 6–11 May 2007; pp. 294–302. [Google Scholar]
  22. Karkhaneh, M.; Clark, B.; Ospina, M.B.; Seida, J.C.; Smith, V.; Hartling, L. Social Stories™ to improve social skills in children with autism spectrum disorder: A systematic review. Autism 2010, 14, 641–662. [Google Scholar] [CrossRef] [PubMed]
  23. Vismara, L.A.; Rogers, S.J. Behavioral treatments in autism spectrum disorder: What do we know? Annu. Rev. Clin. Psychol. 2010, 6, 447–468. [Google Scholar] [CrossRef]
  24. Jacobson, N.S. Specific and nonspecific factors in the effectiveness of a behavioral approach to the treatment of marital discord. J. Consult. Clin. Psychol. 1978, 46, 442. [Google Scholar] [CrossRef] [PubMed]
  25. Grandin, T. How does visual thinking work in the mind of a person with autism? A personal account. Philos. Trans. R. Soc. B Biol. Sci. 2009, 364, 1437–1442. [Google Scholar] [CrossRef]
  26. Moore, D.; Taylor, J. Interactive multimedia systems for students with autism. J. Educ. Media 2000, 25, 169–177. [Google Scholar] [CrossRef]
  27. Ip, H.H.; Wong, S.W.; Chan, D.F.; Byrne, J.; Li, C.; Yuan, V.S.; Lau, K.S.; Wong, J.Y. Enhance emotional and social adaptation skills for children with autism spectrum disorder: A virtual reality enabled approach. Comput. Educ. 2018, 117, 1–15. [Google Scholar] [CrossRef]
  28. Strickland, D.; Marcus, L.M.; Mesibov, G.B.; Hogan, K. Brief report: Two case studies using virtual reality as a learning tool for autistic children. J. Autism Dev. Disord. 1996, 26, 651–659. [Google Scholar] [CrossRef]
  29. Newbutt, N.; Sung, C.; Kuo, H.J.; Leahy, M.J.; Lin, C.C.; Tong, B. Brief report: A pilot study of the use of a virtual reality headset in autism populations. J. Autism Dev. Disord. 2016, 46, 3166–3176. [Google Scholar] [CrossRef]
  30. Li, C.; Yuan, S.; Ip, H. A Case Study on Delivering Virtual Reality Learning for Children with Autism Spectrum Disorder Using Virtual Reality Headsets. In Proceedings of the 10th International Conference on Education and New Learning Technologies (EDULEARN18), Palma, Spain, 2–4 July 2018; IATED Academy: Valencia, Spain, 2018; pp. 728–734. [Google Scholar]
  31. Cruz-Neira, C.; Sandin, D.J.; DeFanti, T.A.; Kenyon, R.V.; Hart, J.C. The CAVE: Audio visual experience automatic virtual environment. Commun. ACM 1992, 35, 64–73. [Google Scholar] [CrossRef]
  32. Cruz-Neira, C.; Sandin, D.J.; DeFanti, T.A. Surround-screen projection-based virtual reality: The design and implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, New York, NY, USA, 2–6 August 1993; pp. 135–142. [Google Scholar]
  33. Slater, M. Immersion and the illusion of presence in virtual reality. Br. J. Psychol. 2018, 109, 431–433. [Google Scholar] [CrossRef]
  34. Lorenzo, G.; Pomares, J.; Lledó, A. Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome. Comput. Educ. 2013, 62, 88–101. [Google Scholar] [CrossRef]
  35. Matsentidou, S.; Poullis, C. Immersive visualizations in a VR cave environment for the training and enhancement of social skills for children with autism. In Proceedings of the 2014 International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal, 5–8 January 2014; Volume 3, pp. 230–236. [Google Scholar]
  36. Cheng, Y.; Huang, C.L.; Yang, C.S. Using a 3D immersive virtual environment system to enhance social understanding and social skills for children with autism spectrum disorders. Focus Autism Other Dev. Disabil. 2015, 30, 222–236. [Google Scholar] [CrossRef]
  37. Bozgeyikli, L.; Bozgeyikli, E.; Raij, A.; Alqasemi, R.; Katkoori, S.; Dubey, R. Vocational rehabilitation of individuals with autism spectrum disorder with virtual reality. ACM Trans. Access. Comput. 2017, 10, 1–25. [Google Scholar] [CrossRef]
  38. Maskey, M.; Lowry, J.; Rodgers, J.; McConachie, H.; Parr, J.R. Reducing specific phobia/fear in young people with autism spectrum disorders (ASDs) through a virtual reality environment intervention. PLoS ONE 2014, 9, e100374. [Google Scholar] [CrossRef]
  39. Maskey, M.; Rodgers, J.; Grahame, V.; Glod, M.; Honey, E.; Kinnear, J.; Labus, M.; Milne, J.; Minos, D.; McConachie, H.; et al. A randomised controlled feasibility trial of immersive virtual reality treatment with cognitive behaviour therapy for specific phobias in young people with autism spectrum disorder. J. Autism Dev. Disord. 2019, 49, 1912–1927. [Google Scholar] [CrossRef]
  40. Bozgeyikli, L.; Raij, A.; Katkoori, S.; Alqasemi, R. A survey on virtual reality for individuals with autism spectrum disorder: Design considerations. IEEE Trans. Learn. Technol. 2017, 11, 133–151. [Google Scholar] [CrossRef]
  41. Bradley, R.; Newbutt, N. Autism and virtual reality head-mounted displays: A state of the art systematic review. J. Enabling Technol. 2018, 12, 101–113. [Google Scholar] [CrossRef]
  42. Lorenzo, G.; Lledó, A.; Arráez-Vera, G.; Lorenzo-Lledó, A. The application of immersive virtual reality for students with ASD: A review between 1990–2017. Educ. Inf. Technol. 2019, 24, 127–151. [Google Scholar] [CrossRef]
  43. Mak, G.; Zhao, L. A systematic review: The application of virtual reality on the skill-specific performance in people with ASD. Interact. Learn. Environ. 2020, 31, 804–817. [Google Scholar] [CrossRef]
  44. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann. Intern. Med. 2009, 151, 264–269. [Google Scholar] [CrossRef] [PubMed]
  45. Brämer, G.R. International statistical classification of diseases and related health problems: Tenth revision. World Health Stat. Q. Rapp. Trimest. Stat. Sanit. Mond. 1988, 41, 32–36. [Google Scholar]
  46. White, S.W.; Richey, J.A.; Gracanin, D.; Coffman, M.; Elias, R.; LaConte, S.; Ollendick, T.H. Psychosocial and computer-assisted intervention for college students with autism spectrum disorder: Preliminary support for feasibility. Educ. Train. Autism Dev. Disabil. 2016, 51, 307. [Google Scholar] [PubMed]
  47. Lorenzo, G.; Lledó, A.; Pomares, J.; Roig, R. Design and application of an immersive virtual reality system to enhance emotional skills for children with autism spectrum disorders. Comput. Educ. 2016, 98, 192–205. [Google Scholar] [CrossRef]
  48. Cox, D.J.; Brown, T.; Ross, V.; Moncrief, M.; Schmitt, R.; Gaffney, G.; Reeve, R. Can youth with autism spectrum disorder use virtual reality driving simulation training to evaluate and improve driving performance? An exploratory study. J. Autism Dev. Disord. 2017, 47, 2544–2555. [Google Scholar] [CrossRef]
  49. Amaral, C.; Mouga, S.; Simões, M.; Pereira, H.C.; Bernardino, I.; Quental, H.; Playle, R.; McNamara, R.; Oliveira, G.; Castelo-Branco, M. A feasibility clinical trial to improve social attention in Autistic Spectrum Disorder (ASD) using a brain computer interface. Front. Neurosci. 2018, 12, 477. [Google Scholar] [CrossRef]
  50. Yuan, S.N.V.; Ip, H.H.S. Using virtual reality to train emotional and social skills in children with autism spectrum disorder. Lond. J. Prim. Care 2018, 10, 110–112. [Google Scholar] [CrossRef]
  51. Simões, M.; Bernardes, M.; Barros, F.; Castelo-Branco, M. Virtual travel training for autism spectrum disorder: Proof-of-concept interventional study. JMIR Serious Games 2018, 6, e5. [Google Scholar] [CrossRef]
  52. Jacques, C.; Cloutier, V.; Bouchard, S. The “Decoding of Social Interactions in Virtual Reality” Tasks for Autism Spectrum People: Development of an Intervention Protocol and Pilot Testing. Annu. Rev. Cyberther. Telemed. 2018, 2018, 148. [Google Scholar]
  53. Politis, Y.; Sung, C.; Goodman, L.; Leahy, M. Conversation skills training for people with autism through virtual reality: Using responsible research and innovation approach. Adv. Autism 2019, 6, 3–16. [Google Scholar] [CrossRef]
  54. Ravindran, V.; Osgood, M.; Sazawal, V.; Solorzano, R.; Turnacioglu, S. Virtual reality support for joint attention using the Floreo Joint Attention Module: Usability and feasibility pilot study. JMIR Pediatr. Parent. 2019, 2, e14429. [Google Scholar] [CrossRef] [PubMed]
  55. Herrero, J.F.; Lorenzo, G. An immersive virtual reality educational intervention on people with autism spectrum disorders (ASD) for the development of communication skills and problem solving. Educ. Inf. Technol. 2020, 25, 1689–1722. [Google Scholar] [CrossRef]
  56. Jialiang, H.; Haiyan, Z.; Huiying, Z. Research on the auxiliary treatment system of childhood autism based on virtual reality. J. Decis. Syst. 2021, 1–18. [Google Scholar] [CrossRef]
  57. De Luca, R.; Naro, A.; Colucci, P.V.; Pranio, F.; Tardiolo, G.; Billeri, L.; Le Cause, M.; De Domenico, C.; Portaro, S.; Rao, G.; et al. Improvement of brain functional connectivity in autism spectrum disorder: An exploratory study on the potential use of virtual reality. J. Neural Transm. 2021, 128, 371–380. [Google Scholar] [CrossRef]
  58. Baker-Ericzén, M.J.; Smith, L.; Tran, A.; Scarvie, K. A cognitive behavioral intervention for driving for autistic teens and adults: A pilot study. Autism Adulthood 2021, 3, 168–178. [Google Scholar] [CrossRef] [PubMed]
  59. Frolli, A.; Savarese, G.; Di Carmine, F.; Bosco, A.; Saviano, E.; Rega, A.; Carotenuto, M.; Ricci, M.C. Children on the autism spectrum and the use of virtual reality for supporting social skills. Children 2022, 9, 181. [Google Scholar] [CrossRef]
  60. Ip, H.H.; Wong, S.W.; Chan, D.F.; Li, C.; Kon, L.L.; Ma, P.K.; Lau, K.S.; Byrne, J. Enhance affective expression and social reciprocity for children with autism spectrum disorder: Using virtual reality headsets at schools. Interact. Learn. Environ. 2022, 1–24. [Google Scholar] [CrossRef]
  61. van Pelt, B.; Nijman, S.; van Haren, N.; Veling, W.; Pijnenborg, G.; van Balkom, I.; Landlust, A.; Greaves-Lord, K. Dynamic Interactive Social Cognition Training in Virtual Reality (DiSCoVR) for adults with Autism Spectrum Disorder: A feasibility study. Res. Autism Spectr. Disord. 2022, 96, 102003. [Google Scholar] [CrossRef]
  62. Hocking, D.R.; Ardalan, A.; Abu-Rayya, H.M.; Farhat, H.; Andoni, A.; Lenroot, R.; Kachnowski, S. Feasibility of a virtual reality-based exercise intervention and low-cost motion tracking method for estimation of motor proficiency in youth with autism spectrum disorder. J. Neuroeng. Rehabil. 2022, 19, 1–13. [Google Scholar] [CrossRef]
  63. Kavale, K.A.; Forness, S.R. Social skill deficits and learning disabilities: A meta-analysis. J. Learn. Disabil. 1996, 29, 226–237. [Google Scholar] [CrossRef]
  64. Christensen, D.L.; Braun, K.V.N.; Baio, J.; Bilder, D.; Charles, J.; Constantino, J.N.; Daniels, J.; Durkin, M.S.; Fitzgerald, R.T.; Kurzius-Spencer, M.; et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years-autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveill. Summ. 2018, 65, 1. [Google Scholar] [CrossRef] [PubMed]
  65. Herrington, J.; Oliver, R. Critical characteristics of situated learning: Implications for the instructional design of multimedia. In Proceedings of the ASCILITE 1995 Conference, Melbourne, Australia, 4–6 December 1995; Australian Society for Computers in Learning in Tertiary Education: Queensland, Australia, 1995; pp. 253–262. [Google Scholar]
  66. Hawton, K.E.; Salkovskis, P.M.; Kirk, J.E.; Clark, D.M. Cognitive Behaviour Therapy for Psychiatric Problems: A Practical Guide; Oxford University Press: Oxford, UK, 1989. [Google Scholar]
  67. Cohn, J. Statistical Power Analysis for the Behavioral Sciences; Lawrence Earlbam Associates: Hillsdale, NJ, USA, 1988. [Google Scholar]
  68. Cooper, H.; Hedges, L.V.; Valentine, J.C. The Handbook of Research Synthesis and Meta-Analysis; Russell Sage Foundation: New York, NY, USA, 2019. [Google Scholar]
  69. Spitzer, R.L.; Kroenke, K.; Williams, J.B.; Löwe, B. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch. Intern. Med. 2006, 166, 1092–1097. [Google Scholar] [CrossRef] [PubMed]
  70. Aronson, E.; Wilson, T.D.; Akert, R.M. Social Psychology; Prentice Hall: Upper Saddle River, NJ, USA, 2005; Volume 5. [Google Scholar]
  71. Ho, B.P.; Stephenson, J.; Carter, M. Cognitive—Behavioural approach for children with autism spectrum disorder: A literature review. J. Intellect. Dev. Disabil. 2015, 40, 213–229. [Google Scholar] [CrossRef]
  72. Kolb, D.A. Experiential Learning: Experience as the Source of Learning and Development; FT Press: Bergen, NJ, USA, 2014. [Google Scholar]