Remember me
Advanced Cardiac Life Support Course (ACLS) developed by American Heart Association (AHA) and Advanced Life Support (ALS) course developed by European Resuscitation Council (ERC) with a similar content to ACLS, aim to train healthcare professionals in managing adult patients suffering from cardiac arrest. The target group consists of medical doctors, nurses, and paramedics [-]. Around 1.3 million candidates take part in ACLS or ALS courses worldwide every year []. These training sessions have to be repeated at certain intervals depending on local regulations or institutional requirements [].
Participants learn how to deliver high-quality cardiopulmonary resuscitation (CPR) to adult patients and to manage cardiac arrest cases. Besides technical skills, another aim of these courses is to provide training for nontechnical skills, which are essential when working as a multidisciplinary team. Blended learning techniques including didactic lecture-based, video-based learning; serious gaming; and simulation-based training modalities are used for ALS training [,,].
Like in other industries, serious gaming is becoming an additional training modality for simulation-based education for healthcare over the last decade. A significant amount of time is spent in lecture-based sessions, which are generally organized before hands-on training for ALS in order to teach cardiac rhythm interpretation, drug dosage, the defibrillation procedure, and differential diagnosis [,]. By using serious gaming modules for this aspect of training, the educators can reduce the time spent on theoretical lectures and alleviate logistical difficulties, such as coordinating the training with a clinician from the hospital []. Another advantage is the ability to minimize the instructors' workload by delivering these training sessions with virtual reality (VR) modules []. Besides immersive features and interactivity, serious gaming modules also provide learners the opportunity to improve their skills in a safe environment anywhere and anytime they want [-]. PC-based, tablet-based, and VR-based serious gaming modules for health care training are available commercially. Due to their immersive effect, VR-based serious gaming modules are now widely used in health care training [-,-]. As VR hardware has already become more affordable, significant growth of VR-based learning is expected []. With technological developments, the new generation of VR headsets is becoming more affordable than the older ones, which cost much more and required a cable connection to a PC with an expensive graphics card to operate. The new generation of VR headsets also eliminate the huge logistical and hardware requirements for using VR. Another advantage of such a VR-based serious gaming module is the ability to be used with wireless VR headsets without the need for a PC and an expensive cable-based VR headset. Therefore, Meta Quest 2 headsets (Meta) were used in this study [].
It is obvious that nontechnical skills are as important as technical skills when performing an ALS procedure as part of the Code Blue team [,]. Nontechnical skills including teamwork, resource management skills, and situational awareness are very important competencies that are required when performing an ALS procedure, as ALS is a team-based procedure [,]. Several serious gaming modules for ALS training are available in the market [,]. The existing VR-based ALS serious gaming modules are mostly focused on technical skills when calculating the total score upon final evaluation of the ALS learning outcomes. Therefore, the development of a new serious gaming module for ALS training has been decided, which can also evaluate and score nontechnical skills of the trainee in parallel with technical skills required to perform ALS. The objective of this study is to investigate whether a VR-based ALS serious game module has the potential to replace classroom-based ALS lectures, the latter being part of existing ALS training protocols in addition to skills training.
This study was designed as a randomized controlled trial comparing the results of a VR-based serious gaming module and conventional training for ALS. Pretest, posttest, preference survey outcomes and hands-on training scores of the participants, which is the main performance indicator of the study, were compared.
RecruitmentIn total, 30 third-semester students of Acibadem Mehmet Ali Aydinlar University’s Vocational School for Anesthesiology volunteered to participate in this study. Simulation-based ALS training is part of their regular curriculum. The participants were randomly divided into 2 groups, with 15 participants each in the conventional (hereinafter called “group C”) and VR-based (hereinafter called the “VR group”) training groups. The VR headsets were cleaned with an antiseptic solution before each training session.
Data ExclusionExclusion criteria for the study were having previously experienced VR-induced motion sickness or other medical conditions such as vertigo. One participant from the VR group was not able to attend at the study due to personal reasons and was excluded from the study.
Statistical AnalysisMann-Whitney U and Wilcoxon signed-rank tests were used to analyze the 2 groups’ performances in this study.
VR-Based Serious Gaming Module for ALSThe serious game module for ALS has been developed for this study and named “3DMedsim ALS VR.” During the initial phase of development, the latest versions of international ALS or ACLS protocols from the AHA and ERC were reviewed together with the clinicians and software developers [,]. Crisis resource management (CRM) criteria adapted from those used in aviation and used for team-based training were also reviewed [,,].
In the next phase, the development process of the VR-based gaming module was initiated. The VR-based serious game module developed for this study was designed to be compatible with the 2020 ALS algorithm of the ERC and AHA [,]. It comprises a learning management system and a learning record store (LRS), which allow users' credentials to be stored in a shared database [-]. The 3D visualization engine is also included and is integrated with the LRS to track the users' actions and generate experience application programming interface (xAPI) calls for each action carried out []. When the user triggers a predefined interaction with a significant object in the virtual environment, a xAPI event is automatically generated [,]. To enable this, a software library has been developed as a Unity extension, which makes the necessary xAPI web service calls to the LRS servers over the HTTP protocol. All user actions were defined in terms of actor, verb, and object parameters. The library automatically generates xAPI calls for each action, which are recorded by the LRS. The library can automatically generate xAPI calls for actions that have not been performed within a given time limit or in the proper order. Finally, the library also implements security and user authentication features required for record-keeping using the basic access authentication method included in the HTTP protocol standard.
To accelerate the development process and increase flexibility subsequently, a scenario building Unity plugin was developed []. This tool helps create and edit serious game scenarios as logic nodes outside of the virtual environment, independent from 3D objects. Thus, scenarios can be edited without altering Unity objects, which normally take the most time during the development process. The scenario building tool was only used during the development process and cannot be used by trainers to change the scenario flow during gameplay. The scenario building tool returns function calls to Unity in accordance with the logic flow, and these calls are linked to 3D objects as triggers for certain actions. As actions are triggered, they, in turn, trigger the next steps in the scenario until the serious game is over. A part of a scenario flow can be seen in .
Figure 1. Screen capture of scenario logic flow. The serious game consists of 3 different stages in multiple languages: basic training, advanced training, and a test mode. Initially, the serious game consisted of only training and test modes. However, during early testing, it became clear that a third stage was necessary, as the increase in difficulty and complexity was too much between the 2 modes. Another slightly more difficult training mode was added for this purpose. These stages can be completed in a virtual hospital or an examination room environment as seen in .
Figure 2. Screen capture from the virtual reality–based serious game for advanced life support. The first stage, the basic training mode, is for beginners who wish to learn the fundamentals of the ALS algorithm. This version provides visual and audio guidance to the user at every step of the algorithm. An announcer verbally directs the user to perform specific actions, while corresponding buttons are highlighted on the interface. The user is not allowed to interact with anything that is not highlighted, and the risk of failure is thus eliminated. Additionally, time limits are removed, and the user is reminded to undertake the next step if user stays idle for a particular period.
The next stage is advanced training, which is intended for users who have completed the beginner-level training of the game. The advanced training version does not provide any visual or audio assistance, and the user is free to take any action; thus, mistakes are now possible. If users make significant mistakes or miss a time constraint, they receive a warning, but the game is not terminated.
The third stage is the test mode. The users' actions are evaluated on the basis of their order and timing during this mode. The evaluation is carried out under 2 categories: technical and CRM skills. The scores can be accessed by using the review section. Unlike the training stages, if the user commits a significant error, such as selecting the wrong rhythm, the game terminates with a warning indicating the mistake. The technical and CRM scores are segregated as shown in . The subcategories of these grades are specified to pinpoint the users’ errors during review.
Figure 3. Screenshot from the serious game showing technical, crisis resource management, and total scores of a trainee. CPR: cardiopulmonary resuscitation; CRM: crisis resource management. The total score of a participant was calculated as the sum of the technical and CRM scores during the game. The maximum total achievable score was 100, that for technical skills was 70 points, and that for CRM skills was 30 points. In order to avoid risk of failure due to problems concerning the content, scenario flow, and scoring, the first version of the game was presented to clinicians for evaluation. Depending on the feedback and critiques from the content experts, changes were made. The version used in this study is the second version of the software optimized depending on user feedback.
Study FlowThe details of the study flow are shown in .
Figure 4. The study flow of the conventional training group and virtual reality (VR)–based training group. During the initial stage of the study, all participants were asked to read the educational material sent to them. Then, both groups were asked to complete the pretest form shown in .
The pretest and posttest consisted of the same multiple-choice questions, and the test content is aligned with the curricular content required for ALS training at Acibadem Mehmet Ali Aydinlar University. This test is being used to assess of ALS training at our university and is compatible with the content of the latest version of the ERC’s training algorithm. The two-rounded Delphi technique was used with content experts when generating the pretest and posttest questionnaires. Group C participants took part in an interactive lecture with the instructors. VR group participants took part in a VR familiarization session and played 1 round of the VR beginner training mode followed by 1 round of the VR advanced training mode (). The total time spent for these 2 rounds with VR training was equal to the time the other group spent during the classroom lecture.
Figure 5. Real-time video of the trainee and virtual reality–based advanced life support serious game screen recorded during training sessions. Afterward, participants of both groups took part in a skills training session in order to learn effective CPR and ventilation with a CPR manikin (CPR Lilly Pro+, 3BScientific GmbH; ) [].
Figure 6. Hands-on training sessions performed using Basic Life Support manikin (CPR Lilly Pro+, 3BScientific GmbH). After the skills training, both groups took part in simulation-based ALS scenario using a patient simulator (Apollo Patient Simulator, CAE Healthcare; ) []. The participants were divided into Code Blue teams, each consisting of 5 participants during each simulation session. The content and flow of the scenario were identical to the scenario used for the VR-based ALS serious gaming module. The manikin-based simulation sessions of all participants were video recorded for evaluation. Two independent instructors evaluated the video-recorded performances of the participants during the manikin-based simulation scenario with the same scoring criteria used for the VR-based ALS serious gaming module. Following the manikin-based simulation session, all participants were asked to complete the posttest, which was identical with the pretest. Group C participants were given the opportunity to try the VR-based module at the end of the study; however, as the VR group participants were attending classroom-based lectures during their standard educational program, there was no need for them to take part in a classroom-based lecture at the end of the study. At the end of the study, all study participants were asked to answer the preference survey form shown in .
Figure 7. Manikin-based simulation of an advanced life support scenario with a high-fidelity patient simulator (Apollo Patient Simulator, CAE Healthcare). Ethical ConsiderationsThis study has been approved by Acibadem Mehmet Ali Aydinlar University’s Ethical Committee (approval number 2022-19/04). After being informed about the content of the study, the participants signed informed consent forms. The collected data were anonymized in order to safeguard participant information. No compensation was provided to the participants.
The VR group’s pretest scores ranged from 21 to 79 (mean 43.36, SD 15.47); those of group C ranged from 21 to 79 (mean 46.53, SD 15.05). The pretest scores did not differ significantly between the 2 groups (P=.62). The VR group’s posttest scores ranged from 32 to 89 (mean 53.79, SD 14.01); those of group C ranged from 47 to 84 (mean 64.20, SD 9.96). The posttest scores were significantly higher in group C than in the VR group (P=.01). The results of the pretest and posttest are summarized in .
Table 1. Evaluation of pretest and posttest results (N=29).aVR: virtual reality.
bGroup C: conventional training group.
cMann-Whitney U test.
dWilcoxon signed-rank test. Overall difference in pretest and posttest scores: P=.001; difference in pretest and posttest scores in the VR group: P=.02; difference in pretest and posttest scores in the group C: P=.002.
A comparison of the mean pretest and posttest scores is shown in . There was no significant difference in the pretest scores between the 2 groups (P=.62). The increase in posttest scores compared to that in the pretest scores was significant in both groups (P=.001). The increase in the posttest scores compared to that in the pretest scores was significant in the VR group (P=.02). The increase in the posttest scores compared to that in the pretest scores was significant in group C (P=.002). A significant difference was found between the posttest scores in the 2 groups, and the scores of group C were found to be significantly higher than those of the VR group (P=.01).
The score of the manikin-based simulation session, which was considered the main performance indicator in this study, was evaluated with the same scoring criteria of the serious gaming module. The scores of manikin-based simulation session are summarized in . The total score of each participant consists of the CRM score and the technical score depending on their performances in the manikin-based simulation session. The maximum achievable CRM score was 30, and the maximum achievable technical score was 70. The total score was calculated as the sum of the participant’s technical and CRM scores.
Table 2. Evaluation of manikin-based simulation session scores (N=29).aVR: virtual reality.
bGroup C: conventional training group.
cMann-Whitney U test.
dCRM: crisis resource management.
The VR group’s CRM scores ranged from 13 to 16 (mean 14.45, SD 1.23); those of group C ranged 3 to 16 (mean 11.33, SD 5.37). CRM scores were not significantly different between the 2 groups (P=.23). Mean values of the CRM and technical scores of both groups during the manikin-based simulation session are shown in . The technical scores of the VR group ranged from 44 to 60.5 (mean 53.80, SD 7.63); those of group C ranged from 51 to 68.25 (mean 60.20, SD 8.13; ). The technical scores of the conventional group were significantly higher than those of the VR group (P=.03). The VR group’s total scores during the manikin-based simulation session ranged from 57 to 76.5 (mean 68.25, SD 8.81); those of group C ranged from 64 to 85 (mean 71.53, SD 9.89; ). The total scores of both groups during the manikin-based simulation session do not differ significantly (P=.53).
At the end of the study, a preference survey was sent to the participants to evaluate VR as a learning modality. The outcome of this survey is shown in . More than 65% of the participants agreed or strongly agreed that they would prefer VR-based serious game–based training instead of classroom-based lectures before manikin-based simulation sessions. The survey also revealed that more than 80% of the participants agreed or strongly agreed that VR-based training is more effective for them in terms of teaching CRM skills. More than 70% of the participants agreed or strongly agreed that VR-based training also increased their level of self-confidence during the manikin-based simulation scenario. More than 90% of the participants enjoyed participating VR-based gaming.
Figure 8. Results of the preference survey. It has been investigated whether the use of serious gaming modules could replace lecture-based sessions that are mostly organized prior to medical simulation sessions. Several studies have compared serious games with lecture-based sessions in terms of providing knowledge to the trainees [,,]. Several studies have revealed that learning outcomes in the serious gaming groups were significantly better than those in the lecture groups [,,]. Depending on our results, the increase in the posttest scores compared to that in the pretest scores was significant in both VR group (P=.02) and in group C (P=.002). A significant difference was found in the posttest scores between the 2 groups, and the scores of group C were significantly higher than those of the VR group (P=.01). A significant difference was found between the technical scores during the manikin-based simulation session between the 2 groups, and the scores of group C were higher than those of the VR group (P=.03). CRM scores during the manikin-based simulation session were not significantly different between the 2 groups (P=.23): the mean score of the VR group was 14.45 (SD 1.23), whereas that of group C was 11.33 (SD 5.37). As the maximum achievable CRM score was 30, both groups had low CRM scores, indicating that participants may require extra additional training to improve their CRM performances.
The total score of manikin-based simulation session was considered the main performance indicator of the study for measuring the participants’ learning outcome. The VR group’s total scores during the manikin-based simulation session ranged from 57 to 76.5 (mean 68.25, SD 8.81), while those of group C ranged from 64 to 85 (mean 71.53, SD 9.89). According to the statistical analysis of the 2 groups’ performances, the total scores during the manikin-based simulation session did not differ significantly (P=.53).
Comparison With Prior WorkThe advantages of using serious gaming as an additional training modality for simulation-based education in health care have been revealed in several studies [,,,,]. As CPR performances of trainees deteriorate after a certain time, refresher training is required to maintain knowledge []. It has been shown that VR-based serious gaming for ALS is an adequate method to sustain knowledge [,]. Serious game–based ALS training has the potential to prevent deterioration of knowledge between updates due to its potential to be used anytime or anywhere []. However, for skills training, there is still the requirement to take part in simulation-based ALS training at regular intervals to maintain motor skills. A unique feature of the VR-based serious gaming module, which has been developed for this study, is that the serious game automatically scores CRM skills of the trainees apart from technical scores. When serious gaming modules for ALS training available on the market are compared with the module developed for this study, the available serious gaming modules on the market are not capable of evaluating nontechnical skills in addition to technical skills.
LimitationsThe first limitation was that only limited gameplay time for the ALS serios gaming module could be given to the VR group participants due to the busy schedule of the training center. The second limitation was that only the heads of the avatars could be displayed during the game, as wireless VR headsets available on the market can only provide limited processing power and memory capacity. The third limitation of the study was the limited number of participants. The fourth limitation was limited generalizability of our results due to this being a single-center study and recruiting only 1 type of participants.
ConclusionsAfter comparing posttest and pretest outcomes, all participants improved their knowledge, but the posttest scores of group C were higher than those of the VR group. Both groups’ overall performances in the manikin-based simulation session did not differ significantly. The preference survey reveals that the majority of the participants would prefer a VR-based ALS serious gaming module instead of lecture-based education. The benefits of using a VR-based serious gaming module are the scalability of trainees’ performances and its flexibility to be used anywhere and anytime by the users. Further studies are required to reveal the learning outcomes of VR-based ALS serious gaming. Future studies could further reveal whether the performance of the VR group may improve if more time were given for users to interact with the VR-based serious game.
The authors would also like to thank biomedical support team of Center of Advanced Simulation and Education of Acibadem Mehmet Ali Aydinlar University for their valuable support during this study. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
The data sets generated during or analyzed during this study are available from the corresponding author on request.
MEA was responsible for study conceptualization, methodology, and resources and drafting, reviewing, and editing the manuscript. AEÖ was responsible for the software and methodology. DK was responsible for study conceptualization, methodology, and reviewing and editing the manuscript. TU was responsible for the study methodology and data collection. The authors did not use generative AI to write any portion of the manuscript.
None declared.
Editorial notice: This randomized study was retrospectively registered. The editor granted an exception from ICMJE rules mandating prospective registration of randomized trials, because the risk of bias appears low and the study was considered formative, guiding the development and initial evaluation of the application. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness, as retrospective registration does not prevent authors from changing their outcome measures retrospectively.
Edited by T Leung; submitted 03.03.23; peer-reviewed by T Erdem, N Moore, I Bindoff; comments to author 06.07.23; revised version received 04.08.23; accepted 30.08.23; published 28.09.23
©Mehmet Emin Aksoy, Arun Ekin Özkan, Dilek Kitapcioglu, Tuba Usseli. Originally published in JMIR Serious Games (https://games.jmir.org), 28.09.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.
Comments (0)