How VR Is Being Used to Help Children With Learning Disabilities, Autism
By Emily Gera
LOS ANGELES (Variety.com) – The United States Department of Education is betting on virtual reality to help students with high-functioning autism and learning disabilities in schools across the country. This month the Office of Special Education and Programs announced its investment of $2.5 million toward a new program that will use VR to nurture social skills in students with disabilities – an extension on earlier funding for versions of the program designed for desktop and tablets in 2011.
The project is called VOISS: Opportunities to Implement Social Skills, and is developed by researchers at the University of Kansas’ Center for Research on Learning and Department of Special Education – a collective of academics and Ph.D.s who are treading the waters that intersect virtual reality and education. In addition to students and teachers participating in the project, KU researchers will work in collaboration with educational leaders at the Ohio Center on Autism and Low Incidence Disabilities and a team of computer scientists at Western Illinois University.
Students with disabilities often do not learn social skills or pick up on social cues at the same pace as their peers. So, posit the researchers, could these behaviors be cultivated in a safe, controlled, but virtual environment?
“With previous funding, we developed and did initial testing of our virtual reality tool to develop and improve social competencies with students with high-functioning autism and learning disabilities,” says Sean Smith, professor of special education and co-principal investigator of the project. “Based on that work, we’ve wanted to expand and further investigate this tool with students and teachers in schools, and this grant will allow us to do that.”
With this second injection from the Department of Education, researchers will develop a five-year program to implement social-centric and provide evidence that the system actually works. VOISS will be taken to a minimum of 17 schools throughout the Midwest, where middle-school students will be able to interact with avatars in virtual school hallways, lunchrooms, locker rooms, buses, and classrooms, and react to school-based social situations – A conversation, a response to questions asked, or problem-solving in the everyday. Soon new settings will be added, including a movie theater and sports arena.
“Once the user puts on the VR HMD, [they] will be able to walk around the various environments exploring different situations,” says Smith. “The user will be able to walk up and interact with computer-driven avatars. This interaction will allow the user to understand consequences or positive or negative interactions. For instance, if the user is constantly looking down, the avatars may cut the conversation short and walk away. However, if the user makes eye contact, then the avatar will respond positively by smiling and conversing with the user.”
“Our idea is to use virtual reality to explicitly teach students these social competency skills and how to generalize them in their natural environments. Likewise, we want to equip teachers with ways to implement virtual reality and support student generalization of skills into the classroom setting,” writes Amber Rowland, Ph.D. in the official press release for the University of Kansas.
Rowland is an assistant research professor with the Center for Research on Learning and Co-Principal Investigator on the project. “The development has always focused on school-based access and thus, we developed the virtual experience for what K-12 school would in respect to technology access,” she tells Variety. “At the time of our initial development, desktop and laptops were the norm. We were able to develop for the iPad in 2015.”
In its current state, the program can be used on any computer or mobile device and in fact doesn’t require the use of a VR headset, says Rowland.
“We are purposefully developing in the web and in headsets so that there is greater access to the benefits of VR and also because many students with Autism do not want things on their face. So, students will be able to use the system wherever they have access to a web-enabled device. At this time, high powered VR headsets are not commonplace in schools or many homes.
But it’s only a matter of time before virtual reality becomes a mainstream preoccupation, Rowland continues. “We anticipate during the five years of this grant that will shift. At first, we would anticipate these sorts of VR headsets being kept at school and used there, but as systems become more prevalent, we anticipate students owning their own.”
And yet why use VR as a teaching environment at all?
According to researchers, the safe and controlled environments of virtual reality can indeed be ideal for students with autism, students whose perception isn’t just different but often superior.
“According to reduced generalization, perceptual processing in autism can be explained as reduced processing of the similarities that are held between stimuli and situation,” says Justin Ehrlich, Ph.D., Associate Professor of Computer Science at Western Illinois University and Co-Principal Investigator. “VE seem ideally suited to making the classroom more home-like (or vice-versa), and so VE can reduce the amount of processing required to go between stimuli and situation. In theory, the more realistic and immersive, the less processing is required.”
“The advantage of creating a realistic experience is this provides an ideal learning environment since all aspects of virtual reality can be controlled in a safe environment,” he continues. “There are many studies that suggest that HMDs are a fantastic medium for teaching due to providing a sense of reality. Even though users have the illusion that the experience is real, they know it’s not actually happening. This allows for individuals with phobias to work through these fears utilizing the realistic environments.”
From the viewpoint of cognitive science, studies have found that virtual experiences improve cognitive flexibility outside of the digital intervention. These studies suggest that the user’s brain can form new anatomical functionality, their neural connections rerouting while in the virtual environment. This is what’s known as the brain’s functional plasticity, and this not only improves the ability to generalize what’s been learned while in the virtual world but also allows for those improved skills to be perpetuated outside of it.
But for a virtual world to be an effective learning tool, it needs to manage both to saturate the senses and maintain a sense of realism – what Ehrlich refers to as the level of presence in a virtual environment, the weight and reality of simply
being there
.
“There has been much work on determining ways of increasing this sense of reality,” says Ehrlich. Research by virtual environments professor Mel Slater identifies three major factors needed to recreate that sensation: Immersion, Place Illusion and Plausibility Illusion.
“Immersion is the technology driving the experience and includes the screen resolution, type of display,” says Ehrlich. “HMD versus traditional desktop display, screen rate, sensorimotor contingencies, and latency of feedback. The higher the screen resolution, sensorimotor actions, and screen rate, the better the presence. Likewise the lower the latency of feedback, the more the mind believes what is happening is real. A head-mounted display is much more immersive than a traditional desktop.”
“The other two factors, Place Illusion (Pi) and Plausibility Illusion (Psi) both describe the scene being viewed. Place illusion is the realistic nature of the scene. This is how real it looks and describes the sense of being in the scene. Psi is the illusion the interaction within the scene is real, or what is happening is actually happening. If avatars in the scene react realistically to each other, or to the user, Psi is maintained. If there is a high level of Place Illusion and Plausibility Illusion, and the user is experiencing the scene using a technology with a high level of immersion, then the user will sense that the simulation is real and can lead to realistic responses and behavior from the user.”
“If avatars in the scene react realistically to each other, or to the user, Plausibility Illusion is maintained. If there is a high level of Plausibility and Place Illusion, and the user is experiencing the scene using a technology with a high level of immersion, then the user will sense that the simulation is real and can lead to realistic responses and behavior from the user.”
“We found that students who received the virtual reality experienced increased in their understanding of social skills,” Rowland continues. ”This increase was significantly different from students that did not have access to the virtual reality experience. We also found that students who learned from the virtual reality experience were also able to generalize their understanding to non-virtual environments. Finally, students expressed a level of understanding and presence in the virtual reality experience enhancing the learning experience and understanding of the social skill being described and in which the student interacted.”
“There are a wide variety of social encounters that are based on our targeted users’ own social encounters,” she explains. “This includes both school-based encounters, like classrooms, cafeterias, and will soon include daily life encounters like movie theaters, sporting events, etc.”
“The social encounters vary in complexity based not only upon the user’s age and specific social need but also upon negative or positive feedback within the environments itself. If the user is progressing, the difficulty is increased to assure the user is always being challenged. In addition to the level of complexity of each situation, the situations are categorized, allowing for two-dimensional complexity optimization.”
“In theory, the more realistic and immersive, the less processing is required,” says Ehrlich.
“VR adds [that] additional level of presence, or the feeling of realism, to the environment, which should allow the users to generalize the skills learned in the virtual environment to the real world. Just as we have found that running the program on a tablet is more effective than using flashcards, we hypothesize that running it in a VR HMD will lead to a greater level of effectiveness. The goal is to give the user a feeling that the virtual environment is a real place, and the interactions are plausible. If we succeed in this, then the skills learned in the environment should carry over to the real world.”
“If the virtual experience is realistic, the user is able to use this pretense to link the environment to the real world, which increases the chance of generalizing skills to the real world.”
“The potential for improving learning in VR is huge.”
