AR VR

Hello all, welcome back to our course on digital accessibility. And today we'll talk about a very uh interesting emerging technology which is called um augmented reality and virtual reality. And we'll look at various aspects of this emerging technology and how it can be utilized to make accessible and assisted technology experiences as well as we'll try to look at it from a perspective of creating inclusive AR and VR experiences. Right? So what are the basic design principles to do that? So let us all uh begin with first trying to understand what is virtual reality versus what is augmented reality. So in a virtual reality experience the entire environment is in the virtual space. um you are interacting with that virtual world via different variable uh technologies input output devices I'm sure you might be aware of um you know the VR headsets or um uh you know haptic gloves or there may be touch sensitive um foot pads and all of those things with remote controls etc and you are basically interacting with entirely interacting with that virtual world in a virtual reality setting. However, in an augmented reality, we are actually present in the real environment. But the real environment is augmented. That means that there are some additional information. And it is enhanced using some additional visual cues, audio cues or uh tactile cues uh through your devices and uh you are able to experience the reality in addition to these digital cues and that is what we call augmented reality. Right? So uh basically just to cover a gist about it in virtual reality we are utilizing headmounted displays which are your headsets or sensors to immerse users in fully stimulated digital environments. locking out the real world all together and enabling um whole interaction through controllers, gestures, motion tracking, all of those things. While in augmented reality, the technologies is enhancing the real world environment by overlaying digital elements such as text images which you may be able to experience through smart AR glasses or tab through your phone, tablet, etc. You can place 3D images. I'm sure you might have experienced uh you know uh these things that you scan here and there is a cartoon which starts dancing and uh most of you might have also experienced the VR roller coaster at the your nearby malls. If you haven't please go and try it today just to get a just to get a hang of what is the state-ofthe-art in this technology. This kind of technology is not just happening in the gaming or entertainment or uh navigation and all of these informative uh aspects. It is also enhancing um experiences and finding applications in the healthcare as well as assisted technology sector. uh and um there are several use cases of uh uh VR in particularly rehabilitation, counseling, uh other healthcare uh aspects. So of course um PTSD related, we'll talk about it in some of the upcoming slides. mental health related therapy. Some of the people may be um you know hesitant to open up to a human counselor and also the human counselors may have their own biases uh because they're also humans and uh but a computerized uh programmed therapist who can actually be a virtual therapist is also becoming uh much of um um an interesting domain. Similarly, particularly in pediatric care, developmental disorders, kids can also be a little bit scared to interact with strange uh strangers. In the case of uh autism rehabilitation or uh educational rehabilitation uh but they may be you know um happy to interact with say a robot. So all of this is u more of under research. a lot of it's finding its um you know uh these applications in the clinical stages of trials etc. Uh then there are of course training related uh simulations you might be seeing it every day in your um uh in all of these sectors actually. So any risk uh related or any uh dangerous uh situation related um work which involves humans going into danger the training can happen through simulations in the VR environment. So be it flying a plane, be it um you know mining for gold, be it you know um surgeries, training doctors for uh surgeries um you know medical training um uh you know talking to um mental health patients. All of that training can also uh be enabled through these technologies and they reduce that risk of um um you know that training with because it's a learning phase and people might uh engage themselves just in the learning curve itself. Other very interesting aspect of uh um assistive tech u ARVR usage is in special education uh and uh assistive technologies. So particularly in special education it is finding a lot of uh applications. So one is uh social and emotional learning. So, uh, VR simulations are able to help students with autism or anxiety practice practice, uh, social skills or life skills in a virtual simulated environment like shopping or public transport. And you can increase uh, you know, in a programmed uh, efficient manner. You can start off with uh, people who have social anxiety. You can start off with a grocery shop simulation where there are not a lot of people, maybe fewer customers. Then if they are able to successfully do it, you can gradually increase the density of people in the place so that they are eased into that social environment and in a controlled setting. Then uh motor skills development. So interactive VR tasks and help in the development and enhancement of fine motor skill coordination. So the lot of VR games are finding u uh you know application in uh this domain you know spatial awareness particularly for students with physical disability or cerebral policy or all of those behavioral training AR can be used to simulate cause and effect scenario to teach self-regulation particularly emotional self-regulation uh to children with autism or um uh uh you know ADHD they can help them in making decisions they can help them in honing their decision making skills as well. Um so I'm sure you might be aware that such situational training is uh used to happen in SSC and uh uh you know army cramps but uh such kind of behavioral training can also be simulated in a VR environment in a controlled setting with levels of difficulty and uh you can tone down the level increase the level according to uh the students um um um you know uh skills. Then of course there is the visual and auditory learning. So AR and VR environments can deliver visually rich and interactive lessons uh supporting uh students with learning disabilities who struggle with say traditional uh teaching methods. So people with dyslexia who are unable to you know read and decipher a lot of details. a good um um a lot of people are exploring say three-dimensional um models of uh augmented in augmented reality threedimensional models of body parts etc. so that people can really visually experience uh you know those body parts which are very difficult to describe in words and more so difficult to read and understand for a young student. Then there can be some very interesting real world examples. So, Florio for example uses VR to teach social and life skills. It's an application. Then there is a application called Tiltbrush. So, which allows students. So, this is an example of Florio. This is more of a gamified uh interaction and uh it als it kind of presents different types of social situations in which uh which allows the students to take certain decisions on the go and they have to um they're presented with different use cases. They're presented with different tools and they have to kind of pick the right tool, pick the uh try to make their way through different situations, right? Um similarly then there is an application called tilt brush which allows students with motor limitations to paint and express themselves in a 3D space. So you can actually paint around yourself and you can you're not confined to a two-dimensional flat surface because that requires a certain way a certain posture certain um orientation of the hand or or fingers or uh the arm itself. If it's a three-dimensional open space, if you are if you want to start if your hand if you're comfortable keeping your hand this way and you want to start drawing here, it's okay, right? It will be able to draw. So I think this this is a very uh interesting uh application and of course you can go um and you know find some more application and we can have uh you know an interesting discussion in the forums as well about it. There's a lot of innovation going on in this uh domain and if you're an entrepreneur uh who would like to explore uh this domain particularly with emerging technologies like AR and VR um creating applications for um persons with different kinds of special needs can be a very good uh gap which you can aim to address. Now we uh try to um you know talk about the importance of accessibility in AR and VR. So for example uh how to create applications. So so far we were kind of talking about how AR and VR is enabling um you know solving certain problems for um certain kind of use cases as assistive technology solutions. Then there is another aspect of accessibility in virtual reality. Accessibility in augmented reality, right? So which refers to design and implementation of virtual environments that accommodate users with diverse abilities and needs. So the virtual environment itself should have uh various design aspects so that it more and more people can experience that virtual environment. So for example um uh for mobility impairments you may require adaptive controls and alternative interaction modes. For visual impairments, you may need to highlight uh the need for more nonvisual feedback. So it can be haptic or audio. And now we are also uh I think moving into alactory. Um alactory is through the sense of smell. So we can uh you know explore all of these hearing impairments make captions uh subtitles essential for immersive uh content access. So uh it is important it may need it may seem futile to add captions sometimes in a virtual environment because in a real environment you you don't all all the time have uh close captioning available right so uh and it may seem like dilution of uh reality in a virtual reality experience um but uh it should be an option at least uh to make um sub to make subtitles and captions appear in case uh the user demands it. And then cognitive disabilities call for simple intuitive interfaces that have minimal complexity and u you can utilize all of the other visual design elements that we have discussed in our previous sessions which apply to u you know or your web interfaces or your digital interfaces. All of your same uh principles can now be extended into a virtual environment as well. How not to make it cluttered, how not how to uh enable affordances so that call to action is uh present in your interfaces all of those aspects. So coming to uh challenges for users with different disabilities in ARVR environments and applications. So for physical and mobility barriers, many ARVR systems require body movement uh gestures or heavy equipment which they may not be able to lift and operate uh systems uh in an appropriate manner. So placement of interfaces inside the virtual environment because it's um it is um it is kind of a digitally created virtual environment right so I mean it is possible to make it adaptable so it is it is always um good to have adaptive positioning of say control panels or so that doesn't uh necessitiates the use of a controller in a certain position aiming at a certain angle and then only you are able to control uh the buttons or the interface. I think that is limiting uh um you know in its approach and one should not go for that. So adaptive placement of interfa interfaces or dynamic also I mean it can be that you can choose if it is not adaptive it can be dynamic you can allow people to choose their frame of reference and then accordingly the whole uh environment shifts basically the ground shifts if somebody is actually lying on a head and they want to experience the environment but you have created it with respect to your uh horizontal of the real world and it's fixed in that manner then it is not possible for them to interact with the environment. uh so rather uh they it should be allowed for the user to choose their own frame of reference rather than the designer or the developer fixing it for them in the um in the virtual world. I think that is the flexibility with the virtual world enables and the real world does not right. We cannot change the horizontal with respect to the earth but in the virtual environment we can do that. So why not explore it in that and make it a more inclusive environment. Then maybe v visual or other sensory barriers. So VR really heavily depends on visual content primarily in gaming and entertainment. We've seen that it's largely visual primarily because the VR headsets were the initial uh headpoint and of course more visual information kind of is built into creating that world. Uh but often it lacks there's no qualms about not creating visual content for the virtual world but it should not lack audio descriptions or tactile feedback right because in the real world you are able to uh you know kind of place your hand on the table not just see the table but actually feel it as well. So all of those aspects should not be missing uh from your uh virtual environment as well. And there can be cognitive barriers uh fast-paced interactions particularly in gaming uh maybe or or um entertainment. Too many complex instructions in trainings etc. All appearing in one go or non-customizable interfaces can overwhelm users with dyslexia, ADHD or autism. lack of inclusive design. So, absence of accessibility features can lead to exclusion, frustration rather than engagement and immersion. So current uh state-of-the-art um uh is like improving at a very fast pace with platforms like Oculus Quest and V uh HTC Vive offering adjustable controls, height settings, onehanded play also is possible. These features in increase flexibility inclusion, enable users with different physical abilities to navigate the virtual environments. uh therapeutic VR applications example for anxiety or PTSD show strong potential by supporting diverse users through immersive experiences. Accessibility implementation is in inconsistent across platforms resulting in fragmented user experience. So I mean this is again a challenge but how can we create um inclusive uh more inclusive ARVR experiences. So some of the recent innovations um are um have made paved the way for more uh enhanced interactions. So recent innovations in VR, AR technologies are making virtual environments more inclusive primarily with the technological development of more advanced uh interactive variable devices. Um eyetracking enables hands-free navigation. So you don't need a remote uh um at all times to interact with the virtual environment. just based on the gaze, just based on where your eyes are looking, uh the system is able to understand what you want to do. Uh and you can gaze on a button for a few seconds in order to enable it. All of those kind of interactions are uh being possible. Voice recognition allows user to uh control VR experiences through speech. Haptic feedback adds tactile cues that enhance immersion and support navigation. Integrating all of these technologies promote equal access. So then uh let us talk about the basic design principles for creating accessible VR. So first is it based on all of these discussions if we have to summarize the design principles. So basically interfaces in ARVR should be flexible and customizable. So allowing user to say adjust text size, color, the amount of uh interface they want to see at a given time and all of those things should be customizable user controlled experiences. So, so you support personalizations through adjustable content or there can be agentic avatars um who can you know kind of assist you in the controls. Try to reduce sensory overload. Uh offer options to limit visual and audio and haptic intensity. Uh if I want to see only a few buttons at a time, it should be possible. include breaks. The duration and the frequency of break should also be customizable. Um, and use clear and intuitive cues. So, but like what what is the meaning of an intuitive cue is also debatable. I mean, uh, it is it is more of a research question. Uh, and it there's there's a lot lot of research going on in this domain as well. what what do we uh really associate meaning with? So not just intuitive but also semantic cues, semantic audio cues, semantic visual cues is something which is becoming a very interesting area uh of research for people who are working in the domain of interaction design and cognition. Uh accessible hardware and control. So design of ergonomic adjustable headsets and controllers with customizable buttons. So hardware also needs to have uh customizable designs um and ergonomic kind of shapes and forms so that people are able to hold it in a much uh easier and comfortable manner. uh adjustable headsets that has become very uh the norm with a lot of straps and all of those things. Lightweight headsets is also something people are now working on because it it after a point you're unable to uh kind of you know keep it on your head. Clear navigation and universal uh usability. So uh simple pathways, strong visual cues and spatial orientation. This part as I as we discussed earlier maybe the frames of reference can also be adaptive. Uh that would be the ideal situation or customizable also. So how we can try to approach uh enabling all of these design uh aspects in our VR applications. So uh diverse approaches uh are being used. I mean one is trying to tailor uh features to specific user needs uh to ensure that individuals with different abilities can effectively engage. So this is of course um you know in line with most of the companies or most of the products what they're doing is that they set these you know whatever these customizable and all of those things to a particular setting and then create the virtual environment accordingly based on a specific target users needs. So for example, if I want to create um based on the FIO example, if you want to create a gamified social skill learning platform for a child with autism. So this is a well-defined problem and then based on that you can create the virtual environment. You can create uh you can enable the settings in the uh in the headset in the virtual environment in the controllers that primarily a child is going to use it who has uh autism who is on the autism spectrum needs help in um developing social skills. So such situational social environments need to be created all of those is there. So this is one commonly uh used approach and this kind of a customization uh plays a key role in improving comfort and control and overall user experience. So at least in currently uh in the new and up and cominging products which are tailored around uh VR and AR experiences. This is the most common approach which is being used and they establish an important industry standards and best practices for accessibility in virtual reality and then uh they are also trying to enable multimodal interactive elements. So and the the aspect is that pe the companies the companies who are making um the headsets or uh who are making platforms for creation of virtual environments. they are uh developing um uh you know their products uh with a lot of possibilities just by making some you know customizations and tweaking the package according to a use case and they are enabling a lot of multimodal interactive elements. So if you if you want to buy there like a full bodysuit is also available uh which can enable you to interact uh in the virtual environment. Of course, I'm not going to talk about the costs at this point because of course we all can agree that they're quite expensive. Headsets and controllers have now kind of come down in costs uh a little bit and they have become more accessible in um in in a lot of use cases we that we see. So from a community perspective uh on accessibility in AR and VR uh user experiences from people with disabilities play a crucial role in shaping more access inclusive and usable virtual environments. So feedback is very important in as discussed in a lot of previous experiences of websites of applications and similarly that extends that formula extends to creation of virtual environments as well. And uh feedback from people with uh different use cases and different abilities can help you enhance the systems in a much much better way. and they uh play a crucial role in shaping more inclusive and usable virtual environment. Visually impaired users can highlight challenges uh regarding poor navigation emphasizing the need for clear audio, haptic feedback, multimodal feedback, etc. Consistent feedback shows the importance of multi-ensory design to improve orientation, immersion, and overall accessibility. And developers are increasingly adopting user centric design approach by engaging directly with the communities of special users from the early stages of development itself in order to make uh successful products. To summarize this session, uh ARVR can be a good platform for engaging users with varying disabilities. It is now finding a lot of applications around assistance, rehabilitation, health care and a lot more. Um, creating inclusive AR and VR experiences is the need of the hour which requires human- centered design approach and involving uh users from the beginning uh till the development process is complete. Uh so in this session we've not spoken a lot about compliance because uh so far the compliance standards are still not talking a lot about uh we are experience compliance. Probably in the near future with WAG 3.0 we'll be able to see more on uh compliance related to virtual uh and augmented reality experiences as well. Uh that's all for this session. See you all in the next session.