AI and Accessibility

Hello all, welcome back to our course on digital accessibility. And today we'll talk about a very up and cominging emerging topic of um uh today's day and age which is AI and accessibility. So I am sure we are all aware of the different ways in which AI is improving accessibility. So one is of course uh providing tools that can enhance digital experiences for people with disabilities or AI powered tools that can also make testing for accessibility more intelligent and efficient. These are two broader umb umbrellas. uh but we'll also talk about what kind of applications what are the different ways AI can intervene in in making um digital products and interfaces more accessible and inclusive. The first important intervention that can happen is in content creation and conver uh conversion. Right? So AI can now uh automatically transcribe audio and video text uh into text, create autogenerated audio descriptions for images or um videos which can make uh images and videos accessible to screen readers and hence to people who are visually impaired. Convert text into various other multimodal formats like braille um etc. making information accessible to those with visual impairments or hearing impairments. So, content creation is one big area which I'm sure you're all aware uh how AI is uh interjecting. It is also now able to autogenerate uh close captioning with ease. And uh now it is very easy for content creators themselves to integrate these um AI tools in their videos as well. And um um even earlier if they are not aware of accessibility as uh as a issue they can now easily integrate those plugins in their uh created content which makes it available for a wider uh set of audiences hence uh enabling wider reach for their content as well. The second important uh domain where um AI can find key application is assisted technologies. So AI enhances tools like voice assistants because based based on large language models it is able to enhance the robustness of voice assistance. how it is able to recognize individual voices, individual accents uh across the globe is powered by large language models today and thus it is able to enhance the user experience and the robustness related to voice assistants making them more capable of understanding diverse speech patterns. uh accents is one aspect but there may be list lips lisps stutters all of those aspects uh speech impairments related aspect which is it's now able to understand right AI powered hearing aids can now use machine learning to distinguish speech from background noise it can also in real time translate one language to another and skip the need for a human translator to be present uh at all times. So hearing aids are now capable of realtime language translation as well which is again powered by uh AI and LLM digital accessibility testing. Of course, this is one of the domains we have already spoken about testing and evaluation tools which makes uh auditing, evaluation of assistive technologies, evaluation of any digital product from an accessibility perspective easy. But with the use of AI based plugins, it will be much more efficient. It will be able to crossch checkck your product across competitive uh platforms across uh other digital products serving similar objectives or similar purposes. It can cross-check your um uh nuanced accessibility criteria as well which um which it was not automated tools like wave were not able to do earlier. Now with AI based plugins they will be able to do that as well. So things like whether the all text provided. So earlier it was whether there all text is present or not. So there was no semantic analysis right? So uh even if you write alt um yes there is a description uh in the alt text uh in in an evaluator it would say that yes there is an alt text present. it was largely binary. So, but whether that alt text is able to really convey the meaning of the image, right? Whether it the semantics of the image is mapped to the description or not that required human uh intervention that required human analysis which is now uh possible through image recognition image analysis based u models which can understand extract um uh semantic meaning of the images or videos and then either generate description or check from an evaluation perspective or check whether the all text provided is actually making sense. It's actually delivering the meaning of the image or not. So um so basically techniques to analyze digital products more comprehensively uh mimicking realw world user interactions to find issues that uh automated tools might have missed earlier. So let us see let us talk about some very interesting upand cominging examples which have now been integrated in most of our use cases. So Microsoft has come up with a dictate tool which is um um you know you can find it on your word dashboard. So, dictate is um is basically a voicetoext software now embedded in your word um MSWord um um you know office um dashboard itself. So those who struggle to type earlier can also access the features of uh word and create new documents with ease uh by using dictate. And uh it can also make lives easier for um people with temporary disabilities. Say you are um you want to type but you have injured fingers or some other hand injury etc. and for some reason you are unable to type, you can uh use this dictate dictate features and uh still create uh documents with ease. Then again MS uh there is a read aloud tool which can read out a document or presentation helping people to understand and process the information and it can help with concentration also. So for example if the document is very lengthy. So of course the direct use case is that a read aloud is can benefit persons who are visually impaired. The other use case can be that if the document is very loud uh very long sorry if the document is very long then the read aloud um tool can help uh keep reading the document and it can help you concentrate. You don't need to keep looking at the screen in order to read. It will save you um uh with all the screen uh time or uh aspect uh especially if it's a you know lengthy document. So you can use these um tools in embedded in word itself. Other of course is uh as we talked about earlier audio descriptions which can explain what is happening uh can help those in a video or in an image which is which is basically describing an image. So now it is it is all of these tools of auto generating all text are easily available throughout the internet and you can uh basically they are AI based image or video analyzers which can generate or uh text descriptions which you can add as all text thus making your content your images your um videos accessible to screen reader and hence people with visual impairment. So they largely use uh these kind of models where you input an image uh the models are able to extract some of the features they kind of do an interpretation and then they generate description. So even if it is not entirely 100% correct, it gives you uh you know a draft which you can you know kind of edit which makes the process a lot faster and with the kind of uh innovations happening with the LLMs and AI models I'm sure the generated descriptions will become even more robust and even more um uh more semantically aligned in the near future. Then some of the other commonly known tools which uh such as co-pilot. So co-pilot also uses like large language models uh which can help summarize and spotlight key information from meetings uh etc. So people can refer to or digest information in alternative format. Right? So tools like copilot has the potential to remove challenges and make work easier while improving the quality and efficiency. So particularly in if you're attending an online meeting or if you're attending um um you know a meet you can ask copilot also to attend uh as a separate as your assistant and uh it will basically summarize make of minutes uh minutes of the meeting for you uh summarize the points made by different stakeholders in the meeting and convey to you in May be other multimodal formats uh um which you can access u maybe later as well if you have neurode divergence uh related concentration related uh issues if you have hearing related issues probably um then if you want to refer back to those comments after after a while it is it is able to do that uh for view. So I think that is another very important uh role which an LLM and AI based solution can uh play. So models like these or uh tools like these are able to now make it possible for people with uh permanent sensory disabilities also or people with neurode divergence also get a seat at the table and which was not the case earlier. there were um very limited opportunities available primarily because of access right so and even organizations did not know how and what to include um in their ecosystems in order to enable it's not that they wanted to exclude but uh the ecosystems and the plugins were not so easily available or they were um uh you know not as efficient in actually making the environment inclusive. So now with these technologies even organizations can feel free to include um people with multiple kinds of limitations despite their limitations so that they can add value to the organization or project or the project they are brought on to um contribute to. So I think uh these kind of technologies can be really helpful. Then of course shad GPT I'm sure you're all aware of it. So LLMs like chat GPT and copilot can rewrite content in a more casual or easier way uh which can help people understand. So if but primarily if the uh conversation is around um you know terminology heavy um points if there is um techni technological terminology or management related terminology which for example you're unable to understand uh right in the moment. uh you can uh use these models to either describe them to you in real time or it can generate minutes of meetings etc in a more easy to understand format which you can refer to after the meeting as well. So uh accessing um libraries, accessing uh the internet and like finding meanings in the real time is is something which is a very up and cominging technology. It's an it's an important emerging technology. Real time translation tools, real time uh summarization tools, real time semantic analysis tools uh and they will keep becoming more robust as we uh go. This can help people to read uh write navigate information easier. Um remove the reliance on physical or sensory capabilities. Uh and this can also enable access to information in a multimodal format. Then another common uh example of use of AI for making um technology more accessible is transcripts and captions. So um so you can easily switch on captioning uh in meetings or include them within videos to support people who may be struggling to uh hear for example or people who are accessing the content in a noisy environment or I mean if this image is also depicting it can be realtime language translation as well. So, uh close captioning um open uh uh video captioning or uh real time translations. make making it easier for content creators to add close capt captioning as well in their videos. Thus expanding uh the user base, the subscriber base to a wider set of audiences and also the messaging can reach across um globe and uh I mean the barriers of language etc are no longer there. there can be uh more global participation in all sorts of conversations and all sorts of uh tables uh because of such realtime translation technologies we don't need to rely or we don't need to learn wait to learn other languages right so it if we are speaking if I am I am a researcher based in India I am speaking in Hindi the realtime translator on say um a Microsoft meet is able to translate it in real time and communicate it in English to the other person who is joining in from London for example. So there is no barrier uh per se but the robustness and the efficiency may not be um you know very high currently but in the near future I think this can be on and these are some of the aspects in which if you are a engineer you are a UX designer you work in an organization which looks at AI as an opportunity to solve uh access related issues issues. You can also uh you know look at it as an opportunity to uh work on such uh domains and make the systems a little bit more robust uh by adding your uh bit. So um this is um uh Jenny Le Flurry and uh she is uh an accessibility expert with Microsoft and uh there are a couple of videos which I would like you to uh see particularly some sections uh which can help you in understanding uh some of the key aspects of how accessibility can help with um uh can be improved with the help of AI. So let us now uh watch some video clips uh of Jenny Lelay Flurry where she talks about u how accessibility is impacted in the age of AI and she is the chief accessibility officer at Microsoft and I'm really glad that Microsoft is leading uh change in terms of accessibility and inclusion and they have positions like this uh in their organization which means that they are very uh inclined towards inclusion top down. Uh so let us check it out >> and I can select live captions. Live captions come in. I can also then figure out which language I want to use. I can if there's a Chinese speaker, a German speaker, I can change it to that language. I can have multiple languages happening at the same time. So if a person is Chinese and is speaking, the captions come out in English. It's a really powerful illustration of how we're bridging between accessibility and localization. >> Another aspect of uh AI and accessibility is looking it from looking at it from the other uh lens and uh thinking about how accessibility is shaping the future of AI. And uh again here we have the same uh person Jenny who is the chief accessibility officer at Microsoft. Um in talking about uh this aspect you can again go ahead and watch the whole video but here I'm just showing a small clip >> in our case are people with disabilities. Yeah, >> that's our biggest learning. Um, if you boil it down, which has never been more important as we think about AI and what is possible across the spectrum of humanity, but particularly disabled people, which is 1.3 billion people around the world. 83% of us will experience disability at some point in our lives. >> Exactly. And I think what's super important as well is that when you take into consideration allowing for people of disabilities, building for people with disabilities, you make the world a better place for everyone. Right? If you think about ramps and curb cuts, how many of us use those and you don't identify with having a disability, right? So many organizations are introduced to accessibility as a compliance, right? Why should enterprises see it as a driver of productivity, talent, market growth, and AI innovation? >> I just love how you frame that question. >> And you mentioned history, like curve cuts. >> Yes. >> Last night I got to nip out to Berkeley to visit the Ed Roberts Center over there, which is kind of the epitome of that. Um, disability rights, >> which began right there, right? Um it be began actually here in San Francisco and over the years over the decades over the centuries people with disabilities have been innovating. Most people don't realize the electric toothbrush was designed for people with limited mobility. So you again you think about that potential. >> If you're not designing with and for humans including disability, you are quite frankly missing out. If your website, product, agent, um, app is not accessible or including accessibility, you're missing out on up to 20 to 25% of your potential customer base and your employees. >> Right. >> And remember, 75% of disability, non-apparent. You cannot see it. You don't know that I'm disabled just by looking at me. Exactly. >> Um, so it's really important to include the basics and design for making sure that you breathe in regulation. Regulation is important and it sets the minimum bar. But go after changing paradigms which is even more possible with AI right now. >> Right. Thank you for sharing that. I think for a lot of people watching like it's super important look at the history right behind things like I mentioned curb cuts regulations because how many of you especially coming here to Ignite are dragging your suitcase across the road and you're like oh I'm gonna use the curb cut. Right? If we weren't innovating and building for people with disabilities, that wouldn't exist. Now, take a moment and like marinate on that, right? And I think you have a mouse to show us some designs, right? >> Oh, right. Yeah. I mean, we've got so many toys that we can talk about. This is the Microsoft adaptive mouse. Um, and again, it was designed by people for people. M >> so uh I think this this paints a pretty nice picture of where accessibility uh particularly not just in AI but also around digital product development is heading uh when we talk about adaptive uh gaming consoles and adaptive mouses. We will also look at those aspects in another session where we'll talk about emerging input devices, emerging output devices as well. uh but uh this session and you can again uh go ahead and watch the entire video where she uh discusses various aspects of accessibility which has shaped the technology as we know it today. Right? Uh and we have also throughout our course uh spoken about these examples such as curb cuts which was being talked about such as other uh aspects of uh technology development like a digital toothbrush or um autocomplete algorithms and all of those things which were initially designed for persons uh with um access issues but now have been integrated in the mainstream use case and a wider consumer is actually benefiting uh from those enhanced experiences. Uh the same is being uh thought about in the development of AI and how accessibility can shape uh the different kinds of use cases, the different kinds of uh LMS and the need to develop those LLMs uh in the future. And um again you can go ahead and watch the entire video. I think that is the primary context. Now moving on uh to some of the considerations that uh need to be looked at when we are thinking of AIdriven accessibility. Uh the first is of course bias and equity. So we have to make sure that the AI solutions developed must be uh adaptable by diverse populations. They should be developed with by diverse populations in mind uh to avoid creating or reinforcement of biases which may exist uh because of non-ignorance of uh individual human beings. Right? So uh similarly uh another aspect is user interface design and we've been talking uh about this throughout our um course. It is crucial to ensure that AI tools themselves are accessible and that features like editing transcripts for example are not overly burdensome uh cognitively overloading for users with cognitive disabilities for example or people users who are elderly. Then of course privacy has been one of the major concerns uh when development of AI is being looked at. uh and AI for accessibility requires definite uh a careful considerations in the um aspect of data privacy and security especially when dealing with sensitive user data uh their medical history uh access related uh things which the user may not be willing to reveal and also uh being sensitive with the kind of terminologies used in questionnaires etc. onboarding processes. Uh so all of those aspects are quite important. So other than that we can also uh look at some examples which are using AI agents etc. in order to uh integrate uh those agents into an assisted technology use case. So let us look at one of this uh it's called Cboard and um you can you can actually you can actually play with it. So it's primarily an educational learning app. Uh it's an educational learning app with for children as well as adults for with speech uh and language impairments. Uh maybe it helps in aiding communication, helps in um um communication with symbols. It's also very use very useful for people who are aging. Enables easy access to text to speech. uh you can browse through uh this link and you can also access uh the app easily. It is available for free. Uh you can access it uh some trial version is available for free. You can access it uh by creating an account right here and uh you can try uh using it. So to summarize, we would like to say that AI has the potential to transform accessibility by offering tools that can support various challenges um faced by neurodeivergent individuals and people with sensory disabilities uh making technologies more inclusive etc. assisted technologies like voice to text, audio descriptions and AIdriven tools like co-pilot are already improving access to information for those with diverse needs. So uh as part of this course what we are trying to do is trying to kind of study how those tools are enhancing uh uh interface access and how implementing uh AI based tools can enhance your product as well so that a wider user base can benefit. The future of AI and accessibility is definitely promising with the possibility of integrating different assisted technologies to create a seamless and effective interaction for all kinds of users. uh so uh it is important in today's day and age to understand the power of AI and LLMs and how um access to those uh um aspects can enhance usability of your device as well. So um that's all for this session. Thank you for joining us. We shall continue in the next session.