Accessibility Testing - Part 1

Hello everyone, welcome back to our course on digital accessibility and today we will talk about accessibility testing. Okay, some of it we have already spoken about in a uh brief manner in one of the earlier weeks and we've also briefly shared about some of the accessibility evaluation tools etc. uh in one of the earlier sessions. Uh what we are going to do this week is kind of dive deeper into accessibility testing audit methodologies related to it and what are the different ways industry um is uh you know employing accessibility testing and how is it uh different from other usability testing. So the core question lies in um how does it work. So the traditional testing methodology answers uh asks how does it work? So the core principle behind both usability testing and accessibility testing may differ at its core itself. And while traditional or usability testing asks how does it work, accessibility testing tries to answer how does it works work for everyone. Right? So we are talking about inclusion in its core principle and in traditional testing methodologies which is employed in uh most of the IT companies we are talking about design and development of uh some solution. Then there is requirements gathering. The code is developed. Then there is a testing which is manual plus automated and the deployment. Right? But where is the shift is that it is from validating functionality to validating inclusion particularly in the case of accessibility testing. So the next important question kind of comes when we say who are we really testing for. So traditional testing methods assumes uh certain capabilities of uh your users. So for example, we assume uh the design assumes perfect vision whenever we are designing uh you know visual output and all of the elements uh pertaining to front-end design. All of the elements pertaining to web front-end design or app front-end design which are visual in nature may be assuming all of these capabilities when they are um you know designing the the solution or the application in the first place. Right? Similarly, we are assuming that uh the audio outputs people are able to hear, they are able to navigate. So mobility assumptions are also there that they are able to navigate through uh your website through your application through um your um you know forms or whatever uh enga kind of engagement your product is expecting them to do. Similarly, we may assume. So, these are some assumptions which we are making at a user level in a traditional testing methodology, right? And um most of our users who are working professionals particularly in the IT sector may want to agree that when we even when we do user testing are we actually you know assuming all of these things when we are doing the user testing. If the answer is yes then what we talking about will make sense. Then uh we also assume at a infra or uh an equipment level is that they might have mouse or keyboard available. Their touchpad itself is working and all of those are also uh equipment level assumption. Right? So first first was our user level assumption then we are also having some equipment level assumptions when we are doing user testing or when we are doing your traditional testing that happens before deployment then we are also assuming that there are infrastructural or um um you know other u facilities which are available to them. So for example, optimal conditions may mean um you know good lighting or good Wi-Fi uh good connection right good Wi-Fi good connection um or you know good conditions particularly to use that piece of software. So all of those people might be assuming. However, on the other hand, when we're talking about or or I mean again they might have a str stand standard browser itself, right? Which again may fall somewhere in between your equipment as well as infrale u uh assumptions. Infrale assumption right. So uh in an accessibility testing uh what we are trying to do is that we are saying there are 1.3 billion people with uh disabilities globally and uh uh which is almost 16% of the total population. So these people may include your permanently disabled people, your temporary disabled people or your situational disabled people, right? Although this percentage uh is not just um maybe just covering the permanently disabled numbers, situational and temporary disability will increase this number largely and maybe take it up to 90s uh of the percentage. Right? So that means that 100% of the people almost 100% of the people face some or the other accessibility barriers at some point in their life. So when we include situational uh when we include temporary or we may include um age related issues uh we may be accounting for almost the entire population of on the globe. So this is where uh accessibility testing becomes rather important and critical before deployment of any solution. So what we are trying to do is we are trying to go beyond the does it work framework and we are trying to ask more fundamental questions which are related to accessibility. So the POR framework which we have spoken about earlier in detail as well uh can come in handy when we were talking about design uh itself when we were talking about design principles when we're talking about um universal design and inclusive design in the first two weeks uh we also spoke about the PUR framework the same framework can also be deployed in testing phase, right? So in a perceivable we are asking the question can users detect it, right? Can they perceive it? So carousal loads for s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c s c sited users so for example this moving carousal banners which are there on the website are quite common right so um when when uh such a website is accessed by a person who is visually impaired the screen readers just keep saying image image image right so back to back they are they are describing even if it is a short description of the image then it becomes uh cognitively overloading if there's a lot of audio feedback back to back despite uh h and plus the user does not has any control uh over stopping it right and if they do stop it they may be missing out on some relevant information so then these carousel loads It's become very critical particularly from an accessibility perspective. uh a how to kind of integrate them for access through screen readers and b we are also saying that uh we have to think from a perspective whether to include very relevant or important information or communicate those information through the carousel or not or where that particular information should be placed on the website. if is carousel the right place for it and all of those questions right so this is just one example I'm sure there might be others so overarching question under the perceivable category that is that uh whether users can detect all relevant information then the second principle is operable operability talks about uh how easy it is to operate. And in this when we are testing we can turn it around to ask can users control your product? Can users control the elements present in your piece of software or application. So for example, dropdowns work with mouse. they may not work with uh touchscreen. They the drop-down arrow uh you know itself may be too small for placing a finger and it may uh become very difficult. Then again if keyboard users who use screen readers and keyboard kind of an interaction may become trapped and they would never land at the drop-down. Right? So this is again one of the examples which can cause issues particularly from an accessibility perspective. Now the third question from the PUR framework we have arrived at the understandability aspect and now under this we are asking can users comprehend it? Can they understand it? Right? So for example, there's some error displayed and it says error 4721 in red. Who are we telling this to? What is the user supposed to do? And even today this error 404 kind of happens and but the user has no idea what to do beyond that, right? So uh who are we communicating this to and what is this um you know kind of piece of information helping helping the users with right helping they're not helping the users with anything. The fourth aspects fourth aspect comes when we talk about the robustness or the R in the PUR framework. Here we are asking does it work everywhere? Right? So for example um if there is a plug-in does it only work on a Chrome desktop but how about other assistive tech or how about other relevant informationbased uh applications which may not work a uh say for example on a Safari browser right so then that is kind of creating a barrier to access then we may also ask uh about whether this website is is loading in a low connectivity setting, right? So say if we want uh um you know wider audience sub uh urban as well as rural audience to our product. But the the application is heavily dependent on a large number of web based data sets or whatever. And then uh if there is low connectivity then it becomes very difficult to interact with your product then it becomes very uh kind of nonaccessible to a large number of audience particularly in countries like India. And also another aspect is whether your solution is compatible with assistive technology right so assisted technology right like screen readers etc whether your solution is compatible or not. So when we ask all of these questions in the accessibility testing, we may be able to identify gaps and we may be able to do it before uh fix them before deployment itself and rather than relying on uh uh you know feedback or rejection of the product later. So let us uh think about a scenario or um you know some common use cases. So a simple submit button which is a very common kind of an interaction which may be there in a lot of websites or in a lot of applications. So traditional QA tests there is only one test case where it says only talks about functionality. So it says click button the form is submitting then it is pass it is passing your traditional testing. So again this is from an industry IT benchmark right we're not defining it but this that's that most of the time they are only talking about the functionality of the interface right so the submit button if you press it the form is submitting so it is able to pass the quality analysis test. In a accessibility quality analysis test, there may be six plus test cases. So, can the keyboard users tab to it? Can they just by pressing tab without the use of a mouse are able to land onto this button? That is the first use case. The other use case is the focus indicator visible. So the focus indicator that means the word submit is it visible the signifier is visible or not. Does the screen reader announces its purpose? Does the screen reader the all text or the tagging of this button does it say button or does it say submit button? So from an screen reader if you are listening only button there may be mult 10 m multiple buttons on the screen which one is used to submit the form I may not know. So if there is no semantic signifier and it doesn't say it's a button named submit or it's a submit button uh then it becomes very difficult or inaccessible for people who are accessing your uh website or application using a screen reader. Then is the button large enough for motor impairments right? So whether it is um you know the submit button is in wherever it is placed on your interface. If that needs to be interacted with using hands, your fingers, if it is placed on on a mobile phone, then the size should be big enough that people with even with motor disabilities are able to press it and not in the process of pressing it, they are not uh um you know clicking onto nearby buttons as well. Does the color contrast meets WAG standards? So the color of the button against the background or the color of the signifier font the signifier text against the buttons button color. It should be as per the color contrast uh guidelines defined by WCAG. Can voice control users activated? So this is also another use case that if I am not interacting with uh the uh the website either through um you know mouse or gesture navigation touchscreen navigation or from a tab tab uh process through keyboard through screen reader I may still be able to submit a form uh if I am kind of navigating it using voice control as Well, so in one button itself we there may be several accessibility dimensions and this kind of goes beyond the traditional testing dimensions. So then uh you might also be thinking the moment you say that all right that probably um when we say that all right we now it is also possible to do uh automation and largely a lot of uh again industry uh relies on automated testing where actually users are not interacting but QA team just uh makes some test cases and those test cases are largely functional right similar to this one um and they're not educated about accessibility related test cases otherwise they can uh you know do this as well so I think it is first it is important that uh you create these test cases otherwise the um the solution may be usable but not accessible then uh in terms of automation most of the testing can be auto automated if there is a clear pass or fail criteria and which again is largely possible in functional testing. However, in accessibility testing automation only 30 to 40% can be automated. Right? what automation cannot catch is something to be concerned about and maybe uh more humanbased testing may be required for it to really pass the accessibility test cases. So for example, quality of the alt text is it meaningful? Is there a semantic value to it? Then uh there may be some u logic to the reading order. Right? So what do we mean by order? So again if when we are designing a layout in on a web page or on an application the layout is rather visual there may be still guidelines which are visual design guidelines which your company or your design team might have followed in order to arrive at a layout which is visually offering um you know means to understand the information and navigate through it. But say if the user is interacting with the information through a screen reader then it becomes sequential right it will happen in a sequential manner. So then uh what should be that order in which the screen reader accesses it? And there are multiple uh lit there is multiple literature as well as I have already shared the name of a book which is called mobile accessibility rituals by Ashoto and uh he also talks about this fact that there can be parallel information architectures one which feeds into your GUI your graphical user interface and the other that can feed into an alternative uh accessible assisted technology. ology interface which is maybe a screen reader and there the logical reading order becomes very critical. So the for example in a visual sense even if there are like pop-ups or there are ads in between we can visually assess and scroll through it scroll past it however it may not be possible for the screen reader to do it right. So then the architecture itself has to be different. Then similarly the keyboard navigation flow right. So if there is um automated readout automated read aloud kind of feature which we are using then the reading order has to be uh considered then in a keyboard navigation uh when we are talking about a keyboard navigation then u when we are pressing tab where which next button or which next tab it is going to becomes very important. Screen reader user experience again is of value and uh we have to focus on context and then manage the content. So the difference again here is that automated accessibility tools find technical violations and manual testing may find usability barriers as well as accessibility barriers. Right? So and many a times there may not be clear uh pass fail criteria. That is why manual testing is also very important or you know more uh sophisticated adaptive models might be uh something that can also answer this question. uh but I mean they are still under development and hopefully we might see some development in that area as well. So now based on this understanding of what is required in a functional quality analysis or functional testing versus what all skills are require what all happens in an accessibility testing. Right? So there in the methodology we were you know kind of shed some light about what how these two methods function. Now from that perspective we can also kind of try to understand what are the gaps or differences in the skill sets that are required in a QA engineer who is working on u you know technical violation related work and how they can ramp up their skills to make themselves more relevant to accessibility and usability testing as well. So a traditional QA engineer uh I mean maybe doing test case design, maybe doing selenium automation, maybe doing bug reporting, cross browser testing, SQL API testing, an accessibility QA engineer, all all of these skills plus they have to have screen reader proficiency, jaws and media, voice over, they have to have uh be acquainted with WAG 2.2 expertise. Now we are also moving into 3 uh.1 and there there are over 60 criteria and maybe you can understand there is again there is an earlier session which talks about WAG and related documents and the criteria how to go through that document how to use it to your benefit there is a earlier indepth session on W CAG which you can go back and refer refer to as well at this point. Then uh there can uh you may want to learn area specific knowledge particularly if it is web development area based coding can be really helpful understanding HTML semantic structure. So earlier we were talking about two kinds of information architecture right. So um if you are familiar with uh HTML structures you will be able to uh deploy two parallel architectures as well uh in this HTML structure itself so that uh one feeds into the GUI and one feeds into assisted technology such as screen readers or read aloud uh kind of tools. Then one should understand keyboard navigation patterns as well because that is something which not just people with visual impairment but a lot of users rely on. Then a basic understanding of disability and assisted technology operation. I think since you are part of this course, you are already doing that basic step and trying to sensitize yourself about disability and assisted technology operations.