Accessibility Testing- tools - Part 1

Hello all, welcome back to our course on digital accessibility. And in the previous session we did talk about accessibility testing and the different methodologies that are utilized in the industry and how accessibility testing can augment the traditional quality analysis methodology which is largely you know commonly used in the IT sector in the IT industry. Uh today we will talk a little bit in detail about accessibility testing tools and um um what in what ways it can enable uh you know testing for the developers as well as for the quality analysis uh engineers and uh some of it you might have already seen in um the wave as well as uh uh earlier evaluation tool tutorial. But uh today we will talk about uh code level development related aspects to it similar to how we were discussing in the previous session. So let's dive into it. So as previously discussed the tool ecosystem particularly I'm sure you are all aware of like genkins etc which is your traditional testing tools uh which are you know much lesser in number and they are the commonly uh used tools across the IT industry. Accessibility testing tools may be many in number because automated tools are not able to do a 100% evaluation of all the accessibility issues in your uh in your product or in your code. So it is important to understand uh that some part of it you can you know use u the evaluation can be done using automated tools some part of it needs manual testing with respect to say screen readers so say 30% coverage with automated tools some uh part of it can be done using screen readers uh manually then there can be specialized tools particularly for color analysis, accessibility insights, etc. Then you may also want to uh test different browser combinations uh as well uh along with different screen reader products and softwares. Right? So basically you cannot achieve uh an accessible product with just automated tools. That is what we are talking about. So why tools matter in accessibility testing? Right? So the challenge is that testing for accessibility manually across all success criteria is impossible. Particularly if we are talking about multiple um you know lengths of codes, many many blocks of codes uh a lot of uh you know elements in your website or on your product, lot of walkthroughs uh then it becomes very difficult. Lots of screens, lot of screen interaction, it becomes very difficult. And a company may also have 10 different products right so uh then it becomes very difficult and as discussed earlier there are around 61 W CAG uh 2.2 into success criteria and each one uh you know may be linked to multiple accessibility testing related uh cases test cases and if you just you know plainly multiply them it becomes a very huge number. Uh there can be multiple access assistive technologies that require testing in terms of robustness and thousands of potential user scenarios may be there. Right? So this is a huge challenge uh how do we actually achieve accessibility testing particularly uh when it is now mandated by law. How do we ensure that for our organization? Uh the reality is that manual testing alone uh can require 40 hours per page of comprehensive coverage. So that means we're looking at either a huge team of accessibility testers who are able to do that kind of work uh in order to meet a shorter deadline or with proper tools also it may be four to six hours per page. So tools don't just don't replace the expertise they just amplifying it they are adding to your productivity. So it is essential even you if you're placing yourself you're very well aware of all the WCAT criteria you have educated yourself about u the criterians and the nuances that we spoke about in the previous session still it may take a long time for one accessibility expert to go through all of that uh material and evaluate it. So how we can augment our expertise with the use of right tools and for right tools or the right reasons is something what we can learn in today's session. So the key insight is that tools catch the what uh the technical violations primarily and humans understand the why user impact that violation uh would create. So what students or what professionals think that maybe I'll just just run just one tool and fix everything. However, what actually happens is that this is not true. We cannot particularly for accessibility we have reiterated it that one tool may not be sufficient. So different tool types, different kind of things it is able to catch. So or different kinds of things it is able it is missing. So automated scanners they are able to catch 30% of the technical violation. It is missing out on more or most of the 70% context UX or meaning semantics related aspects which needs user expertise which needs uh uh knowledge about the user and requires human centered expertise as well. Then uh screen readers user experience issues it may the screen reader testing may be able to catch but it doesn't detect all the technical bugs but because somebody has to manually run it it may take much longer time as compared to an automated tool. manual testing which is like real world usability testing. It is time consuming plus subjective. You may want to incorporate. Uh participants design a study, ask them to come and interact with your uh prototype. Um confidentiality may be an issue and all of those things and then that may take several hours. And user testing again it is where um uh user users are not small in number but you're doing a large scale user testing and it may takes weeks etc and it may be very expensive to conduct those studies and you may get slower feedback. So what automated tools are not able to uh catch that is something we'll try to understand. So automated tools uh say for example it detects an image right. So violation it it may be able to detect a technical violation that it uh there is an alt text or an alt attribute missing. What it cannot evaluate is that even if there is an alt text which says image there is no semantic meaning of that quote what is inside that quote then uh the automated tool actually till now is not able to kind of understand or semantically link what is inside the code. So basically the word image along with the semantic value of that image. So that will actually help you understand whether it's the right all text or not. So but the automated tool will not detect any violation in this case. So but this is image a good all text definitely not. So this is why we cannot rely only on uh the automated tools. So we have to ask all of these questions. Is it decorative? So should should it be alt equal to text? Is it a product? Should um you know describe features? If the image is showing a chart, then you need detail description. Maybe also link to the table. Uh is it a link? Then you should describe the destination. Add the destination uh in the alt text also if it's a clickable image. So automated tools see syntax humans understand meaning. Another example if we take uh so for example there is a button and the automated tool is um you know able to it's able to not detect any issues because if it is able to click uh then it is a valid button. Technically it's a valid button right the screen reader user may feel uh that what is the meaning of click here click here for what maybe I don't know right just by the term click here I don't know so there has to be a semantic signifier uh in order to facilitate accessibility uh keyboard user can't even activate with enter or space so if there is a keyboard missing keyboard event then it's also very frustrating for the user for keyboard users. So tools catch what's broken and uh humans catch what is broken for the users. So meet your testing arsenal right. So the four pillars of uh accessibility testing of course the automated scanners uh so coverage is 30% of the issues speed is very fast right and cost is you know some of the tools are free to use some are you know roughly $100 a month which is not that expensive uh so examples are axe dev tools and wave lighthouse house etc. Best for finding lowhanging fruits catching regressions. Screen reader manual uh testing. So around 40% of user experience issues uh people are able to catch uh with screen readers itself um which are related to all text or missing link or um buttons which don't have semantic signifiers and all of those issues are able to be caught in a screen reader evaluation. Um the speed can be depending on the expertise of the evaluator roughly around half an hour to hour per page and the tools they cost very little in media uh is free but Jaws may be a little bit expensive if you're developing for an iOS uh best for understanding human experience designs uh there may also be a hidden cost of um human resource here because we need uh an expert user, expert evaluator who knows um the use of screen reader. Then we come to the specialized tools. So the coverage is very little so like 10% or whatever because they are only targeting specialized issues. So like a color contrast analyzer etc. uh dragon etc. Best for deep diving into specific problems. Then manual testing the coverage the remaining 20% where um an evaluator is looking at the co context the logic the flow uh the speed can be several hours per page. Um cost can be your time the human resource cost to the company uh best for understanding real world usability. So if we are able to you know use all of these together then only we will be able to achieve a 100% coverage for um you know identifying accessibility and usability issues in our product. So now let us talk about the good, bad and essential of automated scanners. So the promise that we get with an automated scanner is um you know run this tool and find all the access. That's how they are branding it. Uh but what they can do well um is missing all text, contrast violations, uh missing from missing form labels, invalid area, missing language attributes, duplicates, ID, etc. Uh it takes um you know few seconds to run and accuracy can be really high. False positives are usually minimal. And top four automated Dave tool uh automated tools can be the axe go uh Dave tool which is like the gold standard. It is also free. Wave learning tool is also free and uh lighthouse quick check built-in uh uh chrome um u tool which is uh kind of more of a quick check tool. Then then then there is PA1 automation champion. It's also free. What they cannot do is that uh is this text meaningful? Is this heading logical? The keyboard focus in order. Does the screen reader flow make sense? Are dynamic updates or announced? Is the content easy to understand? So this they are not able to cover because we have as we've uh discussed previously all of these issues require human judgment and understanding of the context. How to do a screen reader test? So what what um we can do is maybe roughly a simulation. So maybe try to close your eyes and uh now use the website right uh along with the screen reader output. You can turn that on. It's usually inbuilt on your uh laptop or your uh uh phone and uh you can just check. So actual screen reader output you might be hearing words right clickable button image link button image clickable submit right so but if you if you don't if you're not able to receive that input and you are just uh listening to these words so try to just close and imagine if I tell you button clickable Imagine image, link, button, image, clickable, submit. Are you able to make any sense out of it? Probably not. Right? So then it becomes very difficult for the user. But what you see on the screen, sign up for the newsletter, then there is a button. Read more about our services that is clickable. The all text says clickable. Company logo. All text says image. Subscribe now. Contact us. Featured product image. Right. Clickable. Learn more. Submit form. Here the image uh here the all text just says. So the screen reader will only access the all text. So if we are able to augment the alt text with this content which is actually describing what is the role of that button or that click in the im in the website and uh what the semat the visual signifier is also mentioning but is missing from the alt text then it will make uh the lives of the users uh screen reader users very uh easy. So the problem is that without descriptive labels screen readers here users hear generic elements with no context at all. This is why automated tools are not enough and they detect detect that does just the button exist and the all text exist right the button has no meaningful label. So the most important testing tool can be a screen reader and um um you know uh Jaws which is short for job access with speech uh can it's primarily uh it's a paid uh software and uh it's also you it also it's also used in um uh iOS uh platforms um now it is I think largely uh we are also uh have voice over which is again inbuilt in the OS and similarly talkback is built into Android and NVDA is something which you can use for your Windows which is again free and a lot of Microsoft Office tools etc they also have inbuilt read aloud features and all of that. So this one jaws has a good market share and is also usually an industry standard in enterprises. Uh so probably you can use it in final production testing because it's a paid software. Uh the all the other or development and quality analysis testing a lot of the companies they also use NBDA. Uh it also has a decent market share of 35% growing fast perfect for daily testing and iOS and Mac testing 70% of the is users uh mobile testing for all iOS based applications and website. Uh this voice over testing is mandatory and you have to you know comply with that. Uh similarly for Android or mobile based testing talkback is there. You can just enable it in your settings and try to interact with your phone just to get an idea of how any commonly known app uh is built using your screen reader functionality or not. And actually you may need to test with two or three of these uh in order to get a clear uh picture of uh whether or not your website is functioning well or uh whether or not because nowadays it's most of the usage is cross crossplatforms right so a website people may be opening on the laptop also they may also be opening on your phone so when when they're opening it on the phone then this talkback is functioning. If they're opening it on the Windows or laptop, then the NVDA is functioning. So you as a developer has to check with all of those or multiple of these u screen readers otherwise it will not be uh fully you know kind of analyzed. So axe dave tools I'm sure you are aware if you're a developer particularly or a student who is a CS student it's a developer's best friend and um particularly browser Dave tools accessibility tab uh run axe and you get instant results. Uh what it does is it's able to detect uh over 50% of WAG issues. um shows exact code location uh plus copy paste fixes. It kind of suggests very easy quick fixes and explains user impact also and very little false positives and uh so how you use it. So you write the code, you run the acts and see issues, you get the fixes, you apply and you verify and you commit. So bottom line if you learn one tool make it axe. So automated testing uh with axe is quite easy and maybe this is the script that you need to include works uh with just cris playrite and all of these other pipelines. Then wave uh accessibility tool evaluation tool is something I think we have shared a tutorial earlier and um we can you know share it again if needed. It's a visual learning tool primarily. So what makes it special is uh it shows accessibility issues directly to you on your page with colorcoded icons. So this is something which is also largely used by the front-end designers because it it tells you where the issue is with all of these icons. So it's it's it's a lot uh easier to kind of pinpoint where uh the issue is. So the icon system wherever it says red icons that means errors you should fix immediately. The yellow icons are alerts uh you should check and verify. Green icons are features which are good accessibility practices and blue icons are structural elements. The style toggle styles on you see beautiful uh design accessibility issues overlaid style off. You see the raw structure that screen reader uh reader user experience. So here is the style button. If you turn it off, you will see the the naked uh structure which the screen reader is accessing. Uh so real student example. So uh built a beautiful contact form ran wave. Three red icons appeared. Icons pointed to email field, phone field and message field. Problem all missing labels. Fixed time 3 minutes required. result accessible form was achieved very quickly. So why students love it? Because there is zero setup time. It's a browser extension. You can very easily uh it's it's available on your browser itself on your Chrome and uh visual feedback. It's not like a code report. So it's very uh you know user friendly in that sense, developer friendly in that sense. and uh it's very easy to understand. It also explains the issues. So it's it's um students who are learning about accessibility and accessibility evaluation tools. This can be a good starting point and also of course it is free to use.