Analyst Biases in the AI Era: How to Avoid Wrong Decisions

Hi everyone. Glad to see you here. [snorts] I am Alexander Malena, business and system analyst at Anderson. [snorts] And today we are going to talk with you about some effects that everyone of us who works with requirements data and decision [clears throat] knows very good. No theory or a little bit of theory uh more real but patterns uh more real mistakes and some practical ways how to avoid usual biases. So let's go. Let's start. Uh first of all some uh words about our agenda. Uh here what we'll cover today uh five key areas and about 40 50 minutes to go through. We start with paradox that most of us feel rarely uh but rarely name knew it by name. We will look at how our role is structurally vulnerable. Then we'll talk about six buyers traps and with real BA and SA cases not long ago. Uh some of them uh we speak with you with most of us about uh them at our uh telegram channel at our um LinkedIn channel. There were several um um some small uh text about those tips, traps and biases. uh then uh uh with AI angle we'll talk uh where it makes things worse and where it can actually help us and uh then we'll close with some particular and practical toolkits that can that we can use um starting from today. So let's start [snorts] first of all um some basic overview. Now we are living in a world full of data and uh does it really helps us to make better decisions? Let's start with this question. Um just think about it. What did um when did you make some decisions last time and um had to do redone had to redone them even through your head the right data. Um we live in a world full of um big data growth, exponential growth of data but decisions quality in IT projects is not uh growing so good as this data grows or even sometimes in some uh experiments even declining. More information does not equally mean better decisions. It just mean we can uh make wrong decisions but even faster and with more confident [gasps] AI don't fix this paradox it even amplifies it you may think that AI uh may fix those problems but more and more analysis shows that uh it is not really usually so um research which say usually uh even opposite position. [snorts] For example, uh some of the latest research uh that we can find in Harvard Business Review found that employees spend up to 40% of their time checking and correcting AI outputs. That makes more exhausting what uh than uh creating context from scratch. Uh Fortune magazine describes their endless draft effect. AI generates instantly but their decisions uh bottleneck stays exactly the same. Uh PWC calls it AI paradox. More AI generated data means higher um harder work with all those amount of data uh not easier and needs more analysis. So we have more noise but not more clarity. But why do we trust AI? Of course, it's a question that can be discussed um very uh in a very long uh course uh and still different countries have different AI trust. There are AI enthusiast, there are AI skeptics and this uh metrics uh usually even change from time to time but there are some patterns but usually uh a big amount of people trust AI. Uh why do we still trust it? Because um it need uh answers are not sometimes wrong are sometimes wrong and not not so clear as we want sometimes. Um first of all because AI sound confident it writes clearly it's structured perfectly. Uh and uh we are humans uh we like we will use usually our fast thinking pattern machining and shortcast. Um when the answer sounds right and machines um and match uh what we expected we can stop checking further. Um and this is a core problem. We need to talk about it a little bit more today. And the risk is not that we use AI or use it in the wrong way. The re the use uh the problem and the risk is that uh usually AI seemed uh as it can um not lead us uh or can by default um not lead us with our bias problems and bias questions as we usually have when we just um do our work as we do it later. Um here are some things and some um biases and analytics works um and they are not new. We are always uh brought assumptions into interviews, estimations and metric sessions. But now those uh same biases uh get embedded into a prompt and AI return them back as a clean structured and professionallook artifacts and it getting faster, more convenience and harder to question about them. Today we look and we will look at uh only six but of course there are a lot of them but uh rather dangerous patterns and um how we can overcome them. how can we uh see them in AI in um business analyst and system analyst practice one by one but [snorts] first of all let's uh look at the broader picture about why BA and SA works is um um designed with a very big of uncertainty and why it is so uh let's talk about why our role is especially vulnerable we work uh at the very front of processes uh with incomplete data with uncertain requirements with usually conflicting stakeholders vision. [snorts] Uh three zones uh where biases are hit hardest discovery estimation and stakeholder validation. uh and it it's not uh flow uncertainty is our raw material with the material we work every day and it is usual but it is always mean we make decisions under conditions where uh shortcuts feel uh necessary so uh it's our usual condition to uh to use such kind of information and using AI is >> [snorts] >> um but uh when we use AI today it can uh not always lead us to better decisions and for clear vision because uh it is uh the same biases are still with us and we'll see uh how it looks like. Um the more experienced we are uh usually or sometimes um less uh noticed uh we can less noticed our shortcuts. And here is the paradox of expertise. The more experience we have sometimes the more automatic thinking our thinking becames. It's like in uh Kanimanki um system one and system two uh book uh the system a one uh takes over it's working fast it's working intuitive and usually pattern based um how it looks like if we seen it before um well let's say it will take us two weeks if we are talking about some type of work that we need and our estimation It [snorts] will be uncorren bias. For example, uh during some stakeholder interview, we see the stakeholders are nodding okay, they agreed. It's a confirmation bias. Uh the project succeeded and the approach must be right. It's a uh survivorship. Um, experience makes biases sometimes invisible becomes it works uh most and uh and it works most of our time and uh we are becoming experienced. Yes, but it doesn't mean that we would not make the same biases. They are still with us and we should know how to cope with them. uh the BA and the SA environment doesn't reduce biases. It sometimes implies it and the environment AI environment make it even worse. Time pressure uh means less time to check assumptions. Uh senior stakeholders in the room and sometimes fewer questions can get asked. It's a aerity bias. um workshop with loud voices. It's not only loud. Loud in means that they are too confident. Uh the voice uh set the frame of everyone. It's framing plus uncor biases. Uh and uh once sometimes is written down is clean and we see a very clean document and people trust it as truth [snorts] not just as a hypothesis. It's another kind of bias. This is just artifact traps and AI makes it more powerful and we should always make it into consideration that um AI usually can makes us uh works with the same biases but even faster and that's why um we can use AI uh not only to make our documentations clear but sometimes uh it can help us to make the same mistakes but even faster. AI did not remove biases at all. It gives biases professional voice. Before AI bias were even more visible uh as a personal opinion. Uh you could push them back. But now uh same biases is um embedded in a prompt and AI returns it as a structural artifacts. This is very clear documents. But sometimes it mean uh that they contain the same biases but more confident. AI fill gaps confidently. Uh when you see uh something like an assumptions you see uh a clear sentence that is assumption but AI can uh fill these gaps and make it confident but it uh doesn't uh tell you that uh he just uh fill in the gaps. So um and AI say it becomes uh what AI say it becomes a social uh shortcut and sometimes not nobody but usual wants to challenge um the machine and what is the key shift here biases used the cost one analyst time but now it's replaced by AI speed it's like the same problem the same mistakes but even made more fast. [snorts] Let's talk about some of the very usual biases. First of all, confirmation biased. Uh [snorts] you already know the answer. You just need AI to confirm it. Um it's very usual. It's an for example classical case from BA practice when we are convinced that for example the problem is in UI UIX uh and we write to AI generate uses story for improving the for example registration flow it uh drives a perfect list what should be done team is happy but the real problem is maybe backend bugs and they were not they were never investigated by us and by is um wrong scope from day one and the signal here is that AI produced exactly what we expected uh that's not a win that's sometimes like a red flag uh in one of the songs of u time machine group uh when everything uh is going very good we can say that some something is wrong and we should recheck it the same problem is with with AI sometimes when uh the documentation is too clean uh it not exactly a red flag uh but it doesn't mean that it is um u really [snorts] done in a good manner and without biases without mistakes. Uh the second bias is anchoring bias and it is very widespread too. uh the first number became the only number for example project management say that we'll probably work with this item uh for two weeks and the first meeting us casually and we make it without analysis and uh from that point every estimation is built around this number and when we work with AI uh with this two weeks approach AI just uh would make outcomes and outputs to fit this bias to fit this anchor and do everything around these two weeks. But the real turn uh can be more even six weeks or more even. But uh why it is so because uh the anchor is already on the ground. And uh what should we remember here? The first estimate should never become the baseline without some explicit reset. uh there is all there is different possibilities for us to overcome anchoring bias and we'll see later how to overcome it but the main idea here is to avoid anchoring bias uh at the very beginning even when we have strong understanding that we are very convinced about some numbers certain numbers but uh it's good um sometimes to see that Um maybe it is not so to be not so convinced and then to uh use several other prompts or AI features to overcome this bias. uh false casuality. It's a very uh usual bias and the third trap here when correlations failed like reason but it wasn't uh for example feature was launch or launching regression grows 12 uh retrion grows 12% team decide to scale and to work with it AI builds a convincing logical chain in uh planning why the feature cost uh this grow but the Real cause for example was a parallel marketing campaign and it ends retrition drops feature was already in production but it was near but it was not the real um why we have this retrition growth I explain how it works but that was not the same what actually works it's a false casuality um why AI usually working the same it's you it can say why um A uh can be can leads to B but it doesn't mean that uh situation A exactly leads to situation B. [snorts] It can be false casuality and it is always there are several possibilities how to avoid it uh working with the same exact uh AI uh models but a little bit in another way. one of the possibility uh to work not in uh one flow but to work stepbystep approach and we will see how it works later. Overconfidence bias um very usual too uh it's like our fourth trap when we have a clean dashboard but it wrong conclusions uh all metrics can be green AI summarize it system performance is stable and nobody asks what is outside these green and goodlooking dashboards and maybe two or 3 weeks incident uh is in integration layer and it wasn't mentioned uh in our monitoring and at our dashboards at all. Confidence must uh be um proportional to data coverage not uh to the quality of the visualization made. Um it's very uh it's not so clear bias um but it's it sometimes happens. It's always good to uh go one step further to see the broad picture not only in our dashboard not around our green and good-looking metrics not only within those AI summarizes and good performance and table but uh to work with what is outside and uh sometimes it can happen it can helps us to overcome such kind of biases. is not to be overconfident when everything goes good and when everything goes well uh and even our AI answers say are saying everything is good everything works fine um and um stepbystep approach can helps us uh but not always and we will see why it is so [snorts] Um very usual um survival ship bias. It's our first uh fifth trap. Um we learn from what from who survived and sometimes we ignore what um on from one who failed. Team does not uh does a retrospective on uh successful project. AI uh shape it in a best uh practice guideline. Uh but three uh or even more similar um fail project were not included and nobody talk about uh them but just talking about a succeeded project and nobody talked about uh failed one or some of them and the guideline become a company standard without those um practice and without those fallen projects And um it's good to ask not only who worked on those succeeded project but it's always good to go beyond and to work with um failed project too. Um it's like um about dolphins. Uh we know that it's uh the dolphins can uh help people who are in the sea. Uh and when we see the dolphins, they're playing with people and we see okay, they can help people if they have some problems on water. But uh what if we don't know about people whom uh they don't help and even go further deeper and deeper in the sea, we don't know their voices and this is survivorship bias. Um and uh it's a good idea sometimes to to work uh specifically with failed project and to understand why they failed uh and um to include this information uh in our AI models too and to work with them specifically and uh it's good idea to uh them to be the part of our guidelines and even for standards in the company to understand how to overcome and to and to include to overcome problems that leads to failed projects and to understand um how uh why it was so uh so uh to ask not only how it works and to ask why it not works. framing efforts. Uh it's our sixth trap very usual to same data, different frames and leads to different decisions. Uh for example, um errors occur 5% of cases and we have another vision system works correctly in 95% of cases. It's the same fact, but one version gets the hot fix postponed. What's happening here? AI by defaults usually uh do positive framing unless you explicitly tell us some uh negative um vision uh and we will use this framing efforts and works with only uh this positive vision. It's always good to ask how would this uh sound if we wanted to show the problem to specifically show the problem uh not to and to overcome this framing efforts to works with problem vision not with positive vision. It will helps us and we later a little bit later we see how it will works too. uh framing efforts is um sometimes difficult to overcome because in our real life uh most of us uh not only uh BI professionals but in a real life uh we live in a world full of framing efforts and it is very difficult sometimes to detect them and that's why it's good idea to work specifically with the framing efforts with AI tools to understand to detect them to see what uh the framement efforts can be, how to work them >> [snorts] >> uh how to avoid uh only positive framing and to work specifically with some negative uh framing. Uh at the same time, [snorts] it doesn't clear um all the biases. It scales it usually. [snorts] Um so we seen six traps and now let's talk about what to do uh with them um differently with I and uh with a help or without it. Uh for example three mechanism makes AI dangerous here. The first uh confident tone without source structures create an illusion of validity. [snorts] when we see a lot of things uh very uh concrete very good-looking in AI answers but it is always good to ask about the source and then uh we can find out that there is no real source that it is something that comes from uh AI decision that it may be but with no good real source it's [snorts] it was just an illusion of validity uh the Second thing here is uh uh agreable by design. AI adopts uh to your frame and leading prompt leading answers. Um when we are working with AI, AI um will know better our frames and if we are not asking uh just to help us to overcome our frames sometimes it can lead to the situation well uh our AI model will work with the same frames which are our frames and uh this is a gradable by design pattern and um it's sometimes very uh difficult to overcome but it can be overcome of course. The third thing here is automating assumptions. I fills gaps when you should see just assumptions. Um even now uh when I created this presentation uh at the beginning there were several diagrams and when I was working with them I was uh searching for some um not only good-looking but real examples of some uh scientific um knowledge scientific uh articles to see some um some good examples of uh um biases [snorts] uh with some numbers and then when I asked different AI models to make some diagrams and uh they they were they created diagrams very good-looking diagrams but the main problem then when I then I start asking how it was uh made and uh first uh not of them they made with a good source um none of them were real because they um try to um talk to me in a greetable tone because um I expect some result from those models and uh AI creates good-looking um design, good-looking graphs but with no source and incredible by design and fill gaps when uh some of my words that I um told my AI models were just assumptions and then step by step we should uh go out from those biases. uh but now AI only speed and scale our biases and we should take it into consideration when we are talking with our AI models because in uh BA and the safe practice we are working with uh different BI models in everyday's basis and uh day after day they're becoming more confident and not always um uh good working with source and we still uh should take it into consideration. [snorts] Uh there are some tools uh that can uh surface bias and we can use them. Uh here are some good news that AI um can help us not only to uh make us do a wrong decision. Uh the same tools can works against biases. uh three models uh to use instead of uh just generating context. For example, uh in uh model for uh disconformation um what AI usually does in find some uh counter arguments, we can ask them to find them instead of just looking at one solution that it can give us. We can just prompt them what could be wrong uh with this approach. It's just the second step uh we can use after it give us uh the first result. It will it can be good-looking. It can be very clear without any mistakes in first approach. But if we will then uh start asking about um some disconformation about what can goes wrong in this or that situation. We can analyze that the first result was not so clear as it was in the first uh view. Then we we could ask to to show us some alternative path uh build a third or second option. When everything looks clear, when everything is good, we can just ask to give us that some opposite solution. Why not? um he will think uh maybe it's a good idea at the same time to ask for given some opposite solution another model another chart not only in uh chat uh model we're working specifically but to open another model to open another AI agent and to work with uh uh another solution and to ask for another solution uh to ask for alternative in new models. For example, in perplexity computers, there is like a different agora of models and they will talk to each other for alternative path solutions. Uh right now we can make it by ourselves but new models can uh there is a special uh gora models that can um look and find different alternative paths by default and we can just uh look look at how they will uh work with some alternative path but right now when we are working with uh one model with one AI tool it's always good to ask uh to build a second or a third option and then to discuss like with another analysis with another analyst with uh making a good prompt or for example ask me some questions about my path and maybe build the second option step by step by step approach is always good [snorts] then for example another model is the gap coverage uh for example we can ask uh find some age cases uh when something is good-looking and for example how it would looks like in a prompt uh for example what scenarios are not considering in our chart or in our model and then step by step it like in the previous alternative parts we can work with those edge cases it's always good idea to work in such first of all u asking even after getting very clear solution very clear documents even when it is it is like too clear sometimes and it's good to work with right now it's good to work with different models even when we don't have some um paid subscription even with free subscription it's good idea to work with different models still uh with different uh windows in one model and talk to each other uh one by one to file alternative files to find some uh gap coverage to find age cases and then uh uh step by step work with them. [snorts] Uh so uh every eye use cases uh a safe mode and d so maybe it's like dangerous mode. Uh let's be concrete and every task uh we do with AI um has two versions. Um for example um the such task as requirement drafting. The dangerous mode here is to accept a final artifact as it is. And the safe mode here is to validate it with ourself, validate it uh with another model, validate it with another analytics. And um it's always good to uh treat it always the first as a draft. [snorts] Uh the second is uh root case analysis and dangerous mode here is to um accept AI first explanation as it is and as it is good solution. It's always good to use a safe mode and to ask some counterarguments to discuss to ask another model. It would not take a lot of time but it would make uh them and uh uh explanation more clear and we will see uh what are the another explanation what are the another task to do. Uh the tasks may be about estimation support. Um the dangerous here is um to anchor an AI estimation as it is as in our uh and current bias. uh the same uh incorrect bias can be shown can be uh in AI model as well and uh a safe mode here is to um use uh multiple scenarios um to make it one by one stepby-step approach as well and [snorts] um not to um have like an anchor just the first AI estimation um most of uh BAS used as a task different uh meeting transcription meeting summaries and most of them are now uh made by different AI tools and uh um it's always good to u to know that uh the dangerous mode is to decide that what AI roads is uh our decisions not usually it is not so especially in um teams uh from multiple countries from multiple language uh teams uh because it can contains a lot of specific uh words that uh AI does not understand good in terms of some project specific and um what is safe mode here is to confirm it with participants if it is possible of course but just to read it correctly and maybe at the same time to make prompt to uh ask what are some about some unclear things and not to work with the first AI solution that was wrote by um 1 seconds after meeting was ended. Uh another task is uh metric interpretation and [snorts] dangerous mode here is to trust clear conclusions uh made by AI as a um endpoint. It's always good idea to verify data. Even now when I created this presentation, I see how many nation is still even in a very very um good models are still uh the same gaps, the same biases uh because those models were made by people and people uh can can be full of biases can be full of frames, can be full of framing efforts and those models contain them too and it is good idea to verify data because and um the uh main practice here and the main rule here as higher the stakes uh the more human review you need. [snorts] Uh well uh some um repeatable habits now there are like a tool kits uh five habits simples and uh repeatables. The first is uh one of them. It's always uh prompts for disisconfirmations. What could we can ask models? Uh what can break this? How would uh what uh why can you disagree uh with this or this approach or something like that. [snorts] The second is to separate uh generation uh from validation and never do both it in one step. It's usually good to uh first to generate some response and on the next step or in another model or in another window uh to validate it and then make it not only in uh uh one step always in two steps or in three steps. It's better. Uh number three here it's a first alternative pass. Always request uh a second vision and the second option. It wouldn't takes you a lot of time uh but it is always will help you to see another angle of the problem. Uh the four uh thing here is uh we should label our assumptions. Uh if something has no source, we should tag it as assumption with something that is is not like effect and then uh we should ask our model just to um maybe underline uh we can even ask the model to underline uh what information is only assumption of the model not uh coming from some facts and we it will it will do so. uh five the first uh the fifth um [snorts] two section rule uh we should separate uh observed from source uh from um proposed and inferred uh it's good idea to define uh different parts of our work not only usually or make it step by step um it's not a heroic efforts it's a usual workflow habits Today uh models are rather clever. Of course um a year ago they were working like a very good talented students. uh at the end of the previous year most of people who were working with models uh and with scientific models they said some professors said that uh good a models are working as a uh talented PhD students and uh even can get PhD grade and today uh some best models are working as um those who get PhD and uh they even can uh can make uh PhD dissertations and uh in some spheres of knowledge and it is truth but um it doesn't mean that um models are very clear and uh they are free of biases and free of mistakes. It's not so and that's why to have such kind of uh uh habits is always good and to use sometimes something like a uh checklist of a tool tips how it can look like um like a bias control gate um for example like this or we can use a checklist uh seven questions before AI generate artifacts and we can uh use it as a habits [gasps] um you can ask about uh is it really grounded? Is it accurate? Is the information complete? Is it competent, consistent? Is it uh traceable? Is it accountable? Uh who is the real owner of the truth? Um what questions uh how it can looks like? We can even make uh one prompt and then add it to our usual prompts. We are working with our AI prompts at the end for example or make a special prompts just to um overcome some kind of biases and then uh use such kind of questions to our usual AI uh answers and we'll see uh maybe some something that uh models uh can um uh produce us some uh not very clear information. We can uh it it of course it doesn't mean that there would be no biases but with such kind of our checklists that we can use uh as a adding to our prompts or some separate prompts it can helps us to overcome people's biases first of all and AI biases that um because usually AI uh answers are too clear and uh too good-looking it it doesn't mean that they're wrong of because um most of the time they are too right they are very right and it depends of course uh at uh the information they are they ground their answers but um if there are some questions or some mistakes in information at the beginning it will just reproduce it and uh if we want to overcome such kind of questions uh we should use we can use such kind of checklists and uh ask our models how to overcome them. Um if uh it's a good idea to use such kind of questions to use our own biases and not not to reproduce our biases even faster with AI models. So uh let's wrap up. Uh the goal is not to use AI less but um to make our and people biases visible. Uh those uh things uh you can do very easy even with AI and uh um what is uh we can ask just what is wrong and only this question can be our small prompt to work with our usual model and label our assumption. If we are not um uh clear uh if we not some clear vision we can label it as assumption and then AI model would work its specifically as an assumption. Thank you so much. Thank you for your attention. Here you see links to our communities in LinkedIn uh in to you can share your feedback uh follow this conversation we can see our uh telegram channel for BAS2 so uh let's see uh whether some questions are so um uh yes I I see that people use different kind of uh AI tools for example code and yes sometimes it's one of the best model uh and but still a lot of people use code um one of the best now it helps a lot to create CVS yes and for a lot of other uh things too in BA practice but of course it it good idea to uh use it specifically with such kind of prompts too [gasps] Um yes about false assumption it's a good problem too if our assumption but uh it's always good idea to um make a ladder approach or step-by-step approach if we even too confident in something it's good idea to uh ask uh oh no aren't we too confident what are the other ways even to chat with our AI tools like uh like a mirror maybe Um it even it would not usually repres um get us the same biases but um in um it can give us information to start thinking in another way. Yes, of course. Uh AI is not the magic. Uh and um it can um uh what we see here? Oh, it's good idea to focus on some Yeah. tools too. Yes. Um [snorts] yeah, the council of models. Uh I think uh in the council of models, yes, I saw that it's agora um like a Greek word of models too. uh it makes easier and we see that people are using it already. It's like a console, our own console. Yes, it's a great idea to use it. Um the same idea, yes, we see that applies to some case studies. Um even without asking uh questions, we won't get the right answer. Yes, it is. So um it's um the main idea here is not to use I would maybe um say it one more time. It's not to uh to use less AI but to use it in a way that it can help us and help us specifically to overcome our usual biases. And it is very easy only uh to use such kind of uh checklists to use step-by-step approach uh to ask for alternative uh alternatives to our decisions uh just to have a small conversation or to add in our usual prompts some small prompts uh from this checklists uh to overcome usual biases and it will helps us a lot. Thank you for your time. Thank you that this evening you were with us. See you on another Anderson meetings. Bye.