IDL Lect1A Why AI Needs Explanations | Interpretability in Deep Learning

Indexed: 2026-05-18

[00:00:00]Welcome back student. So, we are beginning with our first lecture on this course. You had seen the introductory course lecture. Now, this is the first lecture and here what we are going to talk about you know, the deep learning is there in our society, in health care, in many places, in financial decision making, autonomous driving, in many places.

[00:00:25]Deep learning is good. It is giving very good performance and accuracy, but this deep learning is in many cases a black box and in this course we will know that why the black box is a problem and why we as a human need interpretation, why a model is giving a decision and what made it to give a decision. So, this course is all about that. So, let's go with it.

[00:00:53]Now, my question to you is that why AI need explanation?

[00:00:58]Means, if a teacher ask you uh 2 + 2, you will answer say four, right?

[00:01:06]Now, if teacher ask you why you said four, you know that 2 + 2 you can say that because of I'm doing addition.

[00:01:16]So, it two addition two is equal to four, that is the basic math we know.

[00:01:23]Now, in terms of AI, it gives the decision uh if you give an input, it tells you this is this or that or some kind of prediction.

[00:01:33]It most of this uh models are there which does not have any internal mechanism to tell in the human understandable terms that why it made a decision.

[00:01:45]And if any kind of failure happen, we as a human definitely need a decision if it is a harmful because of AI the decision did a harmful uh impact.

[00:01:56]So, let's go with it.

[00:01:59]Now, question to you that can we trust an AI system?

[00:02:03]Uh because I said, right? AI system is giving decision without an explanation.

[00:02:09]So, now question is that can we trust an AI system which we cannot if we cannot explain its decision.

[00:02:17]So, what is what are the problem with it is because now AI is getting used to detect disease from uh medical images.

[00:02:27]It is also making uh decision when a particular person applied for loan, reject a loan application. AI is also now getting used to recommend for candidate suitability for job or not. Then, AI nowadays controlling autonomous vehicles.

[00:02:46]You know, the autonomous vehicles are there.

[00:02:49]So, please forgive me moving into the slide scene.

[00:02:53]Uh so, it is doing a lot of thing and without explanation if something goes wrong, uh it needs we need explainability. Let's see.

[00:03:04]The the real problem lies in the, you know, the deep learning is very powerful, but it's often difficult to understand because it is a black box model.

[00:03:16]You give an input, it is going to give a output which is a decision, but we don't know how it works. Means, as a human as a human language, like I won't be able to say just I say you that uh uh world is a beautiful place or today is a Sunday, right?

[00:03:37]Today is a Sunday, you can understand because I'm speaking in a language we as a human understand today is a Sunday, a simple English language.

[00:03:47]But AI is a kind of tools or machine, it learns in a kind of language which AI understand, human don't understand.

[00:04:00]Now, if you give an input, it is going to give an output.

[00:04:03]Now, when it gives an output, we don't know how internally it is working and that's give us a challenge that yes, it is a very wonderful system, uh it works, but most of the time it is difficult to understand. Now, that brings to our condition that understand. So, there are two way, interpretability and explainability. So, now let's take it what is the interpretability?

[00:04:35]Means, AI is making a decision which human should be able to understand. So, interpretability means that if AI is making any kind of decision which is understandable to the human that why it is making that kind of decision.

[00:04:51]So, that's the thing that we need to understand.

[00:04:55]Now, why this matters because you know, the if it can if we have an AI system which can explain things for its action, then it can be trusted, then it also be safe. Let's say, if AI system is getting used autonomous driving and it is doing wrong kind of driving and if we can find out why it is doing a wrong kind of driving because of any reason in the algorithm, because of any reason in the data, at least we can correct them, right?

[00:05:26]Then, fairness. Means, a job has been applied and AI is making a decision and it start making unfair decision to even a suitable candidate, it is not able to it is said rejected the applicant.

[00:05:41]So, that fairness need to be uh with uh maintain. Accountability. That, you know, the when AI system is working, there are it is impacting many people.

[00:05:53]Who is uh responsible for correcting it?

[00:05:57]Who is responsible for any kind of legal action? Who is going to do what in the this whole AI [music] system? Very much like our society. You know, the teachers are there to do teaching. The teachers are also there to do examination.

[00:06:13]Now, there are people who are also there to maintain that exam uh system is conducted fairly.

[00:06:20]So, it's it's called accountability, right? And then, regulation.

[00:06:24]So, regulation should also be there.

[00:06:26]Now, AI is getting regulated in the EU.

[00:06:28]So, if you are interested, you can follow my course on uh gender and diversity policy in AI, which is all about EU AI Act and how the AI regulation is important, how it can be regulated, how AI can be made safe, and what kind of rules and regulation will be done. So, you can follow that course.

[00:06:46]It is there. And so, this because of all this reason it is matter. So, let's take an example of why what is the interpretability health care in AI will help, right?

[00:06:59]So, this is an AI system which is taking the X-ray images and patient record and there are different lab results and after that it says that the person has possibly has pneumonia.

[00:07:13]Now, saying someone has a pneumonia is a very high risk thing.

[00:07:17]You are detecting the person has pneumonia and based on that if it is automated, then medicine will be injected.

[00:07:25]Now, the medical fraternity and government and everyone will ask, "What is the evidence?"

[00:07:34]So, AI is going to tell you that this particular place I can see pneumonia highlighted in the X-ray.

[00:07:42]Now, this place AI is giving the attention and in terms of low and high impact based on the colors, the more red the color, it is saying that place the problem is. Now, is this sufficient?

[00:07:58]Now, it also have the Now, if it is giving the I'm predicting possible pneumonia at this place. Now, if it gives the reasoning that why the model predicted the pneumonia, let's say it detected opacity uh whitish area or the the color of the X-ray was not correct. You can see here opacity is there.

[00:08:25]And if it was not correct, then in the lower lung plus it has the lab result like uh the white blood corpuscle was abnormal and also the inflammation marker CRP or infection marker CRP is high. Based on that, it also find out this, then it says I have the I detected a pneumonia.

[00:08:50]Now, can if it gives with the confidence combining its uh what it attention, what it AI model is finding where is the problem and other parameter, if it is saying that I'm 87% confident that this is a possible a pneumonia, then a doctor can sit and go through this and can get final decision. So, doctor is going to use it as a auxiliary uh tools and it is going to use AI output plus evidence reasoning plus confidence and it the doctor's own clinical judgment, then this system is going to be well trusted. Now, if so, AI is getting used in this way, then it is going to make the AI more trustworthy because it is supporting the decision making with interpretable evidences.

[00:09:45]So, this is one example that why interpretability is going to be useful to build the trust and confidence and accountability. Now, let's take the another case.

[00:09:55]AI is getting used for driving, right?

[00:09:58]Now, so autonomous driving means there there are cars which can drive automatically.

[00:10:03]And there are already cars are there.

[00:10:05]Some are already operating commercially.

[00:10:07]I recently saw in the San Diego, San Francisco, Los Angeles in the United States just last month.

[00:10:14]And so what it is doing, let's say this is autonomous driving model and it made a decision which was a dangerous. It did a late breaking and wrong action or missed pedit- pedestrian uh unable to detect the pedestrian.

[00:10:33]Now, it did kind of thing. Now, if we do not know why it did late breaking or wrong action or any kind of missed pedestries pedestrian detection and it hit or it did a risky driving there. So, if we and the kind of data it is taking is [music] camera image, then lidar image, it's a kind of sensors are there in the car, the road condition, it was a hazy from weather condition and also from its own detection and the sensor system, then the maps and its own GPS location.

[00:11:09]Now, if we if we can find out that whether the data had a not enough training data, whether the model itself misclassify even everything was there and it was did the wrong reasoning, whether the sensor was working correctly or not means this autonomous driving is made a wrong decision, but the model is limited with the data it is getting inside when it is working. So, if the sensors are not functioning correctly uh because of rain, fog, or any kind of occlusion in the sensors.

[00:11:45]Uh like I'm in the Norway and here uh it snows 6 month. Even now I'm recording today is a 3rd of May and it's snowing.

[00:11:58]Just now I came by under the snow and the mountain is full of snow.

[00:12:03]So, in this kind of suddenly snow is happening in the daytime and the light condition went off and that can block the sensor etc. So, you know, this kind of scenario can happen.

[00:12:14]Now, now another thing also can be that the AI was trained and the kind of scenario happened that was not was was not in the training time. So, if you can know all this reason that why a problem happened, then we can make a uh if we can improve, we can analyze why failure happened, we can improve safety and we can if needed we can retrain the system.

[00:12:41]So, this will help in safety review and making the AI system more safe and usable for the world. Okay?

[00:12:50]So, you can see the whole scene now.

[00:12:54]Now, so in this course in this lecture the expected learning is that so I will be breaking the lecture this is a lecture one, but I have broken to four parts and I will take you will you can find like lecture 1A, 1B, 1C, 1D. So, similarly lecture 2A, 2B, 2C.

[00:13:15]So, you can understand when it is 1A, B, C, D they are related. So, you can take as a uh chunk A, B, C, D. Okay? So, for your ease of following.

[00:13:26]Now, uh what you are going to expect from this course that by the end of this course uh you will be able to understand why deep learning models are black boxes and how explanation are generated, why explanations are useful or misleading, how interpretability is applied in uh different kind of task, computer vision, natural language processing, time series data, how interpretability connects with ethics, responsible AI.

[00:13:56]And there are different metrics are there in this first lecture, we will go through it. Now, I tell you ask you a question. Okay? You have to think that you have a medical diagnosis system, you have a loan decision making system, a job selection AI system, autonomous driving system, or education assessment, your exam assessment.

[00:14:19]Which AI decision should always be explainable? Means out of A, B, C, D, which system you think or this is E actually. A, B, C, D, E. Which system you think you need a decision when AI is making a decision, you expect it also be explainable. So, stay tuned.

[00:14:40]We will come to the next lectures. Now, this course is based on the book written by me and co-author Interpretable Deep Learning. This is very expensive book. Uh publisher has made it very expensive.

[00:14:53]Uh uh and it is if you are interested you can buy, but uh to follow this course uh you can simply follow the lecture slides, uh lectures and also the hands-on uh lectures will be there where I will record the hands-on thing and I won't be conducting a live session unless requested by student. So, in the next lecture what you are going to find in lecture 1B, uh why AI what is the difference between AI, ML, deep learning and they are all interrelated term and often it is confusing and its foundation. So, stay tuned for it and if you want to see the whole this was the this is a conducted as a classroom course and if I go by the lectures of this uh general flow is lecture 1 to 15 and but in the YouTube version I have divided into small small section. So, you just follow the YouTube lecture. Uh but the course uh in the detail will it's going to be recorded like this.

[00:15:58]So, stay tuned for next lecture and see you again. Thank you.