Mentora School
HomeASSEB AI HubUnit 5: Explainability in AI / XAI
Class XIIASSEB-527Unit 5: Explainability in AI / XAI

Unit 5: Explainability in AI / XAI

Master official ASSEB Class XII AI Unit 5: Explainable AI (XAI), Model Interpretability, SHAP, LIME, Grad-CAM, and Accuracy vs Interpretability Trade-offs.

Theory: 8 Marks
Practical: 5 Marks
12 min read
Subscription Gate Active
AHSEC / ASSEB Higher Secondary Syllabus

Unlock Unit 5: Explainability in AI / XAI

Access verified code errata manual, 50 external viva questions, period lesson scripts, and practical examination rubrics.

256-Bit Encrypted Payment • Supports UPI, Cards & NetBanking
Prefer annual board pass?View All Passes & Billing
Official Textbook Exercises & Master Solutions

Class XII • Unit 5 Exercise Solutions

Human-crafted academic answers for all exercise questions (Pages 26–29) formulated to secure full marks in the ASSEB Board Examinations. Includes step-by-step calculations, diagrams, and examiner scoring tips.

Q1Section A: Fill in the Blanks (1 Mark Each)1 Mark

Explainable Artificial Intelligence (XAI) refers to techniques that make AI decisions ______ to humans.

Q2Section A: Fill in the Blanks (1 Mark Each)1 Mark

In AI, ______ refers to how easily a model’s decision-making process can be understood.

Q3Section A: Fill in the Blanks (1 Mark Each)1 Mark

Models such as decision trees and linear models are considered ______ models because their internal working is clear.

Q4Section A: Fill in the Blanks (1 Mark Each)1 Mark

Neural networks and ensemble methods are examples of ______ models.

Q5Section A: Fill in the Blanks (1 Mark Each)1 Mark

The trade-off between ______ and interpretability is an important consideration in AI systems.

Q6Section A: Fill in the Blanks (1 Mark Each)1 Mark

______ explainability refers to understanding the overall behaviour of a model across all data.

Q7Section A: Fill in the Blanks (1 Mark Each)1 Mark

______ explainability focuses on explaining individual predictions made by a model.

Q8Section A: Fill in the Blanks (1 Mark Each)1 Mark

LIME explains predictions by analyzing ______ cases around a given input.

Q9Section A: Fill in the Blanks (1 Mark Each)1 Mark

SHAP explains predictions by assigning a ______ to each feature.

Q10Section A: Fill in the Blanks (1 Mark Each)1 Mark

Explainability techniques help in identifying important features and detecting possible ______ in AI models.

Q1Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Explainable Artificial Intelligence (XAI) mainly focuses on:

Improving the speed of AI models
Reducing the training data
Making model decisions understandable
Eliminate human involvement
Q2Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Which of the following is a transparent model?

Neural Network
Decision Tree
Ensemble Model
Deep Learning Model
Q3Section B: Multiple Choice Questions (1 Mark Each)1 Mark

“Black box” models are difficult to interpret because:

They use less data.
They are slow.
Their internal workings are complex.
They are inaccurate
Q4Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Interpretability in AI refers to:

Speed of prediction
Accuracy of prediction
Ease of understanding the model
Size of dataset
Q5Section B: Multiple Choice Questions (1 Mark Each)1 Mark

The trade-off in AI models is mainly between:

Speed and cost
Accuracy and interpretability
Data and computation
Input and output
Q6Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Which of the following best describes local explainability?

Understanding the entire model
Explaining predictions for a single instance
Training the model
Reducing model complexity
Q7Section B: Multiple Choice Questions (1 Mark Each)1 Mark

LIME is primarily used to:

Train machine learning models
Increase model accuracy
Explain individual predictions
Reduce dataset size
Q8Section B: Multiple Choice Questions (1 Mark Each)1 Mark

SHAP explains model predictions by:

Removing unnecessary features
Assigning contribution values to each feature
Increasing training data
Simplifying the dataset
Q9Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Explainability in AI helps in:

Increasing hardware performance.
Detecting bias and improving trust.
Reducing training time.
Eliminating data completely.
Q10Section B: Multiple Choice Questions (1 Mark Each)1 Mark

Why are explainability techniques important in healthcare AI systems?

To reduce computation.
To improve hardware.
To understand and verify decisions.
To avoid using data.
Q1Section C: Short Answer Questions (2–3 Marks Each)2 Marks

Define Explainable Artificial Intelligence (XAI). Why is it important in modern AI systems?

Definition: Explainable Artificial Intelligence (XAI) refers to the suite of machine learning techniques, tools, and design principles that make the decisions, internal logic, and predictions of AI systems transparent and understandable to human stakeholders.

Why Important: As complex black-box models are deployed in high-stakes societal domains (medicine, justice, lending), explainability is essential to build human trust, detect algorithmic biases, verify safety, and comply with legal regulations (e.g. GDPR Right to Explanation).

Q2Section C: Short Answer Questions (2–3 Marks Each)2 Marks

Differentiate between explainability and interpretability with suitable examples.

Criterion

Interpretability

Explainability

Nature

Intrinsic / Passive transparency built directly into model architecture.

Post-hoc / Active explanation applied to complex models after training.

Example

A shallow Decision Tree or Linear Regression (weights directly reveal feature impact).

A deep ResNet whose decisions are explained using LIME superpixel masks or SHAP values.

Q3Section C: Short Answer Questions (2–3 Marks Each)2 Marks

What are transparent models? Give one example and explain why they are easy to interpret.

Transparent Models (White-Box Models): Machine learning algorithms whose internal mathematical calculations and decision logic are simple and transparent enough to be directly inspected and comprehended by humans.

Example: A shallow Decision Tree.

Why Easy to Interpret: Decisions follow a series of intuitive, hierarchical IF-THEN rules (e.g., "IF Income < $30,000 AND Credit < 600 THEN Reject"). A human can trace the decision path from root to leaf effortlessly.

Q4Section C: Short Answer Questions (2–3 Marks Each)2 Marks

What is meant by a black-box model? Why are such models difficult to understand?

A Black-Box Model is an AI system that takes inputs and produces outputs without providing any visible or easily understandable internal reasoning for how the decision was reached.

Why Difficult: Architectures like deep neural networks and gradient-boosted ensembles contain millions of parameters, dense matrix multiplications, and nested non-linear transformations. The decision boundary exists in a high-dimensional space that defies mental visualization.

Q5Section C: Short Answer Questions (2–3 Marks Each)2 Marks

Explain the concept of model complexity in AI systems with reference to a real-life example.

Model Complexity: Refers to a model's architectural capacity—measured by the number of trainable parameters, degrees of freedom, and non-linear interactions it can represent.

Real-Life Example: Predicting house prices. A low-complexity model uses a straight line (Price = w₁·Size + b). A high-complexity deep model includes polynomial interactions of square footage, school ratings, crime indices, and aerial photos. While the complex model captures subtle pricing trends, it becomes much harder to explain how any single feature influenced the final quote.

Q6Section C: Short Answer Questions (2–3 Marks Each)2 Marks

What is the trade-off between accuracy and interpretability in AI? Why is it important?

The Accuracy vs. Interpretability Trade-Off is the fundamental inverse relationship in machine learning:

As model complexity increases to achieve maximum predictive accuracy on complex data (e.g. deep neural networks), the model's intrinsic human interpretability diminishes.

Why Important: Practitioners must choose their model based on the operational stakes. In high-stakes healthcare or parole hearings, choosing a slightly less accurate but transparent model is often preferable to deploying an un-auditable black box.

Q7Section C: Short Answer Questions (2–3 Marks Each)2 Marks

Distinguish between global explainability and local explainability.

Dimension

Global Explainability

Local Explainability

Scope

Explains the overall, macro-level behavior of the model across the entire dataset.

Explains the micro-level reasoning behind a single, specific prediction instance.

Question Answered

"What features are most important in general across all loan applicants?"

"Why was applicant Ramesh's specific loan application rejected today?"

Q8Section C: Short Answer Questions (2–3 Marks Each)2 Marks

What is LIME? How does it help in explaining AI predictions?

LIME (Local Interpretable Model-agnostic Explanations): A post-hoc explanation technique that approximates the local decision boundary of any black-box model around a specific input.

How It Works: It generates synthetic variations (perturbations) around the input instance, obtains the black-box model's predictions for them, weights them by proximity to the original instance, and fits a simple interpretable sparse linear surrogate model whose weights reveal local feature importance.

Q9Section C: Short Answer Questions (2–3 Marks Each)2 Marks

What is SHAP? How does it differ from LIME in explaining model predictions?

SHAP (SHapley Additive exPlanations): A game-theoretic framework that computes the fair marginal contribution of each feature to a prediction using Lloyd Shapley's (1953) Shapley Values.

Key Difference from LIME: While LIME uses heuristic local perturbations that can be slightly unstable across runs, SHAP is grounded in solid cooperative game theory, guaranteeing mathematical consistency and the Efficiency Axiom (attributions sum to the exact difference from the baseline).

Q10Section C: Short Answer Questions (2–3 Marks Each)2 Marks

How do explainability techniques help in detecting bias in AI systems?

Explainability tools expose the hidden features driving algorithmic decisions:

Detecting Unfair Proxies: XAI reveals if a model is secretly using proxy features (e.g. zip codes or schooling background) as illicit stand-ins for protected demographic attributes (gender, race, caste).

Catching Spurious Correlations: Tools like LIME and SHAP expose shortcut learning (such as a model classifying wolves based on background snow rather than animal anatomy).

Q1Section D: Long Answer Questions (5 Marks Each)5 Marks

Explain the concept of Explainable AI (XAI). Discuss its role in building trust, ensuring fairness, and improving decision-making in AI systems.

  1. 1.
    Concept of Explainable AI (XAI)

Explainable AI (XAI) encompasses the methods, algorithmic architectures, and evaluation frameworks designed to make the outputs and internal reasoning of artificial intelligence systems transparent, interpretable, and accountable to human users. Rather than treating machine learning as an opaque oracle that outputs unexplained decisions, XAI provides human-understandable justifications for why specific outcomes were produced.

  1. 1.
    The Three Core Pillars of XAI's Societal Role

Building Justifiable Human Trust:

Domain experts (such as oncologists, bank underwriters, and air traffic controllers) cannot legally or ethically accept black-box recommendations without validation.

XAI provides evidence-based explanations (e.g. highlighting tumor margins on an MRI), giving human experts justifiable confidence to adopt automated recommendations.

Ensuring Algorithmic Fairness & Regulatory Compliance:

Machine learning models trained on historical societal data frequently absorb and magnify human prejudices.

XAI allows auditors to verify that sensitive attributes (e.g. gender, religion, caste) do not influence high-stakes decisions, ensuring compliance with non-discrimination laws and the EU GDPR's "Right to Explanation".

Improving Decision-Making & Debugging:

XAI exposes Clever Hans shortcut learning, where models exploit background artifacts rather than true causal relationships.

By pinpointing which features drove an erroneous prediction, engineers can debug data pipelines, patch biases, and refine model architectures effectively.

Q2Section D: Long Answer Questions (5 Marks Each)5 Marks

Describe the difference between transparent models and black-box models. Why is explainability particularly important in black-box models?

  1. 1.
    Transparent vs. Black-Box Models

Dimension

Transparent (White-Box) Models

Black-Box Models

Internal Mechanics

Simple mathematical structure; explicit parameters with clear physical meaning.

Deep hierarchical layers with millions of non-linear parameters.

Simulatability

High; a human can trace calculations manually.

Zero; calculations are too complex for human mental arithmetic.

Representative Architectures

Linear Regression, Logistic Regression, Shallow Decision Trees, Rule-based systems.

Deep Neural Networks (CNNs, Transformers), Random Forests, XGBoost.

Data Suitability

Well-suited for low-dimensional tabular data.

Exceptional performance on high-dimensional unstructured vision and language.

  1. 1.
    Why Explainability is Vital for Black-Box Models

Mitigating Opacity Hazards: In high-stakes applications, an un-audited black box can fail silently on edge cases (e.g. autonomous vehicles misidentifying white trucks against bright skies).

Legal and Ethical Accountability: Institutions must provide actionable justifications when denying critical services (loans, insurance, bail), which black-box models cannot supply without post-hoc XAI tools.

Enabling Post-Hoc Auditing: Techniques like LIME and SHAP open the black box from the outside, extracting feature attributions without compromising model performance.

Q3Section D: Long Answer Questions (5 Marks Each)5 Marks

Explain the concept of model complexity and its impact on interpretability. Discuss the trade-off between accuracy and interpretability with suitable examples.

  1. 1.
    Model Complexity & Interpretability

Model complexity is dictated by the number of trainable weights, the depth of layered transformations, and the capacity to capture high-order feature interactions. As complexity increases, a model's intrinsic interpretability declines exponentially.

HIGH INTERPRETABILITY | [Linear Models, Shallow Trees] | | \ [Generalized Additive Models, KNN] | | \ [Random Forests, Ensembles] | LOW INTERPRETABILITY | \ [Deep CNNs, LLMs, Transformers] +--------------------------------------------------- LOW ACCURACY HIGH ACCURACY

  1. 1.
    The Accuracy vs. Interpretability Trade-Off

Low-Stakes / High-Complexity Scenarios: In movie recommender systems or image upscaling, predictive accuracy is paramount while the cost of error is trivial. Deep black-box networks are deployed without demanding complete interpretability.

High-Stakes / Low-Complexity Scenarios: In judicial sentencing risk tools or pediatric drug dosing, an unexplained error can devastate human lives. Practitioners intentionally accept simpler linear or shallow tree models because full auditability is non-negotiable.

The XAI Solution: Post-hoc methods (SHAP, LIME) allow deploying complex models at peak accuracy while providing local explanations that fulfill interpretability requirements.

Q4Section D: Long Answer Questions (5 Marks Each)5 Marks

Describe the working of LIME and SHAP as explainability techniques. Compare their approaches and explain when each method is preferred.

  1. 1.
    Working Mechanism of LIME

Local Interpretable Model-agnostic Explanations (Ribeiro et al., 2016):

Local Perturbation: Samples synthetic neighboring points by toggling features or superpixels around target instance x.

Black-Box Querying: Passes perturbed points through the black box to observe output shifts.

Proximity Weighting: Weights samples using an exponential distance kernel (closer neighbors get higher weights).

Surrogate Fitting: Fits an interpretable sparse linear model on the weighted samples whose weights serve as local feature attributions.

  1. 1.
    Working Mechanism of SHAP

SHapley Additive exPlanations (Lundberg & Lee, 2017):

Treats features as players in a cooperative game and computes Shapley Values:φi = Σ{S ⊆ F \ {i}} [|S|!(|F|-|S|-1)! / |F|!] × [f(S ∪ {i}) - f(S)]

Efficiency Property: Satisfies exact attribution, where the sum of all Shapley values equals the prediction minus the base average: Σ φ_i = f(x) - E[f(x)]. (As proven in Module 5: Income +22.5 and Credit +17.5 sum to exactly 40.0).

  1. 1.
    Direct Comparison & Selection Guide

Criterion

LIME

SHAP

Mathematical Foundation

Heuristic local linear surrogate approximation.

Rigorous axiomatic cooperative game theory.

Consistency

Can yield slight variations due to random sampling.

Mathematically unique, consistent, and provably fair.

Compute Speed

Fast; ideal for rapid exploration on complex data.

Computationally intensive (exponential unless approximated via TreeSHAP).

Recommended When

Explaining high-resolution images/text quickly.

Auditing regulated tabular financial or clinical risk models.

Q5Section D: Long Answer Questions (5 Marks Each)5 Marks

Consider an AI system that predicts whether a student will pass or fail based on study hours, attendance, sleep, and previous scores. Explain how LIME and SHAP can be used to interpret the prediction. What insights can be obtained from such explanations?

  1. 1.
    Scenario Overview

Consider a student, Rahul, whom the black-box classifier predicts has an 82% probability of Passing (Baseline average = 50%). The inputs are: Study Hours = 6 hrs/day, Attendance = 92%, Sleep = 5 hrs/night, Previous Score = 58%.

  1. 1.
    Interpreting with LIME

LIME generates synthetic student profiles by perturbing Rahul's features (e.g. 5 hrs study, 85% attendance).

It queries the black-box model and fits a local linear regression line around Rahul's profile.

LIME Output: Reveals that in Rahul's immediate local context, Attendance (>90%) contributed +0.25 and Study Hours (>5) contributed +0.18 toward passing, while Sleep (<6 hrs) had a minor negative weight of -0.06.

  1. 1.
    Interpreting with SHAP (Waterfall / Force Plot Analysis)

Base Value: 0.50 (50%) + Attendance = 92% [+0.20] -----> (Pushes toward Pass) + Study Hours = 6 hrs [+0.15] -----> (Pushes toward Pass) - Sleep = 5 hrs [-0.05] <----- (Pushes toward Fail) + Previous Score = 58% [+0.02] -----> (Pushes toward Pass) Final Output: f(x) = 0.82 (82% Pass Probability) Sum: 0.50 + 0.20 + 0.15 - 0.05 + 0.02 = 0.82 (Satisfies Efficiency Axiom)

  1. 1.
    Actionable Educational Insights Obtained

Validation of Success Factors: Rahul's strong attendance and dedicated daily study hours are the primary drivers boosting his pass probability by +35% above the class average.

Targeted Remedial Advice: The negative SHAP attribution for sleep (-0.05) provides an actionable well-being insight: Rahul can further increase academic performance and memory retention by improving sleep hygiene to 7–8 hours per night.

Absence of Bias: Confirms that the model evaluates legitimate academic effort rather than extraneous student background variables.

© ASSEB Class XII Artificial Intelligence (Subject Code: 985) • Master Facilitator Answer Series • Bamunimaidam, Guwahati