Why Black-Box Models Are a Regulatory Problem in Indian Insurance
When an Indian insurer declines a commercial risk or loads the premium by 40%, it must be able to explain why. This obligation predates AI: Section 45 of the Insurance Act 1938, as amended in 2015, prohibits insurers from repudiating a policy after three years on grounds of misstatement unless the insurer can demonstrate that the misstatement was material and that the policyholder knew it was false. The burden of explanation sits with the insurer. IRDAI's claims adjudication guidelines extend this logic to every stage of underwriting: the decision to accept, decline, load, or conditionally cover a risk must be supportable with reference to the specific facts of the case.
A gradient boosting model trained on 10 years of motor fleet loss data and 80 input features does not naturally produce this explanation. It produces a prediction. The prediction may be more accurate than a human underwriter's judgment, but accuracy is not the same as accountability. When a broker asks why a client's fire premium was loaded by 35% relative to the previous year, the answer cannot be 'the model said so.' IRDAI's Information Security Guidelines 2023 require that AI systems used in consequential decisions maintain audit-ready documentation of how outputs are produced, and that decision rationales be explainable to non-technical reviewers including policyholders, surveyors, and IRDAI inspection teams.
The specific regulatory hook is paragraph 7.4 of the Information Security Guidelines 2023, which requires insurers to implement controls ensuring that automated systems operating on policyholder data produce outputs that are traceable, interpretable, and reviewable. The word 'interpretable' is doing significant work here. IRDAI's inspection teams have, since mid-2025, begun asking specific questions about how AI-assisted underwriting decisions are documented and what processes exist for policyholders to contest them. Insurers without explainability infrastructure have found themselves scrambling to retrofit documentation onto production systems.
The practical consequence of unexplainable denials extends beyond IRDAI scrutiny. Under the Consumer Protection Act 2019, a policyholder denied coverage or subjected to an unexplained premium loading can file a complaint with the IRDAI Ombudsman or the Consumer Disputes Redressal Commission. The Ombudsman has historically sided with policyholders in cases where the insurer cannot articulate a specific factual basis for its decision. AI-generated decisions without attached explanations are particularly vulnerable to this outcome, because the insurer cannot reconstruct the decision rationale from a prediction score alone.
What Explainable AI Actually Means for Underwriting Models
Explainable AI (XAI) is not a single technique. It is a category of methods that aim to make model predictions interpretable, either by designing models that are inherently transparent or by attaching post-hoc explanation methods to opaque models after training. For insurance underwriting in India, where the dominant model architectures are gradient boosted trees (XGBoost, LightGBM), neural networks for image and unstructured data, and increasingly large language models for document analysis, post-hoc explanation is the primary practical approach.
SHAP (SHapley Additive exPlanations) is the most widely deployed XAI method in Indian insurance underwriting as of 2026. SHAP assigns each input feature a contribution value for a specific prediction, expressed in the same units as the model output. For a commercial property underwriting model, SHAP might indicate that the presence of a thatch roof contributed INR 18,000 to the predicted annual premium, while a fire NOC from a certified authority reduced it by INR 7,500. These feature contributions sum to the model's output, and the explanation is faithful to the model's actual computation rather than being a simplified approximation. SHAP values for tree-based models can be computed exactly and efficiently, making them the default choice for the property, marine, and motor underwriting models deployed at ICICI Lombard and HDFC Ergo.
LIME (Local Interpretable Model-Agnostic Explanations) takes a different approach. Rather than deriving explanations from the model's internal structure, LIME builds a simple interpretable model (typically a linear regression) on a set of perturbed inputs in the neighbourhood of the prediction being explained. The simplified model approximates the complex model's behaviour locally and provides feature weights as the explanation. LIME is more computationally expensive than SHAP for tree models but is useful for explaining predictions from neural networks and language models where SHAP is harder to compute. HDFC Ergo's cyber underwriting model, which uses a neural network trained on breach incident data and company profile features, relies on LIME for generating adjuster-facing explanations.
Attention weights are a third approach used specifically for transformer-based models including LLMs. When a language model analyses a proposal form and produces a risk assessment, the attention weights indicate which tokens (words, phrases) in the input received the highest weight in the model's decision. Attention is not a perfect proxy for causality, and researchers have documented cases where high-attention tokens are not the causally important features, but it provides a starting point for underwriter review and is more tractable than post-hoc methods on very large models.
For practical deployment, Indian insurers use SHAP for their primary underwriting models because it is computationally efficient, produces stable explanations, and integrates cleanly with the Python-based model development pipelines used across the industry. LIME is reserved for specialist cases. Attention visualisation is used in document analysis tools rather than in core pricing models.
The three explanation audiences
Effective explainability infrastructure must serve three different audiences with different information needs. The underwriter needs a feature contribution summary that allows them to quickly understand the dominant risk factors, verify that the model has picked up on the same cues they would have considered manually, and make an informed override decision if required. The policyholder or broker needs a plain-language summary of the factors that drove the premium or the decline decision, expressed without statistical jargon. The IRDAI inspector or compliance team needs the full audit trail: model version, input features, SHAP values, any overrides applied by the human underwriter, and the final decision with its rationale. These three needs are distinct and an XAI implementation must address all three.
IRDAI's Information Security Guidelines 2023: Specific Explainability Obligations
IRDAI's Information Security Guidelines 2023, issued in November 2023, are the primary regulatory instrument governing AI transparency in Indian insurance. The Guidelines are structured around six control domains: governance, risk management, data security, system security, operational security, and third-party management. AI explainability obligations sit primarily within the governance and risk management domains.
Paragraph 5.2 of the Guidelines requires insurers to constitute an Information Security Committee at the board level with responsibility for overseeing AI and data-driven decision systems. The Committee must review, at least annually, the insurer's inventory of AI systems, the risks they pose, and the controls in place. For underwriting AI, this means the board-level committee must be able to satisfy itself that the models used are appropriate, that their outputs are interpretable, and that processes exist for challenging and overriding model-generated decisions.
Paragraph 7.4 is the most operationally specific provision for XAI. It requires that automated decision systems maintain records that allow reconstruction of how a specific decision was reached, that these records be retained for the period specified in the insurer's data retention policy (typically 7 to 10 years for underwriting decisions), and that they be available to authorised IRDAI inspection teams on request. The standard 'export the SHAP values to a database table keyed by the quote or policy number' is the most common technical implementation of this requirement.
Paragraph 8.1 addresses model change management. Any material change to a model in production, including re-training on new data, updating model architecture, or adjusting decision thresholds, must go through a formal change management process including impact assessment, testing in a controlled environment, and sign-off from designated risk and compliance officers. This provision prevents the silent model drift that has been a problem in early Indian AI deployments, where a model's behaviour changed substantially between training cycles without the underwriting team being aware.
The IRDAI Regulatory Sandbox framework, updated in 2024, provides a parallel channel for testing novel XAI approaches. Sandbox participants can test explanation methods not yet established in Indian regulatory practice, with IRDAI receiving structured outcome data. At least three sandbox applications in the 2025 cohort involved XAI for underwriting: one testing SHAP-based explanation dashboards for motor commercial underwriting, one testing counterfactual explanations ('what would have changed to make this risk acceptable'), and one testing natural language generation of explanation summaries from SHAP values. The sandbox results, expected to inform IRDAI's 2026 draft AI guidance, will likely set more specific technical standards for explanation methods.
How ICICI Lombard and HDFC Ergo Approach Model Documentation
The two largest private-sector general insurers in India, ICICI Lombard and HDFC Ergo, have published enough in investor presentations, industry conferences, and technical hiring job descriptions to allow a reasonable reconstruction of their XAI approaches, even where they have not published detailed technical specifications.
ICICI Lombard's underwriting technology strategy, outlined in its FY2025 and FY2026 investor communications, describes a 'model card' approach to AI governance. Model cards are structured documentation artefacts that specify for each production model: the model's intended use and scope, the training data characteristics, performance metrics on held-out validation sets, known failure modes and edge cases, the explanation method used, and the human oversight process. ICICI Lombard has adopted model cards as a standard governance artefact across its underwriting AI portfolio, and the cards are reviewed quarterly by the technology risk function. The SHAP explanation pipeline runs at prediction time and stores feature contribution vectors alongside the quote record in the underwriting system's database, making it possible to retrieve the explanation for any individual decision long after the fact.
HDFC Ergo's approach, described in technical presentations at the Insurance Summit India 2025, uses a layered explanation architecture. The first layer is a real-time SHAP summary displayed to the underwriter at the point of quote generation: the top five positive and top five negative contributing features, expressed in plain language. 'Manufacturing location classified as Zone 3 seismic risk: +INR 12,400.' 'Installed sprinkler system with FM Global certification: -INR 8,200.' The second layer is a full SHAP waterfall chart accessible to the underwriting team for quality review and override justification. The third layer is the complete audit record, including model version, all 60 input features, all SHAP values, the model's raw output score, the decision rule applied to that score, and any human overrides, stored in an append-only audit table.
Both insurers use model performance monitoring infrastructure that flags prediction distribution shifts on a weekly basis. When the distribution of SHAP values for a specific feature drifts materially from its historical pattern, the monitoring system alerts the model governance team. This is operationally important because distribution shifts often indicate data quality problems or real-world changes not yet reflected in the model (a new building code, a change in fire hazard classification), and unexplained shifts in explanation patterns can indicate that the model is starting to rely on spurious correlations.
Override governance
Both insurers require underwriters to document the reason for any override of a model-generated decision. When an underwriter accepts a risk that the model would have declined, or prices a risk materially below the model's recommendation, the override record must include the underwriter's specific rationale. These override records are analysed quarterly to identify systematic override patterns, which can indicate either that the model needs retraining or that specific underwriters are not applying the model's inputs consistently. This closed-loop process, where explanation data informs model governance and override patterns inform model improvement, is the operational infrastructure that makes XAI genuinely useful rather than a compliance checkbox.
Practical Implementation Patterns for Underwriters
Implementing XAI in a production underwriting environment requires decisions across four areas: explanation architecture, output format, integration with workflow tools, and governance processes. Each area has implementation choices that materially affect whether the explanation infrastructure is used effectively or becomes a documentation exercise that nobody reads.
Explanation architecture should match the model type. For XGBoost and LightGBM models, use SHAP's TreeExplainer, which computes exact SHAP values efficiently without sampling. For neural networks, SHAP's DeepExplainer or GradientExplainer provides approximate values. For linear models (useful as audit-friendly fallbacks), direct coefficient interpretation is sufficient. Avoid approximate SHAP methods like KernelExplainer for production primary models because the approximation variance is unpredictable on individual predictions and creates unreliable audit records.
The explanation output must be translated from model-space to business-space before it reaches underwriters. A SHAP value of +0.23 on a log-odds scale means nothing to a commercial lines underwriter reviewing a cement plant risk. The explanation layer should translate this to 'Dust exposure in manufacturing process: estimated premium loading of INR 15,600' using the same premium units that appear on the quote. This translation requires a mapping layer that converts model features and SHAP values into business descriptions and monetary terms. Building this mapping layer is typically 30 to 40% of the XAI implementation effort.
Workflow integration determines whether XAI is used. If the SHAP summary is available only in a separate analytics dashboard that requires a separate login, underwriters will not use it. The explanation must be embedded in the quote screen or the underwriting workbench at the point of decision. Most Indian insurers have built this integration through API calls from the underwriting platform to the explanation service, with the SHAP summary rendered inline. The latency budget for the explanation service call is typically 500 milliseconds, which is achievable with pre-computed SHAP values for common feature combinations or with efficient tree SHAP computation.
Counterfactual explanations are a powerful complement to SHAP for underwriter communication. A counterfactual explanation answers: 'What specific changes to this risk would bring the premium down by 20%?' For example: 'Installing a fire suppression system rated by a IRDAI-recognised testing laboratory would reduce the estimated premium by INR 42,000 annually. Upgrading electrical wiring to Bureau of Indian Standards IS 694 specification would reduce it by a further INR 18,500.' Counterfactuals give the underwriter a conversation tool with the broker or insured, and they turn the model's output from a static verdict into an actionable risk improvement dialogue. Generating high-quality counterfactuals requires constrained optimisation over the feature space, which is more computationally intensive than SHAP, but the user value is substantial for mid-to-large commercial risks.
Governance processes must include three elements: periodic model validation against holdout data with explanation consistency checks; a process for underwriters to flag explanation outputs that seem incorrect or misleading; and a process for incorporating flagged issues into model improvement cycles. The explanation consistency check is particularly important: a model explanation that systematically misattributes premium loads to the wrong features is worse than no explanation, because it creates false confidence and incorrect override decisions.
Regulatory Risk of Unexplainable Denials Under Section 45 and IRDAI Complaint Processes
The intersection of AI-generated underwriting decisions and Section 45 of the Insurance Act creates a specific compliance risk that Indian legal teams and underwriting managers are increasingly alert to. Section 45, as amended in 2015, limits an insurer's ability to challenge the validity of a policy after three years on grounds of non-disclosure or misrepresentation. But the section's broader context, combined with IRDAI's adjudication guidelines, creates obligations at the point of original underwriting: the insurer must be able to demonstrate that its risk assessment was based on the material facts disclosed, not on opaque algorithmic factors that the policyholder could not have known would be material.
When an AI model declines a commercial risk based on features that the policyholder did not provide (inferred features derived from geospatial data, third-party credit bureau signals, or web-scraped information), the legal position of the insurer becomes precarious if the decline is challenged. The insured's broker can argue that the decline was based on information the insured never had an opportunity to correct or supplement. IRDAI's Ombudsman Rules 2017 and the subsequent 2021 amendments provide a formal channel for challenging such decisions, and the Ombudsman has the power to direct the insurer to issue coverage, reverse a premium loading, or pay compensation for wrongful denial.
Three specific risk scenarios have emerged in 2025 and 2026 from IRDAI inspection reports and Ombudsman determinations:
First, decline decisions where the SHAP analysis retrospectively reveals that a protected attribute (most commonly geography as a proxy for caste or religion in certain rural risk contexts) was a dominant explanatory factor. IRDAI's emerging fairness guidelines, previewed in the 2025 sandbox consultation, would make such declines directly challengeable as discriminatory underwriting.
Second, premium loadings imposed by AI models on renewal policies where the feature driving the loading is a claims indicator that the insurer cannot disclose to the policyholder without compromising fraud prevention. The tension between the requirement to explain and the requirement not to disclose fraud-detection signals requires careful policy design.
Third, coverage gaps created when an AI underwriting model introduces an exclusion or a sub-limit that was not present in the prior year's policy, without the underwriter providing a specific explanation to the policyholder. If the exclusion later causes a claim shortfall, the policyholder can challenge whether the exclusion was adequately disclosed, and an unexplained AI-generated exclusion is difficult to defend.
The practical mitigation for all three scenarios is the same: document the explanation for every material underwriting decision, ensure the explanation references only disclosed or disclosable features, and maintain an audit trail that can reconstruct the decision from the explanation record alone. Insurers that have invested in XAI infrastructure for operational efficiency find that the same infrastructure also provides their primary regulatory defence against unexplainable denial claims.
Building an XAI-Ready Underwriting Function: Roles, Skills, and Tooling
Deploying XAI in production underwriting requires capabilities that sit between the data science team and the underwriting team and are not native to either. The data scientist knows how to compute SHAP values; the underwriter knows what a meaningful risk factor looks like; neither alone can build explanation outputs that are technically correct and operationally useful. The emerging role bridging this gap in Indian insurers is the underwriting analytics translator: a function that holds both insurance domain knowledge and quantitative skills, whose job is to validate that the model's explanations align with underwriting logic, to build the business-language mapping layer, and to train underwriters on using explanation outputs effectively.
The technology stack for XAI in Indian insurance has converged around a set of open-source Python libraries. The SHAP library (version 0.44 and later) provides TreeExplainer, DeepExplainer, and KernelExplainer. MLflow is used for model tracking and versioning, ensuring that the model version used for any prediction is recorded alongside the SHAP output. Feature stores (Feast or custom implementations on cloud platforms such as AWS SageMaker Feature Store) ensure consistency between training features and inference features, a known failure mode where feature computation differs between model training and production prediction, causing SHAP values to misrepresent the actual feature contributions.
Explanation quality testing is a distinct engineering discipline from model accuracy testing. Standard model validation measures accuracy on held-out samples. Explanation quality testing measures whether the explanations correctly identify the features that the model relies on, using techniques such as perturbation tests (systematically changing the value of a feature and checking whether the SHAP value correctly predicts the change in model output), consistency tests (checking that the same feature produces consistent SHAP values across similar inputs), and sanity checks against domain knowledge (checking that features known to be relevant from actuarial experience produce high SHAP values in the appropriate direction).
Vendor tooling from Fiddler AI, Arthur AI, and WhyLabs provides monitoring infrastructure for explanation drift in production. These platforms compute SHAP values on production predictions, monitor the distribution of feature contributions over time, and alert when explanations start behaving anomalously. Go Digit has publicly described using a combination of in-house SHAP computation and third-party monitoring for its motor underwriting models. Bajaj Allianz uses a custom explanation dashboard built on top of open-source tools, integrated with its policy administration platform.
The investment timeline for a greenfield XAI implementation in a mid-sized Indian insurer is approximately 6 to 9 months from decision to production for a single model, including model audit, explanation layer build, workflow integration, and underwriter training. Retrofitting XAI onto an existing production model without retraining typically takes 3 to 4 months. The primary cost driver is not the XAI library licensing (SHAP is open source) but the engineering and product effort to build the business-language explanation layer and integrate it into the underwriting workflow.