Explainable AI

Explainable AI (XAI) is the set of methods and practices that make an AI system's predictions understandable to humans — identifying which inputs drove which outputs. XAI is required by regulators in lending (ECOA adverse-action notices), healthcare (clinical decision support validation), and under the EU AI Act for high-risk systems.

Common XAI techniques include SHAP (feature importance), LIME (local linear approximations), attention visualization, and natural-language explanation generation. For LLMs, XAI remains an active research area; current production approaches rely primarily on citation, grounding, and structured output constraints rather than mechanistic interpretability.