Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation

The gap LAD fills

Concept-based explanations fall into families that each give something up. Post-hoc discovery methods (ICE, CRAFT, FACE) are faithful to the model, but their factors are unnamed, and the same factor index can mean different things in different images. Concept bottleneck and neuron-naming methods attach names, but only by retraining the classifier head or without checking that a named concept actually drives the prediction.

Abstract

Deep neural networks are widely deployed in high-stakes visual applications where interpretability is critical, yet existing explanations face a trade-off: post-hoc concept methods recover factors that are faithful to a model’s behavior but unnamed, while naming and by-design methods attach human-readable concepts only by retraining or altering the classifier. We propose Language-Anchored Decomposition (LAD), a post-hoc framework that delivers concepts which are simultaneously named, faithful, and obtained without modifying the model. For each class, a large language model proposes a concept vocabulary that CLIP-based similarity maps localize across image regions. Inverting standard non-negative matrix factorization, LAD fixes these language-grounded maps as the coefficient matrix and learns only a concept basis that reconstructs the frozen encoder’s activations, so naming becomes a structural constraint and the model’s own feature geometry determines which concepts are retained. Removing this anchor preserves accuracy but collapses attribution faithfulness. Across natural-image, scene, and medical-imaging benchmarks, LAD produces spatially precise explanations that are decision-relevant under both concept insertion and deletion, while uniquely providing stable, human-interpretable concept names.

Invert the factorization

Standard NMF learns both factors of A ≈ UW^⊤, so the coefficients are unconstrained and each factor must be labelled after the fact. LAD fixes the coefficients to a language-grounded matrix S and learns only the basis:

Ā ≈ S W^⊤,   S ∈ ℝ₊^(nhw)×r fixed,   W ≥ 0 learned

1. Vocabulary. An LLM proposes r = 25 visually grounded concepts per class (parts, textures, shapes, context). Filler terms, class-name leakage and near-duplicates are filtered automatically. No manual annotation.
2. Localize. At each cell of a grid aligned to the encoder’s feature map, a small red circle is overlaid and CLIP image–text similarity to “a photo of concept” is recorded. Stacking these maps gives S.
3. Reconstruct. Encoder activations are unfolded over space (each location is a row, so the sample count grows from n to nhw), and W is fit by projected gradient descent on ½‖Ā − SW^⊤‖²_F from an NNDSVD initialization. The classifier is never touched.
4. Explain. For a new image, non-negative least squares gives Ŝ with W fixed. Each column of Ŝ, reshaped to h×w, is a heatmap for a named concept.

Predictive preservation and faithfulness

All methods keep the classifier’s accuracy on reconstructed activations, so faithfulness is read from concept insertion (C-Ins: are the concepts sufficient?) and concept deletion (C-Del: are they necessary?). Baselines use the same backbones, preprocessing, splits and concept count r.

Named concepts, and the same name across images

NMF baselines highlight coarse regions under indices like “factor 1”. LAD returns concepts such as maple neck, fretted scale and tuning pegs, and each name localizes to the same part in every image of the class.

For a single input, LAD produces a per-image named-concept rollout: the concepts the model used, localized, ranked, and annotated with their share of the reconstruction ĉ_k. FACE returns factors of similar contribution but no names, and its leading factor lands on a different region in each image.

The encoder, not CLIP, decides which concepts survive

Before optimization, concept scores mirror the CLIP initialization and rank fairly uniformly. After fitting the basis to the encoder’s activations, the distributions spread out: concepts that the encoder’s features do not support get small basis weight and are suppressed, which limits the influence of known CLIP biases.

Retinal fundus photographs

On ODIR-5K (five ocular-disease classes), the explained model is a DenseNet-121 fine-tuned to 0.755 balanced accuracy, and concepts are disease-specific attributes localized with BiomedCLIP. Disease evidence here is spatially focal, and LAD’s named concepts anchor to it: removing them degrades the prediction more than any baseline’s factors (C-Del 0.954 vs. 0.863).

The language anchor is functional, not cosmetic

Replacing the fixed language-grounded coefficients with learned ones recovers standard unsupervised NMF on the same activations. Accuracy does not move, but deletion faithfulness drops from 0.902 to 0.507. The anchor changes which basis directions are found.

Concept budget r

Accuracy is saturated at every budget; C-Ins and C-Del rise monotonically as the vocabulary grows. The main experiments use r = 25 to match the baselines.

BibTeX

Copy@article{akash2026lad,
  title   = {Naming the Concepts Classifiers Rely On: Language-Anchored
             Decomposition for Faithful Explanation},
  author  = {Akash, Ahsan Habib and Bhusal, Dipkamal and Jones, Stacey and
             Adjeroh, Donald A. and Bhattarai, Binod and Gyawali, Prashnna Kumar},
  journal = {arXiv preprint arXiv:2607.07264},
  year    = {2026}
}