Artificial intelligence systems often produce highly accurate predictions, yet the reasoning behind those decisions can remain opaque. This lack of transparency creates challenges in high-stakes domains such as medical diagnosis or autonomous driving, where users must understand why a model reached a particular conclusion. Researchers at MIT have introduced a new method that improves the ability of computer-vision models to explain their predictions in terms humans can understand.
The approach builds on a technique known as concept bottleneck modeling (CBM). In traditional CBMs, an AI model predicts a set of human-interpretable concepts before producing a final decision. For instance, a system analyzing medical images might identify features such as irregular pigmentation or clusters of brown dots before predicting melanoma. These intermediate concepts allow users to trace the reasoning behind a prediction.
However, existing methods often rely on concepts predefined by human experts or generated by language models. These concepts may not align with the patterns actually used by the neural network, which can reduce both interpretability and predictive accuracy. In some cases, models also rely on hidden features not included in the concept set, a problem known as information leakage.
The MIT team addressed this limitation by extracting concepts directly from the trained model itself. Their system first uses a sparse autoencoder to identify key internal features the neural network has learned during training. A multimodal large language model then converts these features into concise, human-readable descriptions. The resulting concepts are used to train a concept bottleneck module that forces the model to base its predictions only on those extracted concepts.
To improve clarity, the researchers limit each prediction to a small number of concepts, ensuring explanations remain concise and understandable. Experiments involving tasks such as identifying bird species and diagnosing skin lesions showed that the method produced more accurate predictions and clearer explanations than existing CBM approaches.
Although traditional black-box models can still achieve slightly higher accuracy, the researchers believe their framework represents a major step toward trustworthy AI. By revealing the concepts underlying predictions, the method could help users evaluate whether AI systems are reliable enough for real-world decision-making.