Concept-based learning (CBL) is a machine learning technique that involves using high-level concepts derived from raw features to make predictions. It enhances both model interpretability and efficiency. Among the various types of CBLs, the concept-based bottleneck model (CBM) has gained prominence. It compresses input features into a lower-dimensional space, capturing the essential data and discarding non-essential information. This enhances explainability in tasks such as image and speech recognition. However, CBMs often require deep neural networks and extensive labelled data.
A different approach, called Multiple Instance Learning (MIL), labels groups of data with unknown individual labels, thus identifying clusters of image patches and making inferences about individual patch labels based on overall image labels. A groundbreaking method of CBL, known as Frequentist Inference CBL (FI-CBL), has been developed by researchers at the Great St. Petersburg Polytechnic University. This method involves segmenting concept-labelled images into patches and encoding them into embeddings using an autoencoder. The embeddings are then clustered to identify groups associated with specific concepts.
FI-CBL analyses the frequency of patches associated with each concept value to determine concept probabilities for new images. What sets this method apart is its ability to integrate expert knowledge through logical rules. This enhances the transparency, interpretability, and efficacy of the model, especially in situations with limited training data.
CBL models are important in a variety of applications and are particularly crucial in medicine. These models normally incorporate a two-module structure which separates the learning of concepts and their impact on target variables. The integration of expert knowledge has enhanced the performance of these models significantly.
FI-CBL also integrates expert rules, which adjust the concepts’ prior and conditional probabilities. By integrating logical expressions provided by experts, the model is able to refine its predictions. This process can lead to more nuanced understanding of medical imaging data and improve both diagnostic accuracy and interpretability. The integration of expert rules into FI-CBL allows it to blend domain expertise with statistical modelling effectively, which results in a model that is both reliable and insightful.
FI-CBL has several advantages over neural network-based CBMs, including better transparency and interpretability. It can provide detailed explanations of predictions and works well even with small training datasets. However, FI-CBL’s effectiveness depends on accurate clusterization and optimal patch size selection. Despite these challenges, FI-CBL shows promise for enhancing interpretability and performance in machine learning tasks. The model’s flexibility and ability to integrate expert rules effectively make it a promising approach.
