Predicting Healthcare Insurance Risk via Tabular Foundation Models and In-Context Learning

Abstract

In machine learning (ML) and data mining, especially in scenarios such as medical fraud detection, financial risk control, credit assessment, and anti-money laundering, the problem of extremely unbalanced table data is very common. There are very few samples of minority classes (such as fraud, breach of contract, and abnormal transactions) in this kind of data, and the proportion of the majority class is extremely high, which makes the model more likely to be biased towards the majority class during the training process. This seriously weakens the ability to identify key minority classes and brings actual business risks even when the overall accuracy is high. To solve this problem, research is mainly carried out from the data layer and algorithm layer: data layer methods include random undersampling, oversampling, and synthetic minority class sample technologies such as SMOTE; algorithm layer methods include cost-sensitive learning, category weight adjustment, and loss function improvements such as Focal Loss to enhance the model’s focus on difficult-to-classify samples. At the same time, ensemble learning methods such as random forests (RF) and gradient-boosted trees are widely used in this type of task due to their strong robustness and ability to adapt to imbalanced data. In extremely imbalanced or even weakly labeled scenarios, anomaly detection and semi-supervised learning methods also show good results. In terms of model evaluation, traditional accuracy is no longer reliable, and indicators such as ROCAUC, PR-AUC, F1-score, recall rate, and KS statistics are usually used to more comprehensively measure minority class recognition capabilities. Overall, improving minority class detection capabilities while ensuring generalization performance is still a core issue in this field. In the future, it is expected to combine deep learning, table-based models, self-supervised learning, and interpretability methods to further improve practical application effects.