Doklady Mathematics General Mathematics

FoCAT: Foundation Model for Estimating the Conditional Average Treatment Effect

S. R. Kirpichenko, A. V. Konstantinov, L. V. Utkin
Springer Nature · 2025-08-01 · DOI 10.1134/s1064562425700280

Abstract

Abstract The paper presents a novel foundation model, FoCAT (Foundation Causal Adaptive Transformer), developed for estimating the conditional treatment effect. The model addresses several key challenges inherent in causal inference tasks, including a limited sample size in the treatment group, the impossibility of simultaneously observing patient outcomes before and after intervention, and difficulties in testing models on real data. FoCAT employs a hypernetwork architecture. Unlike existing approaches that predict separate outcome functions for control and treatment groups, FoCAT directly estimates the conditional treatment effect. The model allows for control of the context informativeness through specialized classification tokens. Numerical experiments on synthetic and real-world datasets demonstrate superiority of FoCAT in estimation of the treatment effect. The code implementing FoCAT is publicly available.

Read at publisher →