To Örebro University

oru.seÖrebro University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Beyond Causal Accuracy: Evaluating Representation Quality in Deep Treatment Effect Estimation
Örebro University, School of Science and Technology. AILS. (AASS)ORCID iD: 0000-0002-9341-7803
Örebro University, School of Science and Technology.ORCID iD: 0000-0003-4026-7490
Örebro University, School of Science and Technology. AILS. (AASS)ORCID iD: 0000-0003-3958-6179
2025 (English)Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

Recent advances in treatment effect estimation (TEE) leverage deep neural networks to learn latent representations that balance treatment and control groups. However, despite their empirical success, we know remarkably little about the learned representations themselves—what structure they capture, why they lead to improved performance, and how far we can trust them. Existing evaluations focus almost exclusively on causal outcome metrics such as PEHE or ATE bias, leaving the representational mechanisms largely unexplored. This lack of understanding limits our ability to explain why models perform well, to identify when they fail, and to design more interpretable or trustworthy causal systems. In this work, we present a comprehensive analysis for representation-level evaluation of TEE methods. We benchmark CFR, DRCFR, TEDVAE and modern disentanglement-driven models using a unified suite of quantitative and graphical measures. Quantitatively, we assess AUC(T | Z) for confounding separability, Mutual Information (MI) retention for informativeness, MMD2 and SW1 for balance, CH-ratio for compactness, and Participation Ratio (PR) for effective dimensionality. Graphically, we analyze bias removal through embedding visualizations, kNN-mixing histograms, covariance heatmaps, and outcome-smoothness histograms—providing an interpretable view of latent geometry. Our analysis shows that modern VAE-based disentangling models yield more structured, balanced, and informative representations, which also correlate strongly with causal reliability. We provide a practical way to see inside TEE representations—revealing hidden biases, interpreting latent behavior, and understanding when conventional embedding properties (such as neighborhood preservation) may actually hinder bias mitigation. Overall, our findings emphasize that evaluating how models learn is essential to understanding why they perform well, paving the way toward more interpretable and trustworthy causal inference.

Place, publisher, year, edition, pages
2025.
National Category
Computer and Information Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:oru:diva-126246OAI: oai:DiVA.org:oru-126246DiVA, id: diva2:2028024
Conference
EurIPS, Copenhagen, Denmark, December 2-7, 2025
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Available from: 2026-01-14 Created: 2026-01-14 Last updated: 2026-01-14Bibliographically approved

Open Access in DiVA

Abstract(79 kB)39 downloads
File information
File name FULLTEXT01.pdfFile size 79 kBChecksum SHA-512
c994c42da03dfc1f8213fc42d7c9e3d0ae87322e659f16929dc77e8e8cf8fc16cf9dac54f3c75ea868d3aa4ad4b045f2b4d74f075232f053153cae16b6b52958
Type fulltextMimetype application/pdf

Authority records

Khan, Ahmad SaeedSchaffernicht, ErikStork, Johannes Andreas

Search in DiVA

By author/editor
Khan, Ahmad SaeedSchaffernicht, ErikStork, Johannes Andreas
By organisation
School of Science and Technology
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 11280 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf