Distilling Knowledge from Multiple Foundation Models for Accurate and Efficient Spatial Gene Expression Prediction

Spatial transcriptomics (ST) enables spatially resolved profiling of gene expression within tissue, but experimental assays remain costly and labor-intensive. We propose a lightweight knowledge distillation framework that predicts spatial gene expression from hematoxylin and eosin-stained (H&E) histology through multi-teacher distillation from diverse pathology foundation models (FM). The student model, leveraging efficiently distilled pathological feature representations, is further integrated into a downstream spatial gene expression prediction pipeline. We evaluate the method on large-scale H&E-ST repositories spanning multiple tissue cohorts and demonstrate that the distilled student model consistently matches or exceeds end-to-end model training and raw feature concatenation baselines in predictive accuracy and shows robust generalizability when tested on unseen cohorts, with significantly reduced model complexity and inference cost. The student model (around 11 M parameters) generalizes across different ST techniques and is scalable and easily deployable.