Overcoming model mis-specification

Summary

Modeling biological systems in drug development and clinical trials requires statistical simplifications due to their inherent complexity. Even high-dimensional models are mis-specified to some extent, as they cannot fully represent critical processes such as patient responses to treatment. This mis-specification can result in predictive models that fail to align with the downstream decision-making task, particularly in contexts where different types of errors have unequal implications.

To address this challenge, recent advances in statistical learning have highlighted the importance of incorporating task-specific loss functions that reflect the intended use of a model. Such approaches allow for optimization based on the expected utility or costs (disutility) associated with the model output, rather than traditional likelihood-based or generic machine learning loss functions. This research theme focuses on extending and applying these methodologies to problems relevant to drug development and clinical trials design. Key areas of investigation include:

1. Estimation of treatment response: Developing methods to infer treatment response signatures from multi-omic and multi-modal data while accounting for imbalanced classification objectives, such as prioritizing the accurate prediction of non-responders to guide treatment strategies.
2. Optimizing recruitment models: Incorporating downstream consequences, such as trial costs and statistical power, into recruitment models to better align trial design with operational and regulatory constraints.
3. Quantification of model errors: Establishing formal methods to identify and quantify systematic errors in pre-specified predictive models when repurposed for new contexts, such as trial design.

References

• Bissiri, P. G., Holmes, C. C., & Walker, S. G. (2016). A general framework for updating belief distributions. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 78(5), 1103–1130
• Karapanagiotis, S., Benedetto, U., Mukherjee, S., Kirk, P.D.W., Newcombe, P.J. Tailored Bayes: a risk modeling framework under unequal misclassification costs, Biostatistics, Volume 24, Issue 1, January 2023, Pages 85–107
• Alban, A., Chick, S.E. and Forster, M., 2023. Value-based clinical trials: selecting recruitment rates and trial lengths in different regulatory contexts. Management Science, 69(6), pp.3516-3535.
• Feizi, A., Orfanoudaki, A., Saghafian, S. and Hudgson, N., 2023. Vertical Patient Streaming in Emergency Departments.