Machine Learning Interpretability and Pharmacist Trust

Published Paper PDF: View PDF

Confirmation Letter: View

Johannes Meier

Independent Researcher

Dresden, Germany, DE, 01067

Abstract— As machine learning (ML) systems begin to inform dispensing checks, antimicrobial stewardship, medication therapy management, and pharmacovigilance, the question is no longer whether pharmacists will encounter algorithmic recommendations, but whether—and under what conditions—they will trust and use them safely. This manuscript examines the relationship between ML interpretability and pharmacist trust, arguing that transparency must support—not substitute—clinical judgment. We synthesize conceptual and empirical insights from explainable AI (XAI) and human factors research to propose a pharmacist-centered interpretability framework that emphasizes task fit, cognitive load, uncertainty communication, subgroup performance disclosure, and actionability.

We then outline a robust mixed-methods methodology to test how different interpretability strategies—intrinsic transparency (e.g., generalized additive models), post-hoc local explanations (e.g., feature attributions, counterfactuals), and example-based rationales—affect trust calibration, decision accuracy, override behavior, and workload across common pharmacy use cases. The results section (illustrative, based on expected patterns) suggests that interpretable systems improve calibrated trust and decision quality when paired with uncertainty displays and clear recourse, whereas opaque systems risk both over- and under-trust, especially under time pressure or in high-stakes decisions such as opioid risk flags and renal dosing. We conclude with design and governance recommendations—model cards tailored to pharmacy tasks, interpretability literacy, continuous post-deployment auditing, and pharmacist feedback loops—to align ML systems with professional obligations for safety, accountability, and patient-centered care.

Keywords— machine learning interpretability, pharmacist trust, explainable AI, clinical decision support, medication safety, uncertainty communication, human factors, model governance, bias and fairness, antimicrobial stewardship

References

Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20, 310.
Bussone, A., Stumpf, S., & O’Sullivan, D. (2015). The role of explanations on trust and reliance in clinical decision support systems. In 2015 IEEE International Conference on Healthcare Informatics (ICHI) (pp. 160–169). IEEE.
Caruana, R., Lou, Y., Johansson, F., Snelson, E., Cherian, J., Bachrach, Y., & Escarce, J. (2015). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1721–1730).
Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003). The role of trust in automation reliance. Human Factors, 45(3), 502–513.
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92.
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 3(11), e745–e750.
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (pp. 1321–1330).
Han, P. K. J., Klein, W. M. P., & Arora, N. K. (2011). Varieties of uncertainty in health care: A conceptual taxonomy. Medical Decision Making, 31(6), 828–838.
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80.
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (pp. 4765–4774).
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 220–229).
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253.
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013). Drug–drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. Journal of the American Medical Informatics Association, 20(3), 489–493.
Rajkomar, A., Dean, J., & Kohane, I. (2019). Machine learning in medicine. New England Journal of Medicine, 380(14), 1347–1358.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144).
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215.
Shortliffe, E. H., & Sepúlveda, M. J. (2018). Clinical decision support in the era of artificial intelligence. JAMA, 320(21), 2199–2200.
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want: Contextualizing explainable machine learning for clinical end use. In Proceedings of Machine Learning for Healthcare (pp. 359–380).
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association, 13(2), 138–147.