Each year, over 60,000 people in the United States are diagnosed with head and neck cancers. Treatment strategies are dynamic and complex and can involve multiple courses of therapy with different potential outcomes. With head and neck cancer numbers increasing every year, partly due to viral infections, clinicians aim to develop customized treatment strategies tailored to each individual patient, under a healthcare model termed “precision medicine”. Precision medicine proposes the customization of healthcare, with medical decisions, practices, and products being tailored to the individual patient. The customization is based on individual factors collected from cohorts of patients who are similar to the patient under consideration. These factors may include disease markers, treatment options, demographics, and genetic, lifestyle, environmental, laboratory, or quality of life data. These data are heterogeneous, large-scale, and locally sparse. Making similarity-based precise recommendations for a specific patient requires novel approaches which blend biomedicine with complex quantitative methodology. We describe a first application of deep-Q-learning (DQL) and simulation as a digital twin dyad to optimize multi-step treatment of head and neck cancer patients, and to predict patient survival and toxicity outcomes. The treatment decision DQL digital twin and the patient’s digital twin were created, trained and evaluated using Google Colab for the HPC solution on a dataset of 536 oropharyngeal squamous cell carcinoma (OPC) patients with the goal of, respectively, determining the optimal treatment decisions with respect to survival and toxicity metrics, and predicting the outcomes of the optimal treatment on the patient. The models were trained on a subset of 402 patients (split randomly) and evaluated on a separate set of 134 patients. The main outcomes and measures include accuracy in predicting the final outcomes of a treatment sequence, and improvement in predicted outcomes rates of the DQL when compared to the observed physician outcomes. On the validation set, we obtained 87.09% mean and 90.85% median accuracy in treatment outcome prediction. Given the prediction accuracy and predicted improvement on medically relevant outcomes yielded by this approach, this digital twin dyad of the patient-physician dynamic treatment problem has the potential of aiding physicians in determining the optimal course of treatment and in assessing its outcomes.

Keywords: digital twin dyad, reinforcement learning, head and neck cancer

URL

Tardini, E., Zhang, X., Canahuate, G., Wentzel, A., Mohamed, A.S.R., Van Dijk, L., Fuller, C.D., Marai, G.E., A digital twin dyad with deep Q-learning in head and neck cancer treatment, Fourth ISC Workshop on HPC Applications in Precision Medicine, June 18th, 2021. https://ncihub.org/groups/hapm21/2021_workshop_program_overview