Cancer treatment courses which rely on imaging and spatially-dependent therapy involve making multiple treatment decisions (e.g., radiotherapy alone, radiotherapy plus chemotherapy, induction chemotherapy) over time. These decisions depend on complex factors, including the tumor location with respect to sensitive organs and its response to treatment, laboratory data, toxicity, anticipated side effects and survival probability. In this project, we design and develop novel statistical methodology for dynamic and personalized treatment decisions with specific application to head and neck cancer radiotherapy planning. We extend the state of the art by developing methods which take into account both imaging data features (such as radiation dosage and spatial distribution) and nonspatial data (such as demographics and toxicity) in the development of predictive models for anticipated outcomes and of the optimal treatment rules. We introduce novel similarity metrics over these hybrid, high-dimensional and sometimes incomplete patient data, and we extract informative, relevant and non-redundant features to identify similar patients. We further combine statistical models for outcome prediction with local models built over cohorts of patients. These ensemble models are used to further improve the reinforcement learning process to account for patient-specific preferences regarding side effects and survival outcomes. Specifically, we extend q-learning, a type of reinforcement learning, to account for competing, multiple outcomes and different preferences on the probability of short versus long-term outcomes. We validate our precision-driven model against known toxicity outcomes using a retrospective head-and-neck cancer dataset.

Our specific aims are to:
- Create quantitative methodology for extracting and selecting spatial relationships from medical images as well as non-imaging features, and generate a hybrid similarity metric which can identify similar cohorts of patients.
- Create an ensemble model that combines global static models with dynamic local models over similar cohorts of patients with locally-sparse data.
- Create dynamic reinforcement learning methodology which uses spatial and non-imaging features and metrics and accounts for patient preferences of multiple treatment outcomes.

The project defines novel reinforcement learning techniques which account for multidimensional outcomes and patient preferences. Previous examples of reinforcement learning in the statistical literature have utilized examples with a small number of well-defined features, which is not applicable in the case of head and neck cancer. This project creates novel hybrid techniques for extracting informative and relevant features and similarity from imaging and nonimaging data. The project further introduces novel predictive modeling using a combination of global and dynamic local models over multiple outcomes. The proposed techniques leverage Big Data that is not only heterogeneous but also high-dimensional and locally sparse, in order to derive a novel precision-medicine approach.

The empirically-derived treatment rules developed in this project have the potential to improve the standard of care (i.e., treatment plans chosen by the tumor treatment board) and the quality of life of surviving patients. The methods developed in the proposal may be used to derive optimal treatment strategies across not only a variety of spatially-dependent cancer diagnoses, but also other chronic conditions including mental health disorders, substance abuse diseases, or diabetes, that require making multiple decisions that must weigh the tradeoffs between efficacy and toxicity.