To find the right design for our neural network model, we first attempt to solve each task separately. Subsequently, we run the multi-task experiments first using only one auxiliary task and later both. Each experiment is run once with and once without the gradient normalization. Next, we compute the first principle components of the deep features. This is because we need to represent each brain structure with a single variable. In order to evaluate the deep features we focus on phenotypes which are associated with specific brain structures. For example, we know there is a correlation between the age of a subject and the volume of their cerebrospinal fluid. This is because the brain goes under neurodegeneration as the subject ages. So in this scenario, using a linear regression model, we can test the deep features which were extracted from the cerebrospinal fluid annotations against the volume of the cerebrospinal fluid.
One of the most well-known causes of neurodegeneration is Alzheimer's disease. Using the same deep features and volumetric measurements as before, we can try to predict whether a subject is diagnosed with Alzheimer's disease or not. This will be via a logistic regression model.
To evaluate our approach in a multi-modal analysis, we perform a genome-wide association study (GWAS). In GWAS, we investigate the correlation between the genetic variations within a population and a phenotype (volume of a brain structure). When a statistically significant number of individuals who carry the same genetic variant have similar phenotypic measurements, that genetic site and phenotype are shown to be associated.
 Z. Chen, V. Badrinarayanan, C. Lee, and A. Rabinovich. GradNorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In ICML, (2018).
 M. Kirchler, S. Khorasani, M. Kloft, and C. Lippert. “Two-sample Testing Using Deep Learning”. In: arXiv preprint abs/1910.06239 (2019).
 S. Ruder. “An Overview of Multi-Task Learning in Deep Neural Networks”. In: arXiv preprint abs/1706.05098 (2017).
Kirchler, M.,Khorasani, S.,Kloft, M., Lippert, C. (2019). "Two-sample Testing Using Deep Learning". In: ariXiv: 1910.06239
Konigorski, S., Khorasani S., & Lippert, C. (2018). Intergrating omics and MRI data with kernel-based tests and CNNs to identify rare genetic markers for Alzheimer’s disease. 32nd Conference on Neural Information Processing Systems (NeurIPS), arXiv:1812.00448.
Masoudi, R., Mazaheri-Asadi, L., & Khorasani, S. (2016). Partial and complete microdeletions of Y chromosome in infertile males from South of Iran. Molecular Biology Research Communications, 5(4), 247–255, PMCID: PMC5326488.