Quantifying the Predictive Uncertainty of Regression GNN Models Under Target Domain Shifts


Yurekli S., Demirtas M. A., Rekık I.

5th International Workshop on Predictive Intelligence in Medicine (PRIME MICCAI), Singapore, Singapore, 22 September 2022, vol.13564, pp.149-159 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Volume: 13564
  • Doi Number: 10.1007/978-3-031-16919-9_14
  • City: Singapore
  • Country: Singapore
  • Page Numbers: pp.149-159
  • Keywords: Predictive uncertainty estimation, Domain shift, Regression GNNs, Brain connectivity, INTELLIGENCE, ALGORITHM
  • Istanbul Technical University Affiliated: Yes

Abstract

Predicting cognitive scores (e.g., intelligence quotient (IQ)) from functional brain connectomes enables the analysis of the underlying connectivity patterns that determine such abilities. In this context, recent works addressed IQ prediction from connectomes by designing graph neural network (GNN) architectures for regression. While effective, existing studies have two important drawbacks. First, the majority of these works train and evaluate regression GNNs on data from the same distribution. Thus, the performance of the models under domain shifts, where the target training and testing behavioral scores are drawn from different distributions, has not been considered. Second, the proposed architectures do not produce uncertainty estimates for their predictions, limiting their usage in critical real-world settings where data distribution may drastically change and render the predictions unreliable. To cope with this, a few studies proposed proposed Bayesian neural networks for estimating predictive uncertainty. However, these require heavy computation of the training process and have not been applied to regression GNNs. To address this problem, we unprecedentedly propose a deep graph ensemble of regression GNNs for estimating predictive uncertainty under domain shifts. Our main contributions are three-fold: (i) forming ensembles of regression GNNs for estimating their predictive uncertainties, (ii) simulating domain shift between training and test sets by applying clustering algorithms in the target domain, (iii) designing a novel metric for quantifying the uncertainty of GNN ensembles. We believe our study will inspire future research on the performance and uncertainty of GNNs under domain shifts, allowing their use in real-world scenarios.