This is the work of my group project in the Deep Graph-based Learning module in MSc AI at Imperial. For complete comprehension, please refer to our report and repository.

Introduction

The popularity of image super-resolution (SR) research has grown significantly, especially with the adoption of Deep Learning (DL) techniques. SR methods increase the resolution of data such as brain connectivity matrices (BCMs)—compact representations of the strengths and patterns of the structural connections between brain regions. The large-scale acquisition of high-resolution BCMs from raw neuroimaging data is costly in terms of time, money and computational resources. Brain SR could significantly speed up acquisition, via the generation of of BCMs from existing ones, thus reducing the cost of research and increasing the availability of brain data for clinical applications—for instance Alzheimer’s detection.

BCMs are naturally represented as graphs, where the nodes are represented by the brain regions and the edges by the brain connections. This representation makes the use of Graph Neural Netowrks (GNNs) more intuitive and efficient than traditional DL approaches. Recently, many graph-based frameworks have been proposed for brain graph SR. Inspired by GraphUNet (Gao & Ji, 2019), we propose a framework to learn the generation of high-resolution brain graphs inductively: GraphNETbyNET (GNByN).

Results

The final model scores a Mean Absolute Error (MAE) of 0.13 on the testing data, underscoring the precision of our model’s high-resolution brain image generations.

Model Architecture

GNByN uses a deep Graph Neural Network (GNN) featuring two main components, the UpChanger and the DownChanger, each containing seven blocks of Graph Convolution Netowrk (GCN) layers. This model incrementally adjusts node counts in adjacency and feature matrices to learn mappings between Low Resolution (LR) and High Resolution (HR) brain connectivity encodings. To address over-smoothing (Rusch et al., 2023), it incorporates the GraphCON (Rusch et al., 2022) framework, which introduces oscillatory behaviour to maintain differentiation among node features in deep networks. The UpChanger progressively increases nodes from 160 to 268, while the DownChanger reduces them in reverse order. Inspired by the GraphUNet architecture, this structure ensures effective dimensionality changes and enriched information propagation. Moreover, the model uses GATv2Conv (Brody et al., 2022) layers for attention mechanisms and LSTM-like gates to manage long-term dependencies and node embeddings.

The LR and HR data are passed into the UpChanger and the DownChanger, respectively, to obtain the two sequences of adjacency matrices over all the steps. Between the two sequences, the adjacency matrices with the same number of nodes are compared to compute the average MSE loss. This loss is backpropagated to update the DownChanger’s parameters while keeping the UpChanger frozen. In the next stage, a weighted average MSE loss is computed, with the final HR projection from the UpChanger given increasing importance as training progresses, defined by the weight \(\alpha_{t}=\frac{(2-\exp(\frac{-t}{5})}{2}\). This loss updates the UpChanger’s parameters while keeping the DownChanger frozen, and are aimed to focus more on the correctness of the final HR projection. Finally, a reconstruction loss is calculated between the HR projection from the UpChanger and the ground truth, which updates the parameters of both the UpChanger and DownChanger.

High-level architecture and step-level comutational graph. Batchnorm, laynorm and dropout layers are omitted. Learnable parameters (and dimensions) excluding VAEs and standard layers:\(D_{A,t} \ (d_{t+1} \times d_t), D_{A,t} \ (d_{t+1} \times d_t), D_{Z,t} \ (\text{channels} \times d_{t+1}),\) \(\beta_{A} \ (d_{t+1} \times 1), \beta_{Z} \ (d_{t+1} \times 1), i_t \ (d_{t+1} \times 1),\) \(f_{t} \ (d_{t+1} \times 1)\)

Authors

Konstantinos Mitsides
Moritz Hauschulz
Timothy Wong
Sadegh Emami
Dilay Ercelik

We propose Graph-Coupled Oscillator Networks (GraphCON), a novel framework for deep learning on graphs. It is based on discretizations of a second-order system of ordinary differential equations (ODEs), which model a network of nonlinear controlled and damped oscillators, coupled via the adjacency structure of the underlying graph. The flexibility of our framework permits any basic GNN layer (e.g. convolutional or attentional) as the coupling function, from which a multi-layer deep neural network is built up via the dynamics of the proposed ODEs. We relate the oversmoothing problem, commonly encountered in GNNs, to the stability of steady states of the underlying ODE and show that zero-Dirichlet energy steady states are not stable for our proposed ODEs. This demonstrates that the proposed framework mitigates the oversmoothing problem. Moreover, we prove that GraphCON mitigates the exploding and vanishing gradients problem to facilitate training of deep multi-layer GNNs. Finally, we show that our approach offers competitive performance with respect to the state-of-the-art on a variety of graph-based learning tasks.

We consider the problem of representation learning for graph data. Convolutional neural networks can naturally operate on images, but have significant challenges in dealing with graph data. Given images are special cases of graphs with nodes lie on 2D lattices, graph embedding tasks have a natural correspondence with image pixel-wise prediction tasks such as segmentation. While encoder-decoder architectures like U-Nets have been successfully applied on many image pixel-wise prediction tasks, similar methods are lacking for graph data. This is due to the fact that pooling and up-sampling operations are not natural on graph data. To address these challenges, we propose novel graph pooling (gPool) and unpooling (gUnpool) operations in this work. The gPool layer adaptively selects some nodes to form a smaller graph based on their scalar projection values on a trainable projection vector. We further propose the gUnpool layer as the inverse operation of the gPool layer. The gUnpool layer restores the graph into its original structure using the position information of nodes selected in the corresponding gPool layer. Based on our proposed gPool and gUnpool layers, we develop an encoder-decoder model on graph, known as the graph U-Nets. Our experimental results on node classification and graph classification tasks demonstrate that our methods achieve consistently better performance than previous models.

Introduction

Results

Model Architecture

Authors

References

2023

2022

2019