GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators

Mattia Rigotti, Nicholas Thumiger, Thomas Frick

IBM Research

NeurIPS 2026

Abstract

Neural operators on irregular meshes face a fundamental tension. Spectral positional encodings, a natural choice in graph learning for capturing geometry, require cubic-complexity eigendecomposition and inadvertently break gauge invariance through numerical solver artifacts; existing efficient approximations typically sacrifice gauge symmetry by design. Both failure modes break discretization invariance: models fail to transfer across mesh resolutions of the same domain, and similarly across different graphs of related structure in inductive settings. We propose GIST (Gauge-Invariant Spectral Transformer), a scalable neural operator that resolves this tension by restricting attention to pairwise inner products of efficient approximate spectral embeddings. We prove these inner products estimate an exactly gauge-invariant graph kernel at end-to-end O(N) complexity, and establish a formal connection between gauge invariance and discretization-invariant learning with bounded mismatch error. To our knowledge, GIST is the first scalable graph neural operator with a provable discretization-mismatch bound. Empirically, GIST sets state-of-the-art on the AirfRANS, ShapeNet-Car, DrivAerNet, and DrivAerNet++ mesh benchmarks (up to 750K nodes), and additionally matches strong baselines on standard graph benchmarks (e.g., 99.53% micro-F1 on PPI).

Results

CFD (ground truth) GIST prediction Error (GIST − CFD)
Loading the 3D car…
Pressure
Error
Surface pressure on a DrivAerNet test car. Drag to rotate, scroll or pinch to zoom; the three views move together.

Surface-pressure prediction on unseen geometries, lower is better. GIST: mean ± standard deviation over 3 seeds on AirfRANS and ShapeNet-Car.

ModelAirfRANS (MSE)ShapeNet-Car (rel. ℓ2)
Transolver0.01427.45%
GIST0.0017 ± 0.00027.02% ± 0.05
ModelDrivAerNet (MSE)DrivAerNet (rel. ℓ2)DrivAerNet++ (MSE)DrivAerNet++ (rel. ℓ2)
RegDGCNN0.090128.49%0.082927.72%
Transolver0.053722.52%0.071523.87%
FigConvNet0.043820.98%0.049920.86%
TripNet0.042320.35%0.045520.05%
GIST0.040020.03%0.036318.60%

Node classification (accuracy, micro-F1 on PPI; higher is better), against the best of the baselines in the paper for each benchmark. GIST: mean ± standard deviation over 6 seeds.

ModelCoraCiteSeerPubMedPhotoogbn-arxivPPI
Best baseline85.5073.4081.1295.4872.6399.53
(method)GCNIIGCNIIGCNSpecFormerSGFormerGCNII
GIST84.25 ± 0.6573.85 ± 0.6382.60 ± 0.3795.65 ± 0.4672.31 ± 0.2999.53 ± 0.01

Bold marks the best mean. Baseline numbers, their sources and all other baselines are in the paper.

Architecture

Diagram of Gauge-Invariant Spectral Self-Attention and of the GIST block with its feature, local and global branches
Left: Gauge-Invariant Spectral Self-Attention uses the spectral embeddings as queries and keys and the node features as values, so the embeddings set the attention weights and never enter the values. In the code, the softmax drawn here is linear attention with a ReLU kernel: each head subtracts a learned threshold from the embeddings, applies a ReLU and normalizes the weights by their sum. Right: every GIST block runs three branches in parallel (feature, local and global); only blocks followed by another use Gauge-Equivariant Spectral Self-Attention to refine the embeddings for the next block.

Scalability

GPU memory and forward plus backward time of GIST as a function of the number of nodes, for several hidden dimensions
GPU memory (left) and forward plus backward time (right) of a 3-layer GIST as a function of the number of nodes, for hidden dimensions 64, 128 and 256, on DrivAerNet meshes subsampled by random node dropout.

In industry

With Dallara, GIST predicts the aerodynamics of an LMP2-class race car in seconds, where simulation takes hours, on a CFD dataset validated by Dallara's aerodynamicists: see the Faster by Design paper (AI & PDE workshop, ICLR 2026) and the IBM Research blog.

Code

The repository holds the model and one config per benchmark, with the commands, data and compute needed to reproduce each result.

Needs uv; pip works too, see the README.

curl -LsSf https://astral.sh/uv/install.sh | sh  # installs uv, if missing
git clone https://github.com/IBM/gist-neurips-2026.git
cd gist-neurips-2026
uv run python scripts/generate_embeddings.py cora --embed_dim 512 --deg 128
uv run python -m gist.train configs/cora.yaml

Citation

@inproceedings{rigotti2026gist,
  title     = {{GIST}: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators},
  author    = {Rigotti, Mattia and Thumiger, Nicholas and Frick, Thomas},
  booktitle = {Advances in Neural Information Processing Systems},
  volume    = {39},
  year      = {2026}
}