TL;DR. FlatClip tests a simple hypothesis: cortical geometry may account for part of the gap between ROI and voxel representations. It feeds surface flatmaps to a frozen SigLIP2 encoder and trains only a lightweight probe. Results are competitive on HCP and ADNI, but weaker on PPMI and below the strongest voxel models overall.
How much can the input representation do on its own?
Regional averages are efficient but discard spatial organization. Voxel models retain that organization at the cost of specialized architectures and expensive pretraining. FlatClip explores a surface-level representation between these two scales.
The question is deliberately simple: if cortical activity is rendered as a geometry-aware image, how much can a frozen vision encoder already do? Keeping the encoder fixed helps probe the contribution of the input representation.
Render the cortex, freeze the encoder, train a probe
FlatClip reuses an image encoder without fMRI-specific encoder pretraining. A small downstream predictor, or probe, learns from its frozen features. This creates a useful test of whether preserving the layout of cortical activity helps prediction.
Method overview. The figure positions flatmaps between regional averages and dense voxel representations. A flatmap preserves cortical layout in two dimensions, allowing a frozen image encoder to extract features from that organization. Source paper ↗
Render cortical activity as flatmaps
Activity on the cortical surface is mapped to a two-dimensional representation that retains spatial organization. A frozen SigLIP2 image encoder extracts features; only a lightweight downstream probe is trained.
Summarize resting recordings
For resting-state tasks, each subject contributes 40 flatmap frames. Their image features are averaged over time to produce a 768-dimensional subject representation.
Match the view to a visual task
For visual-fMRI decoding, stimulus-level GLM response maps become flatmaps. Experiments compare the whole cortex, visual cortex and NSD-provided task-active cortex, alongside controls that alter spatial organization.
Geometry helps, especially when it matches the task
The most informative result is the pattern across tasks and controls. FlatClip does not improve every benchmark in the same way.
A competitive middle ground
FlatClip outperforms ROI baselines on HCP and ADNI tasks, but is weaker on PPMI and remains below the strongest voxel-level models overall.
Task-aligned cortex helps
For visual decoding, restricting the input to visual or NSD task-active cortex improves performance over using the whole cortex.
Spatial controls explain part of the effect
Disrupting flatmap geometry reduces performance. Adaptive patches in highly activated regions improve several voxel prediction metrics.
A baseline that helps ask a more precise question
FlatClip separates the value of the spatial representation from the value of fMRI-specific encoder training. The result is a practical surface-level baseline and a design lesson: which cortical regions are shown, and how they are arranged, can matter as much as the choice of downstream probe.
Explore the project
Start with the public repository for surface rendering, frozen feature extraction and downstream probes. FlatClip reuses Google’s SigLIP2 image-encoder weights; it does not require fMRI-specific encoder pretraining.
FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning
Research post by fMRI Atlas, based on the original study linked above.
Read an abstract overview
FlatClip investigates whether cortical surface images let a pretrained vision model capture useful spatial information in fMRI. It renders cortical activity as flatmaps, keeps SigLIP2 frozen and learns a small task-specific predictor. The results place it between regional and voxel representations: HCP and ADNI favor this baseline over ROI methods, whereas PPMI remains challenging. Visual decoding improves when the input focuses on task-relevant cortex. Perturbation experiments further test the contribution of spatial arrangement, with anatomical layouts performing better than vertex permutations across three color mappings. See the linked arXiv paper for the full abstract and experiments.
BibTeX
@misc{wang2026flatclip,
title={FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning},
author={Wang, Mo and Ye, Wenhao and Ning, Zihan and Zuo, Jiayu and Xia, Junfeng and Wen, Hongkai and Liu, Quanying},
year={2026},
eprint={2609.31204},
archivePrefix={arXiv},
primaryClass={cs.CE},
url={https://arxiv.org/abs/2609.31204}
}