TL;DR. BrainWorld generates whole-brain 4D fMRI conditioned on anatomy and functional context. Its 22-dataset evaluation reports stable trajectories up to 400 frames, downstream gains from generated training examples and transferable features from the generative backbone.
What would it mean to model a changing brain?
Representation learning asks how to summarize a scan for a prediction. BrainWorld instead asks how to generate a sequence of whole-brain activity under a set of conditions. This requires modeling change over time, not just producing a subject-level feature vector.
The study treats structural MRI as subject-level anatomical context. Structural information guides the denoising process that generates functional activity, rather than serving only as a separate input branch for classification.
Generate dynamics in a compressed 4D space
A whole-brain fMRI recording has three spatial dimensions and a time dimension. Directly generating every voxel at every frame is expensive. BrainWorld separates the problem into a compressor that represents the recording and a diffusion model that learns its conditional dynamics.
Animation from the author’s GitHub ↗ · Schematic BOLD activity.
Method overview. The VAE compresses fMRI into latents; conditional diffusion models their evolution; the decoder returns a voxel-level sequence. The conditioning paths supply anatomy, past functional connectivity and optional sensory context. Source paper ↗
Compress the 4D signal
A pretrained 4D variational autoencoder maps standardized whole-brain fMRI into a latent representation. Generation takes place in this compressed space.
Condition the dynamics
A conditional Diffusion Transformer uses anatomical context and past functional connectivity, with optional visual or audio context, to model future activity. The structural prior enters the denoising process.
Decode or reuse the representation
The VAE decoder converts generated latents back into voxel-level fMRI sequences. Intermediate DiT features can also support downstream tasks, connecting generation with representation learning.
Evaluate the sequence, then test what it is useful for
The experiments examine three uses of the same model: long-horizon generation, training-data augmentation and downstream feature transfer.
Up to 400 frames
Across a 22-dataset evaluation, the paper reports stable generated 4D trajectories reaching 400 frames.
Generated examples aid prediction
The study reports downstream improvements from adding generated examples to training data. This evaluates generation through its usefulness for subsequent prediction.
Transferable features
The learned multimodal representations outperform the compared baselines in the reported transfer experiments, giving the generative model a second use beyond producing sequences.
From a generated sequence to a reusable representation
The study links generation and prediction through one conditional model. Anatomical context guides the dynamics, while the learned features can be reused for downstream analysis. The reported 400-frame stability and augmentation gains are distinct tests of usefulness; they do not by themselves establish an individual’s actual future trajectory.
Explore the project
Explore the code for the preprocessing, latent generation and feature extraction workflow. A working public checkpoint link is not currently listed in this collection, so check the repository’s release instructions before planning inference.
A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics
Research post by fMRI Atlas, based on the original study linked above.
Read the paper’s abstract
Whole-brain 4D fMRI generation is valuable for modeling functional brain dynamics, yet existing fMRI foundation models mainly target representation learning and downstream prediction rather than conditional predictive generation. We introduce BrainWorld, a structural-prior-conditioned generative model for whole-brain 4D fMRI dynamics. BrainWorld uses sMRI as subject-level anatomical context to guide future fMRI generation, integrating structural information into the denoising process rather than treating it as a parallel modality. Evaluated on 22 datasets spanning diverse cohorts and brain states, BrainWorld generates stable 4D fMRI trajectories up to 400 frames, improves downstream performance through generated-example augmentation, and learns transferable multimodal representations that outperform baselines. Together, these results establish BrainWorld as a condition-aware generative framework for long-horizon brain dynamics modeling and multimodal representation learning. Code is available at the code repository.
BibTeX
@misc{brainworld2026,
title={BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics},
author={Xia, Junfeng and Ye, Wenhao and Zhang, Junxiang and Pan, Xuanye and Wang, Mo and Liu, Quanying},
year={2026},
eprint={2606.17742},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.17742}
}