NeuroVLM

NeuroVLM#

NeuroVLM maps between neuroimaging activation maps and neuroscience text.

Installation

Install the package and run a minimal quickstart.

Installation
Tutorials

Step-by-step notebooks to learn the workflow.

Tutorials
Paper Figures

Preprint-v3 figure map and reproduction guidance.

Preprint Figure Reproduction
API Reference

Function and class reference for the public package API.

API Documentation

Preprint#

R. P. Hammonds, J. Aguirre-Chavez, B. Omoma-Edosa, A. Patel, and B. Voytek, “NeuroVLM: A generative vision-language framework for human neuroimaging,” bioRxiv, preprint v3, July 1, 2026. [Online]. Available: https://www.biorxiv.org/content/10.64898/2026.02.06.704508v3

bioRxiv preprint v3

Ongoing and historical research notebooks remain available under Experimental Notebooks.

Install#

ation

Minimal, inference-only installation:

pip install neurovlm

With optional dependencies needed to train and reproduce analyses:

pip install "neurovlm[full]"

Installation take a couple minutes. After installation, calling neurovlm.data.fetch_data() will fetch datasets and models from huggingface, which will be slower.

Demo#

See here for the introductory notebook that walks through using all NeuroVLM models. In short:

Fetch NeuroVLM’s datasets and models:

from neurovlm.data import fetch_data
fetch_data()

Use the four text/brain inference paths:

from neurovlm import NeuroVLM
from neurovlm.data import load_latent

nvlm = NeuroVLM(device="cuda") # use device="cpu" if GPU not available

# Text-to-brain generation (MSE)
brain_map = nvlm.text("auditory processing").to_brain(head="mse")
brain_map.plot(0, threshold=0.1)

# Brain-to-text generation (QFormer)
auditory = load_latent("networks_neuro")["Du"]["AUD"]
description = nvlm.brain(auditory).to_text(head="qformer")
print(description)

# Text-to-brain retrieval (InfoNCE)
brain_matches = nvlm.text("auditory processing").to_brain(head="infonce")
df_text_to_brain = brain_matches.top_k(3)

# Brain-to-text retrieval (InfoNCE)
text_matches = nvlm.brain(auditory).to_text(head="infonce")
df_brain_to_text = text_matches.top_k(3)

Select either model family with the structured inference API. CNN autoencoders always default to the mixed-source baseline; domain-specific contrastive and text-to-brain heads use the mixed baseline unless variant="finetuned" is requested explicitly:

from neurovlm import load_pipeline

autoencoder = load_pipeline(family="cnn", task="autoencoder")
contrastive = load_pipeline(
    family="cnn", task="contrastive", domain="pubmed"
)
text_to_brain = load_pipeline(
    family="cnn", task="text_to_brain", domain="nilearn"
)

# The same task-level surface loads a standardized local training run.
local = load_pipeline(
    family="cnn", task="contrastive", domain="pubmed",
    from_run="runs/<run-id>",
)

Training uses typed configs and automatically writes reproducible config, provenance, best/last checkpoints, metric CSVs, plots, and logs:

from neurovlm.training import ContrastiveTrainConfig, train_contrastive

result = train_contrastive(ContrastiveTrainConfig(domain="neurovault"))
print(result.run_dir / "metrics/history.csv")

See the atlas-free CNN tutorial and technical guide for all PubMed, Nilearn, and NeuroVault switches; MLP/CNN reconstruction, retrieval, and generation; resume; and explicit local-run chaining.

Documentation#

See the docs for the API, tutorials, and paper figure reproduction.

Reproducibility#

Analyses are organized as Jupyter notebooks:

  1. docs/01_tutorials: User-facing examples

  2. docs/02_data: Data loading and preprocessing

  3. docs/03_models: Model training and development

  4. docs/figures: Preprint-v3 evaluation and figure reproduction

  5. docs/experimental: Retained exploratory evaluation, atlas-free CNN, and data-preparation research

License#

Apache-2.0 (LICENSE).