API Documentation

API Documentation#

API reference for the neurovlm module.

Table of Contents#

Inference#

The legacy high-level interface remains available for text, NIfTI, and retrieval workflows.

NeuroVLM([datasets, device])

Unified interface for text-to-brain and brain-to-text.

BrainSearchResult(scores, metadata, latents, ...)

Container for brain retrieval or generation outputs.

TextSearchResult(scores_by_dataset, ...)

Container for text retrieval scores and metadata.

BrainTopKResult(table, parent)

Chainable wrapper around a top-k brain result table.

Structured runtime#

Task-oriented tensor inference selects the family, task, domain, variant, and released checkpoint or local run explicitly. CNN domain tasks default to the mixed baseline; fine-tuning is explicit.

load_pipeline(*[, family, task, domain, ...])

Resolve and load an inference pipeline.

NeuroVLMRuntime(*, spec, model, device, source)

A resolved model pipeline with consistent tensor-level methods.

RuntimeMetadata(canonical_name, family, ...)

Resolved inference selection, suitable for logs and manifests.

Data#

Fetches from huggingface and loads.

Fetching#

fetch_data([datasets, models, cache_dir])

Fetch NeuroVLM data from Hugging Face repositories.

load_dataset(name)

Alias to _load_* functions in retrieval resources.

Embeddings#

Pre-computed latent vectors for text and neuroimages.

load_latent(name)

Alias to _load_latent* functions in retrieval resources.

Masker#

Nifti masker need to resample and mask neuroimages.

load_masker()

Masker alias.

Atlas-free CNN datasets#

Published split JSONLs and their shared volume tensor. Legacy per-row local paths are ignored.

AtlasFreeCNNDataset(split, *[, source, ...])

A split view over the shared atlas-free CNN volume tensor.

AtlasFreeCNNDataProvider(*[, source, ...])

Lazily provide validated train, validation, and test dataset views.

atlas_free_cnn_splits(**kwargs)

Return all three atlas-free CNN split views with shared resources.

Models#

Base models for autoencoder, projection heads, and specter. Pretrained models return from load_model or calling .from_pretrained on model classes.

NeuroAutoEncoder([seed, out, dim_neuro, ...])

Autoencoder for neuro-vectors.

ProjHead([latent_in_dim, hidden_dim, ...])

Align latent tensors.

Specter([model, adapter, orthgonalize, ...])

Wrapper for Specter model.

load_model([name, family, task, domain, variant])

Load a packaged model by legacy name or structured fields.

The structured selectors are defined in the model registry:

ModelFamily(value[, names, module, ...])

Supported model architecture families.

ModelTask(value[, names, module, qualname, ...])

Public task identifiers shared by training and inference code.

ModelDomain(value[, names, module, ...])

Datasets with released domain-specific CNN checkpoints.

ModelVariant(value[, names, module, ...])

Canonical variants of released model artifacts.

ModelSpec(family, task, variant, loader[, ...])

An immutable, fully resolved model artifact specification.

resolve_model_spec([name, family, task, ...])

Resolve a legacy alias or structured selection to a canonical spec.

Atlas-Free CNN#

Installable 3D CNN architectures and conversion helpers for the MLP and CNN input spaces. Pretrained instances are returned by load_model.

CNNContrastiveModel(brain_encoder, ...)

Brain and text encoders aligned in a shared normalized space.

CNNTextToBrainModel(text_projection, autoencoder)

Generate a dense brain volume from a precomputed SPECTER2 embedding.

atlas_free_volume_to_mlp_flat(volume, *[, ...])

Convert cropped CNN volumes to the packaged MLP masker's flat space.

mlp_flat_to_atlas_free_volume(flat)

Scatter packaged MLP flat vectors into cropped CNN volume space.

Loss Functions#

The pretrained models used InfoNCELoss or MSELoss. Additional options include FocalLoss or TruncatedLoss.

InfoNCELoss([temperature, ...])

Compute symmetric InfoNCE loss between paired image/brain and text embeddings.

FocalLoss([alpha, gamma])

TruncatedLoss([percentile, base_loss])

Training#

Standardized task runners#

Typed runners share artifact, metric, checkpoint, provenance, and resume conventions.

AutoencoderTrainConfig([output_root, ...])

Typed configuration for mixed pretraining or domain fine-tuning.

ContrastiveTrainConfig(domain[, ...])

Configuration for one domain-specific Stage 3 branch.

TextToBrainTrainConfig(domain[, ...])

Configuration for one domain-specific Stage 4 projector branch.

MLPAutoencoderTrainConfig(output_root, ...)

MLPContrastiveTrainConfig(output_root, ...)

MLPTextToBrainTrainConfig(output_root, ...)

MLPBrainToTextRetrievalTrainConfig([...])

Train the retained contrastive heads for brain-to-text retrieval.

BrainToTextGenerationTrainConfig(...)

train_autoencoder(config, *[, provider, model])

Train and evaluate a mixed or explicitly fine-tuned CNN autoencoder.

train_contrastive(config, *[, provider, ...])

Train, select, and fully evaluate one CNN contrastive branch.

train_text_to_brain(config, *[, provider, ...])

Train a fresh Stage 4 projector with a frozen, provenance-bound AE.

train_mlp_autoencoder(config, *, provider[, ...])

train_mlp_contrastive(config, *, provider[, ...])

train_mlp_text_to_brain(config, *, provider)

train_mlp_brain_to_text_retrieval(config, *, ...)

Train symmetric InfoNCE while selecting the brain-to-text direction.

train_brain_to_text_generation(config, *, ...)

Train only Q-Former parameters against a frozen causal LM.

Model comparison#

Shared MLP/CNN reconstruction, retrieval, and generation comparisons. The default matrix uses mixed-baseline CNN checkpoints; fine-tuned rows are an explicit opt-in.

ComparisonSelection(family, task[, domain, ...])

ComparisonResult(summary, by_source, ...)

default_comparison_matrix(task, *[, ...])

Return MLP plus mixed-baseline CNN selections for a task.

evaluate_reconstruction_comparison(*[, ...])

Compare AE reconstruction in each family's declared spatial space.

evaluate_contrastive_comparison(*[, ...])

Evaluate paired full-split retrieval with family-native text preprocessing.

evaluate_text_to_brain_comparison(*[, ...])

Compare generated maps with paired rows and family-native text inputs.

Metrics#

Performance metrics.

recall_at_k(cos_sim, k)

recall_curve(latent_text, latent_image[, step])

dice(img_a, img_b)

Compute dice score.

dice_top_k(y_true, y_prob[, k])

Compute dice score of top k.

bernoulli_bce(y, p[, eps])

Elementwise Bernoulli negative log-likelihood (cross-entropy), in nats.

bits_per_pixel(y_true, logits, *[, ...])

y_true: (N, D) floats in [0,1] logits: (N, D) raw logits from decoder (before sigmoid)