API Documentation

API Documentation#

API reference for the neurovlm module.

Table of Contents#

Inference#

The high-level interface for inference.

NeuroVLM([datasets, device])

Unified interface for text-to-brain and brain-to-text.

BrainSearchResult(scores, metadata, latents, ...)

Container for brain retrieval or generation outputs.

TextSearchResult(scores_by_dataset, ...)

Container for text retrieval scores and metadata.

BrainTopKResult(table, parent)

Chainable wrapper around a top-k brain result table.

Data#

Fetches from huggingface and loads.

Fetching#

fetch_data([datasets, models, cache_dir])

Fetch NeuroVLM data from Hugging Face repositories.

load_dataset(name)

Alias to _load_* functions in retrieval resources.

Embeddings#

Pre-computed latent vectors for text and neuroimages.

load_latent(name)

Alias to _load_latent* functions in retrieval resources.

Masker#

Nifti masker need to resample and mask neuroimages.

load_masker()

Masker alias.

Models#

Base models for autoencoder, projection heads, and specter. Pretrained models return from load_model or calling .from_pretrained on model classes.

NeuroAutoEncoder([seed, out, dim_neuro, ...])

Autoencoder for neuro-vectors.

ProjHead([latent_in_dim, hidden_dim, ...])

Align latent tensors.

Specter([model, adapter, orthgonalize, ...])

Wrapper for Specter model.

load_model(name)

Alias to .from_pretrained methods in model classes.

Loss Functions#

The pretrained models used InfoNCELoss or MSELoss. Additional options include FocalLoss or TruncatedLoss.

InfoNCELoss([temperature])

Compute InfoNCE loss between image and text embeddings.

FocalLoss([alpha, gamma])

TruncatedLoss([percentile, base_loss])

Training#

Convenience wrapper for training: a standard PyTorch training loop.

Trainer(model, loss_fn, lr, batch_size, ...)

Training loop.

which_device()

Determine the device to move models and tensors to.

Metrics#

Performance metrics.

recall_at_k(cos_sim, k)

recall_curve(latent_text, latent_image[, step])

dice(img_a, img_b)

Compute dice score.

dice_top_k(y_true, y_prob[, k])

Compute dice score of top k.

bernoulli_bce(y, p[, eps])

Elementwise Bernoulli negative log-likelihood (cross-entropy), in nats.

bits_per_pixel(y_true, logits, *[, ...])

y_true: (N, D) floats in [0,1] logits: (N, D) raw logits from decoder (before sigmoid)
Contents