neurovlm.evaluation.evaluate_contrastive_comparison

neurovlm.evaluation.evaluate_contrastive_comparison#

neurovlm.evaluation.evaluate_contrastive_comparison(*, selections=None, data=None, provider=None, lookup=None, mlp_text_encoder=None, split='test', domains=('pubmed', 'nilearn', 'neurovault'), device='cpu', batch_size=64, include_finetuned=False)[source]#

Evaluate paired full-split retrieval with family-native text preprocessing.

Every family sees the same atlas-free map/text rows. CNN models consume the immutable published normalized cache; MLP models re-encode the raw positive text with their released SPECTER2 adhoc_query convention. This paired protocol is intentionally distinct from historical family-native benchmarks that used different sample cohorts.

Parameters:

  • selections (Sequence[ComparisonSelection] | None)

  • data (Dataset | None)

  • provider (AtlasFreeCNNDataProvider | None)

  • lookup (AtlasFreeTextEmbeddingLookup | None)

  • mlp_text_encoder (Callable[[Sequence[str]], Tensor] | None)

  • split (str)

  • domains (Iterable[str])

  • device (str | device)

  • batch_size (int)

  • include_finetuned (bool)

Return type:

ComparisonResult