neurovlm.training.train_brain_to_text_generation

neurovlm.training.train_brain_to_text_generation#

neurovlm.training.train_brain_to_text_generation(config, *, provider, qformer=None, causal_lm=None, qformer_loader=None, lm_loader=None, collator=None, generation_callback=None, semantic_metric_callback=None)[source]#

Train only Q-Former parameters against a frozen causal LM.

Parameters:

  • config (BrainToTextGenerationTrainConfig)

  • provider (Any)

  • qformer (NeuroQFormer | None)

  • causal_lm (Module | None)

  • qformer_loader (Callable[[], NeuroQFormer] | None)

  • lm_loader (Callable[[], Module] | None)

  • collator (Callable[[Sequence[Any]], Any] | None)

  • generation_callback (Callable[[Module, Module, BrainToTextBatch], Sequence[str]] | None)

  • semantic_metric_callback (Callable[[Sequence[str], Sequence[str | None], Sequence[Mapping[str, Any]]], Mapping[str, float]] | None)

Return type:

BrainToTextGenerationTrainResult