Training Tokenizers

medtokenizers ships a Trainer with a callback system for training tokenizers from scratch.

python scripts/train.py --help

The trainer supports continuous (reconstruction + KL) and discrete (reconstruction + quantization/commitment + optional entropy) objectives, mixed precision, and checkpointing. Callbacks such as EarlyStopping, Checkpoint, LRScheduler, and Logger hook into the training loop.

See Training for the full API.