medtokenizers

Tokenization pipelines for volumetric medical imaging – continuous and discrete tokenizers (VAE/AE, VQ/FSQ/LFQ/ResidualFSQ, TiTok, RAE, MAISI) with supporting quantization modules, training, and evaluation infrastructure.

Ecosystem

medtokenizers is one of three packages that together form a toolkit for generative modeling of medical images:

• medrs – fast medical-image I/O (NIfTI/DICOM) and preprocessing primitives.

• medtokenizers – continuous and discrete tokenizers that compress 2D/3D scans into latents or token grids (this package).

• medlatents – generative models (autoregressive, MaskGIT, diffusion, flow matching, Bayesian flow) that learn over those tokens.

Typical pipeline:

images -> medrs (load) -> medtokenizers (tokenize) -> medlatents (generate) -> medtokenizers (detokenize) -> images

Getting Started

• Getting Started
  • Installation
  • Reconstruct a volume (MAISI)
  • Discrete tokens (TiTok)
  • Evaluation

Guides

• Choosing a Tokenizer
  • Continuous vs. discrete
  • Specialized tokenizers
  • Rule of thumb
• Training Tokenizers
• Evaluating Tokenizers

API Reference

• Tokenizers
  • BaseTokenizer
  • ContinuousTokenizer
  • DiscreteTokenizer
  • MAISITokenizer
  • TiTokTokenizer
  • RAETokenizer
• Quantizers & Building Blocks
  • Quantizers
  • Encoder / Decoder
• Evaluation
  • TokenizerEvaluator
  • Metrics
• Training
  • Trainer
  • Callbacks
  • Configuration
• I/O & Preprocessing
  • Loading tokenizers
  • Saving & loading tokens / latents
  • Preprocessing

Project Info

• Contributing

Indices and tables

• Index

• Module Index

• Search Page