maxtext.input\_pipeline.tokenizer module
========================================

.. automodule:: maxtext.input_pipeline.tokenizer
   :members:
   :show-inheritance:
   :undoc-members: