maxtext.models.mixtral module

Decoder layer definition for mixtral.

class maxtext.models.mixtral.MixtralDecoderLayer(*args, **kwargs)

Bases: Module

Transformer decoder layer that attends to the encoder.

Parameters:

• args (Any)

• kwargs (Any)

Return type:

Any