Concept
Mixture-of-Transformers
Mixture-of-Transformers (MoT) is a sparse architecture where model parameters are split by modality rather than shared across all inputs. It is like Mixture-of-Experts, except the routing is deterministic: instead of a learned gate picking experts per token, MoT sends…
The rest of “Mixture-of-Transformers” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→