Concept
Mixture-of-Experts (MoE)
Mixture-of-Experts (MoE) is a neural network design where a layer holds many parallel sub-networks (the "experts"), and a small gating network decides which experts process a given input. The key idea is conditional computation: instead of every parameter touching…
The rest of “Mixture-of-Experts (MoE)” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→