Concept
Hierarchical softmax
Hierarchical softmax is a way to approximate the softmax output layer so that computing word probabilities costs O(log V) instead of O(V), where V is the vocabulary size. The standard softmax normalizes over every word: to get one probability you compute a dot product…
The rest of “Hierarchical softmax” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→