Paper
Efficient Estimation of Word Representations in Vector Space
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
arXiv:1301.3781 · 0▲ · cs.CL
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
This is the original word2vec paper. It introduces two log-linear model architectures, Continuous Bag-of-Words (CBOW) and Skip-gram, that learn dense word vectors by removing the expensive non-linear hidden layer used in prior neural language models and training on billions of words.
Why it matters
Word vectors became a reusable building block: instead of treating words as opaque vocabulary indices, you get dense representations where similarity and analogies (king - man + woman = queen) fall out of vector arithmetic. Training that previously took weeks on complex neural nets now runs in about a day on a single machine, and later the released C++ code hit billions of words per hour.
Practical takeaway
If you need cheap semantic embeddings for search, classification, or clustering and are not spinning up a full transformer, CBOW/Skip-gram (word2vec) is still a fast, low-cost baseline. Watch for the word offset trick: subtract and add vectors to solve analogy-style relationships.
Key result
On the authors' Semantic-Syntactic Word Relationship test set, a 1000-dimensional Skip-gram model trained on 6B words reached 65.6% total accuracy (66.1% semantic, 65.1% syntactic) in 2.5 days on ~125 CPU cores, versus 50.8% for their own NNLM. Note the benchmark and evaluation metric are defined by the same authors and require exact-match answers, so absolute numbers are self-reported.
Jargon
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.