Concept
RoPE
RoPE (Rotary Position Embedding) is a way to inject token position into a transformer's attention, used in most modern LLMs (LLaMA, Qwen, and the like) and increasingly in video and diffusion models. Instead of adding a learned or sinusoidal position vector to each token…
The rest of “RoPE” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→