Concept
ReLU
ReLU (Rectified Linear Unit) is the activation function f(x) = max(0, x): it passes positive inputs through unchanged and clamps everything negative to zero. Think of it like a diode, except applied elementwise to a tensor of activations.
The rest of “ReLU” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→