Concept
Effective rank
Effective rank measures how many dimensions a matrix (or a batch of activations) actually uses, as opposed to how many it nominally has. Ordinary matrix rank counts nonzero singular values, so a 1024 x 1024 weight matrix with one tiny-but-nonzero singular value still has…
The rest of “Effective rank” is a premium feature: every concept in the library gets a precise, practitioner-focused write-up like this one, cross-linked straight from the paper summaries that use it.
Log in to unlock→