One Number Predicts When Cosine Similarity Fails Your Embeddings

Cosine similarity is the default metric for text embeddings. It ships in every vector database, every retrieval tutorial, every RAG starter kit. The assumption baked into that default is that cosine is universally the best parameter-free choice. It is not, and the condition that breaks it has been sitting in the geometry of the embedding space the whole time.

The geometric property at fault is anisotropy: when an encoder concentrates its variance into a few dominant directions rather than spreading it evenly across the full embedding space. In a well-spread, isotropic space, cosine similarity measures what it is supposed to measure, directional alignment between semantically similar points. In an anisotropic space, those dominant dimensions pull every vector toward them, compressing the angular differences that cosine relies on and flattening the signal. Rank-based and L1-type metrics survive this distortion because they operate on ordinal relationships rather than raw angular geometry, making them less sensitive to a handful of directions that dominate the variance budget.

The study tested nineteen parameter-free similarity metrics across nineteen encoders, ranging from compact sentence transformers to seven-billion-parameter language models, over seven datasets. One number predicts the outcome: the fraction of total variance held by the single most dominant dimension. That statistic correlates with metric advantage at rank correlation 0.86 and linear correlation 0.95 across all nineteen encoders. The relative gain from switching to rank-based metrics averages around 20% on anisotropic encoders, which sounds modest until you note that cosine starts low on those models, making the relative improvement the largest where the default choice hurts most. The causal direction is confirmed by a projection test: remove the dominant directions, and cosine recovers while the advantage of alternative metrics nearly disappears, but only on the encoders that were anisotropic to begin with. For teams running retrieval pipelines, the takeaway is direct: compute the dominant-dimension variance fraction for your encoder before choosing a similarity metric, not after you have scaled to production.

We're thinking: We find the training-versus-geometry result the most operationally important piece here. The study shows that where training method and embedding geometry disagree about which metric should win, geometry wins. That means fine-tuned embedders optimized for retrieval, which tend to be more isotropic, are probably fine with cosine as-is. The real exposure is for teams using large general-purpose LLMs as off-the-shelf encoders without checking their geometry first. Those models are far more likely to be anisotropic, and a 20% relative retrieval gain is not a rounding error when it compounds across millions of queries. The one-number diagnostic makes this audit cheap enough that skipping it is no longer defensible.

Key takeaways:

• Embedding space anisotropy, specifically the variance fraction held by the single most dominant dimension, determines whether cosine or rank-based metrics perform better, and this geometric property overrides training method as the deciding factor.
• Rank-based and L1-type metrics beat cosine by roughly 20% relative on anisotropic encoders across 19 models and 7 datasets; the caveat is that the absolute gain is modest and the effect largely disappears on the fine-tuned retrieval embedders most teams already deploy.
• Teams using large general-purpose LLMs as text encoders should compute the dominant-dimension variance fraction for each encoder and switch to rank-based similarity where that fraction is high, before scaling retrieval infrastructure.

Source: Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings