KONVENS 2026, Conference on Natural Language Processing, Hamburg
Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations
Gradient-based explanations tell you which training examples shaped a model’s answer, but full gradients of a large language model are far too big to compare. The paper tests two ways to shrink them: an architecturally informed, greedily selected subset of gradient components, or random projection (as in TRAK). On a new retrieval benchmark, selecting components captured training-data influence more effectively and more efficiently than random projection or even the full gradient.