Tiny LLM prunes 68% of RAG context, saving a third of query cost
Kapa inserted a cheap LLM between the retriever and the generator to prune irrelevant chunks. The pruner drops 68% of retrieved context but retains 96% recall, slashing query costs by a third. It lets agents handle massive knowledge bases without ballooning token usage.