R&D / AI: mini-rag evaluation of cross-encoder efficiency

Jun 29, 2026 - 1 min read - 100 words

Brett Dargan

Principal Solutions Architect (AWS) | Tech Co-Founder | Product Strategy & Business Value

R&D / AI: Mini-RAG Cross-Encoder Efficiency Evaluation

Plenty of strong RAG content exists on papers and evaluation (e.g., Miraflow Guide).

Forked florianbuetow/mini-rag for local deep dive: how much efficiency gain from the cross-encoder?

Added token test in bdargan/mini-rag + bdargan/mini-rag-runner container. (Output posted below showing savings.)

See rag_token_test.py.

[TOKEN BENCHMARK REPORT]
  • Baseline Prompt Tokens (K=10): 308
  • Reranked Prompt Tokens (K=3):  112
  • Absolute Token Savings:         196
  • Efficiency Gains:               63.64%
PASSED

Key takeaways

Code from Gemini impressed again.
Follow-up: evaluate with Ragas