I wrote about basic RAG in Spring AI a little while ago: chunk your documents, embed them, store the vectors, retrieve the nearest ones, put them in the prompt. That post ends with a modular RetrievalAugmentationAdvisor and a couple of seams I waved at — query transformation before retrieval, reranking after it — without opening them.
This series opens them. It started from Cole Medin’s all-rag-strategies catalogue, which surveys eleven techniques in Python. I’ve rebuilt the territory in Spring AI, checked the headline claims against their primary sources, and found that the numbers vary by an order of magnitude in ways the usual “advanced RAG” listicle flattens completely.
Before any of that, though, two questions worth settling.
Do you even need retrieval any more?
It’s a fair thing to ask in 2026. Context windows are 1–2 million tokens now. If your entire corpus fits in the prompt, why build a retrieval pipeline at all?
Sometimes you shouldn’t. If you’re answering questions about a 40-page handbook that changes twice a year, put the handbook in the prompt and go do something else. I said this about prompt stuffing in the earlier post and it’s more true now than it was.
But the “RAG is dead” position doesn’t survive contact with the numbers:
- Cost. Retrieval is roughly three orders of magnitude cheaper per query than stuffing a million-token window. You pay for every input token on every request, and a large context is a large bill repeated forever.
- Accuracy degrades with position. Long-context models lose substantial accuracy when the relevant passage sits in the middle of a large window rather than near either end. Filling the window doesn’t guarantee the model reads it.
- Adoption says otherwise. Retrieval is still in the majority of production LLM applications, and usage went up, not down, as context windows grew.
That last one confused me until I understood the pattern. Long context didn’t replace retrieval — it changed what retrieval is for. You no longer need to squeeze the perfect three chunks into a 4k window. You retrieve to narrow a corpus from millions of tokens to tens of thousands, then let a long window reason across all of it. Precision matters less than it did; recall matters more. Miss the relevant document and no amount of context window saves you.
So: retrieve to narrow, then let the model reason. That’s the shape of the thing, and it’s why the strategies in this series skew towards “find more of the right stuff” rather than “find fewer, better things”.
How basic RAG actually fails
“Improve your RAG” is not an actionable instruction. The useful move is to name the failure you’re actually seeing, because different failures have completely different fixes — and most teams reach for a fix without diagnosing first.
Five failure modes cover almost everything I’ve run into.
The chunk boundary split the answer. Your chunker cut between a definition and the thing it defines, or between a table’s heading and its rows. Neither half retrieves well, because neither half means much alone. This is an ingestion problem and no amount of query cleverness fixes it.
The chunk is a fragment with no context. A chunk reads “This threshold defaults to 30 days.” Which threshold? The document knew, the chunk doesn’t. Its embedding is nearly meaningless, so it never surfaces for the question it actually answers. Also ingestion.
The vocabulary doesn’t match. The user asks about “sending it back”, the document says “refunds”. Embeddings handle this case well — that’s their whole point. What they handle badly is the opposite: exact tokens. Error code ORA-01555, SKU 4417-B, a surname. Semantic similarity actively works against you when the query term is an arbitrary string that means nothing.
There was no good answer, and you got one anyway. Without a similarity floor, a question your corpus can’t answer still returns your five least-bad chunks, and the model answers confidently from irrelevant text. I covered the similarityThreshold fix in the earlier post; it remains the single highest-value one-line change in RAG.
Lost in the middle. You retrieved the right chunk, ranked it fourth of ten, and the model paid most attention to the first and last. This one is real enough that Spring AI’s own DocumentPostProcessor javadoc names it as a problem the component exists to solve — alongside context-length limits and noise reduction.
Note how these split. Two are ingestion problems, two are query problems, one is a data-model problem. That’s the structure of the series.
The three places you can intervene
Every technique in this series changes one of three things:
- Ingestion time — what you index. Chunk differently, enrich chunks before embedding, store parent-child relationships, use a different embedding model. Expensive to change (you re-process the corpus), and it fixes problems nothing downstream can.
- Query time — how you search. Rewrite the query, run several variants, rerank the results, grade them and search again, or let an agent choose the tool. Cheap to change and cheap to remove; costs latency and tokens on every single request.
- The data model — what “search” even means. Add keyword search alongside vectors. Add a graph. Different retrieval substrate, not a tweak to the existing one.
Parts two, three and four take those in order.
Here’s the thing worth internalising early: Spring AI’s support is heavily lopsided. The query-time strategies are nearly all first-class — MultiQueryExpander, RewriteQueryTransformer, ConcatenationDocumentJoiner, DocumentPostProcessor, ContextualQueryAugmenter are all in org.springframework.ai.rag.* and compose into RetrievalAugmentationAdvisor. The ingestion-time strategies have no equivalent at all; Spring AI gives you an ETL pipeline and you supply the strategy. That asymmetry is worth knowing before you plan a sprint around it.
Measure first, or you’re guessing
This is the part people skip, and skipping it makes the rest of the series useless to you.
Every strategy ahead costs something — latency, money, or complexity, usually all three. You cannot make that trade without a number. And you cannot get the number from vibes, because RAG failures are quiet: a wrong answer looks exactly like a right one.
The metric that matters most is recall@k — of the documents that genuinely answer the question, how many appear in the top k retrieved? Not whether the final answer read well. Retrieval and generation fail differently and you have to separate them.
Anthropic’s contextual retrieval work, which part two leans on, measures exactly this and reports it as 1 − recall@20: the share of relevant chunks that fail to make the top 20. I like the inversion, because it counts failures rather than successes, and failures are what you’re trying to remove.
You need a golden dataset: real questions paired with the chunks that should be retrieved. Thirty to fifty is enough to be useful. Build it from actual user questions if you have them, and write down the expected sources by hand — it’s a couple of hours of unglamorous work that pays for itself the first time it stops you shipping a regression.
Then wire it into CI, the way I described for prompt regression testing. Retrieval quality is a number that can go down silently when someone changes a chunk size.
One discipline to hold to for the rest of the series: change one thing at a time and re-measure. Stack five strategies at once and you’ll have a slower, pricier pipeline and no idea which part earned its keep — or which one made it worse. Some of them will make it worse.
What’s coming, and a warning about the numbers
The techniques ahead are not equally valuable. That’s the most useful thing I can tell you, and it’s the thing every “11 advanced RAG techniques” post obscures by giving each one a tidy row in a table with star ratings.
Concretely, and with sources given in the relevant parts: contextual retrieval cuts retrieval failures by 35% on its own and 67% combined with reranking. Late chunking — comparable implementation effort, similar conceptual sophistication — buys about 3.6% relative in its authors’ own benchmarks. Fine-tuned embeddings claim 5–10% and cost you a training pipeline and a full re-embed whenever the model changes.
Those are not the same kind of thing, and treating them as a list of eleven interchangeable options is how teams end up with a baroque pipeline that’s slower than where they started.
So: part two covers ingestion, where the biggest single win lives. Part three covers query time, where Spring AI does most of the work for you. Part four covers hybrid search and knowledge graphs, plus a decision table mapping the symptoms above to the fix that addresses them.
Conclusion
Retrieval isn’t obsolete — its job changed. With a million-token window you’re no longer fighting for space, you’re making sure the right material is in the shortlist at all. Recall over precision.
Before you adopt anything from the rest of this series, do the boring part: name which of the five failures you’re actually seeing, build a golden dataset of thirty questions, and get a recall@k number you trust. Otherwise you’re buying latency and tokens on the strength of a blog post, mine included.
Next: the ingestion half — chunking that respects document structure, and the one technique with numbers big enough to justify a per-chunk LLM call.
More info: docs.spring.io/spring-ai/reference/api/retrieval-augmented-generation