From RAG to context engineering: the training
and eval data behind modern LLM apps

For two years, "just add RAG" was the default answer to almost every enterprise LLM question. Retrieve some relevant chunks, stuff them into the prompt, generate an answer. It worked well enough to launch thousands of pilots — and well enough to expose exactly where naive retrieval falls apart: stale chunks, lost context across a multi-step task, an agent that retrieves once when it needed to retrieve five times, or not at all. The reaction has been a wave of "RAG is dead" headlines. That framing is wrong, but it points at something real.

What is actually happening in 2026 is a reframing, not a funeral. Classic single-shot RAG is being absorbed into a broader discipline the field now calls context engineering: the deliberate design of what an agent knows, sees, and remembers at the moment it acts. Retrieval is still central — it is just no longer a one-time lookup bolted to the front of a prompt. It has become an intelligent, agent-driven capability with memory attached. And here is the part that gets lost in the architecture debates: every one of these newer patterns is only as good as the labeled data underneath it. That data is the work H2L India is built to do.

From single-shot RAG to a context engine

The clearest way to understand the shift is to look at how "retrieval" is changing shape. Classic RAG treated retrieval as a fixed step. Agentic retrieval treats it as a decision the model reasons about — whether to retrieve, what to retrieve, and when to stop. Around that sits persistent memory, so an agent doesn't re-derive the same context on every turn.

Two forces are driving this. Context windows have grown enormously — models now handle hundreds of thousands to millions of tokens — which changes the calculus of how much you retrieve versus how much you keep resident in memory. At the same time, agent workloads have exposed the gaps: a longer window doesn't help if you fill it with the wrong things. Enterprise buyers noticed. Industry trackers reported that intent to adopt hybrid retrieval roughly tripled in early 2026, and retrieval optimization overtook evaluation as the top investment priority. RAG didn't die; it grew up into a context engine.

The data work hiding under every layer

Architecture diagrams make this look like a pure software problem. It isn't. Each capability in a modern LLM or agent stack is trained or evaluated on human-labeled data, and the quality ceiling of the system is set by the quality of those labels. Here is the actual annotation work that sits beneath the buzzwords:

• Entity, intent, and NER labels. Before an agent can route a query or populate a knowledge graph, someone has to mark what the entities are, what the user actually wants, and how spans relate. This is the unglamorous backbone of intent classification and structured retrieval.
• Retrieval relevance and reranking judgments. A reranker is only as good as the graded query-document pairs it learns from. Human judges score whether a retrieved passage is truly relevant, partially relevant, or noise — the difference between an agent that cites the right paragraph and one that confidently cites the wrong one.
• Tool-use and function-call traces. Agentic systems learn to call tools from annotated traces: given this state, was calling this function with these arguments correct? Labeling the good, the redundant, and the outright wrong calls is what teaches an agent when to retrieve versus when to stop.
• Preference pairs for RLHF. The alignment layer — "response A is better than response B" — is human comparison data at its core. This is the lineage that runs back to the original instruction-tuning work and still underpins how models are shaped to be helpful and honest.
• Evaluation and red-team datasets. You cannot improve what you cannot measure. Held-out eval sets and adversarial red-team prompts — carefully written and labeled to probe hallucination, jailbreaks, and failure modes — are how teams know whether the new retrieval pipeline is actually better or just different.
• Document-layout labels for parsing. Most enterprise knowledge lives in PDFs, tables, and forms. Bounding boxes and region labels for headers, tables, figures, and reading order are what let a parser turn a messy document into clean, retrievable text in the first place.

Why annotation discipline transfers across domains

H2L India's deepest track record is in agritech. We label the field imagery that trains autonomous laser weeders and the POTECTOR crop-disease robot on real Dutch farms — pixel-precise segmentation of potato plants, weeds, and disease, verified with multi-pass QA and consensus IoU agreement across roughly three-times overlap. That is not a different craft from text annotation; it is the same craft applied to different pixels and tokens.

The transferable core is the guideline-plus-agreement loop. In vision work, we measure whether independent annotators drew the same mask (IoU); in text and retrieval work, the equivalent is inter-annotator agreement on whether a passage is relevant, whether an intent label is correct, or whether response A genuinely beats response B. The machinery is identical: a tight annotation guideline, multiple independent passes, a measurable agreement metric, and adjudication where annotators disagree. A studio that has learned to keep IoU high on a moving crop canopy already knows how to keep labeling consistent when the task is subjective — which is exactly where retrieval-relevance and preference data get hard.

We are honest about where we stand. H2L India is expanding into NLP, medical imaging, and autonomous-vehicle data — that is capability and method, not a decade-long client roster in every vertical. What we bring to a text or LLM-data project is the operational discipline proven on agritech: an in-house team of 15+ trained annotators, Label Studio workflows, and QA that treats agreement as a number to defend rather than a box to tick.

When to use what

None of this means every project needs an agentic stack. Match the architecture to the problem, and match the data investment to the architecture:

• Bounded, static knowledge? Classic RAG plus a well-labeled eval set is often enough. Don't over-build.
• Multi-hop questions or open-ended tasks? Agentic retrieval — and now you need reranking judgments and tool-use traces to make it reliable.
• Long-running agents with memory? Full context engineering, which raises the stakes on eval and red-team data because the failure surface is larger.

The move from RAG to context engineering is genuinely exciting, but it doesn't reduce the need for careful human labeling — it multiplies it. If you're building an LLM or agent application and the data layer is where you're feeling the strain, that is precisely the problem we like to work on. Talk to H2L India about the annotation, relevance, and evaluation data behind your stack.

Frequently asked

Is RAG obsolete in 2026?

No. Classic single-shot RAG is being absorbed into a broader discipline called context engineering, where retrieval becomes an intelligent, agent-driven capability paired with persistent memory. For bounded knowledge bases, plain RAG is still the right, cost-effective choice. The 'RAG is dead' framing overstates a reframing, not a replacement.

What kind of labeled data does an agentic LLM app actually need?

Several distinct datasets: entity/intent/NER labels for routing, graded query-document pairs for retrieval and reranking, annotated tool-use traces for function calling, human preference pairs for alignment (RLHF), evaluation and red-team sets for measuring quality and safety, and document-layout labels so parsers can turn PDFs into clean retrievable text.

How does agritech annotation experience transfer to NLP data work?

The transferable core is the guideline-plus-agreement loop. High-quality vision labeling depends on tight guidelines, multiple independent passes, and a measurable agreement metric (IoU). Text and retrieval work uses the same machinery — inter-annotator agreement on relevance, intent, or preference judgments — so the operational discipline carries over directly even though the data type differs.