Context Is King. But Relevance Is Everything.

In this blog post I am exploring an analogy about context engineering.

I am sure you have heard the phrase:

When I first started studying large language models, something quickly stood out to me: tools like ChatGPT and Claude are excellent at figuring out what you are trying to achieve. They do not just respond to words in isolation. They use the surrounding context, infer intent, weigh possibilities, and then shape a response around what they think will be most useful.

But there is a problem.

They are still generating responses probabilistically. They are designed to be helpful, agreeable, and fluent, which means the output can sound convincing even when it is shallow, generic, or wrong. Unless you understand the field you are working in, it can be difficult to judge whether the answer is actually good.

This is why so much AI-generated content starts to feel the same. Marketing copy begins to blur together. Blog posts open in familiar patterns. Sales messages sound polished but empty. Without meaningful context, large language models tend to produce outputs that look and sound alike.

And yet, we have all seen enough to know there is real value here.

The value is not automatic. It has to be extracted.

The more I think about how AI systems create value from business data, the more mining feels like the right metaphor.

Our businesses sit on enormous amounts of context. Some of it relates to products. Some to marketing. Some to sales, support, operations, finance, delivery, and customer experience. Every day we generate more data, and hidden inside that data is more useful context.

Context might be:

• marketing campaigns and sales numbers

• customer support conversations

• product feedback

• popular colours, flavours, features, or trends

• tone of voice and brand position

• buying patterns and customer objections

• operational knowledge buried in documents, tickets, and chats

All of this context has potential value. It can help teams create better campaigns, write stronger ad copy, support customers more effectively, make better product decisions, and spot opportunities earlier.

The problem is that business data is rarely structured for reasoning. It is scattered across systems, duplicated, outdated, inconsistent, and filled with noise.

That is why I think the mining analogy works so well.

But this creates a tension:

In reality, the opposite often happens.

That idea has driven a lot of excitement around larger context windows. If a model can read more documents, more chat history, more product data, and more customer conversations in one go, surely the answers should improve.

But real-world AI systems quickly expose the flaw in that thinking:

A larger context window gives the model more material, but not more discernment. Poor-quality context, duplicated information, outdated documents, and irrelevant material all compete for attention.

So the problem is not just whether the model has enough context.

The problem is whether it has the right context.

To understand why, it helps to think less about AI as a source of answers, and more about AI as an extraction system: one that must find valuable signals hidden inside vast amounts of data.

The Mountain of Data

Every organisation now sits on a mountain of valuable information:

• documents

• PDFs

• tickets

• emails

• source code

• chat history

• APIs

• databases

• telemetry

• operational knowledge

This mountain grows every day.

The instinctive response when building AI systems is usually:

Larger context windows. Bigger prompts. More retrieval chunks. More tokens.

But this creates a problem.

The model is now standing in front of an entire mountain trying to find a few grains of gold.

The Gold Panning Phase

This is where many AI systems are today.

They behave like prospectors standing in a river with a pan:

• scooping large amounts of sediment

• filtering through pebbles and sand

• hoping something valuable appears

• repeating the process again and again

Sometimes they find valuable context. Sometimes they don’t.

The bigger the river becomes, the harder it gets.

This is exactly what happens when we overload LLMs with raw context.

As context grows:

• relevance density falls

• noise increases

• attention becomes diluted

• important signals disappear inside irrelevant information

The system becomes slower, more expensive, and less reliable.

This is not intelligence.

It is probabilistic gold panning.

Why Bigger Context Windows Are Not the Answer

There is an assumption that larger context windows solve this problem.

They help — but only partially.

A larger context window simply allows the model to process more terrain at once.

But processing more terrain does not mean finding more value.

If your retrieval strategy is poor, all you have done is:

• increase compute cost

• increase latency

• increase distraction

• increase the amount of irrelevant material competing for attention

The issue is not:

The issue is:

That is an entirely different engineering problem.

Vector Databases: Geological Surveying for AI

This is where vector databases fundamentally change the game.

Instead of blindly searching the mountain, the terrain becomes surveyed.

Patterns emerge.

Relationships become visible.

The system starts identifying where valuable seams are likely to exist.

This is much closer to modern geological surveying than gold panning.

A vector database:

• embeds information into semantic space

• maps similarity and relationships

• narrows the search space

• prioritises likely relevance

The model no longer searches randomly.

It is directed toward promising regions.

That is a profound shift.

The goal is no longer:

The goal becomes:

MCP Servers: The Mining Infrastructure Layer

But even retrieval is only part of the story.

Knowing where the gold is matters. Being able to extract it reliably matters even more.

This is where MCP (Model Context Protocol) becomes important.

If vector databases are geological surveys, MCP servers are the mining infrastructure:

• roads

• drilling equipment

• excavation machinery

• logistics

• operational control systems

MCP gives models structured, governed access to:

• databases

• APIs

• internal tools

• live systems

• document stores

• operational platforms

The model stops behaving like an isolated chatbot generating probabilistic answers and starts behaving more like an operator coordinating specialised systems, retrieval pipelines, and live business infrastructure.

This is the difference between:

• a prospector with a pan

• an engineered mining operation

Cognitive Observability

And like any industrial extraction system, observability matters.

Most AI systems today have extensive telemetry around infrastructure:

• GPU utilisation

• token counts

• latency

• throughput

• cost

But they have almost no telemetry around context quality itself.

We rarely measure:

• which context actually influenced the answer

• which retrieved documents were useful

• where attention collapsed

• how much duplicate or irrelevant information polluted the system

• whether retrieval genuinely improved reasoning quality

As AI systems mature, cognitive observability may become as important as infrastructure observability.

Because the real bottleneck is increasingly not compute.

It is attention.

The Real Shift in AI Architecture

The future of AI systems is not:

It is:

Modern AI systems scale through:

• semantic retrieval

• structured access

• tool use

• workflow orchestration

• relevance optimisation

Not brute-force prompting.

This is why the industry is moving rapidly toward:

• RAG architectures

• vector search

• MCP servers

• tool-enabled agents

• retrieval pipelines

• hybrid reasoning systems

The intelligence increasingly comes from the system architecture around the model — not just the model itself.

Better Context Beats Bigger Context

This is the key insight.

The objective is not:

The objective is:

Because raw context is not value.

Raw context is sediment.

The value comes from:

• finding the right information

• at the right time

• with the right tools

• in the right format

That is what modern AI systems are actually solving for.

Final Thought

The next generation of AI systems will not win because they have the largest context windows.

They will win because they:

• retrieve better

• connect to better systems

• operate with better tooling

• reduce noise more effectively

• deliver higher relevance with lower friction

The future of AI is not gold panning.

It is engineered extraction.