Insights

Prompt, context, harness, loop and graph engineering.  A practical framework for diagnosing where agentic AI systems actually fail

Why this matters to you

Every leader who has sponsored an AI agent project has heard some version of the same complaint: "the model is inconsistent."
It sounds like a single problem. It rarely is.

When an engineering team investigates, they typically find that "inconsistent" is masking several different failure modes, each requiring a different fix, a different owner and often a different investment.

This matters because many organisations are still funding AI initiatives as if better prompting is the whole discipline. It isn't. Treating it that way is one reason so many pilots stall between an impressive demo and a dependable production system.

These layers are connected, but they are not interchangeable.

They also describe a progression in capability. Prompt and context shape cognition. Harness gives the system agency. Loop design introduces autonomy. Graph design determines how that autonomy is orchestrated across a wider system.

That distinction matters because the consequences of poor design increase as you move through the layers: a prompt problem may create a poor answer; a harness, loop or graph problem can create poor actions at scale.

This article walks through all five using one running example: an AI agent handling customer refund requests.

The shape of the problem

Picture a customer messaging your support system:

"I want a refund for order #4471. It arrived damaged."

A capable agent needs to:

• understand what the customer is asking;
• know what happened with that specific order;
• know the relevant refund policy;
• be authorised to take appropriate action;
• check whether its action succeeded;
• know when to stop;
• know when the case needs to move to a human or another system.

Each of those needs sits at a different layer.

Layer 1: Prompt - what you're telling the model to do

It is also limited by everything around it.

A brilliant prompt sitting on top of the wrong information or the wrong permissions will still fail.

For the refund agent, the prompt might say:

Technical depth. Prompt engineering covers instruction clarity, role framing, output formatting, examples and reasoning scaffolds.

A poorly written prompt creates real failure modes: ambiguous goals, missing edge cases and unclear escalation criteria.

But prompt engineering has a ceiling. It can shape how the model uses what it has been given. It cannot supply missing information, grant system permissions or repair broken workflow logic.

When teams treat every failure as a prompting problem, they often end up repeatedly rewording instructions to compensate for weaknesses elsewhere in the system.

Layer 2: Context - what the model actually sees

Context engineering is about getting the right information to the model, at the right moment, in a form it can reliably use.

For the refund agent, that might include:

• the order record;
• the original purchase receipt;
• the policy applying to that product category;
• the customer's account history;
• and the specific clauses covering damaged goods within the return window.

The distinction matters. Giving the model a generic refund policy is not the same as giving it the information relevant to this refund.

Technical depth. Context engineering includes four major decisions:

Retrieval: finding the right records and documents rather than merely finding related ones.

Structuring: presenting those records in a format the model can interpret reliably instead of dumping raw JSON or extracted PDF text into the context window.
Recency and relevance: filtering stale, duplicated or contradictory material.

Context-window management: deciding which information deserves inclusion when everything relevant cannot fit at once.

A common production failure is not that the model "doesn't know". It is that the system showed it several conflicting versions of the truth and left the model to choose between them.

That is primarily a context problem, not a prompt problem.

Layer 3: Harness - what the model is actually allowed to do

This is where governance, security and risk become concrete system design.

For the refund agent, the harness might provide:

• read access to the order management system;
• write access to the payments API up to a fixed refund amount;
• no permission to alter customer account details;
• mandatory human approval above the refund threshold.

An agent with no meaningful limits is a liability. An agent with restrictions that are too tight cannot perform useful work.

The challenge is controlled agency.

Technical depth. Harness engineering includes tool and API design, authentication, permission scoping, sandboxing, spending and rate limits, approval gates and observability.

A reliable harness applies the same principle of least privilege that security teams already use for people and service accounts: the agent gets exactly the access required to perform its role, with high-consequence actions explicitly controlled.

Just as importantly, the system should log what the agent attempted to do, not merely the final result.

That turns "the AI decided to do X" from an alarming statement into an auditable system event.

Layer 4: Loop - how the agent iterates until it's actually done

Rather than receiving one instruction and returning one answer, an agent can observe the situation, plan a next step, act, inspect the result and decide what to do next.

For the refund agent, the loop might be:

The important point is that the system does not simply keep going. It knows what successful completion looks like.

Technical depth. Reliable loops usually depend on four things:

"Stop when the task is done" is not a useful stop condition.

"Stop when the refund is confirmed processed, or after two failed eligibility checks, whichever comes first" is.

Layer 5: Graph - how agents, tools and humans organise together

It determines how agents, tools, knowledge sources and humans connect: who hands work to whom, where approval occurs, what information travels between components and how failures are contained.

If the first four layers are about making one agent competent, the graph is about making the wider system coherent.

For the refund workflow, the graph might:

• allow straightforward, low-value refunds to be resolved end to end
• route high-value or disputed cases to a human reviewer
• hand suspected fraud cases to a specialist fraud-detection agent.

Technical depth. Graph engineering includes routing logic, agent-to-agent handoffs, shared versus isolated context, human-in-the-loop checkpoints and failure containment.

It also introduces an important design question: when one part of the system is wrong, how far is that error allowed to travel?

A poorly designed graph can allow one mistaken assumption to propagate across multiple agents and systems.

A well-designed one contains failure, makes escalation explicit and gives each participant a clear role.

This is still an emerging discipline. There is less consensus around graph design than there is around prompt or context engineering.

A useful way to think about it is as an organisational structure: deliberate reporting lines, responsibilities and escalation paths rather than a web of agents calling each other simply because they can.

What this means for how you organise the work

The value of the framework is not academic. It changes how you diagnose problems, invest and assign accountability.

Diagnosis gets faster

When an agentic system misbehaves, ask which layer is actually failing:

• Wrong kind of answer, with the right information → prompt
• Right kind of answer, but using wrong, incomplete or outdated information → context
• The agent knows what action it should take but cannot take it → harness
• The agent repeats itself, stops too early or keeps going indefinitely → loop
• Individual agents work independently but fail to hand off, escalate or coordinate → graph

That distinction can prevent engineering teams spending days fixing the wrong problem.

Investment gets better targeted

Prompt engineering attracts attention because it is fast, visible and relatively inexpensive.

Context and harness work is usually less visible, but it is where much of production reliability is won or lost.

As systems become increasingly autonomous, loop design becomes critical because it determines how behaviour unfolds over time.

And once multiple agents and systems begin working together, graph design becomes a strategic concern because it determines how capability, control and failure propagate across the organisation.

The investment requirement therefore changes as the system matures:

The more capable the system becomes, the more consequential architecture becomes relative to wording.

Accountability gets clearer

The layers also create a useful division of responsibility.

Prompt and context often sit closest to the team that understands the business process and information requirements.

Harness decisions sit closer to security, platform engineering and governance.

Loop design belongs with the engineers building the agent's operating logic.

Graph design increasingly belongs with whoever owns the end-to-end process the agents are supporting.

Naming these layers gives those groups a common language.

Instead of five people using the word "prompt" to describe five completely different problems, teams can identify exactly which part of the system needs attention.

The bottom line

Prompting started the discipline, but it was never going to be the whole discipline.

As agentic AI moves from pilot to production, the important question is no longer simply:

"How do we write a better prompt?"

It is:

"Which layer is actually causing the problem and who owns fixing it?"

Prompt and context determine how well an agent can reason with the information available to it.

The harness determines what it can do.

The loop determines how independently it can operate.

The graph determines how that capability behaves across an organisation.

That progression from cognition, to agency, to autonomy, to orchestration... is why agentic AI becomes more valuable as systems mature, but also why reliability, governance and design become increasingly important.

The next competitive advantage in agentic AI is unlikely to come from cleverer wording alone.

It will come from designing the whole system well.