There is a pattern I keep seeing when designing Agentic AI systems.
We start by asking:
- Which LLM should we use?
- Should we use LangGraph?
- Where can MCP fit?
- Should we build multiple agents?
- Do we need RAG?
- Should we add memory?
- Should every step be handled by an autonomous agent?
These are useful questions.
But they are often asked too early.
The result can be an architecture that is technically impressive but operationally difficult, expensive, slow, and surprisingly hard to trust.
A better approach is to reverse the order:
Start with the product outcome. Define the constraints. Then design the architecture. Choose the tools last.
I have found a useful way to structure those constraints around four dimensions:
LCFE
L — Latency
C — Cost
F — Failure
E — Evaluation
This is not a framework that says every agentic system must look the same. It is a way of forcing architectural decisions to start with the realities of the product rather than the capabilities of the technology.
In this article, I’ll walk through a concrete incident-automation example and show how starting with constraints can completely change the architecture.
1. The “backwards” way of designing an agent
Imagine we want to build an AI Incident Resolution Assistant for an engineering organization.
The goal sounds straightforward:
When a production incident is raised, the AI should investigate the incident, gather context, identify the likely cause, recommend or perform remediation, and verify the result.
Now imagine the team starts with the technology.
The first architecture might look like this:
User / Incident
|
v
┌──────────────┐
│ Triage Agent │
└──────┬───────┘
|
v
┌────────────────┐
│ Research Agent │
└───────┬────────┘
|
┌──────────────┼──────────────┐
v v v
Logs Agent Metrics Agent Knowledge Agent
| | |
└──────────────┼──────────────┘
|
v
┌─────────────────┐
│ Remediation │
│ Agent │
└────────┬────────┘
|
v
┌─────────────────┐
│ Validation Agent│
└────────┬────────┘
|
v
Resolution
Enter fullscreen mode Exit fullscreen mode
It looks sophisticated.
We have agents.
We have tools.
We have MCP.
We have orchestration.
We have reasoning.
We have memory.
We have autonomous remediation.
Technically, there is nothing wrong with building this.
The problem is that we don’t yet know whether the product needs it.
We have designed the solution before defining the constraints.
2. Start with the product outcome
Before selecting an LLM or orchestration framework, define the actual outcome.
For our incident system, maybe the product requirement is:
Resolve common production incidents within 10 minutes, while reducing manual engineer effort and keeping high-risk actions under human approval.
That statement is much more useful than:
Build a multi-agent incident-resolution system.
Now we can ask the questions that actually drive architecture.
Latency
What response time is acceptable?
For example:
- P95 time to initial diagnosis: < 30 seconds
- P95 time to recommended remediation: < 90 seconds
- P95 time to automated resolution for eligible incidents: < 5 minutes
Cost
What can a successful resolution cost?
For example:
- Average cost per incident: < £0.20
- Expensive frontier-model calls reserved for complex cases
- Avoid unnecessary agent loops
Failure
What happens when something goes wrong?
For example:
- Logs API unavailable
- Metrics delayed
- Kubernetes API timeout
- Tool returns malformed data
- Model generates an invalid action
- Diagnosis is uncertain
- Remediation fails
- Incident remains unresolved
Evaluation
How do we know the system is actually useful?
For example:
- Correct incident classification
- Correct root-cause hypothesis
- Correct tool selection
- Safe remediation
- Successful verification
- Reduction in mean time to resolution
Now we have something much more valuable than a technology stack.
We have engineering constraints.
3. The first architectural surprise: not everything needs an agent
This is where the design can change dramatically.
Suppose historical incident data shows:
Incident Type Percentage Typical Resolution High CPU 25% Scale service Pod crash loop 20% Inspect logs + restart Certificate expiry 10% Renew certificate Database connection pool 15% Restart / tune service Deployment regression 10% Roll back Unknown / complex 20% Deep investigationSuddenly, the idea of having every incident go through a fully autonomous multi-agent workflow looks questionable.
For many incident types, the process is already known.
For example:
Incident
|
v
Classify
|
+---- Known pattern? ---- Yes ----> Deterministic workflow
|
No
|
v
Agentic investigation
Enter fullscreen mode Exit fullscreen mode
This is a major architectural insight:
An agent should exist where reasoning variability exists.
It should not exist simply because the technology makes it possible.
4. The new architecture
After applying the constraints, our architecture might become:
Incident Event
|
v
┌─────────────────┐
│ Fast Classifier │
└────────┬────────┘
|
┌───────────┴────────────┐
| |
v v
Known Incident Complex / Unknown
| |
v v
Deterministic Workflow Agent Runtime
| |
| ┌─────────┼──────────┐
| v v v
| Logs Tool Metrics Tool KB/Search
| | | |
| └─────────┼──────────┘
| |
| v
| Diagnosis
| |
| v
| Policy / Guardrails
| |
| ┌───────┴────────┐
| | |
| Safe High Risk
| | |
| v v
| Auto Remediation Human Approval
| | |
└────────────────┴────────────────┘
|
v
Verification
|
v
Outcome / Escalation
Enter fullscreen mode Exit fullscreen mode
Notice what disappeared.
We may no longer need:
- five autonomous agents
- an LLM call for every incident
- an agent deciding every deterministic operation
- unnecessary memory
- multiple reasoning loops
The system is actually simpler.
But it is also more production-oriented.
5. L — Latency: design backwards from the P95 budget
Latency is one of the easiest constraints to ignore during an AI prototype.
A demo can take 45 seconds and still look impressive.
A production incident-response system may not have that luxury.
Suppose our requirement is:
P95 time to diagnosis < 30 seconds
Now work backwards.
A possible latency budget:
End-to-end P95 = 30s
Classification 2s
Retrieval 4s
Tool calls 10s
LLM reasoning 10s
Orchestration 2s
Buffer 2s
--------------------------
Total 30s
Enter fullscreen mode Exit fullscreen mode
Now imagine someone proposes adding a reranker.
The reranker adds another 3 seconds.
We should not automatically say:
“Reranking improves retrieval, so let’s add it.”
Instead ask:
Does reranking produce enough evaluation improvement to justify 3 seconds of our latency budget?
Suppose evaluation shows:
Without reranking:
Diagnosis accuracy = 91%
With reranking:
Diagnosis accuracy = 91.8%
Enter fullscreen mode Exit fullscreen mode
An extra 3 seconds may not be worth it.
But if it changes:
91% → 97%
Enter fullscreen mode Exit fullscreen mode
the architectural decision becomes much easier.
This is the important mindset:
Every piece of complexity has to earn its place in the latency budget.
The same principle applies to:
- additional model calls
- reflection loops
- reranking
- extra retrieval stages
- agent-to-agent communication
- long context windows
- MCP calls
- retries
6. C — Cost: optimize cost per successful outcome
Agentic systems can become expensive surprisingly quickly.
Imagine a naïve incident workflow:
Triage LLM call
+
Research LLM call
+
Reasoning LLM call
+
Tool-selection LLM call
+
Remediation LLM call
+
Validation LLM call
+
Retries
Enter fullscreen mode Exit fullscreen mode
Now multiply that by thousands of incidents.
The important metric is not:
“How cheap is our LLM call?”
It is:
“How much does it cost us to successfully resolve an incident?”
Consider two architectures.
Architecture A
Every incident
↓
Large model
↓
Multiple agent loops
↓
Multiple tools
↓
Expensive reasoning
Enter fullscreen mode Exit fullscreen mode
Average cost:
£0.80 / incident
Architecture B
Classifier
↓
Known pattern?
↓
Yes → deterministic workflow
No
↓
Stronger model
↓
Agentic investigation
Enter fullscreen mode Exit fullscreen mode
Average cost:
£0.18 / incident
Suppose both achieve similar overall resolution rates.
Architecture B is clearly more attractive.
This naturally leads to a tiered model strategy:
Incoming incident
|
v
Small / cheap model
|
┌────────┴────────┐
| |
Simple Complex
| |
v v
Workflow model Strong model
|
v
Deep reasoning
Enter fullscreen mode Exit fullscreen mode
Use the expensive reasoning capability where it creates measurable value.
Not everywhere.
7. F — Failure: assume everything will fail
This is where production systems differ from demos.
A demo assumes:
Input
↓
LLM
↓
Tool
↓
Success
Enter fullscreen mode Exit fullscreen mode
A production system assumes:
Input
↓
LLM
↓
Tool
↓
Timeout
↓
Retry
↓
Malformed response
↓
Validation failure
↓
Fallback
↓
Human escalation
Enter fullscreen mode Exit fullscreen mode
For an incident agent, failure scenarios might include:
Tool failure
The metrics API is unavailable.
The agent should not invent metrics.
Retrieval failure
The knowledge base returns nothing.
The agent should explicitly recognize missing evidence.
Invalid tool call
The model produces:
{
"service": "payment-api",
"replicas": "many"
}
Enter fullscreen mode Exit fullscreen mode
But the tool requires an integer.
The system must validate the tool request before execution.
Dangerous action
The agent wants to execute:
DROP DATABASE
Enter fullscreen mode Exit fullscreen mode
The system should never treat the LLM’s intention as permission.
The permission boundary belongs to the runtime.
Endless reasoning
An agent can continue:
think → tool → observe → think → tool → observe
Enter fullscreen mode Exit fullscreen mode
without making progress.
So we need:
- maximum iterations
- time budgets
- retry limits
- tool timeouts
- action policies
- schema validation
- circuit breakers
- human escalation
This leads to a principle that is particularly important for agentic systems:
The model proposes actions. The runtime decides which actions are allowed.
The model should not become the security boundary.
8. E — Evaluation: measure the task, not just the answer
This is perhaps the most important shift when moving from an LLM application to an agent.
For a simple chatbot, evaluation might look like:
Was the answer helpful?
For an incident agent, that is not enough.
We need to evaluate the trajectory and outcome.
For example:
Incident
↓
Correct classification?
↓
Relevant evidence retrieved?
↓
Correct tools selected?
↓
Reasoning supported by evidence?
↓
Safe action selected?
↓
Action executed successfully?
↓
Incident actually resolved?
↓
Outcome correctly verified?
Enter fullscreen mode Exit fullscreen mode
A response such as:
“The issue appears to be high CPU.”
may sound intelligent.
But if the service remains unhealthy, the agent hasn’t completed the task.
The real evaluation might therefore include:
Task outcome
Did the incident get resolved?
Tool-use accuracy
Did the agent choose the appropriate tools?
Evidence quality
Did its diagnosis rely on relevant evidence?
Safety
Did it avoid disallowed actions?
Efficiency
How many tool calls and model calls were needed?
Reliability
Does the same scenario succeed repeatedly?
This is why agent evaluation needs to go beyond traditional prompt evaluation.
A useful production evaluation strategy has two layers.
Pre-release evaluation
Before deployment:
Benchmark scenarios
↓
Tool-use evaluation
↓
Trajectory evaluation
↓
Safety tests
↓
Regression suite
↓
Release gate
Enter fullscreen mode Exit fullscreen mode
Post-release evaluation
After deployment:
Real traffic
↓
Tracing
↓
Success / failure signals
↓
Production evaluation
↓
New failure cases
↓
Added to benchmark suite
↓
Next release
Enter fullscreen mode Exit fullscreen mode
This creates a feedback loop:
Production failures become future evaluation cases.
That is one of the most important pieces of a mature agentic architecture.
9. The role of MCP changes when constraints come first
MCP is a good example of why tool-first thinking can be dangerous.
If we start with:
“We have MCP, where can we use it?”
we are already designing backwards.
Instead ask:
“Which capabilities does the product need, and what is the safest and most efficient interface for those capabilities?”
Maybe MCP is a good fit for:
- standardized access to operational tools
- reusable tool interfaces
- connecting agents to external systems
- reducing custom integration work
But that doesn’t mean every operation needs to pass through MCP.
A deterministic internal workflow may call a service API directly.
A highly sensitive operation may need a tightly controlled internal execution service.
The correct question is not:
“Can MCP do this?”
It is:
“What interface best satisfies the product’s latency, cost, failure and security constraints?”
Then choose MCP when it fits.
10. The role of LangGraph also changes
The same logic applies to orchestration frameworks.
A graph framework can be extremely useful when you need:
- stateful execution
- retries
- branching
- checkpoints
- human-in-the-loop
- durable workflows
But using a graph framework does not automatically make a system more production-ready.
For our incident system, the architecture might contain:
Deterministic workflow
+
Agent runtime
+
Policy layer
+
Tool layer
+
Evaluation layer
+
Observability
Enter fullscreen mode Exit fullscreen mode
LangGraph might be part of the implementation.
Or it might not be.
The architecture should determine the tool choice.
Not the other way around.
11. Where memory actually belongs
Memory is another feature that is often added because an agent “should remember things.”
But what should it remember?
For an incident platform, maybe we need:
Short-term state
Current incident context:
Incident ID
Service
Recent logs
Metrics
Recent deployment
Actions already attempted
Current hypothesis
Enter fullscreen mode Exit fullscreen mode
Long-term knowledge
Reusable information:
Known failure patterns
Runbooks
Service documentation
Historical incidents
Operational policies
Enter fullscreen mode Exit fullscreen mode
These are not necessarily the same thing.
And some information should not be treated as permanent truth simply because a previous agent generated it.
That is a failure-mode question:
How stale can memory become before it becomes dangerous?
Again, the constraint drives the architecture.
12. Human-in-the-loop should be policy-driven
A common mistake is to treat “human approval” as one generic feature.
Instead, classify actions by risk.
For example:
LOW RISK
Read logs
Read metrics
Read deployment status
↓
Autonomous
MEDIUM RISK
Restart pod
Scale service
Clear cache
↓
Policy-based
HIGH RISK
Rollback production
Change configuration
Database operation
↓
Human approval
Enter fullscreen mode Exit fullscreen mode
Now autonomy becomes a policy decision, not an LLM personality trait.
This is a much more reliable way to build agentic systems.
13. The final architecture is often simpler
Notice what happened.
We started with:
Multi-Agent Everything
Enter fullscreen mode Exit fullscreen mode
and ended with:
Incident
|
v
Classification
|
┌───────────┴───────────┐
| |
Known pattern Complex case
| |
v v
Deterministic flow Agent runtime
|
Tools + Retrieval
|
Reasoning
|
Policy / Guardrail
|
┌────────────┴────────────┐
| |
Auto-action Human approval
| |
└────────────┬────────────┘
|
Verification
|
v
Outcome
Enter fullscreen mode Exit fullscreen mode
This architecture has fewer moving parts.
But it has something more important:
Each part exists for a reason.
14. A practical architecture decision sequence
When starting a new Agentic AI system, I now prefer to work through this sequence.
Step 1 — Define the outcome
What business or user outcome are we trying to improve?
Not:
“Build an AI agent.”
But:
“Reduce incident resolution time by 40%.”
Step 2 — Define the constraints
Write down:
Latency:
P95 target
Cost:
Cost per successful outcome
Failure:
Known failure modes + recovery expectations
Evaluation:
How success will be measured
Enter fullscreen mode Exit fullscreen mode
Also consider:
- security
- authorization
- compliance
- reliability
- availability
- data freshness
Step 3 — Classify the problem
Ask:
Is this deterministic, probabilistic, or mixed?
A useful pattern is:
Deterministic → workflow
Probabilistic → agent
Mixed → workflow + agent
Enter fullscreen mode Exit fullscreen mode
Many real-world systems fall into the third category.
Step 4 — Decide where autonomy is actually useful
Don’t make the whole system autonomous.
Make the right parts autonomous.
Step 5 — Design the failure paths
Before implementing the happy path, define:
Tool timeout
Invalid output
Low confidence
Stale context
Unauthorized action
Loop detected
Provider unavailable
Missing evidence
Enter fullscreen mode Exit fullscreen mode
Step 6 — Define evaluation before implementation
If you cannot clearly describe how you’ll measure whether the system works, you probably aren’t ready to choose the architecture.
Step 7 — Select technologies
Only now ask:
- Which model?
- LangGraph?
- MCP?
- RAG framework?
- Vector database?
- Agent runtime?
- Workflow engine?
At this stage, technology selection becomes much easier because the problem has already constrained the solution space.
15. A useful mental model
The simplest way I think about this is:
PRODUCT OUTCOME
|
v
CONSTRAINTS
┌─────┬─────┬─────┬─────┐
| | | | |
L C F E Security
| | | | |
└─────┴─────┴─────┴─────┘
|
v
ARCHITECTURE
|
v
IMPLEMENTATION
|
┌────────────┼────────────┐
v v v
Models Tools Frameworks
Enter fullscreen mode Exit fullscreen mode
Compare that with the common approach:
Models
↓
Framework
↓
MCP
↓
Agents
↓
Architecture
↓
"Now let's figure out the constraints."
Enter fullscreen mode Exit fullscreen mode
That second approach is where a lot of unnecessary complexity starts.
16. The bigger lesson
The most important lesson isn’t that everyone should use LCFE.
It is the order of thinking.
Agentic AI gives engineers an enormous amount of flexibility.
We can create:
- autonomous agents
- multi-agent systems
- tool-calling loops
- memory systems
- retrieval pipelines
- planners
- critics
- reflection loops
- MCP integrations
But technical possibility is not the same as product value.
A sophisticated architecture can still be the wrong architecture.
The best production system might contain fewer agents, fewer model calls and fewer tools than the original prototype.
And that is not a failure.
It is often a sign that the architecture has finally started responding to the product instead of the technology.
Conclusion
When designing Agentic AI systems, the temptation is to ask:
“What can we build with these tools?”
I think a better question is:
“What is the simplest system that can reliably achieve the required outcome within our constraints?”
Start with:
Outcome → Constraints → Architecture → Tools
Think about:
Latency → Cost → Failure → Evaluation
Then decide where agents, models, MCP, RAG, memory and orchestration actually belong.
Because a production-grade agentic system isn’t the one with the most components.
It is the one where every component has earned its place.
And sometimes, the best agentic architecture is the one where you discover that you don’t need an agent for half the problem.