Stop Designing Agentic AI Systems Backwards: Start With Constraints, Then Choose the Architecture

작성자

카테고리:

← 피드로
DEV Community · Himanjan · 2026-08-27 개발(SW)

There is a pattern I keep seeing when designing Agentic AI systems.

We start by asking:

  • Which LLM should we use?
  • Should we use LangGraph?
  • Where can MCP fit?
  • Should we build multiple agents?
  • Do we need RAG?
  • Should we add memory?
  • Should every step be handled by an autonomous agent?

These are useful questions.

But they are often asked too early.

The result can be an architecture that is technically impressive but operationally difficult, expensive, slow, and surprisingly hard to trust.

A better approach is to reverse the order:

Start with the product outcome. Define the constraints. Then design the architecture. Choose the tools last.

I have found a useful way to structure those constraints around four dimensions:

LCFE

L — Latency
C — Cost
F — Failure
E — Evaluation

This is not a framework that says every agentic system must look the same. It is a way of forcing architectural decisions to start with the realities of the product rather than the capabilities of the technology.

In this article, I’ll walk through a concrete incident-automation example and show how starting with constraints can completely change the architecture.

1. The “backwards” way of designing an agent

Imagine we want to build an AI Incident Resolution Assistant for an engineering organization.

The goal sounds straightforward:

When a production incident is raised, the AI should investigate the incident, gather context, identify the likely cause, recommend or perform remediation, and verify the result.

Now imagine the team starts with the technology.

The first architecture might look like this:

                    User / Incident
                           |
                           v
                    ┌──────────────┐
                    │ Triage Agent │
                    └──────┬───────┘
                           |
                           v
                   ┌────────────────┐
                   │ Research Agent │
                   └───────┬────────┘
                           |
            ┌──────────────┼──────────────┐
            v              v              v
        Logs Agent     Metrics Agent   Knowledge Agent
            |              |              |
            └──────────────┼──────────────┘
                           |
                           v
                  ┌─────────────────┐
                  │ Remediation     │
                  │ Agent           │
                  └────────┬────────┘
                           |
                           v
                  ┌─────────────────┐
                  │ Validation Agent│
                  └────────┬────────┘
                           |
                           v
                       Resolution

Enter fullscreen mode Exit fullscreen mode

It looks sophisticated.

We have agents.

We have tools.

We have MCP.

We have orchestration.

We have reasoning.

We have memory.

We have autonomous remediation.

Technically, there is nothing wrong with building this.

The problem is that we don’t yet know whether the product needs it.

We have designed the solution before defining the constraints.

2. Start with the product outcome

Before selecting an LLM or orchestration framework, define the actual outcome.

For our incident system, maybe the product requirement is:

Resolve common production incidents within 10 minutes, while reducing manual engineer effort and keeping high-risk actions under human approval.

That statement is much more useful than:

Build a multi-agent incident-resolution system.

Now we can ask the questions that actually drive architecture.

Latency

What response time is acceptable?

For example:

  • P95 time to initial diagnosis: < 30 seconds
  • P95 time to recommended remediation: < 90 seconds
  • P95 time to automated resolution for eligible incidents: < 5 minutes

Cost

What can a successful resolution cost?

For example:

  • Average cost per incident: < £0.20
  • Expensive frontier-model calls reserved for complex cases
  • Avoid unnecessary agent loops

Failure

What happens when something goes wrong?

For example:

  • Logs API unavailable
  • Metrics delayed
  • Kubernetes API timeout
  • Tool returns malformed data
  • Model generates an invalid action
  • Diagnosis is uncertain
  • Remediation fails
  • Incident remains unresolved

Evaluation

How do we know the system is actually useful?

For example:

  • Correct incident classification
  • Correct root-cause hypothesis
  • Correct tool selection
  • Safe remediation
  • Successful verification
  • Reduction in mean time to resolution

Now we have something much more valuable than a technology stack.

We have engineering constraints.

3. The first architectural surprise: not everything needs an agent

This is where the design can change dramatically.

Suppose historical incident data shows:

Incident Type Percentage Typical Resolution High CPU 25% Scale service Pod crash loop 20% Inspect logs + restart Certificate expiry 10% Renew certificate Database connection pool 15% Restart / tune service Deployment regression 10% Roll back Unknown / complex 20% Deep investigation

Suddenly, the idea of having every incident go through a fully autonomous multi-agent workflow looks questionable.

For many incident types, the process is already known.

For example:

Incident
   |
   v
Classify
   |
   +---- Known pattern? ---- Yes ----> Deterministic workflow
   |
   No
   |
   v
Agentic investigation

Enter fullscreen mode Exit fullscreen mode

This is a major architectural insight:

An agent should exist where reasoning variability exists.

It should not exist simply because the technology makes it possible.

4. The new architecture

After applying the constraints, our architecture might become:

                    Incident Event
                           |
                           v
                  ┌─────────────────┐
                  │ Fast Classifier  │
                  └────────┬────────┘
                           |
               ┌───────────┴────────────┐
               |                        |
               v                        v
       Known Incident              Complex / Unknown
               |                        |
               v                        v
      Deterministic Workflow      Agent Runtime
               |                        |
               |              ┌─────────┼──────────┐
               |              v         v          v
               |           Logs Tool Metrics Tool KB/Search
               |              |         |          |
               |              └─────────┼──────────┘
               |                        |
               |                        v
               |                 Diagnosis
               |                        |
               |                        v
               |               Policy / Guardrails
               |                        |
               |                ┌───────┴────────┐
               |                |                |
               |              Safe            High Risk
               |                |                |
               |                v                v
               |          Auto Remediation   Human Approval
               |                |                |
               └────────────────┴────────────────┘
                                |
                                v
                           Verification
                                |
                                v
                         Outcome / Escalation

Enter fullscreen mode Exit fullscreen mode

Notice what disappeared.

We may no longer need:

  • five autonomous agents
  • an LLM call for every incident
  • an agent deciding every deterministic operation
  • unnecessary memory
  • multiple reasoning loops

The system is actually simpler.

But it is also more production-oriented.

5. L — Latency: design backwards from the P95 budget

Latency is one of the easiest constraints to ignore during an AI prototype.

A demo can take 45 seconds and still look impressive.

A production incident-response system may not have that luxury.

Suppose our requirement is:

P95 time to diagnosis < 30 seconds

Now work backwards.

A possible latency budget:

End-to-end P95 = 30s

Classification       2s
Retrieval            4s
Tool calls          10s
LLM reasoning       10s
Orchestration        2s
Buffer                2s
--------------------------
Total                30s

Enter fullscreen mode Exit fullscreen mode

Now imagine someone proposes adding a reranker.

The reranker adds another 3 seconds.

We should not automatically say:

“Reranking improves retrieval, so let’s add it.”

Instead ask:

Does reranking produce enough evaluation improvement to justify 3 seconds of our latency budget?

Suppose evaluation shows:

Without reranking:
Diagnosis accuracy = 91%

With reranking:
Diagnosis accuracy = 91.8%

Enter fullscreen mode Exit fullscreen mode

An extra 3 seconds may not be worth it.

But if it changes:

91% → 97%

Enter fullscreen mode Exit fullscreen mode

the architectural decision becomes much easier.

This is the important mindset:

Every piece of complexity has to earn its place in the latency budget.

The same principle applies to:

  • additional model calls
  • reflection loops
  • reranking
  • extra retrieval stages
  • agent-to-agent communication
  • long context windows
  • MCP calls
  • retries

6. C — Cost: optimize cost per successful outcome

Agentic systems can become expensive surprisingly quickly.

Imagine a naïve incident workflow:

Triage LLM call
      +
Research LLM call
      +
Reasoning LLM call
      +
Tool-selection LLM call
      +
Remediation LLM call
      +
Validation LLM call
      +
Retries

Enter fullscreen mode Exit fullscreen mode

Now multiply that by thousands of incidents.

The important metric is not:

“How cheap is our LLM call?”

It is:

“How much does it cost us to successfully resolve an incident?”

Consider two architectures.

Architecture A

Every incident
      ↓
Large model
      ↓
Multiple agent loops
      ↓
Multiple tools
      ↓
Expensive reasoning

Enter fullscreen mode Exit fullscreen mode

Average cost:

£0.80 / incident

Architecture B

Classifier
    ↓
Known pattern?
    ↓
Yes → deterministic workflow

No
    ↓
Stronger model
    ↓
Agentic investigation

Enter fullscreen mode Exit fullscreen mode

Average cost:

£0.18 / incident

Suppose both achieve similar overall resolution rates.

Architecture B is clearly more attractive.

This naturally leads to a tiered model strategy:

              Incoming incident
                     |
                     v
              Small / cheap model
                     |
            ┌────────┴────────┐
            |                 |
         Simple            Complex
            |                 |
            v                 v
      Workflow model      Strong model
                              |
                              v
                        Deep reasoning

Enter fullscreen mode Exit fullscreen mode

Use the expensive reasoning capability where it creates measurable value.

Not everywhere.

7. F — Failure: assume everything will fail

This is where production systems differ from demos.

A demo assumes:

Input
  ↓
LLM
  ↓
Tool
  ↓
Success

Enter fullscreen mode Exit fullscreen mode

A production system assumes:

Input
  ↓
LLM
  ↓
Tool
  ↓
Timeout
  ↓
Retry
  ↓
Malformed response
  ↓
Validation failure
  ↓
Fallback
  ↓
Human escalation

Enter fullscreen mode Exit fullscreen mode

For an incident agent, failure scenarios might include:

Tool failure

The metrics API is unavailable.

The agent should not invent metrics.

Retrieval failure

The knowledge base returns nothing.

The agent should explicitly recognize missing evidence.

Invalid tool call

The model produces:

{
  "service": "payment-api",
  "replicas": "many"
}

Enter fullscreen mode Exit fullscreen mode

But the tool requires an integer.

The system must validate the tool request before execution.

Dangerous action

The agent wants to execute:

DROP DATABASE

Enter fullscreen mode Exit fullscreen mode

The system should never treat the LLM’s intention as permission.

The permission boundary belongs to the runtime.

Endless reasoning

An agent can continue:

think → tool → observe → think → tool → observe

Enter fullscreen mode Exit fullscreen mode

without making progress.

So we need:

  • maximum iterations
  • time budgets
  • retry limits
  • tool timeouts
  • action policies
  • schema validation
  • circuit breakers
  • human escalation

This leads to a principle that is particularly important for agentic systems:

The model proposes actions. The runtime decides which actions are allowed.

The model should not become the security boundary.

8. E — Evaluation: measure the task, not just the answer

This is perhaps the most important shift when moving from an LLM application to an agent.

For a simple chatbot, evaluation might look like:

Was the answer helpful?

For an incident agent, that is not enough.

We need to evaluate the trajectory and outcome.

For example:

Incident
   ↓
Correct classification?
   ↓
Relevant evidence retrieved?
   ↓
Correct tools selected?
   ↓
Reasoning supported by evidence?
   ↓
Safe action selected?
   ↓
Action executed successfully?
   ↓
Incident actually resolved?
   ↓
Outcome correctly verified?

Enter fullscreen mode Exit fullscreen mode

A response such as:

“The issue appears to be high CPU.”

may sound intelligent.

But if the service remains unhealthy, the agent hasn’t completed the task.

The real evaluation might therefore include:

Task outcome

Did the incident get resolved?

Tool-use accuracy

Did the agent choose the appropriate tools?

Evidence quality

Did its diagnosis rely on relevant evidence?

Safety

Did it avoid disallowed actions?

Efficiency

How many tool calls and model calls were needed?

Reliability

Does the same scenario succeed repeatedly?

This is why agent evaluation needs to go beyond traditional prompt evaluation.

A useful production evaluation strategy has two layers.

Pre-release evaluation

Before deployment:

Benchmark scenarios
       ↓
Tool-use evaluation
       ↓
Trajectory evaluation
       ↓
Safety tests
       ↓
Regression suite
       ↓
Release gate

Enter fullscreen mode Exit fullscreen mode

Post-release evaluation

After deployment:

Real traffic
    ↓
Tracing
    ↓
Success / failure signals
    ↓
Production evaluation
    ↓
New failure cases
    ↓
Added to benchmark suite
    ↓
Next release

Enter fullscreen mode Exit fullscreen mode

This creates a feedback loop:

Production failures become future evaluation cases.

That is one of the most important pieces of a mature agentic architecture.

9. The role of MCP changes when constraints come first

MCP is a good example of why tool-first thinking can be dangerous.

If we start with:

“We have MCP, where can we use it?”

we are already designing backwards.

Instead ask:

“Which capabilities does the product need, and what is the safest and most efficient interface for those capabilities?”

Maybe MCP is a good fit for:

  • standardized access to operational tools
  • reusable tool interfaces
  • connecting agents to external systems
  • reducing custom integration work

But that doesn’t mean every operation needs to pass through MCP.

A deterministic internal workflow may call a service API directly.

A highly sensitive operation may need a tightly controlled internal execution service.

The correct question is not:

“Can MCP do this?”

It is:

“What interface best satisfies the product’s latency, cost, failure and security constraints?”

Then choose MCP when it fits.

10. The role of LangGraph also changes

The same logic applies to orchestration frameworks.

A graph framework can be extremely useful when you need:

  • stateful execution
  • retries
  • branching
  • checkpoints
  • human-in-the-loop
  • durable workflows

But using a graph framework does not automatically make a system more production-ready.

For our incident system, the architecture might contain:

Deterministic workflow
        +
Agent runtime
        +
Policy layer
        +
Tool layer
        +
Evaluation layer
        +
Observability

Enter fullscreen mode Exit fullscreen mode

LangGraph might be part of the implementation.

Or it might not be.

The architecture should determine the tool choice.

Not the other way around.

11. Where memory actually belongs

Memory is another feature that is often added because an agent “should remember things.”

But what should it remember?

For an incident platform, maybe we need:

Short-term state

Current incident context:

Incident ID
Service
Recent logs
Metrics
Recent deployment
Actions already attempted
Current hypothesis

Enter fullscreen mode Exit fullscreen mode

Long-term knowledge

Reusable information:

Known failure patterns
Runbooks
Service documentation
Historical incidents
Operational policies

Enter fullscreen mode Exit fullscreen mode

These are not necessarily the same thing.

And some information should not be treated as permanent truth simply because a previous agent generated it.

That is a failure-mode question:

How stale can memory become before it becomes dangerous?

Again, the constraint drives the architecture.

12. Human-in-the-loop should be policy-driven

A common mistake is to treat “human approval” as one generic feature.

Instead, classify actions by risk.

For example:

LOW RISK
Read logs
Read metrics
Read deployment status
        ↓
Autonomous

MEDIUM RISK
Restart pod
Scale service
Clear cache
        ↓
Policy-based

HIGH RISK
Rollback production
Change configuration
Database operation
        ↓
Human approval

Enter fullscreen mode Exit fullscreen mode

Now autonomy becomes a policy decision, not an LLM personality trait.

This is a much more reliable way to build agentic systems.

13. The final architecture is often simpler

Notice what happened.

We started with:

Multi-Agent Everything

Enter fullscreen mode Exit fullscreen mode

and ended with:

                      Incident
                          |
                          v
                    Classification
                          |
              ┌───────────┴───────────┐
              |                       |
        Known pattern             Complex case
              |                       |
              v                       v
      Deterministic flow        Agent runtime
                                      |
                              Tools + Retrieval
                                      |
                                  Reasoning
                                      |
                              Policy / Guardrail
                                      |
                         ┌────────────┴────────────┐
                         |                         |
                   Auto-action              Human approval
                         |                         |
                         └────────────┬────────────┘
                                      |
                                  Verification
                                      |
                                      v
                                   Outcome

Enter fullscreen mode Exit fullscreen mode

This architecture has fewer moving parts.

But it has something more important:

Each part exists for a reason.

14. A practical architecture decision sequence

When starting a new Agentic AI system, I now prefer to work through this sequence.

Step 1 — Define the outcome

What business or user outcome are we trying to improve?

Not:

“Build an AI agent.”

But:

“Reduce incident resolution time by 40%.”

Step 2 — Define the constraints

Write down:

Latency:
P95 target

Cost:
Cost per successful outcome

Failure:
Known failure modes + recovery expectations

Evaluation:
How success will be measured

Enter fullscreen mode Exit fullscreen mode

Also consider:

  • security
  • authorization
  • compliance
  • reliability
  • availability
  • data freshness

Step 3 — Classify the problem

Ask:

Is this deterministic, probabilistic, or mixed?

A useful pattern is:

Deterministic → workflow

Probabilistic → agent

Mixed → workflow + agent

Enter fullscreen mode Exit fullscreen mode

Many real-world systems fall into the third category.

Step 4 — Decide where autonomy is actually useful

Don’t make the whole system autonomous.

Make the right parts autonomous.

Step 5 — Design the failure paths

Before implementing the happy path, define:

Tool timeout
Invalid output
Low confidence
Stale context
Unauthorized action
Loop detected
Provider unavailable
Missing evidence

Enter fullscreen mode Exit fullscreen mode

Step 6 — Define evaluation before implementation

If you cannot clearly describe how you’ll measure whether the system works, you probably aren’t ready to choose the architecture.

Step 7 — Select technologies

Only now ask:

  • Which model?
  • LangGraph?
  • MCP?
  • RAG framework?
  • Vector database?
  • Agent runtime?
  • Workflow engine?

At this stage, technology selection becomes much easier because the problem has already constrained the solution space.

15. A useful mental model

The simplest way I think about this is:

             PRODUCT OUTCOME
                    |
                    v
              CONSTRAINTS
          ┌─────┬─────┬─────┬─────┐
          |     |     |     |     |
          L     C     F     E   Security
          |     |     |     |     |
          └─────┴─────┴─────┴─────┘
                    |
                    v
                ARCHITECTURE
                    |
                    v
             IMPLEMENTATION
                    |
       ┌────────────┼────────────┐
       v            v            v
     Models        Tools      Frameworks

Enter fullscreen mode Exit fullscreen mode

Compare that with the common approach:

Models
  ↓
Framework
  ↓
MCP
  ↓
Agents
  ↓
Architecture
  ↓
"Now let's figure out the constraints."

Enter fullscreen mode Exit fullscreen mode

That second approach is where a lot of unnecessary complexity starts.

16. The bigger lesson

The most important lesson isn’t that everyone should use LCFE.

It is the order of thinking.

Agentic AI gives engineers an enormous amount of flexibility.

We can create:

  • autonomous agents
  • multi-agent systems
  • tool-calling loops
  • memory systems
  • retrieval pipelines
  • planners
  • critics
  • reflection loops
  • MCP integrations

But technical possibility is not the same as product value.

A sophisticated architecture can still be the wrong architecture.

The best production system might contain fewer agents, fewer model calls and fewer tools than the original prototype.

And that is not a failure.

It is often a sign that the architecture has finally started responding to the product instead of the technology.

Conclusion

When designing Agentic AI systems, the temptation is to ask:

“What can we build with these tools?”

I think a better question is:

“What is the simplest system that can reliably achieve the required outcome within our constraints?”

Start with:

Outcome → Constraints → Architecture → Tools

Think about:

Latency → Cost → Failure → Evaluation

Then decide where agents, models, MCP, RAG, memory and orchestration actually belong.

Because a production-grade agentic system isn’t the one with the most components.

It is the one where every component has earned its place.

And sometimes, the best agentic architecture is the one where you discover that you don’t need an agent for half the problem.

원문에서 계속 ↗