Production-grade agentic systems with LangGraph, tool use & evaluation

This page describes how LMXAI designs and builds agentic systems for production: explicit LangGraph control flow, tool use behind typed schemas, retrieval with access control, evaluation before release and tracing in operation. It places that approach in the research on agent reliability, where benchmarks show that current models still fail on long-horizon and multi-turn tasks, and explains when an agent is the right design and what an engagement delivers.

Keywords agentic systems · LangGraph · tool use · evaluation · observability · MCP

LMXAI is an EU-based AI engineering studio that designs and builds production-grade agentic systems with LangGraph, controlled tool use, retrieval, evaluation and observability. The focus is not on autonomous demos, but on bounded workflows that can be tested, monitored and operated safely in real business environments.

Based in Leiden, the Netherlands, LMXAI works across architecture, backend engineering, model integration, RAG, inference and product delivery, so the agent layer is designed together with the systems it must actually use.

The modern agent loop comes from work showing that a language model does better on multi-step tasks when it interleaves reasoning with actions against external sources, so that it can plan, handle exceptions and gather missing information^[1]. A systematic survey organises the systems built on that idea around a common construction framework and reviews how they are evaluated^[2].

Evaluation results explain why LMXAI bounds autonomy. AgentBench, across eight environments, identifies poor long-term reasoning, decision-making and instruction following as the main obstacles to usable agents^[3]. On τ-bench, state-of-the-art function-calling agents completed fewer than half of the tasks and were inconsistent across repeated trials^[4]. The Berkeley Function Calling Leaderboard, which checks calls structurally, finds that models handle single-turn calls well but still struggle with memory, dynamic decisions and long-horizon tasks^[5].

Tool access also widens the attack surface: instructions hidden in retrieved content can steer a model and trigger tool calls without the attacker ever addressing the system directly^[6]. These findings are the reason for explicit graphs, schema-checked tool calls, approval gates and evaluation before release.

A production agent needs more than an LLM connected to tools. It needs explicit control over what the model can decide, what it can access, how failures are handled and how quality is measured.

LMXAI typically treats an agentic system as six connected layers:

1. Orchestration — state, routing, retries, branching and deterministic steps.
2. Model layer — one or more models selected according to task quality, latency, privacy and cost.
3. Tools — internal APIs, search, databases, document services, MCP servers and business actions.
4. Retrieval — relevant internal or external knowledge with source attribution and access controls.
5. Evaluation — repeatable tests for tool selection, task completion, groundedness and failure cases.
6. Observability & governance — traces, logs, permissions, approvals and operational metrics.

LangGraph is useful because it lets us make state and control flow explicit rather than hiding the entire workflow inside a long prompt.

LMXAI uses LangGraph where a workflow benefits from model-driven decisions but still requires engineering control.

Examples include:

• routing a request to the correct tool or knowledge source;
• decomposing a complex task into bounded steps;
• retrieving additional evidence when the first result is weak;
• requiring approval before a consequential action;
• validating tool output before it reaches the next node;
• escalating to a human when confidence or policy conditions are not met.

The goal is not maximum autonomy. The goal is the minimum autonomy required to solve the task reliably.

Tool calling is where an agent starts interacting with real systems, so tool design matters as much as prompt design.

LMXAI designs tools with:

• narrow, explicit schemas;
• validated inputs and outputs;
• scoped permissions;
• idempotency where actions may be retried;
• predictable error contracts;
• auditability of tool calls;
• timeouts and fallback behavior;
• approval gates for high-impact actions.

Tools can be exposed through direct APIs, internal service wrappers or MCP when interoperability is useful. The model should never receive broader access than the task requires.

Agentic systems often need to choose between multiple data sources rather than query one fixed vector index. LMXAI can combine:

• dense and sparse retrieval;
• hybrid search;
• Elasticsearch or other enterprise search infrastructure;
• metadata- and permission-aware retrieval;
• query rewriting and decomposition;
• multi-index routing;
• source verification before response generation.

This turns retrieval into a controlled relevance layer instead of a generic “search tool”.

An agent is not ready because a few manually tested conversations worked. LMXAI builds evaluation around the actual decisions the system must make.

Depending on the use case, evaluation can include:

• task completion rate;
• correct tool selection;
• tool argument accuracy;
• retrieval recall and groundedness;
• citation correctness;
• refusal and escalation behavior;
• loop and retry rates;
• latency and token cost;
• regression tests across model or prompt changes.

For tool-use systems, LMXAI also works with benchmark-style evaluation patterns such as BFCL-derived function-calling tests, combined with domain-specific test sets that reflect the client's real workflows.

Production agents need traces that explain what happened across a workflow. LMXAI can instrument agent runs with technologies such as OpenTelemetry and Phoenix so teams can inspect model calls, tool calls, retrieval steps, latency, failures and token usage.

This is especially important when agents operate in regulated or high-stakes domains. A team must be able to distinguish a model error from a retrieval failure, tool failure, permission problem or orchestration bug.

A typical system may look like:

User or application → API → LangGraph workflow → model/router → retrieval + tools → validation/approval → response or business action

Across the workflow sit identity, permissions, traces, evaluation and policy controls.

The final architecture can run with frontier APIs, self-hosted open models or a hybrid setup. LMXAI is model-agnostic and also builds sovereign inference stacks with vLLM and Kubernetes when data residency or infrastructure control requires it.

LMXAI has built production systems across enterprise, education and allied health, including:

• enterprise AI workspaces with sovereign inference;
• LangGraph-based agentic workflows;
• MCP and custom tool integrations;
• RAG systems with enterprise retrieval;
• multimodal tool-use fine-tuning;
• AI gateways with streaming, isolation and observability.

The engineering stack includes LangGraph, LangChain, MCP, FastAPI, Elasticsearch, vLLM, Kubernetes, OpenTelemetry and Phoenix.

Explore LMXAI's agentic capabilities

Agentic architecture is useful when a task requires dynamic decisions between tools, sources or workflow paths. It is often unnecessary when a deterministic service, fixed pipeline or normal search endpoint can solve the problem more reliably.

LMXAI therefore starts with the business workflow and asks where model reasoning actually adds value. Autonomous complexity is added only when it earns its operational cost.

A production engagement can cover:

• workflow and architecture design;
• LangGraph implementation;
• tool and MCP integration;
• retrieval architecture;
• model selection and routing;
• evaluation datasets and release gates;
• observability and tracing;
• permissions and human approval flows;
• deployment, load testing and production hardening;
• documentation and handover.

• Agentic capabilities project
• Production RAG and agents
• Sovereign vLLM deployment
• Fractional AI CTO

LMXAI can take an agentic product from workflow design to production architecture, implementation, evaluation and deployment under one engineering owner.

Discuss your agentic system with LMXAI