Buyer and engineering answers on production RAG, tool use, evaluation, permissions and agent safety.
RAG retrieves relevant information and adds it to the model context before generation. An agent adds decision-making and tool use: it can choose actions, call APIs, search systems, update records or execute a multi-step workflow. Many production systems use both.
Common causes include weak document parsing, stale indexes, poor chunking, missing access control, low-quality retrieval, no evaluation set and no observability. A demo can look good on hand-picked examples while failing on real user queries.
Measure retrieval quality separately from answer quality. Useful signals include recall or hit rate on relevant evidence, citation correctness, groundedness, answer completeness, latency, cost and failure rate. Evaluation should use representative domain questions rather than generic prompts.
Use tools when the answer depends on fresh, authoritative or private data, or when the system must take an action. Examples include searching an internal database, checking a live order, creating a ticket or retrieving a current policy document.
Use constrained schemas, narrow tool descriptions, deterministic validation, retries where appropriate and an evaluation set that measures tool selection and argument correctness. Tool reliability should be tested explicitly rather than inferred from general language-model benchmarks.
No. It is one orchestration option. The important design properties are explicit state, controllable transitions, retry and failure handling, observability and testability. LangGraph is useful when workflows contain branching, loops, tools and long-running state.
The agent should never have broader permissions than the user or workload it represents. Tool access should be scoped by identity, role, tenant and action. Retrieval should also enforce document-level permissions before content reaches the model.
Record request identity, model and version, retrieved evidence, tool calls, tool results, important state transitions, latency and final outcome. Sensitive payloads may need redaction or controlled retention, but the system should preserve enough evidence for debugging and audit.
Use least-privilege tools, allow-lists, validation, transaction boundaries and approval gates for high-impact actions. The model should propose actions inside a controlled system rather than receive unrestricted access to production infrastructure.
Self-hosting is attractive when data sovereignty, privacy, predictable cost, model control or hyperscaler independence matter. It also allows the organisation to keep retrieval, model inference, logs and tool integrations inside one controlled environment.
Next step. If you want this to ship, LMXAI scopes the integration as a system — not a workshop series.
