Building AI Agents

LangChain, LangGraph, LlamaIndex, MCP, memory, workflows, and AgentOps.

LangChain

Overview

LangChain composes LLM calls, tools, retrievers, and parsers into pipelines—common as building blocks inside larger orchestrators.

Technical Deep Dive

LCEL chains runnables with |. Tools expose JSON schemas; retrievers return Document objects with metadata. Legacy AgentExecutor loops are being replaced by graph runtimes for production control.

Modules: langchain-core (interfaces), langchain-community (integrations). Version-pin everything—public APIs shift between minor releases.

Practical Use Case

Weekend POC: load PDFs → chunk → embed → Chroma → Q&A chain. Year 2: keep only document loaders/retrievers; orchestrate agent logic in LangGraph for checkpoints and human sign-off.

Advantages

  • Largest integration catalog
  • Fast demos
  • Composable LCEL

Limitations

  • Opaque executors
  • Breaking changes
  • Not ideal for cyclic workflows

Best Practices

  • Pin versions
  • Avoid monolithic chains in prod
  • Adopt LangGraph when tools > 5

LangGraph

Overview

LangGraph expresses agent logic as a state machine: nodes mutate shared state; edges route; checkpoints survive restarts.

Technical Deep Dive

Define state with TypedDict + reducers like add_messages. Conditional edges implement ReAct branching. PostgresSaver checkpoints enable interrupt() before emails/payments and resume after crashes.

Subgraphs package multi-agent teams; platform API serves graphs per thread_id.

Practical Use Case

Accounts payable graph pauses at human_approval before ERP posting; weekend batch jobs resume from checkpoint without re-running OCR.

Best Practices

  • Small state objects
  • recursion_limit
  • interrupt() on writes

LlamaIndex

Overview

LlamaIndex optimizes the path from raw files to indexed, queryable knowledge for RAG-centric applications.

Technical Deep Dive

VectorStoreIndex, SummaryIndex, KG indices. Ingestion pipelines chain splitters + metadata extraction. RouterQueryEngine selects among corpora. Agents wrap indices as QueryEngineTool.

Practical Use Case

Compliance team queries 3 indices (regulation, internal policy, enforcement actions); answers must cite paragraph IDs—LlamaIndex metadata supports audit.

Advantages

  • Connectors
  • Citation-friendly nodes
  • HyDE / decomposition

Limitations

  • Less flexible agent graphs
  • Heavier dependencies

Model Context Protocol (MCP)

Overview

MCP is JSON-RPC between an AI host and tool servers—standardizing discovery, invocation, and resource reads.

Technical Deep Dive

Handshake → capability negotiation → tools/call with schema-validated args. Secrets live in server env, not prompts. Compose postgres-mcp + github-mcp concurrently.

Practical Use Case

Internal developer platform registers 6 MCP servers once; product teams stop building custom IDE plugins per system.

Advantages

  • One integration pattern
  • Isolation of credentials

Limitations

  • Process management
  • Immature governance tooling

Reference Implementation

# Register servers in host config; spawn stdio processes

Function Calling

Overview

The model returns machine-readable tool invocations; your runtime is the source of truth for execution.

Technical Deep Dive

Attach OpenAPI-like schemas to chat completion requests. Validate arguments with Pydantic/JSON Schema before IO. Feed results back as tool-role messages. Enable parallel tool calls only when business logic is commutative.

Practical Use Case

Scheduling assistant calls find_slots and get_attendees in parallel, then serial create_event after user taps Confirm.

Advantages

  • Reliable parsing
  • Vendor support

Limitations

  • Schema maintenance
  • Hallucinated parameters

Tool Calling Patterns

Overview

Enterprise tool layers add authorization, sandboxes, quotas, and compensating actions around raw function calling.

Technical Deep Dive

OPA policies on tool + user role. gVisor for code execution. Idempotency-Key header on writes. Saga orchestrator rolls back prior steps on failure.

Practical Use Case

Insurance FNOL agent: photo upload (read) auto; total-loss payout (write) needs adjuster mobile approval within 4 hours.

Best Practices

  • Structured errors back to model
  • Per-tenant rate limits

Agent Memory Design

Overview

Memory is not one vector DB—it is a policy for what to remember, for how long, and with what evidence.

Technical Deep Dive

Working = messages + rolling summary. Episodic = object store of sessions. Semantic = extracted triples/facts with timestamps. Procedural = store successful tool traces as few-shot exemplars.

Practical Use Case

B2B SaaS agent remembers customer's integration stack and open tickets across logins; wipes all tiers on account deletion request.

Advantages

  • Continuity
  • Less repetition

Limitations

  • Wrong memories
  • GDPR complexity

Workflow Design

Overview

Design the state machine on paper before choosing LangGraph vs. CrewAI—clarifies HITL gates and failure handling.

Technical Deep Dive

Document triggers, max iterations, terminal states, and which steps are deterministic code vs. LLM. Attach trace span names to each edge for ops dashboards.

Practical Use Case

Vendor onboarding: collect docs (OCR) → validate tax ID (API) → risk score (ML) → contract draft (LLM) → legal queue (human)—six nodes, six owners.

Best Practices

  • Whiteboard first
  • Cap loops
  • Separate read/write paths

Observability for Agents

Overview

You cannot debug agents with average latency alone—you need per-trajectory forensics.

Technical Deep Dive

Instrument LLM, retrieval, and tool spans with shared trace_id. Export to Langfuse/Datadog. Compare failed vs. successful trajectories side-by-side.

Practical Use Case

Discovered agent always called deprecated Salesforce API—trace showed tool node v3 still wired; fixed routing flag, not prompt.

Production Monitoring

Overview

Treat agents like payment services: SLOs, error budgets, synthetic probes.

Technical Deep Dive

Metrics: task_success, cost_usd, steps_count, human_handoff_rate. Synthetics every 10 min. Page when success < 99% for 15 min on payment agent.

Practical Use Case

Tax season: double synthetic frequency; freeze graph releases except hotfixes when escalations spike.

Agent Evaluation

Overview

Ship agents with datasets that assert both final answers and the path taken.

Technical Deep Dive

Store golden trajectories from best human operators. Score tool-sequence match + rubric on output. Block deploy if regression on either dimension.

Practical Use Case

Data entry agent must call validate_EIN before create_vendor—CI fails if order inverts on 12 test scenarios.

AgentOps

Overview

AgentOps is how platform teams run weekly quality reviews, controlled rollouts, and incident response for graphs—not single model endpoints.

Technical Deep Dive

Tag traces with {graph_version, prompt_hash, model_id}. Automate eval on PR. Kill-switch feature flag. Post-incident: add failing trace to regression pack within 48h.

Practical Use Case

Tuesday production graph release only if Monday nightly eval shows ≥ baseline on 400 tasks; rollback button tested monthly.

Advantages

  • Controlled change
  • Compliance evidence

Limitations

  • Process overhead