What Was Built: The Production Support Agent Architecture
A production customer support AI agent for a B2B SaaS company typically handles three distinct query types: self-service resolutions where the agent can answer fully from the knowledge base, guided resolutions where the agent retrieves a procedural answer that requires the user to take action, and escalation cases where the query requires human judgment, account-specific context, or is outside the agent's trained domain. The architecture that serves all three patterns reliably consists of an intent classification layer that categorises the incoming query before retrieval begins, a retrieval-augmented generation layer that searches a chunked and embedded knowledge base to find relevant source documents, an LLM synthesis layer that generates a response grounded in the retrieved documents with explicit citation links, a confidence scoring mechanism that determines whether to present the response directly, request clarification, or escalate to a live agent, and a full conversation memory context window that maintains thread history so multi-turn diagnostic conversations remain coherent. The knowledge base is built from the company's existing documentation: Intercom articles, Zendesk help centre pages, internal runbooks, product changelogs, and known issue logs. Each document is chunked at the paragraph level, embedded using a high-dimensional text embedding model, and stored in a vector database such as Pinecone or pgvector. Retrieval happens at query time using semantic similarity search, not keyword matching, which means the agent can answer questions phrased in customer language even when the documentation uses technical product terminology.
Why This Architecture Works in Production
The RAG architecture is specifically chosen over a fine-tuned model approach because it separates the knowledge update cycle from the model update cycle. When a product ships a new feature, the relevant documentation is added to the knowledge base and immediately available to the agent without a model retrain. For B2B SaaS companies shipping features on a weekly cycle, this is critical. Fine-tuned models that encode product knowledge in their weights become stale the moment a product update ships, and retraining cycles introduce deployment risk. The intent classification layer upstream of retrieval matters because it prevents the retrieval step from being polluted by off-topic queries. Without intent classification, a customer asking about a billing dispute might retrieve documentation about billing settings rather than triggering the correct escalation to the accounts team. Separating intent from retrieval also allows the system to handle queries that should never touch the knowledge base at all, such as account-specific questions that require a CRM lookup rather than a documentation search.
Tech Stack Used
The production customer support AI agent stack typically includes Next.js for the customer-facing chat widget embedded in the product, a Python FastAPI backend for the agent orchestration layer, LangChain or LlamaIndex for the RAG pipeline orchestration, pgvector on PostgreSQL or Pinecone for the vector store, OpenAI text-embedding-3-large for document and query embedding, Claude or GPT-4o as the synthesis LLM for response generation, Redis for conversation thread state management, Intercom, Zendesk, or Freshdesk via API for live agent handoff and ticket creation, and a Next.js internal dashboard for the support team to monitor agent performance, review flagged conversations, and add knowledge base corrections. CSAT scoring is collected via an in-conversation thumbs up or down mechanism, with low-rated conversations automatically flagged for human review and added to a retraining queue.
Live Agent Handoff: The Critical Design Decision
The handoff mechanism is where most customer support AI agents fail in production. There are three failure modes: the agent hands off too eagerly, routing queries to human agents that it could have resolved, which negates the cost-saving case for the system; the agent hands off too late, leaving frustrated customers waiting through multiple failed resolution attempts before escalating; and the agent hands off without context, dumping the customer into a live chat queue where they have to explain their issue from scratch. The pattern that works is a tiered confidence threshold combined with a frustration signal detector. The confidence threshold is calibrated during a two-week shadow period where the agent generates responses but does not send them, and a human evaluates whether each response would have resolved the query. Frustration signals include repeated rephrasing of the same question, explicit escalation requests from the customer, and negative CSAT on previous turns in the same session. When handoff is triggered, the full conversation transcript, the agent's attempted resolution steps, and any account context retrieved from the CRM are passed to the live agent dashboard so the customer never has to repeat themselves.
What to Replicate from This Pattern
The shadow mode deployment is the most valuable practice to replicate. Running the agent in parallel with the existing support workflow for two weeks before going live catches the failure modes that no amount of internal testing reveals: edge cases in customer phrasing, product-specific terminology gaps in the knowledge base, and confidence threshold miscalibration. The investment in a quality monitoring dashboard is the second most valuable practice. Support agents seeing in real time which conversations the AI handled, which it escalated, and what CSAT scores resulted creates a continuous improvement flywheel where the human team's insights directly feed knowledge base improvements.
What to Avoid
The most common architectural mistake is building a single LLM prompt that handles intent detection, knowledge retrieval, and response synthesis simultaneously. This produces an agent that is expensive per token, impossible to debug when it fails, and difficult to update. Separating the pipeline into discrete, auditable steps is more engineering work upfront but dramatically reduces production debugging costs. The second mistake is treating the knowledge base as a one-time migration project rather than an ongoing content operation. Knowledge bases decay rapidly in SaaS environments where products change frequently. Without a defined process for updating documentation as features ship, the agent's answer quality degrades within weeks of launch.