What LangSmith Provides in a Production AI Application
LangSmith is a developer platform from LangChain that captures and visualises the internal execution of LangChain pipelines. Every run - a chain execution, an agent step, a retrieval call, an LLM call - is recorded as a nested trace. The trace shows the exact prompt sent, the model response received, the token count, the latency and the cost. For RAG applications, the trace shows which documents were retrieved and their similarity scores. For agents, the trace shows every tool call, its inputs and outputs and the reasoning chain. This granularity makes debugging AI applications feasible. When a user reports that the AI gave a wrong answer, LangSmith lets you pull up the exact trace for that request and inspect every step. LangSmith also provides prompt versioning (commit and track prompt changes like code), dataset management (collect representative examples for evaluation), evaluation runs (score model outputs against ground truth or with LLM-as-judge) and deployment with LangServe. For production AI products, LangSmith transforms LLM debugging from guesswork into a systematic engineering discipline.
Setup Steps: LangSmith in a Production LangChain Application
Integrating LangSmith into a LangChain application is primarily a configuration task rather than a code change, which is one of its significant advantages. First, create a LangSmith account and create a project (projects group traces by application or environment - create separate projects for development, staging and production). Second, generate an API key from the LangSmith dashboard. Third, set the following environment variables in your application: LANGCHAIN_TRACING_V2=true, LANGCHAIN_API_KEY (your LangSmith API key), LANGCHAIN_PROJECT (your project name). In a Next.js application, these go in the server-side environment configuration only, never prefixed with NEXT_PUBLIC_. Fourth, verify tracing is working by running a LangChain chain in your development environment and confirming traces appear in the LangSmith dashboard. LangSmith auto-instruments all LangChain components without code changes - chains, agents, retrievers, LLM calls and tool calls are traced automatically. Fifth, configure separate projects per environment with separate API keys - this prevents development traces polluting production data and enables environment-specific alerting. Sixth, set up LangSmith feedback collection: add user feedback buttons (thumbs up/down) in your UI and submit the feedback to LangSmith linked to the run ID, so positive and negative examples accumulate for evaluation. Seventh, configure tracing sampling for high-volume applications: tracing every request on a product with 100,000 daily queries generates substantial data; sampling 10-20% of traces reduces storage while maintaining debuggability. Eighth, set up LangSmith alert rules for high latency, high error rate or unexpected cost spikes.
SpeedMVPs Production Use Case: AI Research Assistant
A UK fintech startup engaged SpeedMVPs to build an AI research assistant that helped analysts query a corpus of regulatory documents, earnings reports and news articles. The system used LangChain with a multi-step retrieval pipeline: query rewriting, hybrid search (Pinecone vector + BM25 keyword), reranking with Cohere and GPT-4o for final answer generation. Without observability, debugging this pipeline was impossible - there were too many steps where quality could degrade. SpeedMVPs integrated LangSmith from the first day of development. Every research query appeared as a complete trace: the rewritten query, the retrieved document chunks with similarity scores, the reranker output, the final prompt and the LLM response. When the client reported that certain regulatory queries produced low-quality answers, the team pulled the traces for those queries, identified that the retrieval step was returning irrelevant chunks (low similarity scores) and pinpointed that the issue was in the chunk size configuration for regulatory documents (which have dense, structured content requiring smaller chunks than news articles). The fix was diagnosed in 20 minutes; without LangSmith it would have required significant instrumentation work. The team also used LangSmith's evaluation runner to test prompt changes against a curated dataset of 50 representative queries before deploying to production - this caught two regressions from prompt updates before they reached users.
Why LangSmith and LangChain Work Together
LangSmith is built specifically for LangChain and treats LangChain's object model natively. Chains, agents, retrievers and LLM wrappers are all first-class trace entities - the dashboard renders them with appropriate visualisation (agent thoughts shown as a reasoning chain, retrieval results shown with scores, LLM calls shown as prompt and completion pairs). This native understanding means the trace data is meaningful without configuration - you do not need to instrument each step manually. For teams building multiple AI products, LangSmith's project structure means each product has its own trace history and prompt library. Prompt versioning in LangSmith means the team can commit, compare and rollback prompt changes with the same discipline as code changes. LangSmith's hub allows sharing prompts across projects or with the broader LangChain community, which accelerates prompt development for common tasks. For EU AI Act compliance (which SpeedMVPs advises on), LangSmith's trace history provides an audit log of AI system behaviour - the exact input, output and reasoning chain for every request - which supports transparency and documentation requirements for higher-risk AI applications.
Limitations and Production Gotchas
LangSmith is designed for LangChain applications; if your application calls OpenAI directly without LangChain (using the raw openai SDK), LangSmith requires the LangChain RunTree API for manual instrumentation, which adds significant code changes. For non-LangChain applications, Helicone is a simpler alternative that works as a transparent proxy for any OpenAI-compatible API. LangSmith traces contain the full prompt content - for applications handling sensitive user data, you must review whether sending prompts to LangSmith's US-hosted service is compatible with your GDPR obligations. LangSmith supports a self-hosted deployment option on their Enterprise plan, which resolves data residency concerns for regulated industries. Trace data volume can be substantial for high-query-volume applications - the LangSmith Pro plan (39 USD per month) includes 5,000 traces; production applications typically need the Plus or Enterprise tier. LangSmith evaluation requires creating ground-truth datasets, which requires human annotation effort - plan for this in your development timeline if systematic evaluation is a requirement. Finally, LangSmith's feedback collection API requires instrumenting your frontend to capture the run_id from each AI response and link it to user feedback submissions - this requires careful passing of metadata from server to client, which adds implementation complexity.