What Is Redis and Why SpeedMVPs Uses It
Redis is an open-source, in-memory data structure store that operates as a database, cache, and message broker. It stores key-value pairs in RAM for sub-millisecond read and write operations, with optional persistence to disk for durability. In the context of AI products, Redis serves several distinct functions: caching LLM responses to avoid redundant API calls, implementing rate limiting to control LLM spend, storing user sessions and conversation history, and providing the backing store for background job queues. SpeedMVPs uses Redis in AI products for cost and performance reasons that become tangible quickly. LLM inference is the most expensive component of most AI SaaS products. At GPT-4o pricing (approximately GBP 0.004 per 1,000 output tokens at current rates), a product with 10,000 daily active users generating similar queries will spend significant money on identical or near-identical completions. Caching even 20 to 30 percent of LLM responses in Redis reduces this cost proportionally while improving response time for cached queries from seconds to milliseconds. Rate limiting via Redis is equally important. Without a rate limiting layer, a single user can exhaust an API budget in minutes by automating requests. Redis counters with expiring keys implement sliding window rate limits in a few lines of code, enforcing per-user, per-organisation, or per-IP request quotas before a single LLM API call is made.
Setting Up Redis in a Production AI Project
Redis setup depends on where the rest of your infrastructure lives. Here are the common configurations SpeedMVPs uses. For Vercel-hosted Next.js applications, Upstash Redis is the standard choice. Upstash provides a serverless Redis instance billed per request rather than per hour, which matches Vercel's serverless execution model. There is no idle cost - you pay only when Redis is accessed. The Upstash REST API works in Vercel Edge Middleware and serverless functions where a standard TCP Redis connection is not supported. Install @upstash/redis and initialise with your REST URL and token from the Upstash console. For Railway-hosted applications, provision a Redis service in the same Railway project. Railway's managed Redis injects a REDIS_URL environment variable automatically. Use ioredis or the native node-redis client with the injected URL. Both support all Redis data structures and commands without additional configuration. For AWS-hosted AI backends, ElastiCache for Redis provides managed Redis with Multi-AZ replication, automated backups, and VPC-level network isolation. ElastiCache is appropriate for enterprise AI products with SLA requirements. It does not support the REST API mode required for edge functions, so use it only with traditional serverless or container-based backends. Regardless of provider, configure Redis with a maxmemory policy appropriate for caching use cases: allkeys-lru evicts the least recently used keys when memory is full, which is correct for LLM response caches. Set an appropriate maxmemory limit based on your expected cache size. For LLM response caching, design a cache key that captures all inputs that affect the response: the model version, the prompt template identifier, and a hash of the variable inputs (user query, selected documents). A cache key mismatch from a minor prompt change will generate cache misses, so version your prompt templates and include the version in the cache key. For semantic caching (caching based on embedding similarity rather than exact key match), RedisVL (the Redis Vector Library) provides a SemanticCache class that stores embeddings of cached queries and retrieves responses for semantically similar new queries above a configurable similarity threshold.
Key Features and Capabilities
LLM response caching is the most impactful Redis pattern for AI cost reduction. Exact match caching is straightforward: hash the full prompt, check Redis for a cached response, return it if present, otherwise call the LLM, cache the response with an appropriate TTL, and return it. For AI products where many users ask similar questions against the same knowledge base (customer support bots, documentation assistants, FAQ tools), cache hit rates of 30 to 60 percent are achievable, directly reducing LLM API spend by the same percentage. Semantic caching via RedisVL extends this to near-duplicate queries. If a user asks "how do I reset my password" and another user asks "I forgot my password, what do I do", a semantic cache with a similarity threshold of 0.92 will serve the same cached response to both queries. This significantly increases cache hit rates for natural language AI products where the same underlying question is phrased many different ways. Rate limiting with Redis sorted sets and atomic INCR operations implements precise sliding window rate limits. A common pattern in SpeedMVPs AI products: each LLM API call increments a Redis counter keyed by user_id with a TTL of one minute. If the counter exceeds the configured limit, return a 429 response before the LLM call is made. This prevents a single user from generating unexpected costs and is required for any AI product sold on a subscription basis. Conversation history storage in Redis is appropriate for AI chat products where history needs to be retrieved quickly on every turn but does not require permanent storage. A Redis list per conversation_id stores message history with a configurable maximum length (using LTRIM to keep only the last N messages). Conversation data in Redis should have an appropriate TTL - SpeedMVPs typically sets 24 to 72 hours for active conversation storage, with longer-term history archived to a relational database. Job queue backing via Redis (using BullMQ in Node.js or Celery in Python) handles async AI tasks: document ingestion, report generation, and any AI workload that should not block an HTTP response. BullMQ provides job prioritisation, retry logic, rate limiting per queue, and a dashboard for monitoring queue health.
Real-World Workflow: Redis in an AI MVP
A concrete SpeedMVPs example: an AI customer support assistant for a UK telecoms company. The assistant answered billing, technical, and account questions via a web chat interface. LLM API costs were a key concern - the client estimated 50,000 support interactions per month at peak. Redis handled three distinct roles. First, LLM response caching for common questions. The top 200 frequently-asked questions (billing dates, data allowances, network coverage queries) were identified during testing. A warm cache populated these responses on deployment, and new responses were cached with a 6-hour TTL. Cache hit rates averaged 42 percent in the first month, reducing LLM API spend by roughly the same fraction. Second, rate limiting. Each user session was limited to 20 LLM calls per 10-minute window. A Redis sorted set tracked request timestamps per session ID. Requests beyond the limit received a polite message asking the user to wait, preventing the small number of users attempting to use the chat interface as a general-purpose AI tool from consuming the shared API budget. Third, conversation history. Each conversation's message history was stored in a Redis list keyed by session_id with a 2-hour TTL. After TTL expiry, the conversation was archived to PostgreSQL for compliance and quality review. Retrieving the last 10 messages of a conversation from Redis took under 2 milliseconds, adding negligible latency to each chat turn. The Redis instance on Upstash cost GBP 12 per month at this usage level - a tiny fraction of the LLM API spend it helped reduce.
Cost and Pricing Considerations
Redis cost depends on the provider and usage model. Upstash Redis for serverless applications starts with a free tier (10,000 commands per day) and scales at USD 0.20 per 100,000 commands on the pay-as-you-go plan. For an AI product making 1 million Redis commands per month (a mix of cache reads, rate limit checks, and session operations), Upstash costs approximately USD 2 per month. Managed Redis on Railway is priced by memory and compute. A 512 MB Redis instance costs approximately USD 5 per month, suitable for most AI MVP caching workloads. A 1 GB instance handles larger conversation history stores and bigger LLM response caches. Redis Cloud (the Redis Labs managed service) and AWS ElastiCache are enterprise options at higher price points with stronger SLAs, Multi-AZ replication, and enterprise compliance certifications. ElastiCache for Redis in eu-west-2 starts at approximately USD 15 per month for a single cache.t3.micro node. The return on Redis investment is straightforward to calculate for LLM caching: estimate your cache hit rate (start conservatively at 20 percent), multiply by your monthly LLM API spend, and compare to Redis cost. A product spending GBP 500 per month on LLM API calls with a 25 percent cache hit rate saves GBP 125 per month, paying for several years of Redis at any pricing tier.
Alternatives to Redis for AI Caching
Memcached is a simpler caching solution for pure key-value caching without the data structure richness of Redis. It does not support lists, sorted sets, or pub/sub, which means it cannot handle conversation history storage, rate limiting with sliding windows, or job queue backing. For teams that only need simple response caching and already use Memcached, it is adequate, but SpeedMVPs defaults to Redis for AI products because the additional data structures cover patterns that emerge in almost every product. Vercel KV (powered by Upstash Redis) is the built-in option for Next.js applications on Vercel. It provides the same Redis API as Upstash with simpler setup. For teams already on Vercel who want to minimise the number of third-party services, Vercel KV is a clean choice. For LLM response caching specifically, LangChain provides a SQLite cache and in-memory cache that work without Redis. These are appropriate for development environments but are not suitable for production: SQLite does not scale across multiple application instances, and in-memory cache is lost on process restart. For any production AI product handling more than a single server instance, Redis is the correct caching layer. For semantic caching specifically, LangChain's Momento Semantic Cache provides managed semantic caching without running Redis. It is a newer service with less production track record than Redis-based approaches, but worth evaluating for teams who want managed infrastructure and are building products where semantic cache hit rate is the primary goal.