architecture

Edge Functions: Low-Latency Compute for AI Products

Serverless functions that run at CDN edge locations geographically close to users, reducing latency for globally distributed applications.

Edge functions are serverless functions that run at CDN edge locations geographically close to the user, rather than in a centralised cloud region. Instead of a user in London sending a request to a data centre in Virginia and waiting for the response to travel back, edge functions run the request processing in a data centre in Frankfurt or London, reducing round-trip latency significantly. For AI products, edge functions are valuable for specific use cases: authentication checks, request routing, personalisation, and lightweight request transformation. However, they have real limitations for AI workloads that are important to understand before designing your architecture around them. The most important limitation for UK AI product teams is the restricted runtime environment. Edge functions run in a lightweight V8 isolate, not a full Node.js process. Standard database drivers, large npm packages, and Node.js-specific APIs are unavailable. This means edge functions cannot directly query PostgreSQL with a standard client, cannot use most AI SDK features that depend on Node internals, and cannot run the kinds of business logic that make up the core of most AI products. What they can do is run extremely fast, with near-zero cold starts, making them ideal for the authentication and routing layer that sits in front of your main application. For UK products, edge functions also have GDPR implications: if personal data passes through an edge function that runs in a data centre outside the UK or EEA, that constitutes an international data transfer that requires assessment. SpeedMVPs configures geographic restrictions on edge middleware execution to keep personal data processing within the appropriate regional boundary on every UK client deployment.

How Edge Functions Differ from Standard Serverless

Standard serverless functions (AWS Lambda, Vercel Functions with the Node.js runtime) run in a specific cloud region. A function deployed to eu-west-1 (Ireland) processes all requests in that region regardless of where the user is located. Edge functions run in a different model: they are distributed across dozens or hundreds of CDN edge nodes globally and execute in the node closest to the user. Vercel's Edge Runtime, Cloudflare Workers, and Deno Deploy are the major edge function platforms. The edge runtime has significant constraints compared to a standard Node.js environment. It does not have access to the full Node.js standard library. Native Node.js modules do not work. Certain npm packages that depend on Node.js internals are not compatible. The available APIs are based on the WinterCG standard: fetch, the Web Crypto API, the Streams API, and similar web-standard interfaces. Execution time limits are typically much shorter (Cloudflare Workers: 30 seconds CPU time; Vercel Edge: 25 seconds). Memory limits are smaller. But cold starts are negligible, often under 5 milliseconds, because edge functions run in a lightweight V8 isolate environment.

What Edge Functions Are Good At

Edge functions excel at work that must happen on every request but does not require complex server-side computation or access to a central database. Authentication and authorisation checks are a natural fit: verifying a JWT token at the edge, checking whether the user is in an A/B test variant, or redirecting unauthenticated users to a login page. These operations involve cryptography (Web Crypto API is available) and token parsing but not database access. Internationalisation and localisation redirects (detecting a user's preferred language from headers or cookies and redirecting to the appropriate page version) are extremely fast at the edge. Bot detection and rate limiting based on IP address and request patterns can be applied at the edge before requests reach your origin server. Feature flag evaluation, for simple flag definitions stored in edge-accessible configuration rather than a database, can gate feature access at near-zero latency. For AI products, the most common edge function use case is middleware: running authentication checks, logging, and request transformation on every request before it reaches the Next.js application.

What Edge Functions Cannot Do Well

Edge functions have limitations that rule them out for many AI workload components. They cannot connect to standard database systems. Connecting to a PostgreSQL or MySQL database from an edge function requires a connection pooler designed for edge environments (Neon's edge driver, PlanetScale's edge driver, or Upstash for Redis). Standard database connection libraries do not work in the edge runtime. For AI processing that calls LLM APIs, the HTTP fetch API is available in edge functions, so calling OpenAI or Anthropic from an edge function is technically possible. However, AI responses can take 5-30 seconds for complex tasks, which approaches or exceeds edge function timeout limits. Additionally, running long AI processing in an edge function provides no inherent advantage over running it in a standard serverless function in a nearby region. For background AI processing, data analysis, or any operation that requires access to a standard Node.js database driver, file system, or native module, standard serverless functions or containers are the correct choice.

Edge Functions in Next.js

Next.js has native support for edge functions through its middleware feature and the Edge Runtime configuration for API routes. Next.js middleware runs on every request before the page or API route is served, and it runs in the Vercel Edge Runtime by default. This makes it the natural location for authentication checks, A/B testing, and request routing logic. Individual Next.js API routes or route handlers can be configured to run on the Edge Runtime using the runtime export. This makes them faster for latency-sensitive operations but imposes all the edge runtime constraints (no full Node.js API, no native modules). Choosing the Edge Runtime for a Next.js API route that calls a PostgreSQL database with a standard pg driver will fail because the pg driver requires Node.js internals not available in the edge runtime. The decision of which runtime to use for which route requires understanding what each route needs to do.

Data Residency and GDPR at the Edge

Edge functions create a specific GDPR complication: if a user in the UK sends a request to a Vercel Edge Function, that function might run in any of Vercel's edge nodes globally, including outside the UK and EEA. If the request or response contains personal data (which most authenticated API requests do, because they include at least a user identifier), this means personal data is being processed in potentially any jurisdiction. Both Vercel and Cloudflare offer configuration to restrict edge function execution to specific geographic regions. On Vercel, Edge Config and region routing can direct traffic to EU-based nodes. On Cloudflare Workers, Smart Placement and geographic restrictions limit execution to specified regions. For UK products processing personal data, configuring edge functions to run only in UK or EU data centres is a GDPR requirement if you consider the request processing to constitute personal data processing. SpeedMVPs configures data residency constraints on edge middleware as part of every UK and EU client deployment.

Practical Edge Function Architecture for AI Products

A practical edge function architecture for an AI Next.js product uses edge functions for the request layer (authentication, routing, bot detection, A/B testing) and standard serverless functions or containers for the data and AI processing layer. The middleware at the edge handles every request cheaply and with minimal latency: checking the auth token, setting the tenant context, and routing to the appropriate page or API. The API routes that perform actual operations, database queries, LLM API calls, and data transformations, run in the standard Node.js runtime where they have access to full database drivers and Node.js modules. This layered approach uses edge compute for what it is good at (fast, stateless request processing) and standard compute for what it is good at (stateful operations requiring database access and full library support).

Frequently Asked Questions

Should I run my AI inference in an edge function?+

Not as a general rule. LLM inference via external APIs (OpenAI, Anthropic) is possible from edge functions since HTTP fetch is available, but edge function timeout limits (25-30 seconds) are tight for complex AI tasks, and you gain no meaningful latency benefit for the LLM API call itself (which goes from the edge node to the LLM provider's servers). The latency reduction from edge functions is in the round-trip between the user and your infrastructure, not in the AI processing time. Run AI inference in standard serverless functions or containers.

What is the cold start time for edge functions?+

Edge functions using the V8 isolate model (Cloudflare Workers, Vercel Edge Runtime) have near-zero cold start times: typically under 5 milliseconds. This is because the isolate model is much lighter than spinning up a full Node.js process. Edge functions do not suffer from the 100ms-3s cold starts that standard Lambda-style serverless functions experience. This is the primary performance advantage of edge functions over standard serverless for latency-sensitive request handling.

Can edge functions access a database?+

Not with standard database connection libraries, which require Node.js APIs unavailable in the edge runtime. There are edge-compatible database clients: Neon provides an edge-compatible PostgreSQL driver that uses HTTP instead of TCP, Upstash provides an edge-compatible Redis client, and PlanetScale provides an edge-compatible MySQL client. These options work within the edge runtime's constraints but typically have higher latency per query than a native TCP connection because they use HTTP.

How do Vercel Edge Functions compare to Cloudflare Workers?+

Both use the V8 isolate model with near-zero cold starts and the WinterCG API set. Cloudflare Workers have a broader global network and slightly lower typical latency due to more edge nodes. Vercel Edge Functions are more tightly integrated with Next.js, making them the natural choice for Next.js middleware. Cloudflare Workers support Durable Objects for stateful edge computation. For a Next.js AI product on Vercel, Vercel Edge Functions for middleware are the default; for standalone edge compute, Cloudflare Workers are worth considering.

Does SpeedMVPs use edge functions in AI product deployments?+

Yes, specifically for Next.js middleware: authentication token verification, tenant routing, A/B test assignment, and bot detection all run at the edge on Vercel. We configure geographic restrictions on edge function execution to ensure personal data processing stays within UK or EU regions for GDPR compliance. AI inference, database access, and business logic run in standard Node.js runtime functions where they have full library support and no edge runtime constraints. Get a free consultation at speedmvps.co.uk

Want the right compute architecture for your AI product, with GDPR-compliant edge configuration? Get a free consultation at speedmvps.co.uk

Get a Free Quote