devops

Feature Flags: Safe AI Feature Rollouts and Continuous Deployment

A software development technique that enables or disables features at runtime for specific users or segments without deploying new code.

Feature flags, also called feature toggles, are a software development technique that separates feature deployment from feature release. Code for a new feature is deployed to production but the feature is hidden behind a flag that can be enabled or disabled at runtime, without deploying new code. For AI products, feature flags solve specific problems that traditional deployment workflows cannot address: enabling a new LLM model for a subset of users before switching everyone, running A/B tests on different prompt templates, gradually rolling out AI features to internal users before public release, and turning off a misbehaving AI feature instantly without a rollback deployment. At SpeedMVPs, a UK-based AI MVP agency in Hemel Hempstead delivering products in 2-3 weeks at GBP 8,000 with full code ownership, feature flags are part of every production build so clients can control AI behaviour after handover without a redeployment. LLM providers update their models on their own schedules, and a flag-controlled model reference lets a team respond in minutes rather than days. UK teams under UK GDPR benefit from consent-gated flags: AI processing features can be restricted to users who have given explicit consent, enforced at the flag layer rather than duplicated across every code path. This guide explains how feature flags work, the main platforms available, and specific patterns for AI product development in a UK context.

How Feature Flags Work

A feature flag is a conditional in your code that checks whether a flag is enabled for the current user or context before showing a feature. The flag value is read from an external source, typically a feature flag platform, an environment variable, or a database record, rather than hardcoded in the application. This means the flag can be toggled without redeploying the application. The simplest feature flag is a boolean: either the feature is on or off for all users. More sophisticated flag implementations support user targeting, where specific users or user segments see the feature while others do not, percentage rollouts where a random percentage of users are assigned the feature, and A/B test allocation where users are randomly split between variants. The flag evaluation happens at runtime on every relevant request or page load, so changes take effect immediately or within seconds of updating the flag configuration. This decoupling of deployment from release is the core value proposition: your CI/CD pipeline can deploy code containing unreleased features at any time without risk of users encountering unfinished work.

Feature Flags for LLM Model Switching

One of the most practical applications of feature flags in AI products is controlling which LLM model processes user requests. When a new model version is released by a provider, you want to evaluate it against your specific use cases before switching your entire user base to it. A feature flag that routes a percentage of users, or a specific internal user segment, to the new model while others continue on the current model lets you run a controlled comparison. Monitor output quality, latency, cost per token, and user engagement metrics for both groups. If the new model performs better across your quality criteria, increase the flag percentage gradually until you are satisfied, then switch all users. If the new model underperforms or misbehaves for your specific use cases, disable the flag with no user impact. This pattern applies equally to switching between providers, such as evaluating Anthropic Claude against OpenAI GPT-4o for a specific workflow, and to testing different model parameters such as temperature, maximum tokens, or system prompt variations.

Prompt Template A/B Testing

Feature flags are the standard mechanism for A/B testing prompt templates in production. Rather than guessing which prompt phrasing produces better outputs for your users, flag-based A/B testing allocates users randomly to different prompt variants and measures the outcome. The outcome metrics for AI prompt A/B tests are more nuanced than standard conversion A/B tests. You might measure user acceptance rate of AI-generated outputs (how often users use the output rather than discarding it), task completion rate for workflows involving AI assistance, user-rated output quality if you have an in-product feedback mechanism, or downstream business metrics such as feature retention for users who engaged with the AI feature. A flag-based A/B test requires consistent assignment: the same user should see the same prompt variant across multiple sessions to avoid confounding the experiment results. Most feature flag platforms support this through stable user ID-based bucketing.

Kill Switches for AI Features

A kill switch is a feature flag used defensively: a flag that is normally on and can be turned off instantly to disable a feature that is misbehaving. For AI products, kill switches are essential safety mechanisms. LLM APIs can experience increased error rates, latency spikes, or unexpected output quality degradations without warning. A kill switch on your AI feature means you can instantly route users to a non-AI fallback, a message explaining the feature is temporarily unavailable, rather than exposing them to broken AI outputs or timeouts while you diagnose the problem and deploy a fix. Kill switches should be implemented for every AI feature that is customer-facing and that has a meaningful degradation of user experience when it fails. The configuration of the kill switch should be accessible outside your codebase, in a feature flag platform or operations dashboard, so it can be activated by someone without a deployment process. Response to a broken AI feature should be measured in seconds to minutes, not in the hours that a code deployment might require.

Feature Flag Platforms for AI Startups

Several purpose-built feature flag platforms are worth considering for AI product teams. LaunchDarkly is the market leader with a mature SDK ecosystem, sophisticated targeting and rollout controls, and strong integration with analytics tools. It is the appropriate choice when feature flagging is a core workflow for a team shipping frequently to a large user base, but the pricing reflects that positioning. PostHog combines product analytics, session recording, and feature flags in a single platform, which is an attractive combination for early-stage AI products that want unified tooling. The feature flag functionality in PostHog is genuinely capable and the integration with analytics makes measuring flag-driven experiments straightforward. Flagsmith is an open-source alternative that can be self-hosted, which may be relevant for AI products with strict data residency requirements. Vercel Edge Config and Vercel Feature Flags provide basic flag functionality natively for teams on Vercel. For very early-stage products, environment variable-based flags are often sufficient before investing in a dedicated flag platform.

Technical Debt and Flag Hygiene

Feature flags accumulate technical debt if not actively managed. Each flag adds conditional logic to the codebase, and flags that are never removed after a feature is fully rolled out create dead code paths that confuse future developers and increase testing surface area. Establish a practice of removing flags once they have served their purpose: either the feature is fully rolled out and the flag can be deleted with the non-flag code path removed, or the feature is permanently abandoned and all associated code can be deleted. Flag expiry dates are a useful mechanism: when creating a flag, set an expected expiry date in the platform and create a ticket to clean up the flag and its code paths after that date. For AI product teams running many prompt experiments simultaneously, flag naming conventions that identify the experiment, the variant, and the expected review date help keep the flag list manageable. Periodic flag audits, reviewing all active flags and retiring those that have reached their intended purpose, should be a regular engineering team activity.

Frequently Asked Questions

Can we use feature flags without a dedicated platform?+

Yes, for simple cases. Environment variables are the simplest form of feature flag: set a variable to true or false and read it in your code. The limitation is that environment variables require a deployment to change. For flags that need to be toggled at runtime without deployment, you need either a database-backed flag store your application reads from, or a dedicated feature flag platform. Most AI products benefit from runtime-togglable flags from relatively early, because the ability to kill a misbehaving AI feature or switch a model without deploying is genuinely valuable once you have real users.

How do we measure the impact of an AI feature controlled by a flag?+

Connect your feature flag platform to your analytics tool. Most platforms including LaunchDarkly and PostHog support sending flag assignment as an analytics event, which you can then use to segment users by which variant they received. Define your outcome metrics before the experiment starts: task completion rate, output acceptance rate, session retention, or feature-specific actions. Analyse the metrics for the flag-enabled group versus the control group after sufficient sample size has accumulated. Statistical significance matters for A/B tests: small sample sizes produce misleading results. For AI product experiments, aim for at least 200-500 users per variant before drawing conclusions.

What is the difference between a feature flag and an A/B test?+

A feature flag is the mechanism: a runtime toggle that shows different code paths to different users. An A/B test is an experiment using that mechanism: you allocate users to variants randomly, measure outcome metrics, and determine which variant performs better. Every A/B test uses feature flags or an equivalent mechanism, but not every feature flag is an A/B test. A kill switch is a feature flag used for safety rather than experimentation. A gradual rollout uses a feature flag to incrementally increase the percentage of users on a new feature without A/B test comparison. A/B tests specifically require random allocation, a control group, defined outcome metrics, and statistical analysis.

How do feature flags interact with caching in AI products?+

Feature flag state should not be cached at layers that would cause inconsistent flag behaviour for the same user across requests. If your CDN caches API responses, flag-controlled content in those responses will be cached in its flag state at the time of first request and served to other users regardless of their flag assignment. Ensure that API responses affected by feature flags carry Cache-Control: no-store or use cache keys that include the flag variant so different variants are cached separately. In Next.js, server-side rendering with flag checks should opt out of full-page caching or use per-user cache keys.

Can feature flags help with GDPR compliance for AI features?+

Feature flags can support a consent-based approach to AI features. If processing user data through an AI feature requires explicit consent under GDPR, a feature flag can enable the AI functionality only for users who have given consent, with the flag driven by the user's consent record in your database rather than a manual toggle. This ensures that users who have not consented never receive AI processing of their data, and withdrawing consent disables the flag-controlled feature immediately without requiring code changes or redeployment.

Want your AI MVP built with feature flags for safe rollouts from day one? We include flag infrastructure in every production-ready build. Get a free consultation at speedmvps.co.uk

Get a Free Quote