Project Overview and Business Context
The client was a digital health startup working with GP practices to reduce avoidable urgent care attendances. Their hypothesis was that patients who could get a structured triage assessment at home would make better decisions about whether to call 111, visit an urgent treatment centre, or book a routine appointment. The existing solution was a static symptom checklist on a website, with no personalisation and no AI interpretation. Conversion to appropriate care pathway was low because patients did not trust a static form to handle their specific combination of symptoms. The new AI symptom checker needed to feel like a conversation, not a form. It needed to ask follow-up questions based on the user's responses, interpret symptom combinations in context, and recommend a specific care pathway with a plain-English explanation. It also needed to operate within tight safety constraints: any symptom combination consistent with a medical emergency (chest pain plus shortness of breath, sudden severe headache, signs of stroke) must immediately direct the user to call 999 or attend A&E, overriding any other logic. MHRA classification of the product as a medical device was a consideration from day one. The client's clinical advisors reviewed the safety rule set and the product positioned itself appropriately within Class I software boundaries by recommending care pathways rather than diagnosing conditions.
Technical Architecture and Stack Decisions
The product is a Next.js application deployed on Vercel, with Supabase handling authentication, session storage, and the audit log database. The symptom checker is implemented as a multi-turn conversational interface, where each user response is passed to a server-side API route that calls Anthropic Claude with the full conversation history and a structured system prompt. The system prompt includes the clinical protocol: a decision tree of symptom categories, red-flag rules that trigger emergency escalation, and instructions for generating follow-up questions that probe symptom duration, severity, and associated factors. Claude returns a structured JSON response: a follow-up question or a care pathway recommendation, a reasoning summary, and a flag if any red-flag symptoms were detected. The red-flag flag is checked server-side before Claude's response reaches the frontend. If it is set, the frontend overrides Claude's recommendation with the hardcoded emergency protocol regardless of what Claude suggested. This two-layer safety architecture, AI recommendation plus deterministic safety override, is the most important design decision in the build. PostHog tracks funnel completion, care pathway distribution, and session drop-off points, giving the client data to optimise the conversation flow. Supabase stores each session with full message history and timestamps for clinical governance review. Data is stored in EU-region Supabase instances to comply with UK GDPR data residency expectations.
Key AI and ML Components
Anthropic Claude handles the conversational intelligence of the symptom checker. The system prompt is the most clinically sensitive component of the build. It was developed in collaboration with the client's GP advisor and went through three review cycles before production deployment. It instructs Claude on how to ask follow-up questions, how to interpret severity descriptors (mild, moderate, severe, sudden onset), and how to map symptom combinations to care pathway categories: self-care at home, pharmacist, routine GP appointment, same-day GP appointment, urgent treatment centre, or 999 emergency. Claude was chosen over GPT-4o for this application because of its stronger instruction-following on structured output requirements and its more conservative behaviour when asked to make health-related assessments. The system prompt explicitly instructs Claude to recommend the more cautious care pathway when symptom presentation is ambiguous, which aligns with clinical safety principles for a triage tool. Claude is not asked to diagnose. It is asked to interpret symptom severity and duration in the context of clinical protocols and recommend a care pathway. That distinction, clear in the system prompt and in the product UI, is important for MHRA classification and for user trust.
Challenges Solved and How
Three challenges shaped the clinical safety design. First, preventing Claude from providing diagnostic statements. The system prompt includes explicit instructions not to name conditions, not to use terms that imply diagnosis, and to frame all outputs as care pathway recommendations rather than medical opinions. A post-processing filter on Claude's output checks for a list of prohibited diagnostic terms before the response reaches the frontend. Second, handling ambiguous or contradictory symptom descriptions. Users often describe symptoms inconsistently. Claude is instructed to ask clarifying follow-up questions rather than making assumptions, and the conversation is designed to allow up to six turns before reaching a recommendation. Third, ensuring the audit trail meets NHS Digital and ICO expectations for health data processing. Every session is logged with user ID (or anonymous session token for unauthenticated users), full message history, timestamps, care pathway outcome, and whether the safety override was triggered. The log format was reviewed against NHS Digital's Data Security and Protection Toolkit requirements. GDPR Article 22 considerations around automated decision-making are addressed by framing the tool as advisory, with users explicitly informed that the recommendation is not a medical opinion.
Outcome and Measurable Results
The client ran a pilot with two GP practices over eight weeks, with 847 completed symptom checker sessions. Of those, 31% resulted in a self-care or pharmacist recommendation, reducing unnecessary GP appointment bookings. 12% triggered the emergency escalation protocol, directing users to call 999 or attend A&E, all of which were reviewed by the clinical team and confirmed as appropriate escalations. GP practice staff reported that patients who attended appointments after completing the symptom checker arrived with clearer symptom histories, reducing average consultation time by approximately 4 minutes per appointment. PostHog data showed an 84% funnel completion rate from session start to care pathway recommendation, significantly higher than the 41% completion rate on the previous static checklist. The clinical governance team used the session audit logs to review a random 5% sample of recommendations, confirming appropriate pathway assignments in 94% of reviewed cases.
Lessons for Similar Projects
Involve a clinician in the system prompt design from day one, not after the technical build is complete. The system prompt is clinical content, not just an engineering artefact. Getting it wrong is not a bug fix, it is a patient safety issue. Design the safety override layer before you design the AI layer. Know exactly which inputs must trigger deterministic emergency protocols regardless of the AI's response, and build that logic first. Be explicit with Claude about what it must not do. Negative instructions in the system prompt, telling the model not to diagnose, not to name conditions, not to provide treatment advice, are as important as positive instructions. Test every edge case in the care pathway decision tree before launch. Symptom combinations that seem unlikely happen regularly in real use. Run structured red-team testing against the system prompt with a clinician reviewing outputs. And finally, plan for MHRA classification from the outset. If your product can influence a clinical decision, you need a view on whether it constitutes a medical device under UK MDR 2002 or the in vitro diagnostic regulations. Getting that view early saves significant rework.