Project Overview and Business Context
The client was a UK-based payment processing startup handling card-not-present transactions for e-commerce merchants. Their existing fraud prevention was a third-party SaaS tool with fixed rule sets. It caught well-known fraud patterns but missed novel attack vectors, and the black-box scoring gave merchants no insight into why legitimate transactions were being declined. Chargeback rates were climbing. Merchant satisfaction scores were falling. The business needed two things simultaneously: better fraud detection accuracy and explainability for declined transactions so merchants could understand and contest decisions. This is a requirement that maps directly to FCA expectations around model explainability for automated financial decisions, and one that consumer credit regulations are increasingly emphasising for any AI-driven outcome that affects customers financially. The project brief was clear: build an MVP that scores transactions in real time, reduces false positives by at least 20% against the legacy tool, and produces a plain-English explanation for every scoring decision. The MVP did not need to be perfect. It needed to be good enough to demo to merchants and to generate the real transaction data needed to improve the model.
Technical Architecture and Stack Decisions
The architecture separates the transaction scoring pipeline from the user-facing dashboard, which is important for latency reasons. Transaction scoring runs as a Python FastAPI microservice deployed on AWS Lambda behind an API Gateway. Each incoming transaction payload is enriched with historical velocity data pulled from PostgreSQL (transaction frequency per card, merchant, IP address, and device fingerprint over 1h, 24h, and 7d windows), then passed to a scoring layer. The scoring layer combines a lightweight statistical model trained on the client's historical transaction data in AWS SageMaker with a GPT-4o call that interprets the enriched feature set and generates a plain-English risk narrative. PostgreSQL handles both the transaction log and the feature store. Redis caches velocity lookups to keep p99 latency under 200ms. The Next.js frontend serves the merchant dashboard: declined transaction review, dispute submission, and a real-time feed of scoring decisions. Stripe webhooks trigger the scoring pipeline for every charge attempt. AWS SageMaker was chosen over a fully managed ML platform because the client wanted model artifacts they owned and could retrain as their data grew. The FastAPI layer is stateless and scales horizontally behind the Lambda concurrency limit. All infrastructure is defined in Terraform and handed over as part of the codebase delivery.
Key AI and ML Components
Two distinct AI layers work in sequence. The first is a gradient boosting classifier trained on the client's 18 months of historical transaction data, labelled with confirmed fraud and confirmed legitimate outcomes. AWS SageMaker handles training, evaluation, and model registration. The classifier outputs a fraud probability score between 0 and 1. It is fast, deterministic, and auditable, which matters for FCA model risk documentation. The second layer is GPT-4o, which receives the enriched transaction features, the classifier score, and a system prompt instructing it to produce a structured explanation: the top three risk factors, a human-readable narrative, and a recommended action (approve, review, decline). This explanation layer solves the explainability problem that the classifier alone cannot address. Merchants receive a plain-English reason for every decline, reducing disputes and building trust. The GPT-4o call is asynchronous for declined transactions, so it does not block the accept/decline decision for latency-sensitive approvals. The combination of a fast ML model for the decision and an LLM for the explanation is a pattern SpeedMVPs uses across several regulated industry projects, because it separates the decision logic (auditable, trainable) from the communication layer (flexible, human-readable).
Challenges Solved and How
Three challenges shaped the build. First, labelled training data was imbalanced: fraud cases represented under 0.3% of transactions. The SageMaker training pipeline handles this with SMOTE oversampling on the minority class and threshold calibration to optimise for recall at a target false positive rate, rather than raw accuracy, which is a misleading metric on imbalanced data. Second, the FCA requires that automated financial decisions affecting consumers be explainable. The GPT-4o explanation layer addresses this directly, but explanations also need to be consistent and not contradict the model's actual feature weights. The system prompt includes the top feature importance scores from the classifier, grounding GPT-4o's explanation in the model's actual logic rather than hallucinated reasoning. Third, latency. Chaining a PostgreSQL velocity query, a SageMaker inference call, and a GPT-4o call in sequence would exceed acceptable response times. The architecture parallelises the SageMaker inference and the GPT-4o explanation call, using the classifier result as input to GPT-4o only after the scoring decision is made, keeping p95 end-to-end latency under 250ms for approve decisions.
Outcome and Measurable Results
The client ran the AI fraud detection MVP in shadow mode alongside their existing tool for six weeks before cutover. During that period, the MVP caught 34% more confirmed fraud cases than the legacy tool at the same false positive rate. After cutover, the merchant chargeback rate fell 28% in the first quarter. False positive rate (legitimate transactions declined) dropped 19%, which had a direct impact on merchant revenue and satisfaction. The plain-English decline explanations reduced dispute resolution time from an average of 4 days to under 24 hours, because merchants could immediately see whether a decline was likely correct or worth contesting. The client used the MVP to secure a Series A conversation with two investors, both of whom specifically cited the explainability layer as a differentiator versus incumbent fraud tools. FCA model risk documentation was produced as part of the build, covering model purpose, training data provenance, known limitations, and the human review process for edge cases.
Lessons for Similar Projects
Build the explainability layer first, not last. Most fraud detection MVP builds focus on model accuracy and treat explainability as a post-launch feature. In regulated fintech, that order of priorities creates compliance debt that is expensive to resolve. Building the explanation layer alongside the model means FCA documentation is a byproduct of the build, not a separate project. Start with a simpler model than you think you need. A well-tuned gradient boosting classifier on good features outperforms a complex deep learning model on limited data, is easier to audit, and retrains faster as new fraud patterns emerge. Invest in your feature engineering. The quality of velocity features, device fingerprints, and merchant-level baselines matters more than model architecture for fraud detection at typical startup transaction volumes. Finally, design for model refresh from day one. Fraud patterns shift. Your training pipeline should be automated, documented, and executable by the client team without SpeedMVPs involvement after handover.