Executive summary Executive summary
Client background
The client is a large provider of prepaid cards and current accounts from the UK. The company has successfully onboarded over 1M customers and processed billions of pounds in payments.
Business challenge
The client needed to centralize the decision-making process of handling transactions. The company wanted to ensure instant payment processing that would remain efficient at handling peak transaction volumes and comply with sanctions, AML, and fraud regulations.
The company decided to develop a solution powered by Machine Learning (ML) that would streamline transaction processing by automatically making decisions on how to handle each payment in real-time. They sought an experienced partner to help them obtain a clear vision and implementation roadmap, and combine their previously developed ML models into a single solution.
Value delivered
N-iX helped the client develop an ML-powered solution that automates and streamlines the company’s transaction processing, reduces manual work, and optimizes operational expenses. Furthermore, after the solution was launched, the client increased their NPS score by 35 points and the number of customers by 20%.
Success story in detail
The client aimed to achieve instant payment processing that could maintain effectiveness during high transaction loads and adhere to regulations related to sanctions, anti-money laundering (AML), and fraud prevention. To achieve this, the company needed to build a solution powered by Machine Learning to consolidate and automate the decision-making process for handling transactions. The client already had nearly 15 ML models, located across multiple systems. Each model was managed separately with separate data preparation per model. Because the number of models grew rapidly, they were difficult to maintain. ML models performed data calculations every 5 minutes, resulting in an increased waiting time for customers submitting transactions. Also, other system components had no clear way to use ML models. The deployment pipeline was entirely manual and required active involvement from the software team during releases.
As a result, their original effort to build a single system to handle the decision-making process was put on hold as they required a more precise and clear vision of the end solution. They sought a partner with deep expertise in Machine Learning to audit the existing decision process, evaluate their needs, create a clear implementation roadmap, and develop a single solution that effectively combines all existing ML models.
N-iX engineers conducted the Product Discovery to determine which system architecture would best match the client’s business, functional, and non-functional requirements and overcome existing constraints. We have outlined the following goals for the project:
- Setting up a single solution to test and update the decision-making logic;
- Building an MLOps pipeline to deliver ML models from the development environment to production;
- Establishing real-time data processing;
- Updating the infrastructure and the CI/CD pipeline to make the solution cloud-agnostic.
We have also determined the main data sources for the ML-powered solution:
- Databases of internal bank systems (analytical and operational);
- Predefined data sets that are generated by the ETL process;
- External systems with API for specific feature data (credit scores, etc.).
As a result of Product Discovery, we delivered an implementation roadmap, outlined assumptions and limitations, delivered a work breakdown structure document, defined a budget and estimation for the scope of work, and formed the development team.
N-iX team has helped the client develop a cloud-agnostic ML-powered solution that automates the handling of transactions. It can detect whether a transaction is fraudulent or risky, not based on the customer’s credit score, credit history, email, IP address, login attempts, etc.
The solution handles ML models in real-time by granting ongoing access to them. It uses ETL processes to prepare and transform data obtained from the client. Other subcomponents send an API request to the solution, which with the help of ML models checks the transaction for fraud and risks and approves or rejects it.
The solution includes an end-to-end MLOps pipeline with ML model serving, API exposure of ML models, and the feature store database that stores all decisions, calculations, and requests. Each model has its own ETL pipeline for data preparation. We have migrated all data calculations to ETL pipelines to reduce the load on the central client’s database and remove the need for an MS SQL data server. The entire solution consists of the following components:
- A system that orchestrates the underlying decision-making process and exposes API used by external consumers;
- Feature Store, which provides a set of API endpoints to get precalculated ML feature values from Feature Store DB;
- ML execution runtime based on BentoML which serves ML models;
- Feature preparation application, which is used to convert Kafka messages with data update details to real-time feature values;
- Machine learning model versioning and repository (MLFlow)—a centralized repository of ML patterned models, used by data scientists and engineers.
We have implemented the Feature preparation application based on Apache Kafka and Debezium CDC to enable real-time data processing. We have also added Apache Flink open source stream- and batch-processing framework to provide real-time data calculations.
The dataflow has been divided into two categories:
- Batch jobs process ML models that do not require real-time calculations and are collected and calculated during a specific time span;
- Real-time data stream processes ML models that require real-time calculations.
The N-iX team has deployed several ML models for batch and streaming processing via Octopus, where the solution is packaged into the Docker containers. Also, we set up the latency to 250 milliseconds per request with a load of up to 1 million requests per month.
Furthermore, we have applied our DevOps expertise to set up infrastructure and the CI/CD pipeline to deploy the solution. We made it cloud-agnostic by running Docker images on multiple virtual machines in a Kubernetes cluster.
N-iX has also ensured that all sensitive data remained protected by working with and setting up strictly anonymized datasets provided by the client.
Finally, we were able to successfully set up the data flow and adjust all data calculations for real-time ML models. We have enabled the preparation and transformation of large amounts of data received from multiple sources, such as the solution’s database or the client’s database.
N-iX has helped the client develop a web-based cloud-agnostic solution powered by ML. After launch, the client saw a 35-point increase in their NPS score and a 20% growth in the number of customers. Additionally, the company has benefited from the solution in several other substantial ways:
- Streamlined transaction processing and removed the need to manually examine declined transactions with an ML-powered solution that provides full information about payments;
- Migrated all data calculations from the central client’s database to the ETL data pipelines, eliminating the need to use MS SQL data server and reducing operational costs;
- Decreased the latency of transaction processing from up to 5 minutes to 250 milliseconds, reducing the time customers had to wait for their transactions to complete;
- Accelerated time-to-market by setting up the infrastructure and the CI/CD pipeline for the solution.