An end-to-end machine learning project for detecting fraudulent insurance claims using predictive analytics and classification models.
- 📖 Project Overview
- Executive Summary
- 💼 Business Problem
- 🎯 Project Objectives
- 📊 Dataset Overview & Provenance
- 🧭 Project Methodology
- 📈 Exploratory Data Analysis & Key Findings
- 🤖 Machine Learning Models & Performance
- 💡 Business Interpretation & Recommendation
- 📁 Repository Structure
- 🛠️ Technologies & Tools
▶️ How to Run the Project⚠️ Project Limitations- 🚀 Future Improvements
- 👤 Author
Insurance fraud costs insurers billions of pounds every year, increasing operational expenses and ultimately driving up premiums for honest policyholders. Detecting fraudulent claims accurately is a challenging task because fraud cases represent only a small proportion of all claims, creating a highly imbalanced classification problem.
This project develops and evaluates multiple machine learning models to identify potentially fraudulent insurance claims using historical claims data. Beginning with comprehensive exploratory data analysis and feature engineering, the project compares Logistic Regression, Random Forest, and a hyperparameter-tuned Random Forest model to determine the most effective approach for fraud detection.
The final recommendation is based not only on predictive performance but also on practical business considerations such as minimising false investigations while maintaining the ability to detect genuine fraudulent claims.
Insurance fraud creates significant financial and operational challenges for insurers. This project develops an end-to-end machine learning workflow for identifying potentially fraudulent insurance claims and supporting more effective fraud investigation.
The analysis covers data familiarisation, data quality assessment, exploratory data analysis, feature preprocessing, model development, model evaluation, and hyperparameter optimisation.
Three classification models were evaluated:
- Logistic Regression
- Random Forest
- Tuned Random Forest
Because fraudulent claims represent a minority of the observations, model performance was assessed using multiple classification metrics rather than accuracy alone, including precision, recall, F1 score, ROC-AUC, and confusion matrices.
The results demonstrate an important operational trade-off. Logistic Regression achieved the highest fraud recall at 90.8%, identifying most fraudulent claims but generating a substantial number of false positives. The baseline Random Forest achieved high overall accuracy but detected very few fraudulent claims.
After hyperparameter optimisation, the Tuned Random Forest achieved a more balanced performance, with an F1 score of 0.262 and ROC-AUC of 0.842. It was therefore selected as the preferred balanced model for this analytical case study.
The project demonstrates how machine learning can support insurance fraud screening while recognising that real-world deployment decisions should also consider fraud losses, investigation costs, operational capacity, decision thresholds, governance requirements, and validation using current insurer data.
Insurance fraud presents a significant challenge for insurers because fraudulent claims can lead to financial losses, increased investigation costs, and inefficient use of operational resources.
The challenge is not simply to identify as many suspicious claims as possible. An effective fraud detection system must balance two competing risks:
- Missed fraud (false negatives): Fraudulent claims that are classified as legitimate may result in avoidable financial losses.
- False alarms (false positives): Legitimate claims incorrectly flagged as fraudulent require unnecessary investigation, increasing operational costs and potentially delaying genuine customer claims.
This challenge is particularly important because fraudulent claims represent only a small proportion of the available data. In such an imbalanced dataset, a model can achieve high overall accuracy while still performing poorly at detecting fraud.
The business problem is therefore to develop a predictive approach that can identify high-risk claims while maintaining a practical balance between fraud detection and investigation workload.
Can historical insurance claims data be used to identify potentially fraudulent claims accurately enough to support fraud investigators while controlling the number of unnecessary investigations?
The analysis addresses this question by comparing multiple classification models and evaluating their performance using fraud-relevant measures including precision, recall, F1 score, ROC-AUC, and false-positive and false-negative outcomes.
The primary objective of this project is to develop and evaluate a machine learning approach for identifying potentially fraudulent insurance claims while considering the operational trade-off between detecting fraud and generating unnecessary investigations.
The project aims to:
-
Understand the claims data
Examine the structure, quality, completeness, and characteristics of the insurance claims dataset before modelling. -
Identify fraud-related patterns
Use focused exploratory data analysis to investigate how fraud varies across relevant claim, policyholder, and vehicle characteristics. -
Prepare the data for predictive modelling
Transform categorical and numerical features into a suitable format for machine learning while avoiding unnecessary or non-predictive variables. -
Develop and compare classification models
Build Logistic Regression and Random Forest models and assess whether hyperparameter optimisation improves fraud detection performance. -
Evaluate models using fraud-relevant metrics
Compare model performance using precision, recall, F1 score, ROC-AUC, and confusion matrices rather than relying on accuracy alone. -
Translate model results into business insight
Assess the trade-off between missed fraudulent claims and false investigation alerts and recommend an appropriate modelling approach for the case study.
A successful solution should demonstrate meaningful ability to distinguish fraudulent from legitimate claims while providing a practical balance between fraud detection capability and investigation workload.
The project uses the Vehicle Insurance Claim Fraud Detection dataset, obtained from Kaggle for analytical and educational purposes.
The dataset contains 15,420 insurance claim records and 33 variables, covering policyholder characteristics, vehicle information, claim circumstances, policy details, and fraud outcomes.
| Attribute | Description |
|---|---|
| Records | 15,420 claims |
| Variables | 33 |
| Target Variable | FraudFound_P |
| Fraud Class | 1 = Fraudulent claim |
| Non-Fraud Class | 0 = Legitimate claim |
| Fraud Prevalence | Approximately 6% |
| Missing Values | No null values identified during the initial audit |
| Data Format | CSV |
The target distribution is highly imbalanced, with fraudulent claims representing only a small proportion of the observations. This imbalance influenced both the modelling strategy and the choice of evaluation metrics.
The dataset was sourced from the Vehicle Claim Fraud Detection dataset published on Kaggle. A separate provenance record is maintained in:
reports/dataset_provenance.md
This documents the dataset source and acquisition details used for the project.
Note: The dataset is used as a portfolio case study and should not be interpreted as representative of the current claims portfolio of any specific insurer.
The distribution highlights the strong class imbalance and demonstrates why overall accuracy alone would be an inappropriate measure of fraud detection performance.
The project follows a structured end-to-end analytics and machine learning workflow, moving from initial data understanding to model evaluation and business interpretation.
- Reviewed the dataset structure, dimensions, variable types, and target variable.
- Examined the meaning and distribution of key claim and policy features.
- Established the analytical context before performing modelling.
- Checked for missing values and duplicate records.
- Reviewed data types, categorical values, and variable consistency.
- Identified variables requiring special consideration before modelling.
- Examined the overall fraud distribution.
- Investigated fraud patterns across selected policyholder, vehicle, and claim characteristics.
- Prioritised analyses with potential business relevance rather than performing exhaustive EDA.
- Removed identifiers and variables unsuitable for predictive modelling.
- Separated predictor variables from the fraud target.
- Created training and testing datasets.
- Applied one-hot encoding to categorical variables using Scikit-learn's
ColumnTransformer.
Two baseline classification approaches were evaluated:
- Logistic Regression — used as an interpretable baseline with class weighting to address class imbalance.
- Random Forest — used to capture more complex and non-linear relationships in the claims data.
The Random Forest model was further optimised using GridSearchCV with five-fold cross-validation.
Hyperparameter selection focused on improving the F1 score, providing a better balance between precision and recall for the imbalanced fraud classification problem.
Models were compared using:
- Accuracy
- Precision
- Recall
- F1 Score
- ROC-AUC
- Confusion Matrix
Particular attention was given to false negatives and false positives because both have important operational consequences in insurance fraud investigation.
The final stage translated model performance into operational terms, considering the trade-off between:
detecting more fraudulent claims
versus
creating additional investigation workload
This provides the basis for selecting an appropriate fraud-screening approach rather than choosing a model solely because it achieves the highest accuracy.
Exploratory analysis focused on identifying claim and policyholder characteristics associated with differences in observed fraud rates. Because the objective is business decision support, the analysis prioritised patterns with potential relevance to fraud screening rather than presenting every variable in the dataset.
Fraudulent claims represent approximately 6% of the dataset, creating a substantial class imbalance.
This means a model could achieve high overall accuracy simply by predicting most claims as legitimate while still failing to identify fraudulent cases. Consequently, recall, precision, F1 score and ROC-AUC were treated as important measures alongside accuracy.
The exploratory analysis identified differences in observed fraud rates across several claim, vehicle and policyholder characteristics. These patterns suggest that fraud risk is not distributed uniformly across the claims portfolio.
Although some categories displayed higher observed fraud rates than others, these relationships should not be interpreted as proof that an individual claim is fraudulent.
Instead, the patterns provide potential predictive signals that can be considered alongside multiple other claim characteristics within a classification model.
Fraud is relatively rare, but observable patterns exist across claim characteristics. The combination of class imbalance and multiple potential risk signals supports the use of multivariate predictive modelling rather than relying on simple rules or overall accuracy.
Three classification approaches were evaluated to understand the trade-off between detecting fraudulent claims and limiting unnecessary investigation alerts.
1. Logistic Regression
Used as an interpretable baseline model with class weighting to account for the imbalance between fraudulent and legitimate claims.
2. Random Forest
Used to capture more complex and non-linear relationships between claim characteristics and fraud outcomes.
3. Tuned Random Forest
The Random Forest was optimised using GridSearchCV with five-fold cross-validation, with the F1 score used as the optimisation metric.
| Model | Accuracy | Precision | Recall | F1 Score | ROC-AUC |
|---|---|---|---|---|---|
| Logistic Regression | 0.637 | 0.132 | 0.908 | 0.231 | 0.814 |
| Random Forest | 0.938 | 0.400 | 0.054 | 0.095 | 0.840 |
| Tuned Random Forest | 0.907 | 0.250 | 0.276 | 0.262 | 0.842 |
The results demonstrate why accuracy alone is not sufficient for evaluating fraud detection models.
-
Logistic Regression achieved the highest recall at 90.8%, detecting most fraudulent claims. However, its low precision indicates that this came at the cost of a large number of false-positive alerts.
-
Baseline Random Forest achieved the highest accuracy at 93.8%, but its recall of only 5.4% meant that it missed most fraudulent claims. Despite its impressive accuracy, this makes it unsuitable as a fraud-screening model in its baseline form.
-
Tuned Random Forest improved fraud recall to 27.6% and achieved the strongest overall F1 score (0.262) and ROC-AUC (0.842). This represents a more balanced trade-off between fraud detection and false-positive control.
The tuned model correctly identified 51 fraudulent claims, while 134 fraudulent claims were missed and 153 legitimate claims were incorrectly flagged for investigation.
The ROC-AUC of 0.842 indicates that the tuned model has useful ability to distinguish between fraudulent and legitimate claims across different classification thresholds.
There is no universally best model across every operational objective. Logistic Regression is preferable when maximising fraud detection is the priority, while the Tuned Random Forest provides a more selective and balanced screening approach when investigation capacity and false-positive workload must also be considered.
The modelling results highlight an important operational reality in insurance fraud detection: the model that identifies the most fraud is not necessarily the model that provides the most practical screening process.
The two strongest approaches serve different business priorities:
| Business Priority | Preferred Approach | Reason |
|---|---|---|
| Maximise fraud detection | Logistic Regression | Identified 90.8% of fraudulent claims |
| Reduce investigation workload | Tuned Random Forest | Produces a more selective set of fraud alerts |
| Balance precision and recall | Tuned Random Forest | Highest F1 score among the evaluated models |
| Rank fraud risk across thresholds | Tuned Random Forest | Highest ROC-AUC at 0.842 |
Logistic Regression may be appropriate where the financial cost of missed fraud is high and the insurer has sufficient investigation capacity to review a larger number of flagged claims.
The Tuned Random Forest provides a more selective alternative. Although it identifies fewer fraudulent claims, it generates substantially fewer false-positive alerts and achieves the strongest balance between precision and recall among the models evaluated.
For this analytical case study, the Tuned Random Forest is selected as the preferred balanced model.
However, a real insurer should not select a production fraud model solely on F1 score or ROC-AUC. The final decision threshold should reflect:
- expected financial loss from undetected fraud;
- cost of investigating legitimate claims;
- available fraud investigation capacity;
- customer impact from unnecessary claim reviews;
- model governance and monitoring requirements; and
- performance on current, representative insurer data.
Rather than automatically rejecting claims, the model would be better positioned as a fraud-screening and prioritisation tool.
Potentially high-risk claims could be referred for further investigation, while claims assessed as lower risk could continue through the standard claims process.
The purpose of predictive fraud analytics is therefore not to replace investigators, but to help focus investigative resources on claims where additional review may provide the greatest value.
Insurance-Fraud-Detection-Analytics/
│
├── assets/
│ └── insurance_fraud_detection_banner.png
│
├── data/
│ ├── raw/
│ └── processed/
│
├── notebooks/
│ ├── 01_data_familiarisation.ipynb
│ ├── 02_data_quality_audit.ipynb
│ └── 03_exploratory_data_analysis.ipynb
│
├── outputs/
│ ├── model_comparison.csv
│ └── tuned_random_forest.pkl
│
├── reports/
│ ├── dataset_provenance.md
│ └── project_brief.md
│
├── visualizations/
│ ├── fraud_distribution.png
│ ├── fraud_rate_by_accident_area.png
│ ├── fraud_rate_by_vehicle_category.png
│ ├── logistic_regression_confusion_matrix.png
│ ├── tuned_random_forest_confusion_matrix.png
│ ├── tuned_random_forest_roc_curve.png
│ └── model_comparison.png
│
├── README.md
├── requirements.txt
└── LICENSE
| Folder | Purpose |
|---|---|
assets/ |
Project branding and README banner |
data/raw/ |
Original source data |
data/processed/ |
Data prepared or generated during the analytical workflow |
notebooks/ |
Main analysis, data-quality and machine-learning workflow |
outputs/ |
Model results and exported analytical outputs |
reports/ |
Project documentation and dataset provenance |
visualizations/ |
Charts and model evaluation figures used throughout the project |
The numbered notebooks provide a clear progression from data understanding → data quality assessment → exploratory analysis and predictive modelling.
The project was developed using the Python data science ecosystem, with Git and GitHub used for version control and portfolio presentation.
| Technology | Application |
|---|---|
| Python | Core programming language for analysis and modelling |
| Pandas | Data manipulation, exploration and preparation |
| NumPy | Numerical operations and array processing |
| Matplotlib | Data and model-performance visualisation |
| Scikit-learn | Data preprocessing, classification, model evaluation and hyperparameter tuning |
| Jupyter Notebook | Interactive analysis and documentation |
| Git | Version control and project tracking |
| GitHub | Repository hosting and portfolio presentation |
- Logistic Regression
- Random Forest Classification
- Class weighting for imbalanced data
- One-Hot Encoding
- Train/Test Split
- GridSearchCV
- Five-fold Cross-Validation
- Hyperparameter Optimisation
- Confusion Matrix Analysis
- ROC-AUC Evaluation
The complete Python package requirements for reproducing the analysis are available in requirements.txt.
git clone https://github.com/Utulu1/Insurance-Fraud-Detection-Analytics.git
cd Insurance-Fraud-Detection-Analyticspython -m venv .venvActivate the environment:
Windows
.venv\Scripts\activatemacOS / Linux
source .venv/bin/activatepip install -r requirements.txtOpen Jupyter Notebook or VS Code and run the notebooks in sequence:
notebooks/01_data_familiarisation.ipynbnotebooks/02_data_quality_audit.ipynbnotebooks/03_exploratory_data_analysis.ipynb
The notebooks document the progression from initial data understanding and quality assessment through exploratory analysis, preprocessing, machine-learning development and model evaluation.
Key project outputs are available in:
visualizations/— EDA and model evaluation figuresoutputs/model_comparison.csv— model performance comparisonreports/— project documentation and dataset provenance
Note: Ensure the required dataset is available in the expected
data/raw/location before running the notebooks.
This project demonstrates an end-to-end fraud analytics workflow using a public insurance claims dataset. The results should therefore be interpreted within the context of an analytical portfolio case study rather than a production fraud detection system.
Key limitations include:
-
Historical public dataset: The data may not reflect current fraud behaviours, insurance products, claims processes, or emerging fraud patterns.
-
Limited business context: The dataset does not provide detailed information about the financial value of individual claims, investigation costs, confirmed fraud losses, or operational investigation capacity.
-
Class imbalance: Fraudulent claims represent approximately 6% of the dataset, making model evaluation sensitive to the choice of performance metric and classification threshold.
-
Default classification thresholds: Model comparisons primarily use standard classification thresholds. In practice, insurers would optimise thresholds according to fraud losses, investigation costs, risk appetite, and operational capacity.
-
No external validation: Model performance was evaluated using a held-out portion of the available dataset but was not validated against an independent insurer dataset.
-
Model predictions are not fraud decisions: A claim flagged by the model represents elevated predictive risk rather than proof of fraudulent activity. Human investigation and appropriate governance would remain necessary.
The reported model performance demonstrates the potential value of predictive analytics for fraud screening and investigation prioritisation, but further validation would be required before any real-world implementation.
The current project provides a strong foundation for insurance fraud screening, but several extensions could improve both predictive performance and practical business value.
-
Classification threshold optimisation
Evaluate alternative probability thresholds rather than relying only on the default 0.50 threshold. This could help achieve a better operational balance between fraud recall and false-positive investigation workload. -
Cost-sensitive model evaluation
Incorporate estimated fraud losses and investigation costs to evaluate models based on financial impact rather than predictive metrics alone. -
Model explainability
Introduce techniques such as feature importance or SHAP analysis to better understand the factors influencing individual fraud-risk predictions. -
Additional model benchmarking
Evaluate selected boosting algorithms such as XGBoost or LightGBM to determine whether they provide meaningful improvement over the current models. -
Independent validation
Test the selected model on a separate and more recent insurance claims dataset to assess generalisability. -
Model monitoring
In a production environment, monitor model performance and changes in claim behaviour over time to identify potential model drift and emerging fraud patterns.
The most valuable immediate extension would be threshold and cost optimisation, because the current results demonstrate a clear trade-off between detecting fraudulent claims and generating false-positive investigation alerts.
This would allow model selection to move beyond purely statistical performance toward a more explicit business-value framework.
Actuarial Science | Insurance Analytics | Data Analytics
This project forms part of my analytics portfolio, focused on applying statistical analysis, machine learning, and business intelligence techniques to practical insurance and business problems.
My background in Actuarial Science, Mathematics and Statistics informs my approach to risk analysis, predictive modelling, and evidence-based decision-making.
- Actuarial Analytics
- Insurance & Risk Analytics
- Predictive Modelling
- Data Analytics
- Statistical Modelling
- Business Intelligence
Python · SQL · Excel · Power BI · Tableau · Git
- GitHub: Utulu1
- LinkedIn: Anthony Utulu
This project is licensed under the MIT License.
DATA • INSIGHT • IMPACT