Combination of different parameters tested

Introduction

As data science students, our inspiration stemmed from a desire to apply our theoretical skills into the context of real-world challenges. We were motivated by the opportunity to tackle complex problems in the insurance industry using data science techniques. Moreover, this endeavour provides us with an invaluable opportunity to assess our compatibility with this dynamic industry.

Data Processing

Dealing with a dataset comprising 304 columns presented significant challenges in preprocessing and feature engineering. Managing such high-dimensional data required meticulous handling to ensure model performance and efficiency, as well as to minimise the curse of dimensionality. Feature selection was carried out after performing exploratory data analysis and visualising correlation plots.

To handle the highly imbalanced dataset, we decided to use SMOTE. Out of 17992 rows, only 710 (4%) of them had value 1 for the target column. As such, undersampling the majority class may not be suitable as we would lose too much data in doing so. Hence, oversampling of the minority class has been adopted to prevent the model from being trained in a skewed manner towards the majority class.

Modelling

For this project, we experimented with various models during the construction of our predictive model. Recognising the high-dimensionality of our dataset, we had to opt for more flexible models. Our approach involved comparison of model performance of different machine learning techniques: Random Forests for its adaptability, and Neural Networks for its ability in recognizing complex patterns.

The data provided was split into a training and validation set. The random forest and neural network models were trained using the training set and the validation set was used for hyperparameter tuning. In the end, the random forest model showed higher overall performance and was chosen as the main structure.

From here, fine-tuning was conducted by testing different combinations of the parameter inputs for the RandomForestClassifier model such as criterion, max_depth, max_features, bootstrap and ccp_alphas.

Performance

We aimed for a high recall rate as we felt that missing potential clients who would purchase insurance carries a greater cost than the resources spent communicating with uninterested clients. Additionally, this approach might also have the unintended positive consequence of increased awareness among such clients about the policies offered by Singlife, potentially converting them into future clients.

In the end, our model obtained a recall rate of 0.7. While there remains room for improvement, the model's performance surpasses chance, especially considering that only a small proportion of 4% of clients in the dataset made a policy purchase. This shows how such a prediction model has merit in decision making processes.

Further Improvements

Moving forward, we plan to further refine our model by incorporating a larger volume of data. As any machine learning approach tends to need a significant amount of data to generalise and capture the underlying variability well, increasing the dataset size beyond our current 17992 rows will enhance the model's performance and robustness.

Additionally, we could also consider an ensemble learning approach. By combining the predictions of multiple models, we can potentially improve the overall performance and reliability of our predictive model.

Learning Points

Through this project, we gained invaluable insights into the nuances of predictive modelling in the insurance domain. We honed our skills in handling large datasets, feature engineering, and implementing advanced machine learning algorithms. We also learned more about how larger, established companies like Singlife would approach data collection and management.