NUS-SDS-Datathon-Singlife-Team 89

Inspiration

Recognising the trend of the increasing use of big data in this digital era, my team is motivated to take the leap of faith in exploring the field for data analyst and experts to improve our skills and knowledge. As such, we have decided to join the NUS Datathon 2024 as a gateway to learning more about data analytics as well as challenging ourselves to put our prior knowledge to good use.

What it does

Our Decision Tree Model works by recursively partitioning the dataset into subsets, followed by considering different features at each node to maximize the separation of the target class. The decision tree evaluates the significance of each factor by assigning weights to the features, allowing it to prioritize and rank them based on their impact on the target class, which is whether a customer will purchase life or health insurance products within the next three months.

How we built it

We first processed the data by removing unnecessary columns ( columns with > 50% of missing data ), afterwards and by imputing the missing values for the remaining columns. Afterwards, we encoded the categorical data via ordinal encoding before balancing the data using SMOTE. We then removed the low variance variables ( columns with observations of only one unique value ) before starting to construct our Decision Tree Model.

Challenges we ran into

Through the process, we were often faced with various errors generated and it was quite challenging to debug some of them as we were not able to understand what some of the errors meant at first glance, which resulted in a lot of time to be taken to debug the codes rather than doing the actual coding.

Accomplishments that we're proud of

We are proud that after 2 days of digesting the new concepts and building the model, we are able to build a decision tree model, with a macro average f1-score of 0.88, which is a commendable result to us.

What we learned

As Year 1 students majoring in either Data Science and Analytics or Data Science and Economics, most of the contents delivered by the SDS workshop team were completely new to us. We learnt alot in the past 2 days, from the basics of Exploratory Data Analysis (EDA) to the different kinds of models and model evaluation methods.

What's next for NUS-SDS-Datathon-Singlife-Team 89

We hope to leverage the insights gained for use in our future projects and we are excited about the prospects of making meaningful contributions to our field through data-driven decision making.