ML Model for Sentence Factual Worthiness

Inspiration

In today's age of information overload, it's crucial to differentiate between factual statements that hold public interest and those that don't. Misinformation can spread rapidly, and having a tool to identify and prioritize statements that require fact-checking can be invaluable. This project aims to develop a machine learning model that determines the worthiness of sentences, helping to focus fact-checking efforts on statements that matter most to the general public.

What it does

The developed machine learning model takes an input sentence and predicts whether it is a factual statement worth fact-checking. The model is trained on a dataset of labeled sentences, where each sentence is labeled as either "Yes" (indicating a check-worthy sentence) or "No" (indicating a sentence that is not check-worthy). The model utilizes a TF-IDF vectorizer for feature extraction and a Logistic Regression classifier for prediction.

How we built it

Data Collection and Preprocessing: Loaded the labeled dataset from the provided CSV file. Converted the 'Category' column to binary labels: "Yes" -> 1, "No" -> 0. Split the data into training and testing sets.
Feature Extraction: Used TF-IDF (Term Frequency-Inverse Document Frequency) vectorization to convert the text data into numerical features. Removed stop words and set a maximum document frequency to 0.7 to filter out less informative words.
Model Training: Trained a Logistic Regression classifier using the TF-IDF features. Optimized the model with a maximum iteration of 1000 to ensure convergence.
Model Evaluation and Saving: Evaluated the model's performance on the testing set using accuracy and classification report. Saved the trained model using joblib for future use.
Prediction: Loaded the trained model from the saved file. Took user input for a new sentence to predict its worthiness. Utilized the trained model and TF-IDF vectorizer to predict and display the worthiness of the input sentence.

Challenges we ran into

Data Quality and Quantity: The labeled dataset might not be comprehensive or representative of all types of sentences, making it challenging to train a robust model.

Feature Engineering: Determining the right set of features and their representation (TF-IDF) to capture the essence of the sentences was crucial for the model's performance.

Model Generalization: Ensuring that the model generalizes well to unseen data and can effectively differentiate between check-worthy and non-check-worthy sentences.

Accomplishments that we're proud of

Developing a Functional Model: Successfully developed a machine learning model that can predict the worthiness of sentences with a reasonable accuracy.

Efficient Feature Extraction: Implemented TF-IDF vectorization to efficiently convert text data into numerical features, aiding in better model performance.

User-Friendly Interface: Created a simple user interface where users can input a sentence to get its worthiness prediction, enhancing the model's accessibility and usability.

What we learned

Importance of Data Quality: The quality and representativeness of the training data are crucial for training a robust and accurate machine learning model.

Feature Engineering Significance: The importance of selecting and engineering the right set of features to capture the relevant information from the text data.

Model Evaluation and Optimization: Learned the importance of evaluating the model's performance using various metrics and optimizing the model parameters for better results.