Scratch Algorithm Machine Learning - Pacmann Project

Step #1 - Select Machine Learning Algorithm

Search for algorithm references from valid sources. Examples of lecture notes/slides, papers, conferences, and books. Lecture notes from CMU Statistics (https://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf).

Step #2 - Create Algorithms from Scratch

To understand the learning components of the Logistic Regression algorithm, I will refer to the provided source, "Logistic Regression" Lecture notes from CMU Statistics (https://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf).

For fitting:
- Optimization: The cost function that is optimized is the Cross Entropy Loss.
- Optimization objective: Minimize the cost function.
- Varied parameters for cost function optimization: Model parameters, i.e., weights and bias.
- Optimization algorithm: Gradient Descent.
For making predictions:
- Predictions are made by calculating the sigmoid function of the dot product of features and weights, plus the bias term. Pseudocode for the Algorithm:
Here is the pseudocode for fitting the Logistic Regression model: C:\ProjectAdvancedML\scratch-algorithm-ML\pseudocode_fitting._logistic_reegression_model.py

Here is the pseudocode for making predictions with the trained Logistic Regression model: C:\ProjectAdvancedML\scratch-algorithm-ML\pseudocode_making_predictions_trained_logistic_regression_model.py

Code Implementation:

Here is the implementation of the Logistic Regression Algorithm in a file named logistic_regression.py: C:\ProjectAdvancedML\scratch-algorithm-ML\logistic_regression.py

Step #3 - Analysis, Conclusion, and References

Once I have implemented the Logistic Regression algorithm from scratch, the next step is to apply it to solve simple data problems. These steps to apply the code:

Prepare a relevant dataset for classification problems.
Perform any necessary data preprocessing, such as normalization or one-hot encoding.
Split the dataset into training and testing data.
Create an instance of the LogisticRegression class from the implemented code.
Call the fit method on the LogisticRegression object, providing the training data as arguments.
Make predictions on the testing data using the predict method.
Evaluate the performance of the model using appropriate evaluation metrics, such as accuracy or a confusion matrix.
Experiment with hyperparameters, such as learning rate and the number of iterations, to improve the model's performance.

References:

"Logistic Regression" lecture notes from CMU Statistics by Cosma Rohilla Shalizi (https://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf)

"Building Logistic Regression from Scratch: Unveiling the Learning Components of Machine Learning"

Framework:

Goal: The primary objective of this project is to equip participants with a comprehensive understanding of the learning components of machine learning, specifically focusing on logistic regression. The project aims to guide participants through the process of building a logistic regression algorithm from scratch, including pseudocode explanation, coding, and implementation.
Impact: This project holds the potential to significantly impact participants' understanding of machine learning algorithms and their underlying learning mechanisms. By delving into the intricacies of logistic regression, participants will gain valuable insights into the gradient descent optimization approach and its role in parameter estimation. This knowledge can serve as a solid foundation for more advanced machine learning concepts.
Challenges: The project presents challenges in breaking down complex concepts of logistic regression into understandable pseudocode and code implementation. Ensuring the accuracy of the algorithm's implementation, particularly in the context of gradient descent, requires careful attention to detail. Participants may also encounter challenges in troubleshooting and debugging during the coding process.
Interesting Findings: The project's exploration of logistic regression from scratch unveils the inner workings of the algorithm's learning components. Participants will discover how gradient descent iteratively updates model parameters to achieve convergence, ultimately enabling accurate predictions on new data. The project's emphasis on encapsulation within a class highlights the importance of modularity and code reusability in machine learning implementations.

By providing a step-by-step breakdown of building logistic regression and emphasizing both the theoretical and practical aspects, this project empowers participants to grasp the essence of machine learning algorithms and encourages them to explore more advanced topics with a strong foundation.

Built With

Created by

In the project "Know How to Use Machine Learning from Scratch and How its Learning Component Works," my contribution involves guiding and facilitating the understanding of machine learning principles and the implementation of a fundamental algorithm. Specifically, I lead the exploration and explanation of the learning component of the Logistic Regression algorithm. I provide pseudocode explanations, outline the coding process, and support the implementation and experimentation of the algorithm in Python.

By breaking down the algorithm into understandable steps and encapsulating its logic within a class, I empower learners to comprehend the underlying mechanisms of Logistic Regression. This approach fosters code reusability, maintainability, and customization, making it a valuable asset for broader machine-learning endeavors. Through this contribution, I aim to demystify machine learning and empower individuals to grasp and apply essential concepts in building predictive models from scratch.

Zulkarnain Prastyo