The idea was to create a platform that combines advanced machine learning models with a user-friendly interface to deliver deep insights from news content. How I Built the Project 1.Set Up the Environment: I used Python as the primary programming language and selected Streamlit for creating the web-based interface. I chose Hugging Face Transformers for leveraging pre-trained language models and FAISS for efficient similarity search.
2.User Authentication: Implemented a basic user authentication system using pickle to store and load user credentials.
3.Data Processing: Created a form in Streamlit for users to input news article URLs. Used UnstructuredURLLoader to fetch and load content from the provided URLs. Split the loaded text into chunks using RecursiveCharacterTextSplitter and created embeddings using HuggingFaceEmbeddings.
4.Building the Vector Store: Utilized FAISS to index the embeddings and saved the index to a pickle file for efficient querying.
5.Query Handling: Implemented a question-answering pipeline using Hugging Face Transformers to process user queries and retrieve relevant answers from the indexed documents.
Built With
- faiss
- huggingface
- langchain
- python
- streamlit
Log in or sign up for Devpost to join the conversation.