NewsPulse

Inspiration

I found a basic problem statement of News analysing on a hackathon page and thought of better features as I started building it.

What it does

It is a platform that can be used in various areas like PR departments, Finance departments, transportation, by individuals, etc. for fast and latest information gain regarding the topics of their choosing in a matter of seconds with little to no effort which normally takes a team of people and hours of hardwork.

How I built it

Started with ElastiSearch and the main scraper and nlp models then made it into an autonomous agent that does everything themselves.

Challenges we ran into

Building a good scraper was difficult. Speed was a major issue as it depends on hardware.

Accomplishments that we're proud of

The scraper, models and the speed is something I'm happy about (made my challenges my achievements :) ).

What we learned

Discovered Elasticsearch, got to learn about various nlp concepts, scraping techniques and how to make a big task work significantly faster on the same hardware by various optimizations.

What's next for NewsPulse

Opt to pay for use services to scale it to a global level and provide personal company to company optimization as per requirement.

Built With

• elasticsearch
• fastapi
• python
• pytorch
• trafilatura