Inspiration
As people who invest, we find that it is difficult to make informed decisions about stocks, especially since we have limited expertise and don't always understand how recent events impact different stocks. With that in mind, we wanted to develop a solution for those in similar positions, so that they are able to make the best decisions for their portfolio.
What it does
Our site assists people in the stock market with making informed decisions by providing them with a detailed analysis of stocks they are interested in, as well as insight of how recent events may impact that stock. We aggregate this information from various news sites across the political spectrum for maximum fairness. We also provide users with our sources in order to ensure that users have agency with their decision making process.
How we built it
Our solution handles simple user on-boarding and login with React and a MongoDB database. In addition, users indicate which stocks they are interested in, and we use the Polygon API to provide them with historical trends of those stocks.
Then, we use Gemini 1.5 Flash to generate 50 key words related to each stock. Using these keywords, we scrape different news sites to find articles containing those keywords, and generate individual predictions (positive, negative, neutral) for the stock based on the scraped articles. Finally, we aggregate these predictions and generate a final conclusion of the predicted trends along with the reasoning.
Challenges we ran into
The first challenge we ran into was that when we were web scraping, we realized that different news sites organize their sites very differently. Thus, it was difficult to figure out how to extract the relevant hyperlinks and information from each site.
Another issue we ran into was the process of tying the various moving parts together. In particular, we struggled to streamline the process of using the Gemini outputs for our web scraping process, as well as extracting the relevant components of the web scraping to our prediction generation.
An unexpected challenge we faced was selecting which information to display, and so we had to brainstorm ideas to display the most information with the least amount of visual clutter, while maintaining full transparency of our analysis.
Accomplishments that we're proud of
We successfully developed a reliable scraping algorithm to extract front-page news related to a given stock. Additionally, we designed a clean and user-friendly interface to display both predictions and headlines of scraped articles. We are also proud of the many moving parts we managed to tie together including the stock history graph and automatic analysis updates. These achievements laid a strong foundation for further improvements in data collection and presentation.
What we learned
During development, we realized that covering all relevant keywords for a given stock was challenging, as financial news often uses varied terminology and our generated keywords weren’t always relevant. We also encountered difficulties in web scraping, including inconsistent website structures, anti-scraping measures, and the abundance of sources needed for comprehensive coverage.
What's next for StockSage
We plan to expand our scraping capabilities by incorporating more reputable news sources for better data diversity. Additionally, we will refine keyword generation by combining Gemini’s AI output with a manually curated list to improve prediction accuracy and relevance. We will also look into developing our own prediction models (time-series analysis). Once perfected, we aim to deploy StockSage to ensure wider accessibility.
Log in or sign up for Devpost to join the conversation.