Inspiration
We were inspired by obsidian's mind map for notes. However, rather than map what you already know we try to map what you might be interested in. We also provide helpful tools to dive deeper into a topic you might find interesting.
What it does
Generates a mind map of similar topics based on a self hosted dockerized conceptnet database blended with a Wikipedia scraper. Based on selected nodes, we pull shorts from YouTube and integrate leaflet to find nearby locations.
How we built it
We self hosted our backend database with docker. Our Wikipedia scraper uses one of their own internal search algorithms to find related terms. We included options to use an LLM for the backend, theoretically providing better output. We utilized leaflet embedding for our map and yt-dlp for the YouTube scraper. For the WebApp, we used HTML CSS and JavaScript.
Challenges we ran into
We ran into numerous challenges during development. The largest was conceptnet's public server being down. Since it is known to be unreliable, we decided to self-host the 300GB 96million edge b-tree in a docker container. Leveraging build times and pipeline errors, the self-hosting was one of the largest roadblocks in our path.
Accomplishments that we're proud of
We are very proud of our frontend and our location and YouTube addons, which integrate smoothly. We optimized for speed, avoiding fetching all related info for a hundred different nodes. Even the extra information related to a topic is not fetched until the node is hovered.
What we learned
We learned a lot about self-hosting databases and fighting through errors with data pipelines. Additionally, we gained a lot of experience in frontend basic design and implementation.
What's next for Semantic
We want to continue expanding with multiple queries, accounts, and a better rabbit hole algorithm.
Log in or sign up for Devpost to join the conversation.