cogwalk

• 

  first page

Inspiration

With the rapid development of modern LLM technology, learning seems easier than ever. With abundant data and large language models as assistants, people are supposed to access knowledge more easily. However, another reality is that media today grows so quickly that people’s attention is constantly drawn to short, new, and fast-consuming content. Many hesitate to read anything “too long” or text-heavy, which they often consider “boring.” Therefore, I came up with the idea of making the reading process more enjoyable.

Moreover, I want to show that texts are not boring at all. Even the most complex or abstract terminology comes from someone’s lived experiences and thoughts. I believe that beyond simply teaching or informing, complex explanatory texts also contain the author’s personality and thinking style. That is why I want to extract the author’s tone.

In terms of structure, whether at the sentence level or the overall organization, it reflects the author’s way of thinking and arranging ideas. This is the key to understanding a person’s perspective. This is also why I chose to visualize a mind map or domain map of their thoughts: I believe that the structure of knowledge is one of the easiest ways to remember information.

That is why I named the project Cogwalk: using the structure of a text as a mapping bridge, allowing users to “walk through” a part of the writer’s cognition.

What it does

It analyzes the uploaded PDF and generates a visualized “mind city,” where each key term becomes a house. The size of the house represents the difficulty of learning that concept. Relationships appear as roads between houses.

By clicking a house’s door, a dialog box pops up showing: the definition of the term, where that term appears in the original text, and an audio explanation generated in the author’s tone using Gemini and ElevenLabs.

By consulting Gemini, it extract the text as a json file which contains: -the author's tone when write the context, -the main goal of this entire conetxt -the important key knowledge terms which includes: key knowledge term name, the difficulty level of this content(since sometime you need to know one thing before the other, for difficulty level show up prequest if it exists), gemini's explaination&example for better understanding toward this term, the orginal text describe the knoledge term -the relationship (hierarchy, dependency, association, sequence, overlap, or influence) between each knowledge term

How we built it

The project is built using TypeScript and React, with Three.js for low poly 3D models, using Gemini and ElevenLabs API.

Challenges we ran into

Designing an effective JSON structure and UI for visualizing the map Configuring the deployment on Cloudflare

Accomplishments that we're proud of

The process of turning the idea into something functional is already a huge accomplishment for me.

What we learned

Technically, I gained a lot from workshops and sponsor packages. Although I could only choose limited technologies for the application, learning about them helped me understand the industry’s focus on product design. I also learned a lot from several failed attempts to implement databases and APIs—they helped me recognize what skills I still lack. For soft skills, time management was the most important. With such a short timeframe, some tasks had to be prioritized and others had to be left behind.

What's next for cogwalk

-A resource database to store previously generated text cities/villages -More creative visualization structures — for example, I once considered a “meal-based” metaphor, but didn’t have time to implement it -More features that extract the author’s thinking patterns, not just tone, including ways to filter out “unique thoughts” -Support for non-educational or casual texts to allow visualization of personal writing styles -More domain-specific interactions, such as: chemical structure diagrams for medical or science textbooks, timeline visualizations for historical texts, or internal reaction animations similar to game mechanics -A fine-tuned model to automatically improve and adapt the mapping system

Built With

• api
• elevenlabs
• gemini
• json
• react
• three.js
• typescript