Inspiration
Difficult to read complicated books. Literature, both academic and poetic, can be dense. I was reading Grapes Of Wrath and I was confused in many parts.
What it does
This app allows you to read ebooks.
Alongside the book, users can select text that they’re confused about. GPT then generates simple annotations for them explaining the text. Also can make image annotations. To visualize settings or items Learning check at the end. GPT generates questions at the end of chapters to see whether you understood them. GPT can also grade your response and provide feedback on how to improve and what’s missing. Helps ensure that people understand what they read
How we built it
We have a flask python backend and a web frontend. The app heavily relies on OpenAI to do both the text and image annotations.
Challenges we ran into
Image generation was hard. DALL-E is not able to understand complicated english. So, we had to use GPT to simplify those requests.
EPUB files are a collection of html files. Since our frontend relies on html objects, the book itself would often interefere with that logic. To get around this, we had to build a sanitizer to make sure our code ran as expected and didnt modify the book itself.
Testing with GPT was slow because it takes time for its API to return responses.
We ran into Cross-Origin Resource Sharing issues where the browser would refuse to load resources. This slowed down development.
Accomplishments that we're proud of
We're proud of alligning annotations with the text in the book. The UI looks amazing and we finished what we planned to do.
What we learned
We learned prompt engineering since it is an integral part of the product. We learned how to use flask and integrate it with javascript. We learned the OpenAI API's
What's next for Napkin
Partnering with schools: our product is geared towards helping a traditional classroom with one instructor and multiple students so we want to build around that. Have educators be able to track progress and provide help to students that really need it.
Log in or sign up for Devpost to join the conversation.