Inspiration
When managing my Gmail inbox(es), I often skim through emails to decide whether to act on them immediately, save them for later, or ignore them entirely — essentially performing inbox review and triage. However, this process takes up a lot of time and I needed a better way to do it.
Most of the existing AI tools focus on summarization using language models (LLMs), but they often fall short when it comes to efficiently surfacing and highlighting critical details, without losing context. Additionally, these tools frequently require sending private information to the cloud, which raises privacy concerns.
What it does
Highlights AI is a chrome extension that reviews emails at the click of a button, identifies key details from Gmail message contents, and visually highlights/marks up emails for faster inbox review and triage i.e., it's an AI assistant that reads emails for you and highlights important details.
How we built it
Built a Chrome extension that leverages Chrome’s built-in AI to process Gmail messages locally, extract key highlights, and visually mark them up. Also in use is some traditional (pre-Gen AI) NLP techniques to augment (pre/post-processing) language model capabilities because email contents are not perfect and come in various formats and with various challenges.
Challenges we ran into
- Parsing email contents is a huge challenge -- email contents are not perfect and come in various formats (text, html, base64 etc.)
- Latency. In-built Chrome AI could feel slow on less powerful devices and thus optimizing for latency was a huge challenge. This is still an area for improvement but feel like it's in a good state at the moment and will be monitoring Chrome team's updates on this front to incorporate more changes and optimize further.
- Working with an early but promising technology i.e., Chrome's built-in AI is still not GA.
Accomplishments that we're proud of
Building something useful and open source using local, private AI; in less than a day.
What we learned
Most important takeaway is on-device language models are still in their early days and might not be as powerful as cloud LLMs. It takes a lot of work to find and build something useful as a result. I believe the key is to identify one specific task and optimize for it vs. trying to compete with cloud LLM capabilities.
What's next for Highlights AI
- Building context from media, attachments etc. in emails to better identify relevant highlights
- While the initial focus is on Gmail, the plan is to extend the solution to other web content, such as blogs and articles, by applying the same principles and learnings from this initial version.
Log in or sign up for Devpost to join the conversation.