Tuned In

The Chrome extension turns any webpage content into a song recommendation, powered by on-device AI and Spotify.

Comment

Inspiration

I think Spotify has a great music recommendation algorithm and appreciated the opportunity to take advantage of that through different content mediums. This local model has provided the chance to develop an application that achieves this goal and has no online overhead or data collection issues.

What it does

Tuned In is built as a Chrome extension that bridges on-device AI summarization and Spotify's recommendation API. It employs local LLM with browser calls to make music recommendations based on the perceived mood of any webpage. (Such that is readable with Mozilla/readability)

How we built it

🧠 Core Technologies

  • Gemini Nano (Local Model) – Used through Chrome's Summarizer API to summarize and analyze webpage content on the user device itself.
  • Spotify API – Directs track lookup, genre matching, and search results by tempo.
  • Readability (Mozilla) – Extracts and parses the body text of the current web page for accurate summaries.

⚙️ Architecture

  1. Content Extraction: The extension, when invoked by the user, uses Mozilla's Readability library to extract useful text content of the currently active tab.

  2. Local Summarization: It is summarized by Gemini Nano with the help of Chrome's on-device Summarizer API. Depending on user preferences, the model can summarize the entire webpage or just the first chunk of text for optimal speed.

  3. Emotion and Tempo Derivation: The summarizer output is processed to anticipate the emotional tone (e.g., relaxed, hyped, sad) of the content by finding a relevant energy level:tempo and context:genres.

  4. Music Recommendation: These affective parameters are sent to a private Spotify API endpoint, which makes a request to Spotify's catalog for matching tracks using genre and BPM filters.

  5. User Interface: The suggested track is displayed in a popup panel with:

    • History list for previous recommendations
    • A Spotify Web Player link for the track
    • A source link to the page analyzed

  6. Local Storage & Privacy: All storage is local: summaries, settings, and histories. No webpage content or summaries are sent to external servers.

Use Cases

Example - Summarizing a Character Bio

Image of Music Summarizer getting a Pokemon related track when analyzing the Bulbapedia page for Bulbasaur In this example the extension recommends a track called "Pokémon" when analyzing the Bulbapedia page for Bulbasaur, this extension has genre validation check which progressively removes what it has perceived to be music genres in the text until a song is found.

Example - Hyperlinks to Original Content

image

Here the extension recommends listening to a calm track for studying and coding when analyzing the Chrome developer page for preparing extensions to the Chrome store.


You may have also noticed the,

"Source:"

field in the music history section. For each item this contains a link back to the webpage that was scanned to generate this music track.

image

The current Suggested Track's song title and artist name is also a hyperlink that takes you to the Spotify Web Player's URL for the track incase you want to quickly share it with friends instead of open in through your Spotify client.


Settings

Tuned In has 3 togglable settings:

image

If you're ever in doubt, hover over them for a tooltip that explains their purposes.

Themes

Theme changes are available using the propscolor library.

colours example

The default colour is gray.

screenshot_1 image


Full Page vs. Text Chunk Summarization

image image

The local model for the Summarizer API is very resource demanding. To avoid page summaries taking longer than 15 seconds, the extension gives you the opportunity to summarize only the first chunk of the webpages text content. This is often enough to get a strong recommendation, especially on encyclopedic webpages like Wikipedia or Fandom.

Quick Summary

screenshot_3

When Full Text checkbox is disabled, a prompt will appear indicating how many characters from the start of the webpage will be submitted into a single Summarizer API call.

⚠️ Text is very long (49,659 characters). Only the first 4,000 characters will be analyzed. Enable "Full Text" to process the entire page (takes longer).

Full Webpages Text

screenshot_4

When Full Text checkbox is enabled, a prompt will appear indicating how many chunks the text will be split into so that it can be summarized using a local model.

⚠️ Full text mode enabled. Text will be processed in 13 chunks (49,659 characters total). This will take longer.

This can be expediated with a hybrid model, but the current version of this extension doesn't implement that.


Size of Text Chunk

image image

The "Chunk:" field controls the character amount of the text that is inputed on each Summarizer API call. Chromes documentation recommends keeping this to 4000 or less, but I found an upper limit of 10,000 to work well as well.

Challenges we ran into

Prompt engineering was by far the hardest challenge. This was my first use of LLMs and it was quite daunting to get back results for genres that didn't exist in Spotify's API or make any sense. Mapping these genres to a backend that expected strict inputs for music genres relies on a cheap fallback method where the genres are dynamically popped from the array until a list of 20 tracks with the right genres is found. Another challenge was using the v1/recommendations API of Spotify which doesn't allow direct searches by BPM. In order to still get some level of momentum or energy from the webpage, I pull 20 tracks and find the closest matching BPM to the suggested BPM received from the model.

Accomplishments that we're proud of

Building and deploying my first Chrome Extension was an awesome experience, what I am most proud of with this project is the simple user interface, consisting of one button. There is nothing else needed for this app!

It truly is the every persons music recommendation extension, anyone who's interested in music can easily tie in their other hobbies and webpage content they keep up to date with, and generate relevant song recommendation based of it locally and quickly.

What we learned

How to prompt engineer! It was fun to see how powerful these things are. I thought this project would never see the light of day because I wasn't able to fine tune the model but through research and reading Chrome's documentary and ai examples, translating these prompts into intuitive experiences became trivial.

Understanding Spotify API without OAuth was interesting as well. It's the first time I've had a live feature as part of an app deployment, as my previous experience is mostly with static software, or applications that wrap other peoples backends.

What's next for Tuned In

  • Getting support using the /recommendations Spotify API to search by tempo and genre as seed_genre and target_tempo, which I was having too many API issues with to finish in time for this submission
  • Hybrid support for full webpage summaries.

Built With

Share this project:

Updates