Orin | Devpost

Hero Page!

Inspiration

Melodies coexist between a Carnatic musician and a pianist, yet are difficult for them to share. On campus, many talented international musicians are unable to join bands because of a disconnect between musical styles. As musicians experienced in multiple kinds of music, we understand the challenges of on-campus musicians in different styles like Korean, Persian, Arabic, Indian, etc. Standard notation may not show a Carnatic ornament or a pitch between piano keys. Orin grew out of that idea of collaboration, using cutting-edge AI tools for on-campus students and beyond.

What it does

Orin connects three tools:

Orin chat uses Gemini to understand a request, refine it into a music spec, answer questions, and explain musical concepts. It also generates musical samples for musicians to understand with ElevenLabs.
Notation Studio accepts audio or typed notes and converts between supported traditions, with confidence information and a note-by-note loss report.
Workspace lets users arrange clips, edit notes, add drum patterns, and export a mix, powered by a Gemini to ElevenLabs pipeline.

Conversions pass through a neutral representation that preserves pitch in cents from the tonic, timing, and measured ornament contours using a Fast Fourier Transform. Supported formats include Western classical and pop, rock, Carnatic, Arabic maqam, and Persian dastgah. Orin reports approximations such as rounded microtones and dropped ornaments.

How we built it

We built a Python music-analysis and conversion library, a FastAPI backend using Render, and a Next.js app. The website is hosted on Vercel with a public API. The audio pipeline estimates pitch, tonic, notes, ornaments, and rhythm, then assembles an intermediate representation that format adapters can render in different notations. A shared conversion engine supports format-to-format conversions and produces the loss report.

Gemini is integrated for conversational intent, music-spec refinement, and explanations grounded in conversion results. The model is configured through the environment. ElevenLabs handles music generation and narration. Built-in stand-ins keep the main flow demonstrable without provider keys if running offline.

Challenges we ran into

Music traditions do not fit neatly into one notation system. We had to preserve information like microtonal pitch, ornament contours, and free timing long enough for each target format to decide what it could express. Technically, it was challenging to create a multi-page Next platform, being able to render API keys with a backend, as well as roadblocks with the Fourier Transform and musical nuance!

Accomplishments that we're proud of

We built an end-to-end experience across chat, notation, and music editing, with handoffs between each part. The same conversion engine handles all supported format pairs through a neutral intermediate representation, and each conversion can explain what the destination notation could not preserve.

We’re also proud that Orin can run in offline mode with labeled stand-ins, and that the interface presents confidence and uncertainty alongside its musical results. The README reports passing offline Python and frontend test suites and a successful production build; those results have not been independently rerun for this pitch.

What we learned

A useful music translation needs more than a plausible-looking score. It needs to preserve important details. We also learned how much work it takes to represent different musical ideas without forcing them into Western notation. We learned a lot about voice and music generation, and interaction between APIs.

What's next for Orin

Have practitioners review the traditional tables, improve recognition on short and mixed-style clips, and expand beyond single-melody transcription. Longer term, Orin can support more traditions and give musicians richer ways to compare and create while giving producers and enthusiasts a way to engage with cultural boundaries. We hope to roll out Orin in educational institutions, cultural festivals, and music producers in the future!