Inspiration
We wanted this project to be able to help us share our love of music with others who may not be able to enjoy it through sound alone. Inspired by our love of music as well as technology, we hoped to make things more accessible for people with our passion for technology.
What it does
Our ASL Music Generator takes in the user's choice of song/artist and returns the lyrics, translating it to American Sign Language
How we built it.
We started by setting up our workspace, working on Replit at the start to make collaboration easier for our team, and in the later stage - GitHub and VSCode. Essentially, we prompt engineered a prompt to call using the Groq API so that we could return song lyrics with the user_inputted variables song_name and artist. We used javascript to connect the front-end and back-end in receiving information. We used Figma to brainstorm the format and structure of our app.
Challenges we ran into
We ran into a lot of challenges with the setup of the project because we had spent a lot of time downloading a lot of packages as we wanted to use a large and complicated model that was able to translate text to pose/animation (American Sign Language). One of the more difficult processes was combining the front end with the back end in terms getting the song lyrics and inputting into the LLM that translates the lyrics into HamNoSys and animation/pose format. We also ran into some challenges when using Replit, it was super helpful when collaborating with a team but we also ran into a lot of efficiency issues when downloading the packages we needed to get the model to run. At the end, we had to transfer all of our code we had made on Replit into VSCode and still had to use GitHub the model we were implementing was too large.
Accomplishments that we're proud of
We're really proud of the fact that we were able to condense such a large and complicated model to fit our project.
What we learned
We learned that we can sleep in any position when we're tired. We also learned how to sign language "We are cooked" (No but we actually had a fun time learning a couple of sign language phrases). Additionally, we gained more experience in full-stack web development and creating a project centered around an AI model.
What's next for ASL Lyric Generator
Our next plan is to successfully deploy the ASL Lyric Generator is to maybe implement some learning and educational features. Right now it's a lyric generator to make music more accessible to people, but we want to make this a learning experience, helping people learn sign-language through music!
Credit for the sign.mt LLM
Translate Model GitHub Repo
Log in or sign up for Devpost to join the conversation.