SeeSay

Inspiration

Drawing inspiration from the incredible human capacity to overcome challenges, SeeSay was born with a vision to transform lives. Imagine a world where visual impairments no longer limit the ability to perceive the beauty around us. Inspired by the unwavering spirit of those who navigate the world with visual challenges.

Our journey began with the determination to bridge the gap between the visually impaired and the visual world. By harnessing the power of language and cutting-edge technology, we envisioned a way for individuals to explore, understand, and connect with their environment like never before.

But our mission extended beyond a single challenge. We saw the untapped potential to serve an even wider audience. As we delved into creating an interface for the visually impaired, we realized the immense value this could bring to the elderly as well. Our older generations, who have contributed so much to society, deserve easy access to the wonders of modern technology. With SeeSay, we found a way to break down barriers and provide a bridge to the advancements that the younger generations take for granted.

What it does

SeeSay empowers users to capture the essence of their surroundings. With SeeSay, users can effortlessly pair their own voice prompts with snapshots, crafting a rich tapestry of auditory and visual experiences. But it doesn't stop there. SeeSay's transformative technology brings these moments to life in an array of languages, ensuring your message resonates globally.

How we built it

Deployment - Azure Cloud Stack - Front End (Angular) , Back End (Flask) APIs - Whisper API , MiniGPT4

Challenges we ran into

Initially we struggled to get the MiniGPT4 api working which is the core of our application but we managed to pull through. Initially we used Angular with Javascript with an interface with python, but we found it extremely tough and we pivoted to using Flask as a backend. Working with Azure cloud is alittle more challenging given the vagueness of the log outputs. We used Azure Cloud as we have free credits (:

Accomplishments that we're proud of

We take immense pride in the accomplishment of Team SoGood—bringing to life a fully functional and deployed web application, SeeSay. This remarkable achievement heralds a new era of accessibility and empowerment for the visually impaired. SeeSay stands as a testament to our commitment to innovation, offering a groundbreaking platform that not only provides a new perspective for the visually impaired but also opens doors to harnessing the cutting-edge capabilities of GPT technology. With SeeSay, we're not just changing technology; we're changing lives

What we learned

In the span of just one day, our journey took us through a whirlwind of discovery, immersing us in the dynamic landscape of cutting-edge technologies. This rapid but profound experience equipped us with the art of seamlessly interacting with emerging tools and innovations. Driven by empathy, we dove into the world of our users, intimately understanding their challenges and aspirations. This immersive process became the crucible for formulating a solution that not only addresses their needs but resonates with their very essence. This one-day expedition has been a transformative chapter, a testament to our unwavering commitment to technical excellence and the profound impact we strive to deliver

What's next for SeeSay

We are looking to further improve the LLaMa model to improve accuracy and enhanced description of the model. We are also looking to add additional features that would help improve the user experience and user interface.

Built With

• angular.js
• azure
• flask
• gtts
• minigpt4
• python
• whisperai