Lighty - Intelligence light

Agent Architecture

Inspiration

We want to find a way for people to interact with intelligence, with minimized efforts. Before computers, we had electricity, and before electricity. We ask ourselves, what we have? What is the most natural way to interact with intelligence that everyone has access already? Then we found the answer: light.

What it does

Lighty is an AI empowered lamp that also projects. It sits on your desk, ready to work with you at anytime. No screen, no apps, just grab a paper and pencil, and lighty will build with you. On the user design side: we focus on the simplest flow. We want lighty to assist in a way that people are doing work already, instead of interrupting it. On the backend, it equips with camera, voice command, and projecting abilities. It will read the paper content, process the questions and build apps in the cloud, then project the buildable back on the desk to turn the desk into an interactive canvas.

How we built it

We put an iphone as camera and taped it on the lamp for content capture and voice recording. On the frontend layer: we used the AWS Bedrock API to interact with Claude model for image processing, Elevenlabs API for speech to text / text to speech processing. On the backend layer: we used AWS Bedrock API again to process the build, integrating with Daynota AI agent for deploying the build and run the apps, also Arize AI to monitor the agent workflow, tracking the error logs and exceptions if there is any.

Challenges we ran into

Turning drawing into contextual information is hard because it requires the correct scale so the model can understand the physical nature of problems.
Building a smooth flow is hard, very hard. There are moments when people need to think and come back and drawing or exploring, how to enable the hardware product work with this habit is another challenge.

Accomplishments that we're proud of

We build a working demo using the hardware that we have. Also we are proud of coming up with this design because it is really intuitive and simple.

What we learned

We learned how to run complex physical simulation in a reliable environment. We learned how to run real life problems into a physical device.

What's next for Lighty - Intelligence light

We really want to improve the workflow of lighty to make it as smooth as possible, also we are looking for partnerships that help us to manufacturer this product and bring it into real life.