WelcomeScreen
App finding item
Home Screen
Select Item
Find Item Storage
AI Analyzing Item
Item Details

Inspiration

The idea for LastSeen AI came from something that happened at home.

One of my roommates has blind spots in his vision that make it difficult to see. One day he couldn't find his TV remote. After searching for several minutes, he started thinking someone else in the house had taken it. We eventually found the remote sitting on the table in plain sight.

Watching such a simple situation turn into frustration, anxiety, and misunderstanding made me realize there had to be a better way.

I started asking myself, what if AI could help people find where they left their things?

Instead of relying on expensive Bluetooth trackers, the idea was to let someone simply take a picture when they set an item down. Google Gemini Vision analyzes the image to understand the scene, identifies the object, and creates an AI-generated description. Later, when the user is searching for the item, the app displays the saved photo, usual location, AI description, and can even launch Smart Finder to help locate the item in real time.

What it does

LastSeen AI helps people find important everyday items without requiring Bluetooth trackers or additional hardware.

Users take a quick photo of an item, and the app uses Google Gemini Vision to analyze the scene and generate an AI description. The user can also save a usual location for the item.

When something is lost, the app displays the saved photo, usual location, and AI-generated description to help the user start searching. If the item still can't be found, Smart Finder analyzes the live camera feed and highlights the object with a yellow box and confidence score when it's detected.

How I built it

SwiftUI for the iOS application
SQLite for fast local storage
Firebase Authentication for user accounts
Google Cloud Functions (Python) for AI processing
Google Gemini Vision API for scene understanding and live object detection
AVSpeechSynthesizer for spoken directions
ChatGPT as second set of eyes

Challenges

One of the biggest challenges was designing an experience that was actually useful instead of simply showing AI predictions. I wanted the app to guide people while searching instead of pretending it always knew an item's exact location.

Another challenge was balancing privacy and accessibility. I decided to store photos locally on the device by default while designing the app to support optional cloud backup in the future.

What I learned

This project taught us that accessibility isn't always about creating brand new technology—it can also mean combining existing AI tools in thoughtful ways to solve everyday problems.

I also learned how powerful multimodal AI can be when paired with a simple user experience. By combining Google Gemini Vision, real-time camera analysis, and natural language descriptions, LastSeen AI helps reduce frustration and gives people more confidence when searching for everyday items.