Inspiration
Befit draws inspiration from the growing power of real-time inference and visual interpretation made possible by multimodal language models and agents like Goose, an open-source project from Block. Since the rise of transformer-based innovations such as BERT and conversational models like ChatGPT, we’ve witnessed an explosion of creative AI applications. Moving beyond simple chatbots and static Q&A tools, Goose—and advanced systems like Anthropic’s Claude Sonnet 4.6—enable agentic experiences that can think, see, speak, and act.
Befit is a v1 application in a broader platform designed to turn everyday queries into meaningful insights and actions. It empowers personalized agents to understand what’s in your refrigerator, pantry, or cabinet, helping you and your loved ones manage wellness in a holistic way. Explore the project at getbefit.netlify.app or dive into the code on GitHub.
What it does
Befit turns what’s on your shelf into a phone-first wellness co‑pilot. Open the web app in Chrome, point your camera at your food or cabinet items, type or ask a question out loud, and get a short “Today’s Plan” card with items detected, 2–3 suggested actions, and safety notes.
How we built it
- Used Perplexity for research and a "second-look" code reviewer
- Built with Goose (agent powered with Claude Sonnet 4.6)
- OpenRouter API/model management
- Uses a hierarchical agent-subagent design
- Human-AI coder and review
- GitHub Copilot (mixture of agents and Raptor Mini) and Copilot (desktop GPT 5.2) code and ideation review
- See repo markdown for detailed insight into planning, decisions, troubleshooting, agent setup, logs and other artifacts of the development journey -https://github.com/professordnyc/befit
Challenges we ran into
- Unexpected Goose v1.25- v1.26 model cost runs (bug submitted, PR under review)
- Temporary fix -- new sessions per sub-feature build -- smaller inputs -- continued file management vs direct prompt edits
- Known issue with voice command responses to audio playback in Chrome for Android
- temporary fix -- action buttons as fallback
- Wrong camera selection during app run -- changed system settings
- Trigger delay issues for queries when mic always on is toggled -- Extended delay and modified response object
Accomplishments that we're proud of
- Multimodal inputs
- Modular system design for multi-agent orchestration
- Deployed working application
- Incorporation of open source tooling for agentic flow
- Real-world demonstration of what's next in touchless, real-world sensing, inference and multimodal input and output
What we learned
- Agentic application development is an exciting field with much promise
- Touchless, gesture-based, multi-lingual, multi-directional and multimodal agency is the future
- Purpose-built agents with orchestration levers and workflows might be the most powerful mix of tools for development, with or without pre-defined skill files
- Model architecture and use are cost drivers. Be careful.
- Test. Lint. Try. Document. Repeat.
- Use WebSpeech API during build and testing so premium model credits are not exhausted before deployment.
- There is room for smaller, specialized models to drive orchestration of purpose-built agents
- Very thankful and immensely grateful for open, innovative culture driven by companies and initiatives like Goose and CreateHer Fest. It creates a shift from quiet curiosity to enabled and empowered systems building.
What's next for Befit
- Befit v2 -- improved latency, multi-model and auto/smart model selection -- database -- security -- additional multimodal options -- social features
- Shelf v1+ -- actions concierge as optional Befit agent or standalone use
- Native and React Native build
- Beta testing
- Online documentation, SDK and API for builders
- Word of mouth and marketing
- Funding and revenue-generation opportunities with forever-free options
Log in or sign up for Devpost to join the conversation.