Frontlinelens

What's next for Frontlinelens

About the Project Inspiration Retail teams often complete opening inspections using paper checklists, messaging apps, or memory. These methods can record whether a check happened, but they rarely help managers understand what matters most or what to do next. We created FrontlineLens to turn the iPhone already in a manager’s pocket into an intelligent store-opening assistant. What It Does FrontlineLens guides managers through five important areas: Entrance Customer walkway Product tags or menus Checkout or service counter Displays, signage, and exits Managers capture photos or short videos as evidence. AI then: Identifies potential operational issues Highlights where attention may be needed Explains the business impact Provides detailed corrective instructions Automatically creates prioritized tasks Generates an evidence-backed Store Readiness Report Managers remain in control and can confirm or reject every AI finding. FrontlineLens does not represent its results as certified legal, safety, or regulatory inspections. How We Built It FrontlineLens is a native iPhone application built with SwiftUI. We used: AVFoundation for photo and short-video capture SwiftUI for navigation, inspection workflows, findings, tasks, and reports OpenAI’s multimodal API to analyze visual evidence and generate structured reports Structured Outputs to convert AI analysis into reliable finding data Keychain Services to protect the API key in the prototype URLSession for secure API communication Local app storage for inspection evidence The AI returns structured information including: { "issue": "Customer walkway partially obstructed", "priority": "High", "recommendedAction": "Move the promotional stand against the wall", "deadline": "Before opening", "confidence": "High", "requiresHumanVerification": true } FrontlineLens transforms this data into visual findings, image highlights, detailed instructions, tasks, and reports rather than displaying raw AI output. Challenges We Faced One major challenge was creating a reliable camera workflow across both physical iPhones and the Simulator. The Simulator does not provide a normal rear-camera feed, so we built valid sample evidence to test the complete workflow without weakening real-device capture. Another challenge was making visual AI output actionable. Detecting a possible issue was not enough. We designed structured responses containing image coordinates, priorities, deadlines, and practical instructions so the app could highlight the relevant area and automatically create a task. We also had to balance AI assistance with responsible product language. Visual findings are presented as potential issues requiring manager verification, not definitive safety or compliance decisions. What We Learned We learned that the most valuable AI experience is often not a chatbot. AI becomes more useful when it is embedded directly into an existing workflow and produces clear actions. We also learned that: Structured AI responses are essential for dependable product interfaces. Evidence is more useful when it stays connected to findings and tasks. Human confirmation is critical when AI interprets real environments. Frontline employees need concise instructions, not long AI explanations. A focused five-step workflow is more effective than a complex analytics system. What’s Next Next, we plan to add: Before-and-after AI verification Team assignments and notifications Multi-store dashboards PDF report export Secure backend-managed AI access Improved text recognition for prices, menus, and signs Historical readiness trends Integration with enterprise task-management systems Our goal is to help every store begin the day with a clear answer to one question: Are we ready to open?