FieldLens

AI turns field photos, voice notes & observations into structured inspection reports in under 5 minutes — powered by Amazon Nova on Bedrock.

Comment

Inspiration

Field inspectors are some of the most underserved workers when it comes to tooling. After talking to people in construction and property
management, a pattern was clear: inspectors spend 2–4 hours after a site visit manually writing up what they already observed. Details get
lost, reports are inconsistent, and supervisors are always waiting. The problem isn't the inspection — it's the paperwork that follows.

I wanted to build something that lets the field worker stay focused on the field, while AI handles the documentation in real time.

## What it does

FieldLens is an AI-powered field inspection platform with two parts:

Mobile app (for inspectors): Walk a site, capture photos, record voice notes in noisy environments, and add text observations. Hit submit — that's it.

Web dashboard (for supervisors): Receive fully structured inspection reports in real time, with AI-classified issues (Critical / Warning / Info), recommendations, and semantic search across all past inspections.

The AI pipeline processes multimodal input — images, transcribed voice, and text — and produces a structured, actionable report in under 5 minutes, compared to the industry standard of 2–4 hours of manual writing.

## How we built it

Mobile: React Native + Expo SDK 54, with a multi-step inspection wizard, real-time voice transcription via WebSocket, GPS auto-location, and offline queue support for poor connectivity.

Web: Next.js 14 dashboard with live inspection feeds, semantic search, PDF export, and analytics.

Backend: Python FastAPI with async SQLAlchemy, PostgreSQL + pgvector for embedding-based semantic search, and AWS S3 for media storage.

AI — exclusively Amazon Nova via Amazon Bedrock:

  • Nova Lite — multimodal report generation, issue detection, and severity classification from photos + voice + text
  • Nova Sonic — real-time streaming voice transcription optimized for noisy field environments
  • Nova Multimodal Embeddings — vector embeddings powering semantic search across inspection history

Infrastructure: AWS EC2, Lambda, S3, RDS, Bedrock, CloudWatch.

## Challenges we ran into

  • Nova Sonic WebSocket streaming was the hardest integration. The bidirectional audio streaming protocol required careful handling of connection lifecycle, audio chunking, and partial transcript merging — especially under flaky mobile network conditions.
  • Multimodal prompt engineering for consistent structured output was non-trivial. Getting Nova Lite to reliably produce JSON-structured reports with correct severity classifications across wildly different inspection types (construction vs. NGO vs. warehouse) took significant iteration.
  • Offline-first mobile UX — queuing submissions when connectivity drops and resuming them transparently, without losing any captured data, required careful state management.
  • Building the full stack solo within the hackathon window while keeping the architecture production-quality.

## Accomplishments that we're proud of

  • End-to-end multimodal pipeline: a field worker can walk a site, speak observations aloud, and receive a fully structured report — without typing a single word.
  • Real-time voice transcription that actually works in noisy environments, using Nova Sonic's streaming WebSocket API.
  • Semantic search that lets supervisors query "find inspections with water damage near electrical panels" and get relevant results — across photos and voice notes, not just text.
  • A complete, deployable full-stack product built solo in a hackathon timeframe.

## What we learned

  • Amazon Nova Sonic is genuinely impressive for real-world audio conditions — the noise robustness is a step above what I expected.
  • pgvector + Nova Multimodal Embeddings is an underrated combination. Multimodal semantic search over inspection history opens up use cases (compliance auditing, recurring issue detection) that keyword search simply can't.
  • The hardest part of AI product development isn't the model — it's the data pipeline: handling unreliable uploads, partial failures, and making sure the AI always has enough context to produce a useful output.

## What's next for FieldLens

  • Recurring issue detection: automatically flag when the same issue appears across multiple inspections at a site over time
  • Compliance report generation: map inspection findings to specific regulatory standards (OSHA, ISO, local building codes)
  • Team collaboration: annotate and discuss findings directly within the report
  • Native iOS/Android builds via EAS for App Store distribution
  • Integrations: push critical findings to Slack, Jira, or project management tools automatically

Built With

Share this project:

Updates