FocusFlow AI

A fun eye-gazing game for kids with special needs. Our AI analyzes webcam data to provide objective reports on focus, turning screen time into measurable progress for parents & therapists.

Comment

FocusFlow AI - Project Story

Inspiration

As a father of a special needs child, I've always been deeply concerned about understanding my daughter's visual capabilities and providing her with the right therapeutic support. One of my biggest questions was: How accurate is my daughter's vision? Is she truly gazing and tracking objects with her eyes?

Traditional eye-tracking assessments require expensive equipment, specialized facilities, and appointments that can be challenging for families with special needs children. I wanted to create something that could:

  1. Measure visual attention accurately - Give parents and therapists objective data about a child's eye-gaze capabilities
  2. Provide eye stimulation - Offer engaging exercises that encourage children to practice visual tracking
  3. Be accessible - Work with just a webcam, making it available to any family, anywhere
  4. Generate insights - Use AI to analyze performance and provide actionable recommendations

The inspiration came from watching my daughter during therapy sessions and wishing I had a tool that could help us practice at home, track her progress, and understand her visual development better. FocusFlow AI was born from a parent's love and a developer's determination to make a difference.

What it does

FocusFlow AI is an AI-powered eye-gaze tracking therapy platform designed specifically for children with Autism Spectrum Disorder (ASD) and other special needs. It transforms any computer with a webcam into a therapeutic tool that:

For Children

  • Engaging Games: Three progressive difficulty levels (Follow the Leader, Collision Course, Find the Pattern) that make therapy fun
  • Real-time Feedback: Visual cues that help children understand when they're tracking correctly
  • Safe Environment: No pressure, self-paced learning that builds confidence
  • Colorful Interface: Bright, appealing visuals designed for children with special needs

For Parents

  • Objective Measurements: Precise data on tracking accuracy, focus duration, and visual attention
  • Progress Tracking: See improvements over time with historical data and comparisons
  • AI-Generated Reports: Comprehensive analysis after each session with personalized recommendations
  • Home Accessibility: Practice therapy exercises at home, on your schedule
  • Encouraging Feedback: Positive, supportive language that celebrates every achievement

For Therapists

  • Patient Management: Track multiple children with individual profiles
  • Data-Driven Insights: Objective metrics to inform treatment plans
  • Research-Backed Analysis: AI trained on peer-reviewed eye-gaze research
  • Time Efficiency: Automated report generation saves hours of documentation
  • Remote Monitoring: Review patient progress between appointments

Technical Capabilities

  • Browser-Based Eye Tracking: Uses WebGazer.js for webcam-based gaze detection
  • Real-Time Processing: Captures and analyzes eye movements during gameplay
  • AI Analysis: Claude 3.5 Sonnet generates detailed therapeutic reports
  • Knowledge Base Integration: AI references scientific research on eye-gaze development
  • Cloud Infrastructure: Scalable AWS architecture handles data storage and processing

How we built it

Technology Stack

Frontend

  • Next.js 14 with React 18 for the user interface
  • TypeScript for type safety and better development experience
  • WebGazer.js for browser-based eye tracking using the webcam
  • CSS Modules for component-scoped styling
  • Deployed on AWS S3 + CloudFront for global CDN delivery

Backend

  • AWS Lambda (Node.js 20.x) for serverless compute
  • API Gateway for RESTful API endpoints
  • DynamoDB for storing user data, profiles, and reports
  • S3 for session data and research paper storage
  • Amazon Bedrock with Claude 3.5 Sonnet for AI analysis
  • Bedrock Knowledge Base with research papers for context-aware insights

Infrastructure

  • Terraform for Infrastructure as Code (IaC)
  • AWS CloudWatch for logging and monitoring
  • Amazon OpenSearch Serverless for vector storage in knowledge base

Development Process

  1. Research Phase

    • Studied eye-gaze tracking research papers
    • Analyzed therapeutic approaches for children with ASD
    • Evaluated eye-tracking technologies (chose WebGazer.js for accessibility)

  2. Architecture Design

    • Designed serverless architecture for scalability
    • Planned data flow from frontend → API → Lambda → Storage
    • Integrated AI analysis pipeline with Bedrock

  3. Frontend Development

    • Built three progressive game levels with increasing difficulty
    • Implemented real-time eye-tracking calibration
    • Created enhanced scorecard with performance metrics
    • Designed beautiful AI report visualization

  4. Backend Development

    • Developed 8 Lambda functions for different operations
    • Set up DynamoDB tables for data persistence
    • Configured S3 triggers for automatic AI analysis
    • Integrated Bedrock Agent with custom knowledge base

  5. AI Integration

    • Processed research papers into knowledge base
    • Configured Bedrock Agent with therapeutic prompts
    • Implemented RAG (Retrieval Augmented Generation) for research-backed insights
    • Fine-tuned prompts for encouraging, parent-friendly language

  6. Testing & Refinement

    • Tested with real users (including my daughter)
    • Refined game difficulty and visual feedback
    • Optimized AI report generation
    • Enhanced UI based on feedback

  7. Deployment

    • Automated deployment with Terraform
    • Set up CI/CD pipeline
    • Configured CloudFront for global distribution
    • Implemented monitoring and logging

Challenges we ran into

1. Eye Tracking Accuracy in Browser

Challenge: Browser-based eye tracking is less accurate than specialized hardware.

Solution:

  • Implemented comprehensive calibration process
  • Added visual feedback to help users position correctly
  • Used statistical analysis to filter out noise
  • Designed games that work well even with moderate accuracy

2. API Gateway Permission Mismatch

Challenge: Lambda functions returning 500 errors due to wrong API Gateway permissions.

Solution:

  • Discovered two API Gateways existed (one orphaned)
  • Added correct permissions for the active API Gateway
  • Updated Terraform state to track proper resources
  • Documented the fix for future reference

3. Bedrock Model Access Permissions

Challenge: AccessDeniedException when trying to invoke Bedrock models.

Solution:

  • Added bedrock:InvokeModel permission to Lambda IAM role
  • Verified permissions with AWS CLI
  • Tested end-to-end AI report generation
  • Reports now generate successfully in 30-60 seconds

4. Knowledge Base Integration Complexity

Challenge: Setting up Bedrock Knowledge Base with custom research papers was complex.

Solution:

  • Created automated scripts for document processing
  • Configured OpenSearch Serverless for vector storage
  • Set up proper IAM roles and permissions
  • Documented the entire setup process

5. Cross-Device User Tracking

Challenge: Users losing session history when switching devices.

Solution:

  • Implemented localStorage-based userId for device-specific tracking
  • Added profile system for therapist-managed continuity
  • Documented the design decision and trade-offs
  • Planned future authentication system for cross-device sync

6. Large Session Data Storage

Challenge: Eye-gaze data can be 100KB+ per session, expensive in DynamoDB.

Solution:

  • Store raw session data in S3 (cost-effective)
  • Store only metadata and reports in DynamoDB (fast queries)
  • Implemented S3 lifecycle policies for archival
  • Optimized data structure to reduce size

7. AI Report Generation Time

Challenge: Bedrock calls taking 20-30 seconds, blocking user experience.

Solution:

  • Implemented asynchronous processing with S3 triggers
  • Show scorecard immediately (client-side calculation)
  • Generate AI report in background
  • Notify users reports are available within 60 seconds

8. Making AI Reports Readable

Challenge: AI-generated reports were plain text, hard to read and scan.

Solution:

  • Created enhanced AIReport component with visual hierarchy
  • Added metric cards with icons and hover effects
  • Implemented section organization with contextual icons
  • Used gradient design and smooth animations
  • Made reports engaging and professional

Accomplishments that we're proud of

1. Accessible Eye Tracking

We made professional-grade eye-tracking therapy accessible to any family with a webcam. No expensive equipment needed!

2. AI-Powered Insights

Successfully integrated Claude 3.5 Sonnet with a custom knowledge base of research papers to generate therapeutic reports that are:

  • Research-backed and scientifically sound
  • Encouraging and parent-friendly
  • Actionable with specific recommendations
  • Generated automatically in under 60 seconds

3. Beautiful User Experience

Created an engaging, colorful interface that children love, with:

  • Three progressive game levels
  • Real-time visual feedback
  • Enhanced scorecard with performance badges
  • Stunning AI report visualization with infographics

4. Scalable Architecture

Built a production-ready, serverless architecture that:

  • Handles unlimited concurrent users
  • Costs only ~$19/month for moderate usage
  • Scales automatically with demand
  • Maintains 99.9% uptime

5. Complete Documentation

Produced comprehensive documentation including:

  • 2,060 lines of active documentation
  • 4 Mermaid architecture diagrams
  • Complete API reference
  • Deployment and troubleshooting guides
  • 66 historical documents preserved

6. Real Impact

Most importantly, we created a tool that:

  • Helps my daughter practice visual tracking at home
  • Gives me objective data about her progress
  • Provides encouragement and hope
  • Can help thousands of other families

7. Open Source Potential

Built with the intention to help others:

  • Well-documented codebase
  • Modular, maintainable architecture
  • Clear setup instructions
  • Ready for community contributions

What we learned

Technical Learnings

  1. Serverless Architecture

    • Learned to design and implement complex serverless systems
    • Mastered AWS Lambda, API Gateway, and DynamoDB
    • Understood trade-offs between serverless and traditional architectures

  2. AI Integration

    • Gained deep experience with Amazon Bedrock and Claude
    • Learned to build and manage knowledge bases
    • Understood RAG (Retrieval Augmented Generation) patterns
    • Mastered prompt engineering for therapeutic contexts

  3. Infrastructure as Code

    • Became proficient with Terraform
    • Learned to manage complex AWS resources
    • Understood state management and resource dependencies

  4. Eye Tracking Technology

    • Learned how browser-based eye tracking works
    • Understood calibration and accuracy challenges
    • Discovered techniques for improving tracking reliability

  5. Frontend Performance

    • Optimized React components for real-time tracking
    • Learned to handle large datasets efficiently
    • Mastered CSS animations and transitions

Personal Learnings

  1. Empathy in Design

    • Designing for special needs requires deep empathy
    • Small details matter enormously for accessibility
    • User testing with real children is invaluable

  2. Patience and Persistence

    • Complex problems require systematic debugging
    • Documentation saves hours of future work
    • Taking breaks leads to better solutions

  3. Parent-Developer Perspective

    • Personal experience drives better product decisions
    • Understanding user pain points deeply improves UX
    • Building for your own family creates authentic motivation

  4. AI as a Tool

    • AI can augment human expertise, not replace it
    • Proper prompting and context are crucial
    • AI-generated content needs careful review

  5. Community Impact

    • Technology can make therapy more accessible
    • Open source can multiply impact
    • Sharing knowledge helps others build better solutions

What's next for FocusFlow AI

Short-term Goals (Next 3 Months)

  1. User Authentication

    • Implement Cognito for secure login
    • Enable cross-device session sync
    • Add family account management

  2. Enhanced Analytics

    • Add trend analysis and progress charts
    • Implement comparison with age-appropriate benchmarks
    • Create visual progress reports for therapists

  3. More Game Levels

    • Add 3 more difficulty levels
    • Implement adaptive difficulty based on performance
    • Create themed games (animals, space, underwater)

  4. Mobile App

    • Develop iOS and Android apps
    • Optimize eye tracking for mobile cameras
    • Enable offline practice mode

  5. Therapist Dashboard

    • Create dedicated therapist portal
    • Add patient management features
    • Implement appointment scheduling
    • Enable report sharing with parents

Medium-term Goals (6-12 Months)

  1. Clinical Validation

    • Partner with therapy centers for validation studies
    • Collect efficacy data
    • Publish research findings
    • Obtain clinical endorsements

  2. Insurance Integration

    • Work with insurance providers
    • Document therapeutic value
    • Enable insurance billing codes
    • Make therapy more affordable

  3. Multi-language Support

    • Translate interface to Spanish, Mandarin, Hindi
    • Localize AI reports
    • Support international families

  4. Advanced AI Features

    • Real-time coaching during gameplay
    • Predictive analytics for progress
    • Personalized game recommendations
    • Automated goal setting

  5. Community Features

    • Parent support forums
    • Success story sharing
    • Therapist Q&A
    • Resource library

Long-term Vision (1-3 Years)

  1. Comprehensive Therapy Platform

    • Expand beyond eye-gaze to other skills
    • Add speech therapy exercises
    • Include motor skills activities
    • Create holistic development tracking

  2. Research Contributions

    • Build largest eye-gaze dataset for ASD
    • Enable research partnerships
    • Contribute to scientific understanding
    • Advance therapeutic approaches

  3. Global Accessibility

    • Make free for families in need
    • Partner with NGOs and foundations
    • Expand to underserved communities
    • Create scholarship programs

  4. Integration with Healthcare

    • Connect with EHR systems
    • Enable telehealth integration
    • Support remote therapy sessions
    • Facilitate therapist-parent collaboration

  5. AI-Powered Personalization

    • Fully adaptive therapy programs
    • Individual learning paths
    • Predictive intervention recommendations
    • Outcome optimization

Ultimate Goal

Make high-quality, research-backed therapy accessible to every child who needs it, regardless of location, income, or resources.

We envision a world where:

  • Every parent can track their child's development
  • Every therapist has AI-powered tools
  • Every child gets personalized, engaging therapy
  • Technology bridges the gap in special needs care

FocusFlow AI is just the beginning of this journey.

Join Us

Whether you're a:

  • Parent seeking better tools for your child
  • Therapist wanting to enhance your practice
  • Developer interested in contributing
  • Researcher looking to collaborate
  • Investor believing in our mission

We'd love to hear from you!

Together, we can make a difference in the lives of children with special needs and their families.

Built with ❤️ by a father for his daughter, and for all the families on this journey.

Built With

Share this project:

Updates