cafe fra.me

Revolutionizing healthcare by replacing spreadsheets with an AI-native and AR-enhanced approach to deliver personalized, context-rich care.

Comment

Inspiration

I remember sitting in a family doctor’s waiting room, frustrated as I watched the doctor’s machine reboot—only to discover it had no record of vaccines I received at another clinic. With all these so-called robust ERP systems and endless spreadsheets, how could this happen? The truth is, the entire healthcare management system is broken. There’s no shared documentation, no privacy, no transparency between physicians and patients, and a clunky lookup process that forces you to rely on specialists like nurses or assistants. It’s a system crying out for change.

What It Does

Cafa Fra.me is a two-part solution designed to revolutionize healthcare management:

  • Software: A conversational AI-driven interface that replaces tedious spreadsheets with dynamic, context-rich interactions.
  • Hardware: Innovative AR glasses that bring critical patient information directly in front of physicians’ eyes, eliminating the need for distracting devices.

How We Built It

There are several key components:

  • Hardware Integration: We partnered with Brilliant.xyz, whose hardware we’ve been experimenting with. The idea was to remove traditional computers and devices that distract from patient care. Now, every piece of critical information is instantly accessible through AR glasses. The Python SDK made integration a breeze.
  • Robust Backend: We developed a powerful backend featuring semantic search capabilities using a vector store. By leveraging agentic chunking, a vector database, and a reranking algorithm, we built a system that accurately extracts information from physicians’ queries.
  • Web3 Security: We built a smart contract on Polygon that employs both asymmetric and symmetric encryption to securely store medical records on-chain. Only authorized personnel can access or update the records. In addition, a React Native app empowers patients to monitor their doctor visits and upload their health data via the Terra API.

Challenges We Ran Into

Integration Complexity: Combining cutting-edge hardware with a robust, secure backend presented unique challenges, particularly in ensuring seamless communication between components. Data Privacy and Security: Balancing transparency with stringent privacy requirements required careful design, especially with the introduction of blockchain-based solutions. User Adoption: Transitioning healthcare professionals from traditional systems to an AR-driven interface meant overcoming resistance to change and ensuring ease-of-use. Performance Optimization: Implementing semantic search and vector ranking in real-time demanded rigorous optimization to maintain a smooth user experience.

Accomplishments We’re Proud Of

Successfully integrating AR glasses to deliver real-time, hands-free access to patient information. Developing a backend with advanced semantic search capabilities that accurately understands and responds to physicians’ queries. Building a secure, decentralized system using smart contracts that ensures only authorized personnel can access sensitive medical data. Creating a user-friendly interface that blends innovation with practicality, making healthcare management more intuitive and patient-focused.

Built With

Share this project:

Updates