Wafri AI

Architecture Diagram
Support Entry made by AI during LIVE DEMO
SMS CONFIRMATION FOR USER

Inspiration

This project started from a personal experience. I was once asked to restock livestock supplies at home while my mom, who is the main person caring for our birds, handed me a list of required items and several distributor contacts to call.

At first, I did not realize how difficult procurement could be. I spent hours calling different numbers, getting redirected, comparing inconsistent prices, and trying to find who actually had stock available before I could complete the list.

When I searched online for help, I found no platform that truly specialized in this workflow. The system was fragmented across calls, chats, and scattered suppliers.

That experience pushed us to investigate deeper. Through additional research and direct conversations with livestock farmers and veterinary clinics, we validated that this is a widespread, high-friction problem: people lose time and money trying to find trusted supplies at fair prices.

What it does

Wafri AI introduces Fatima, a real-time multimodal AI livestock health consultant that sees, hears, and speaks with farmers and veterinary professionals. She is not a replacement for a licensed veterinarian. She is a first-response consultant who listens, asks the right questions, provides control advice based on widely available veterinary knowledge, and knows exactly when to forward a case to a qualified professional.

Farmers can describe symptoms by voice or show animals on camera. Fatima triages the case, reasons against a structured knowledge base of over 70 disease profiles publicly sourced from the MSD Veterinary Manual and enriched with West Africa epidemiological data, explains likely conditions in simple language, and decides whether to recommend treatment products or escalate to nearby clinics.

When a farmer shows an existing medicine bottle or asks for a product, Fatima reads labels, identifies the drug, searches the product inventory accumulated from distributors across the farmer's state, suggests safer or lower-cost options, and places an order linked to the farmer's phone number. Farmers then receive live SMS confirmation.

For veterinary clinics, Fatima works as a procurement copilot: staff can ask for a medicine, compare distributor prices and stock, choose the best option quickly, and place orders without spending hours calling multiple suppliers.

The interface is voice-first but not voice-only. A text fallback is always available for moments when the AI has difficulty understanding a user due to noise or connection issues.

How we built it

The frontend is built with Next.js and streams microphone plus camera context into Gemini Live (gemini-live-2.5-flash-native-audio on Vertex AI), while rendering Fatima's voice interaction and UI state in real time. Sensitive authentication inputs are handled in secure overlays and not passed through the model.

The backend is FastAPI deployed on Google Cloud Run. It manages one live session per farmer, orchestrates ADK tool-calling for diagnosis, product discovery, clinic escalation, and ordering, and emits real-time updates over WebSockets with Redis pub/sub.

Fatima's veterinary knowledge lives in a structured knowledge base of over 70 disease entries covering poultry, cattle, goats, sheep, pigs, dogs, cats, and rabbits. Each entry was built and structured directly inside our Supabase database using an AI-assisted workflow with the Supabase MCP server, then embedded using Vertex AI Text Embeddings (gemini-embedding-001) and stored in Supabase PostgreSQL with pgvector. When a farmer describes symptoms, Fatima calls query_disease_knowledge and the backend runs a cosine similarity search to retrieve the most relevant entries. She speaks from that grounded knowledge, not from hallucination.

Supabase Postgres stores operational data including farmers, products, distributors, carts, and orders. We combine pgvector and full-text search for robust medicine retrieval and use migration-driven schema updates for safety.

Google Maps Geocoding and Google Places API (New) resolve location and nearest clinics. Termii connect the experience to real SMS.

Challenges we ran into

The biggest product challenge was making Fatima genuinely agentic rather than scripted: deciding when to provide control advice, when to sell, when to ask follow-up questions, and when to escalate to a clinic rather than guess.

We also had to enforce secure session isolation without full traditional auth from day one. Early cart and session leakage risks required anonymous JWT sessions, phone plus PIN flows, and strict row-level security policies in Supabase.

Another major challenge was reliability under weak network conditions. We tuned media quality, reconnect behavior, and tool orchestration so the flow remains stable enough for real-world usage and live demo capture.

Extending multilingual support for Hausa, Yoruba, and Igbo without adding pipeline latency was also a deliberate design challenge. We resolved this by leveraging Gemini's existing language understanding capabilities directly, rather than adding an upstream translation layer that would introduce dead air in every conversation turn.

Accomplishments that we're proud of

We built a true "see, hear, speak" livestock health consultant that can assess a sick chicken from live video, explain likely issues clearly using a grounded 100-plus disease knowledge base, identify medicine labels on camera, and place a real order within minutes.

We completed the end-to-end commerce loop: cart creation, intelligent product selection across a multi-distributor inventory, order placement, payment-state updates, and instant SMS confirmation all inside a single live conversation.

We also implemented production-minded security and scale foundations, including session controls, phone plus PIN authentication, Redis-backed rate limiting, and row-level data protection across the entire Supabase schema.

What we learned

The strongest AI products are not judged by model complexity alone; they are judged by whether real people can solve urgent problems with confidence.

We learned that multimodal AI becomes significantly more useful when treated as a stateful agent with tools, memory, and constraints, not just a voice wrapper over static endpoints.

We also learned that operational details, such as security, session handling, and resilience, are what transform a compelling prototype into a deployable system.

Most importantly, we learned that this kind of agent gives time back to the people who keep food and animal health systems running. Reducing procurement friction and diagnostic delays creates compounding value across the local economy.

What's next for Wafri AI

In the near term, we are expanding and improving triage quality across more livestock species and disease profiles with direct local veterinary input, and continuing to strengthen support for Pidgin, Hausa, Yoruba, Igbo, and French.

We are integrating additional payment rails and logistics partners so Fatima can autonomously choose the best distributor and delivery pathway per farmer and clinic.

We are also adding clinic-first procurement workflows, including bulk ordering, repeat order templates, and faster price comparison across verified distributors.

Longer term, we will extend access beyond smartphones through USSD and voice-only paths via Africa's Talking. On those channels Fatima will not be able to see, but she can still listen, ask diagnostic questions, and connect farmers to products and clinics, bringing the same intelligence to the majority of farmers who do not own smartphones.

We are also building an active licensed vet community into the platform. When Fatima sees something through the camera that requires a definitive diagnosis, she will capture a snapshot of what she sees directly from the video stream and post it to a community of onboarded licensed veterinarians, who can review and provide real, qualified diagnosis. Those interactions over time become a curated West African field data source that makes Fatima's future recommendations sharper and more grounded in real cases.

Our broader vision is to lead the next generation of immersive procurement and assistance agents that can be reconfigured across sectors such as veterinary medicine, human pharmacy, and frontline healthcare supply chains and even normal online shopping.

Beyond West Africa, the architecture of Wafri AI is designed to be reconfigured and deployed across different countries and agricultural contexts worldwide. The same voice-first, multimodal agent framework, the RAG knowledge layer, the distributor inventory model, and the multilingual system can be adapted wherever the same fragmentation between farmers, veterinary knowledge, and animal health supply chains exists. Wafri AI is not just a solution for Nigeria. It is a model for how AI can close agricultural health and supply gaps anywhere in the world where the people who feed communities are still underserved.

We believe Wafri AI can contribute directly to key UN Sustainable Development Goals in Africa: