Inspiration

As college students, we understand the struggle of balancing tuition, bills, and social life. That’s why we created Centsible, an AI financial assistant built into iMessage that helps students spend smarter through simple conversations while scrubbing PII to protect sensitive personal information.

What it does

Centsible is an autonomous, proactive AI financial assistant that lives inside Apple iMessage (blue bubbles)—eliminating app fatigue entirely:

  • Multimodal Receipt Parsing: Snap a photo of a crumpled receipt or checkout screen, and Centsible extracts totals, dates, and line items instantly.
  • Smart 50/30/20 Budgeting: Categorizes transactions into Needs, Wants, and Savings, tracking budget pacing in real time.
  • Proactive Outbound Welcomes: Features a "Text-First" engine that initiates contact and welcomes new users via iMessage the moment they sign up.
  • Smart Accountability & Deal Discovery: When users consider expensive impulse purchases, Centsible queries live internet deals to recommend certified refurbished or pre-owned alternatives that save hundreds of dollars.
  • Conversational Brevity: Enforces a strict 2-to-4 sentence conversational standard built for phone screens, avoiding overwhelming text walls.
  • Privacy Guardrails: Reminds users to delete uploaded bank statements from their chat threads and assures end-to-end data encryption.

How we built it

  • Core Intelligence & Vision: Powered by Google Gemini multimodal models to analyze both receipt photos and text queries with sub-second latency.
  • Backend Architecture: Built with Python and FastAPI, featuring multi-model fallback cascades to prevent downtime.
  • Apple iMessage Integration: Connected Apple's messaging runtime directly to our backend using TypeScript, Photon, and the Spectrum SDK.
  • Local Privacy Pipeline: Integrated Microsoft Presidio alongside custom regex pattern matchers to scrub sensitive PII (SSNs, full card numbers, credentials) locally before any data reaches cloud AI models.
  • Persistent Memory: Backed by a local SQLite database to remember user names, budget categories, and spending history.
  • Real-Time Deal Engine: Integrated live web searches to find current prices, certified refurbished alternatives, and coupons on demand.

Challenges we ran into

We initially struggled to settle on an idea, starting with a focus on helping the elderly before recognizing the financial challenges college students face, from overspending to debt and housing insecurity.

  • The "Text-First" Outbound Bottleneck: Standard messaging bots are purely reactive and can only reply to an inbound thread. We wanted Centsible to proactively welcome new clients. We resolved this by diving into the Spectrum runtime internals to invoke dynamic conversation space creation (imRuntime.definition.space.create), unlocking programmatic outbound blue-bubble messages.
  • API Quotas & Grounding Resilience: Early testing against free-tier rate limits and search-grounding quotas resulted in 429 bottlenecks. We engineered a multi-model fallback cascade across Gemini Flash variants and decoupled live shopping queries into a dedicated real-time search engine.
  • Privacy vs. Utility Balance: Scrubbing sensitive information without stripping the transaction amounts and merchant names necessary for budgeting was tricky. We fine-tuned our privacy engine to selectively scrub account credentials while keeping financial totals and merchant context intact.
  • Bot Feedback Loops: Handling incoming and outgoing iMessage webhooks initially caused echo loops. We implemented strict signature filtering and outbound tracking to maintain clean conversational state.

Accomplishments that we're proud of

  • True Blue-Bubble Experience: A fully operational financial assistant operating directly inside native iOS Messages without requiring any third-party app download.
  • Dynamic Outbound Space Creation: Built both a CLI and REST API endpoint enabling the bot to initiate contact with any verified Apple ID phone number.
  • Multimodal Accuracy: Leveraging Gemini's vision allowed us to accurately parse skewed, low-light, and crumpled receipts that traditional OCR engines failed on.
  • Active Privacy Interventions: Created active guardrails that warn users against texting credentials, assure encryption, and prompt them to delete sensitive financial screenshots.
  • Empathetic UX: Crafted a concise, encouraging persona that helps users build healthy spending habits without feeling judged.

What we learned

  • Friction Kills Retention: In consumer fintech, meeting users inside tools they already check dozens of times a day (iMessage) drops onboarding friction from minutes to seconds.
  • Multimodal Vision Trumps OCR: Modern vision models handle real-world receipt noise, tilted angles, and irregular fonts vastly better than traditional text-extraction pipelines.
  • Local Privacy is Essential: Redacting PII locally before sending data to cloud AI services is both technically feasible and a prerequisite for responsible fintech applications.
  • Brevity is King on Mobile: Chatbot interactions on phones demand strict sentence caps—2 to 4 high-signal sentences deliver a vastly superior user experience compared to textbook-length paragraphs.

What's next for Centsible

  • Companion Analytics Web Dashboard: A visual web app displaying long-term spending trends, charts, and budget breakdowns, while keeping daily logging inside iMessage.
  • Group iMessage Budgeting: Expanding Centsible into group chats so roommates and travel groups can seamlessly split rent, groceries, and shared trip expenses in real time.
  • Automated Bank Sync: Optional background syncing with major banks to reconcile iMessage-logged transactions against monthly statement totals.
  • More Robust Secure Storage: Change the storage of user data on the machine to zero trust architecture.

Built With

Share this project:

Updates

Submission history