T.A.M.A : Transaction Analysis & Monitoring Agent

Your AI Financial Bodyguard: An autonomous agent that stops impulsive buying via voice & vision, and actively plans your wealth — powered by Gemini 3 Multimodal Reasoning.

Comment

Inspiration

We live in an era of "Instant Gratification." According to a survey by Bankrate, nearly half of social media users admit to making impulse purchases influenced by what they see—and significantly, nearly two-thirds regret it later. This "Intention-Action Gap" is the primary reason why many fail to secure their pension plans or long-term goals.

Traditional expense trackers are passive—they act like historians, recording your financial regrets after you’ve made them.

We wanted to build something Active. We were inspired by the concept of a "Financial Bodyguard." What if you had an autonomous agent that could see what you are buying, hear your excuses, and intervene in real-time? That is how T.A.M.A (Transaction Analysis & Monitoring Agent) was born.

What it does

T.A.M.A is a multimodal autonomous agent powered by Gemini 3 that acts as a real-time financial firewall. It orchestrates your financial life through three core pillars:

  1. Voice-to-Action (Multimodal Logger)

    • Users don't need to type. They simply speak: "I bought Nasi Goreng for 25k."
    • T.A.M.A listens, translates (Indonesian/English), extracts the price, categorizes the item, and deducts it from the specific budget bucket automatically.

  2. Visual "Roast" Consultant

    • Unsure about a purchase? Snap a photo of the product. T.A.M.A analyzes the item's necessity against your current liquidity.
    • Strict Mode: "You want these shoes? You are 40% behind on your House Goal. Put them back."
    • Supportive Mode: "It’s okay bestie, treat yourself, but let's save more tomorrow!"

  3. Strategic Planner with Math Validation

    • T.A.M.A uses a mathematical engine to detect "Impossible Goals" in real-time. $$\text{Gap} > (\text{Income} - \text{Fixed Exp}) \times \text{Time Left}$$
    • If this condition is met, the Agent triggers a Red Alert and autonomously suggests a strict daily limit (e.g., cutting food budget from 50k to 20k) to get the user back on track.

  4. Auto-Allocation Agent

    • It simulates a "Payday" event where it autonomously allocates a percentage of income to high-priority goals, removing the human temptation to spend it immediately.

How we built it

We built T.A.M.A using a modern, lightweight stack designed for speed and reasoning:

  1. The Brain (Gemini 3 API): We utilized the Gemini 3 Multimodal model for its superior reasoning capabilities. It handles:
    • Vision: Recognizing products and estimating price tiers.
    • Audio: Processing natural language voice logs.
    • Reasoning: The core "Consultant" logic that weighs immediate pleasure vs. long-term pain.
  2. The Interface (Streamlit): Chosen for rapid prototyping, allowing us to build a reactive UI that updates the user's financial status in real-time.
  3. The Voice (gTTS): We integrated Google Text-to-Speech so the Agent can "speak back" to the user, creating a genuine conversational loop.
  4. The Logic (Python): We built a custom financial engine that handles the JSON database, ensuring that the AI's reasoning is grounded by strict mathematical formulas for balance calculations.

Challenges we ran into

  • The 72-Hour Sprint: We discovered this hackathon only 3 days before the deadline. It was a race against time to move from ideation to a deployed agent.
  • Quota Management: We faced several 429 Resource Exhausted errors during testing. This forced us to be clever with our prompt engineering, optimizing our system instructions to be token-efficient while maintaining the AI's distinct "Persona."
  • The "Hallucination" Balance: Initially, the AI would be too optimistic about savings (suggesting saving 100% of income). We had to implement a "Reality Check" layer (Python logic) to force the AI to recognize that humans need money to eat and survive (Living Cost), preventing unrealistic advice.

Accomplishments that we're proud of

  • Multimodal Fluidity: We successfully integrated Voice, Vision, and Text into a single seamless flow. The latency is low enough that it feels like talking to a real consultant.
  • The "Persona" Engine: We are proud of how distinct the "Strict Guardian" feels compared to the "Supportive Bestie." It gives the app personality and makes finance less boring.
  • Functional Autonomy: Beyond the AI capabilities, the core logic of deducting balances, tracking goals, and calculating daily limits works perfectly. It is a functional MVP, not just a demo wrapper.

What we learned

  • Gemini 3 is a Reasoner: We learned that the true power of Gemini 3 isn't just generating text, but understanding consequences. It can look at a transaction history and deduce behavioral patterns (e.g., "User spends too much on coffee") without us explicitly coding a rule for it.
  • Collaborative AI: Building this taught us that AI agents work best when collaborating with hard-coded logic. The AI handles the unstructured (voice/images), while Python handles the structured (math/database). This hybrid approach is powerful.

What's next for T.A.M.A : Transaction Analysis & Monitoring Agent

We believe T.A.M.A has the potential to solve the "Pension Crisis" for Gen-Z. Our roadmap includes:

  1. Geo-Fencing Intervention: Using GPS to detect when a user enters a high-risk spending zone (e.g., a Mall or Coffee Shop). T.A.M.A will proactively send a notification: "Warning: You are near a shopping mall. Remember your House Goal."
  2. UI/UX Refinement: Improving the user flow to make the "Payday Simulator" and "Goal Tracking" more intuitive and visually engaging for non-financial users.
  3. Open Banking API: Moving from manual voice logging to automatic bank mutation tracking for a fully seamless experience.

Built With

Share this project:

Updates