Inspiration

Bison Bank AI Bank Assistant 🦬 An AI assistant engineered to handle institutional banking data, break down complex balance sheets, and answer financial questions using advanced text reasoning pipelines.## What it does In the fast-paced world of banking, analyzing financial sheets, charts, and mathematical models takes a massive amount of time. We wanted to build a centralized assistant that acts as a "knowledge hub." Instead of forcing users to dig through endless database structures manually, we envisioned an AI that could dynamically search through previously logged financial data and instantly calculate answers to complex questions.

How we built it

The application leverages a split-stack architecture to ensure data security, efficient retrieval, and top-tier mathematical processing:

Frontend UI: Built using Chatgpt, React and Bolt and bundled with Vite for a swift, responsive chat and upload interface.

Database & File Management: Powered by Supabase. We utilized Supabase Storage to hold high-resolution financial charts and documents, alongside standard relational tables to map descriptive metadata tags to those assets.

AI Reasoning Core: Integrated the NVIDIA NIM gpt-oss-120b gateway. Since the model is text-only, we constructed a Grounded Caption Retrieval-Augmented Generation (RAG) pipeline. When a text question is submitted, the frontend queries the database for keywords, extracts the mathematical logic, and feeds it as structural text context directly to the AI model.

The Core Math & LogicThe assistant handles various corporate finance computations.

For instance, when analyzing capital structures, it uses the text context to calculate the Weighted Average Cost of Capital (WACC):

$$WACC = \left(\frac{E}{V} \times R_e\right) + \left(\frac{D}{V} \times R_d \times (1 - T)\right)

$$Where: $E$ = Market value of equity $D$ = Market value of debt $V$ = Total market value of financing ($E + D$) $R_e$ = Cost of equity $R_d$ = Cost of debt $T$ = Corporate tax rateBy delivering these variables in structural data packets, the 120-billion parameter model calculates precise solutions without relying on raw vision scanning.

Challenges we ran into

Scope and Architecture Misalignment: Early on, we attempted to process entire files and execute backend training scripts inside Supabase Edge Functions, resulting in execution timeouts. We overcame this by isolating the labor: shifting the PDF parsing logic to the client side and using the database strictly for structured text storage.

Handling Text-Only API Constraints: Our chosen model, gpt-oss-120b, does not process visual pixels natively. To circumvent this, we engineered an automated search bridge using Supabase SQL queries (.ilike) to match text terms dynamically before building the final query payload

Accomplishments that we're proud of

Zero-Vision Image Intelligence: We successfully engineered a system where a text-only, 120-billion parameter reasoning model (gpt-oss-120b) can accurately answer questions about visual charts and documents. By using a smart database tagging strategy, we completely bypassed the need for expensive multimodal vision models.

High-Speed Context Discovery: We created a seamless search pipeline inside our React frontend. Users don't have to manually specify which file they want to look at; the application automatically reads their text intent, scans Supabase behind the scenes, and pulls up the exact context needed.

What we learned

We learned how to connect disconnected infrastructure pieces into a singular pipeline. This project taught us that you don't always need massive multimodal models to process visual or document files; by using smart engineering, client-side extraction, and structured database search tags, text-based reasoning models can deliver incredibly accurate, context-aware answers.

What's Next for Bison Bank AI

AssistantDynamic Vector Embeddings (True Semantic Search): Instead of using basic keyword text matches (.ilike), we plan to transition the Supabase backend to use pgvector. This will allow the application to generate high-dimensional vector embeddings for every image description and document sentence, letting the AI match context based on deep conceptual meaning rather than exact words.

Autonomous multi-step Agents: We want to expand the system from a single question-and-answer assistant into an autonomous financial agent. Given a high-level goal, the assistant will be able to chain multiple calculations together—such as scanning a balance sheet to pull variables, using those variables to calculate capital asset pricing ($CAPM$), and outputting a risk profile.

Native Vision Model Integration: As open-source multimodal infrastructure matures, we plan to implement a hybrid routing model. The app will visually pre-screen complex layout diagrams with a dedicated vision model before passing the refined, heavy reasoning tasks to gpt-oss-120b.

Real-time Financial API Feeds: We intend to connect the pipeline directly to live market data streams. This will enable the assistant to compare the historic, uploaded data in your Supabase storage against real-time stock and bond yields to provide instant, up-to-the-minute valuation adjustments.

Built With

• bolt
• chatgpt
• codex
• gemini