Kodi Tax Agent

Kodi democratizes tax capability with an autonomous AI workspace that investigates records, reasons over tax issues, and delivers grounded, traceable answers for individuals and small businesses.

Comment

Inspiration

Tax is one of those systems where the responsibility placed on an individual is often much greater than the capability available to them. A taxpayer may have bank statements, mobile money records, invoices, receipts, spreadsheets, payroll records, correspondence from the tax authority, and several years of transactions. The law may be somewhere else entirely. Yet the taxpayer is still expected to connect all of that information, understand what matters, calculate correctly, and know when something is missing.

Large organizations can distribute that complexity across accountants, tax lawyers, finance teams, auditors, and compliance staff. An individual or small business often cannot. The information may already exist, but what is missing is the time, expertise, and capacity required to investigate it properly.

That is the problem that inspired Kodi Tax Agent. I did not want to build another chatbot that simply knew a lot about tax. I wanted to explore whether an AI system could actually work on a tax problem.

The mental model that helped me was the way modern coding agents work. A coding agent does not need the developer to tell it every file to open. It receives an objective, inspects the repository, reads documentation, uses tools, runs tests, notices failures, asks questions when necessary, and keeps working until it reaches a defensible result.

I kept asking myself why tax software could not work in a similar way. A coding agent has a repository. Kodi has a Tax Case. A coding agent examines files. Kodi examines taxpayer records and evidence. A coding agent uses development tools. Kodi uses governed tax, document, knowledge, validation, and reporting capabilities. Tests tell a coding agent when something is wrong. Evidence checks and validation tell Kodi when a tax conclusion is not yet supported. Git preserves the history of the work. Kodi preserves the history, evidence, questions, and decisions inside the case.

That became the foundation of Kodi Tax Agent. The goal is not to build AI that simply knows a lot about tax. The goal is to build an AI agent that can work on tax.

What it does

Kodi Tax Agent is an AI-powered tax workspace for individuals and small businesses. A user can ask a normal tax question, but the more important capability is that the user can give Kodi a broader objective and let it work through that objective as a durable Tax Case.

A user might ask Kodi to review 2025 business records, explain a tax demand, investigate a cross-border income question, identify missing evidence, or determine what still needs to be resolved before a reliable tax computation can be prepared. Kodi is designed to move beyond a single prompt and response by investigating available information, retrieving relevant evidence, using tax knowledge where needed, persisting what it learns, asking focused clarification questions, and producing grounded answers with citations.

The Tax Case workspace is separate from ordinary conversation. It has its own objective input, interaction history, live activity, clarification flow, responses, final outcomes, and durable state. This makes it possible for Kodi to work on a case over multiple steps instead of forcing the entire problem into one model call.

Kodi also includes a taxpayer document library. Users can upload business and tax records such as PDFs, spreadsheets, CSV files, Word documents, invoices, receipts, sales summaries, expense ledgers, mobile money registers, payroll records, inventory schedules, asset registers, and other evidence. Kodi processes those records asynchronously and turns them into structured observations and canonical evidence that can later be used in tax reasoning.

The core workflow is simple to describe even though the implementation underneath it is not. The user provides an objective. Kodi understands the problem, investigates the available records, retrieves evidence, retrieves relevant tax knowledge when necessary, reasons about what it finds, uses governed capabilities, persists progress, asks for clarification only when it is needed, and then produces a grounded response.

The product is built around an evidence-first idea. Kodi should not only tell the user what it thinks. It should be able to show what records support the conclusion, what remains uncertain, and what still needs to happen next.

How we built it

For this hackathon, I built Kodi as a complete tax platform around the idea of an autonomous Tax Case. The frontend is built with React, TypeScript, and Vite, while the backend uses Python, FastAPI, Pydantic, PostgreSQL, and a service-oriented architecture deployed on Google Cloud.

The autonomous agent layer uses Google Agent Development Kit. Gemini models run through Vertex AI and are assigned different responsibilities rather than forcing every task through one large prompt. Gemini 3.7 Flash handles autonomous Tax Case planning. Gemini 3.6 Flash handles document understanding and grounded response synthesis. Gemini 3.5 Flash is used for bounded semantic reasoning such as claim relationship judgment, while Gemini 3.5 Flash-Lite handles lighter tasks such as conversation-turn resolution and canonical claim extraction. OpenAI remains in the system for retrieval embeddings only.

I deliberately separated semantic intelligence from application authority. Gemini can plan, interpret, and reason, but Kodi remains responsible for identity, ownership, authorization, evidence boundaries, tool execution, persistence, version handling, validation, and auditability. The principle I used throughout the build was that Gemini provides semantic intelligence while Kodi provides authority.

Document processing is asynchronous because real documents do not become reliable evidence the moment they are uploaded. A document enters the Document AI service, becomes a durable PostgreSQL work item, and is later picked up by a Cloud Run Job triggered through Cloud Scheduler. The worker uses Gemini for document understanding, produces structured observations, validates those observations, assembles canonical evidence, and prepares the material for retrieval. Private source-document bytes are stored in Cloudflare R2.

The document pipeline follows a provenance chain rather than treating model output as automatically authoritative. The source document produces structured observations, which become validated canonical evidence. That evidence can then become CaseEvidence, taxpayer facts, calculations, and finally user-facing conclusions. This lets Kodi preserve where important claims came from instead of adding citations as an afterthought.

The autonomous execution loop is also durable. Each run begins from fresh persisted state, assesses available evidence, determines what actions are allowed, asks Gemini to plan the next step, executes one governed action, persists the observation, reloads the case, and then decides whether to continue, clarify, wait, or answer. PostgreSQL is the durable source of truth for the Tax Case rather than a temporary model session.

Kodi is deployed as a real cloud application. The production backend runs across twelve Cloud Run services, with a separate Cloud Run Job for document processing. Cloud SQL for PostgreSQL stores durable application and case state. Vertex AI provides Gemini inference. Secret Manager, Artifact Registry, Cloud Build, and Cloud Logging support the production environment. The architecture also includes dedicated services for authentication, orchestration, document intelligence, tax logic, knowledge, validation, forms, reports, storage, event persistence, and autonomous agent execution.

Challenges we ran into

The hardest part of the project was not getting a model to produce a fluent answer. The hardest part was making sure the system was looking at the right evidence, understanding the difference between missing evidence and failed infrastructure, preserving progress, and returning conclusions that could be traced back to actual records.

One recurring challenge was working with real document collections instead of a handful of perfect test files. At one point, a multi-document upload failed across every file with a storage PUT 404. That forced me to trace the entire upload contract from the frontend, through the upload initiation endpoint, to the signed private-storage URL, and then back into document finalization. The lesson was that upload reliability is part of the agent experience because an autonomous system cannot reason over evidence that never reached the platform correctly.

A more important failure happened later when I had 27 synthetic 2025 business documents visible in Kodi's Documents library. Eighteen were already searchable and nine were still processing. I opened a Tax Case and asked Kodi to review the records. Kodi responded that there were no 2025 financial records available.

The documents were visibly sitting in the application, so I did not treat this as a prompting problem. I traced the actual execution path. The Tax Case reached Orchestration, but Orchestration was forwarding internal runtime metadata such as runtime_id, conversation_id, and tool_execution_id into Document AI's strict hybrid-search payload. Document AI rejected the request with HTTP 400. The failure was then described as retryable, no usable CaseEvidence reached the case, and the model incorrectly concluded that no records existed.

That bug exposed a distinction that became important to the whole architecture. A failed evidence search is not the same thing as a successful search that found no evidence. Evidence may be missing, processing, ready with limitations, partially available, or temporarily unavailable because a tool failed. Kodi has to preserve those differences rather than collapsing them into a single "nothing found" state.

The same failure also exposed a jurisdiction problem. When evidence retrieval failed, the fallback answer introduced US forms such as 1099-K and 1099-NEC even though the case was Kenyan. That made it clear that final synthesis must be grounded in the actual case jurisdiction, retrieved taxpayer evidence, and governed tax knowledge rather than falling back to generic model assumptions.

Another challenge was separating normal conversation from autonomous work. Earlier versions let the ordinary conversation composer double as the Tax Case objective input. That blurred two very different user experiences. I rebuilt Tax Case as an independent workspace with its own composer, thread, live activity, clarification flow, citations, durable interactions, and final responses. Normal chat remained independent.

Durable state was another major challenge. A tax workflow may need to pause because a document is still processing, two records conflict, the taxpayer needs to answer a question, a tool fails temporarily, or the agent reaches a point where it needs more evidence. I could not let that state exist only inside model context. Tax Case state therefore lives in PostgreSQL and can be reconstructed after refreshes, restarts, retries, and clarification turns.

Knowing when the agent should stop also took repeated work. An autonomous system can easily fall into search and retrieval loops without materially improving the case. I added bounded execution, no-progress detection, action validation, and safe stopping behavior. I also learned that the system must distinguish between a fact that is required for a final tax determination and a fact that is not required for the user's current objective. For example, residency may be important for an authoritative tax computation, but it should not necessarily block a request asking what the business records themselves show.

Calculations created another design challenge. I originally expected the Tax Core to eventually contain most tax calculations, but manually encoding a complete and changing tax system into thousands of jurisdiction-specific functions would become difficult to maintain. Modern models are now much better at extracting figures, comparing records, and performing ordinary arithmetic, so I started separating document-derived and AI-derived figures from authoritative tax determinations. The long-term direction is to let AI understand facts and applicability while deterministic execution and validation provide stronger guarantees where they matter most.

Accomplishments that we're proud of

I am proud that Kodi became much more than a chat interface. It is a working tax platform with authentication, taxpayer-specific document storage, asynchronous document processing, autonomous Tax Cases, durable state, tax knowledge, governed capabilities, validation infrastructure, citations, forms, reports, and a production cloud deployment.

I am especially proud of the evidence model. Kodi does not simply ask a model to read a document and then trust whatever comes back. The platform preserves a path from source document to structured observations, canonical evidence, case evidence, taxpayer facts, and finally the conclusion shown to the user. That provenance is important in a domain where a confident answer is not enough.

The Tax Case architecture is another major accomplishment. Kodi can receive a high-level objective, plan what to do next, use governed tools, persist observations, reload fresh state, ask the user for clarification when necessary, and continue from where it stopped. The case survives browser refreshes and application restarts because the work itself is durable.

I am also proud that the project was repeatedly tested against realistic failures instead of only happy paths. I used a synthetic Kenyan sole-proprietorship document pack with many different record types and processing states. Problems such as broken upload contracts, retrieval payload mismatches, processing-state confusion, citation gaps, jurisdiction fallback, and unnecessary clarification were treated as architecture problems and traced to their real causes.

The production deployment is another milestone. Kodi runs across twelve Cloud Run services with a separate Cloud Run Job for document processing, Cloud SQL for durable PostgreSQL state, Vertex AI for Gemini inference, and supporting Google Cloud services for secrets, builds, artifacts, and logging. The project is not only a local proof of concept.

Most importantly, I am proud that the project remained focused on the original problem. The technology became increasingly sophisticated, but the purpose stayed simple: expand access to tax capability. Kodi can help a taxpayer directly, and the same system can help tax professionals reduce repetitive evidence review, research, reconciliation, and case preparation so they can spend more time on work that truly needs professional judgment.

What we learned

The biggest lesson was that building useful agentic software is much more than connecting an LLM to tools. The model response is often the easiest part. The difficult work is making sure the agent is acting under the correct identity, looking at the correct taxpayer's evidence, distinguishing a failed tool call from missing evidence, preserving progress, handling retries safely, and knowing when enough evidence exists to answer.

I also learned that memory in an autonomous system should not be reduced to chat history. For Kodi, memory is the accumulated state of the work. It includes objectives, evidence, facts, tool observations, clarification questions, user answers, workflow progress, failures, and final outcomes. Persisting that state changed the system from a conversation demo into something closer to a real case-management agent.

The document pipeline taught me that provenance has to begin much earlier than final answer generation. If the system only thinks about citations after the reasoning is finished, it becomes difficult to establish what actually supports a claim. Preserving the connection between source records, evidence, facts, calculations, and conclusions makes the entire system more explainable.

I also learned to be much more careful about the distinction between semantic reasoning and authority. AI is extremely useful for understanding messy documents, natural language, context, ambiguous relationships, and case-specific reasoning. Deterministic software is better at enforcing invariants, permissions, exact execution, and high-stakes validation. The strongest architecture is not one where either side does everything.

Finally, the project changed how I think about tax computation itself. I no longer think the sustainable path is to hardcode an entire tax system into a growing collection of imperative functions. A better long-term direction is likely to combine authoritative legal sources, versioned structured rules, AI interpretation of applicability, structured calculation plans, deterministic execution, and validation. That could make Kodi easier to maintain across changing tax years and eventually across multiple jurisdictions.

What's next for Kodi Tax Agent

The next major area is stronger calculation authority. Kodi can already extract and reason over figures, and modern models can perform useful document-derived arithmetic, but I want important calculations to carry explicit provenance and stronger validation. The goal is to distinguish clearly between a fact taken directly from evidence, an AI-derived calculation, a deterministic calculation, and a fully validated tax determination.

I also want to move tax policy out of a future monolithic Tax Core and toward versioned, declarative rules. Rates, bands, thresholds, reliefs, effective dates, and eligibility conditions should be represented as policy data rather than buried inside application code. AI can help interpret which rules apply to a taxpayer's facts, while a smaller deterministic engine executes the resulting calculation plan.

Cross-document reasoning is another priority. I want Kodi to become better at reconciling bank records, mobile money, sales summaries, invoices, inventory, payroll, and expense evidence as one connected case rather than as isolated documents. That includes identifying contradictions, duplicate transactions, unsupported figures, missing periods, and evidence gaps before they become tax problems.

Professional workflows are also important. Kodi is not intended to remove tax professionals. I want the platform to help them serve more people by reducing repetitive evidence review, research, preliminary calculations, and case preparation. That could eventually support collaboration between taxpayers and advisors inside the same durable case.

Longer term, I want Kodi to support more tax domains, changing legislation, richer audit and explainability tools, and additional jurisdictions without creating a separate hardcoded tax engine for every country. The architecture should make it possible to add new rule packs and authoritative knowledge while preserving the same evidence, reasoning, execution, and validation model.

The long-term goal is not simply a better tax chatbot. It is a system capable of doing meaningful tax work while preserving the evidence, reasoning, and human judgment required to trust the result.

The goal is not to make tax more technically impressive. It is to make the ability to understand and act on tax more accessible.

Built With

Share this project:

Updates

Submission history