AccountantBot

Inspiration:

We work at ComplYant, a company that helps folks simplify small business taxes. We wanted a way for small business owners in Los Angeles to ask questions about taxes and business compliance whenever and wherever they have them. Local regulations are often the trickiest to get information about, so we decided to go deep and specialized on one metro area first, instead of wide and general on the federal level only.

What it does:

AccountantBot is an intelligent chatbot designed to offer personalized accounting assistance. It helps Los Angeles small business owners navigate complex federal, state, and local tax regulations. Users can text questions whenever they need to, receive timely answers, and get directed to where they need to go to get the appropriate paperwork.

How to use it:

Text a small business tax question to +1 (253) 352-4700! Here are few ones to try, if you need some ideas.

• I think I owe sales tax on my baked good sales but I don’t know what to do about that.
• What is business personal property tax and when do I need to worry about it?
• Why is an S-Corp better than a Single-Member LLC?

How we built it:

To move quickly we leveraged as many familiar tools as we could. On the first day we focused on trying to de-risk several concepts and ideas. We landed on what felt like the correct ideas early and began trying to validate them. There is a lot of space in the AI world were we don’t know what we don’t know, so it was important to us to rigidly verify all assumptions as we moved forward.

Early on the second day we ruled out one of the major goals as being unrealistic for the duration of the event. The rest of the product ideas still held value so we moved forward with a refined focus. Build moved quickly after finalizing the scope. We had the ability to tackle our build on three fronts, largely looking at infrastructure, prompt engineering and the workflow system noted in the accomplishments section.

Challenges we ran into:

• Tried to build into it the Los Angeles specific tax authority information, which required too much data and too many new technologies for this hackathon.
• URLs in answers sometimes inaccurate.
• Answers sometimes not entirely complete.
• The non-deterministic nature of the AI model meant answers can be of varying quality.
• The model was difficult to force certain behaviors, like “If you cannot answer, reply with FAILED.”

Accomplishments that we're proud of:

In practice, we find that many of the options available present better solutions for our users’ questions. Even within a single model/parameter set, it is possible to get a bad answer, then rerun the same prompt and get a really helpful one. Taking advantage of this, we developed a couple of stages.

The workflow system works by leveraging a feedback of prompts:

• Send user prompt to a couple of “workers,” each potentially different models/parameters or entirely different engines and platforms.
• Each worker then vets the answer by feeding it into a customized prompt that includes the original question. This ensures the answer appears to be properly relevant to the question.
• For each answer that passed vetting, a final round is done to rank and select the best answer. Again, each vetted answer is combined into a new prompt with the original question. This picks the answer we return for our user.

At the HackAIthon, we focused only on a GPT-3.5 model from OpenAI, however the intention is to extend the system to leverage Bard and others as a possible source of information.

What we learned:

We learned quite a lot about bringing in our own data sources to train the model on. The time limitations of the HackAIthon prevented us from doing so in our submission, however we learned the critical skills needed to implement this.

Prompts have a wild amount of power to control the user experience. The ability to use prompts to refine new prompts was a helpful tool to learn.

What's next for AccountantBot:

• Using our own embeddings to leverage our proprietary knowledge base and data sources.
• Trying it out with more metro areas.
• Building an interface for our Tax Research team to be able to vet responses and mark answers "accountant approved."
• Adding additional models and platforms to our worker sources.
• Performance, performance, performance!

Built With

• amazon-web-services
• filament
• laravel
• openai
• php
• twilio
• vapor