Inspiration

As students we need a good starting point to start our investment carrier quickly and cheaply, in depth Market reports take days or even weeks to compile by even specialists, and thus a student with limited time is severely limited in scope in terms of decision factors which could result in poorly informed financial decisions. We thought AI could have significant contribution to solving this problem however there were some issues we had to tackle.

How we improved accuracy

Every AI tool in the market is susceptible to hallucinations and not being able to validate the legitimacy of sources thus making its outputs not very trustworthy. We aim to eliminate these issues through our custom harness state machine design, which keeps the human in the loop for source validation, as well as multi step deterministic code and agent verifications against the original data source.

How we built it

We built the front end stack with Next JS, using pretext for real time text wrapping in the wayland compositor style window system, and a small api client with pre defined types that connects to the backend. The backend runs on a EC2 instance, and uses bedrock for the models, Bedrock Guardrails to scan fetched text, stores runs and text in DynamoDB, and files in S3. We implemented a 10 step pipeline (plan → analyst approves sources → collect (fetch PDFs/HTML, pull out text per page) → extract (one Haiku agent per page, in parallel, each returning exact quotes) → draft (Sonnet, sees only checked data) → verify → repair → summarize (3-page brief) → verify the summary → finalize.) to ensure our harness catches all of the misbehaviours of the models.

Challenges we ran into

Coordination between multiple members was challenging as we had to maintain a centralised API between the front end and backend while adding new features in, getting the feel of the window system and generative UI just right in the front end needed some fine tuning. In the backend, the citations linking back to financial tables didn't always match exactly because of different heading formats and unit symbols which meant we had to handle alot of edge cases

What's next for Project Axelrod

Next up we would like to expand this project by adding a account system, managing budgeting and cost of generation, as well as the quality given the cost. we would run more benchmarks as well as tune specific models to work even better in our harness.

Built With

  • claude
  • dynamodb
  • ec2
  • nextjs
  • pydanic
  • pymupdf
  • python
  • s3
Share this project:

Updates

Submission history