Personal Portfolio Analyst

Was your time spent picking stocks worth it? The largest retail brokerage (Fidelity) doesn't run the analysis and won't export the data. This app gets the data and delivers the answer.

Comment

Inspiration

I've picked my own stocks at Fidelity for years, and I couldn't answer the most basic question about it: was my time spent picking stocks actually worth it, risk-adjusted, compared to just buying an index fund?

My brokerage couldn't tell me either. Fidelity doesn't offer the analytics and it doesn't export the daily data you'd need to run them yourself (their official Reddit account has confirmed daily performance history simply isn't available for download).

So I built the analyst I wished existed, one that would tell me the truth, including "no, it wasn't worth it."

What it does

Personal Portfolio Analyst starts with one question and follows it to its conclusion.

Getting the data is the first challenge. Fidelity's web app fetches things like daily balance history from its backend APIs but never exposes them for download. The guided import walks you through downloading every export Fidelity does offer, then through capturing the API responses your own logged-in browser session already receives for the data Fidelity doesn't. The app then extracts, transforms, and joins these datasets into an analysis ready daily series. From there, the app compares your returns against an aligned S&P 500 benchmark. A passive replica is also built from your own history, because the S&P 500 maybe not be a suitable benchmark for the assets in the portfolio. The passive replica is mix of up to five ETFs fitted on the earliest 75% of your returns and tested out-of-sample on held-out months. The result allows the user to quickly determine if they can reclaim their time and simply invest in passive ETFs to replicate their returns.

How we built it

From the beginning, the repository was structured for agent legibility so that each new coding session could inherit the project's context instead of reconstructing it from source code or chat history.

A compact AGENTS.md acts as the entry point. It records the product north star, code map, architectural dependency direction, essential commands, and non-negotiable rules. Deeper source-of-truth documents separately own different product boundaries so an agent can find the specific document governing a decision without loading the entire project into context.

Complex changes are developed through versioned execution plans stored alongside the code. Each plan records the intended outcome, non-goals, evidence, constraints, milestones, acceptance criteria, decisions, progress, validation commands, and remaining external inputs. Plans are classified as active or completed and repository checks reject inconsistent plan states and stale references. Future sessions can therefore understand not only what exists, but why it was built that way, which alternatives were rejected, what problems appeared during implementation, and which limitations remain unresolved.

I adopted OpenAI's harness engineering approach: whenever we learned something, we encoded it in the repository rather than leaving it in conversational memory. A discovered edge case became a boundary and regression test. A recurring workflow became a make command. An architectural rule became a mechanical check. Corrections accumulate as durable tests, documentation, scripts, and lint rules that every future agent inherits.

As the codebase grew, this became increasingly important. Large analytics functions, orchestration workflows, React components, and stylesheets became difficult to evaluate through line count alone. I had codex build a cross-language CRAP (Change Risk Anti-Pattern) analysis tool that combines cyclomatic complexity with branch coverage for both Python and TypeScript. Running make crap executes fresh coverage, scores individual functions, ranks the highest change-risk hotspots, and writes a machine-readable reports. A high score does not automatically fail the build; it tells the next agent where focused tests, decomposition, or both will reduce the most risk before a change.

Every form of drift we actually encountered became a mechanical check: custom checkers enforce the architectural dependency direction, keep the API schema and frontend types from silently diverging, validate documentation and plan lifecycles, block private financial data from ever entering the public repository, and smoke-test the built app in a real browser. Versioned migrations and stored-result upcasters protect upgrades the same way as the data model evolves.

The application itself follows the same philosophy of explicit boundaries. Untrusted Fidelity inputs are parsed and normalized before reaching the deterministic Python analytics (pandas, NumPy, SciPy, scikit-learn); raw financial files are processed in memory and discarded; FastAPI owns orchestration and SQLite persistence; and the React/Vite dashboard presents typed analytical results without recalculating them in the browser.

Codex shaped the build from the first conversation, including feature(s) I built and then deliberately turned off:

  1. Scaffolding the stack as a collaborator. Before any code existed, I asked Codex what infrastructure, backend, and frontend stack would fit this kind of application. I adopted its proposal essentially wholesale, with one pushback: it suggested Postgres, I argued SQLite seemed like a better choice for a single-user, local-first tool, and it agreed. That exchange set the working pattern for the whole project: the agent proposes, the human owns the judgment call.
  2. A Codex ETL for data Fidelity won't export, built, then disabled. Repo-scoped agent skills turned Codex into a guided acquisition engine: driving the investor's authenticated Fidelity tab to trigger the internal balance-history API and download every export automatically. It worked. Then we used Codex for a legal review of Fidelity's Terms of Use against our own codebase and the review found that while our manual import path is sound, the agent-driven path falls inside the Terms' prohibition on software that "automates or otherwise assists" downloading, with an explicit clause voiding user authorization as a defense. So I disabled the agent skills and rebuilt guided import around user-driven capture. Codex has the technical capability to be the ETL engine; it also told me why it shouldn't be.
  3. Drafting return assumptions for view-based optimization. For Black-Litterman allocation, an explicit Codex handoff lets the agent research and draft expected-return views into a loopback-only workspace. Codex can propose, only the investor can review and apply the same propose-review-decide pattern we used to build the app, now built into it.
  4. Compliance as part of the build loop. The import path in this submission is the one that survived our own agent-assisted review.

The result is a repository that functions as durable shared memory and a continuous feedback system. Agents working in different sessions inherit the same “organizational context” and the division of labor stays deliberate: the harness gives agents durable context and hard boundaries, agents handle judgment-heavy preparation and review, and every number on screen comes from deterministic and tested code.

Challenges I ran into

When engineering stopped being the bottleneck, curation became the hard problem. Codex delivered roughly nine tasks in ten on the first attempt and most of the rest with one more push. Features that in a past life cost a week or more of implementation arrived in under an hour, often in parallel. The scarcity inverted: the expensive question was no longer "can we build it?" but "should this exist, and where?" Cheap features accumulate into an incoherent interface, and the old instinct to not remove something that looks expensive to build misleads when it took an hour. I ended up spending more time subtracting than adding: distinguishing happy-path value from edge-case clutter, and reshaping seven workspaces into one product. The commit history shows the shift, many feature commits are followed by "simplify," "remove redundant," or "hide."

Accomplishments that we're proud of

  • One person, plus a team of Codex agents. A product with this surface area historically required a team: multiple engineers, a designer, a product manager, a lead to coordinate them and legal counsel to keep it compliant. Here, every one of those seats was filled by agents working under one person's direction, including legal counsel: it was an agent's review of Fidelity's Terms of Use that led us to disable our own agent-driven import feature. The human job is reduced to the part that can't be delegated: product judgment.
  • Day one: the feature Fidelity users have been requesting for years. Public threads dating back years, several with official Fidelity responses confirming "not available", asking for full portfolio tearsheets. The working core (guided import, daily cash-flow-aware returns, benchmark comparison and risk/return metrics) existed by the end of our first day.
  • Day two: analytics that used to be someone's job. Ten-dimension factor analysis, efficient frontiers, bayesian view-based portfolio allocation adjustments, capabilities that institutional firms staff with dedicated analysts, and that no retail brokerage offers at all. Running locally for the self-directed investor by our second day.
  • A QuantStats-class tear sheet in one day. Our Performance workspace implements the full-report architecture of QuantStats, the standard open-source portfolio analytics package, in development since 2019, as a paired Benchmark | Strategy report: comparative chart rail, calendar-year results, rolling diagnostics, and a searchable, pinnable, reorderable metric catalog where QuantStats generates a static page.
  • Six days, 100+ commits, one complete product. Import → verdict → risk → factors → out-of-sample passive replica → rebalance, plus the validation harness (architecture checks, API contracts, real-browser tests, CRAP analysis) that kept quality from collapsing at that speed.

The recurring feeling of the week: Every feature we could imagine clearly, we could have within hours. The craft was in organizing them into an application that doesn’t overwhelm the end user.

What we learned

  • Agentic coding moves the scarcity. When implementation is nearly free, product judgment becomes the binding constraint. What to build, what to remove and what belongs on the happy path. The discipline needed most wasn't prompting; it was subtraction.
  • Repository memory compounds. Agents got dramatically more effective when every lesson was encoded as a test, a check, a document, or a make command. The harness is what made session one hundred smarter than session one.
  • Agents fill team roles on demand. This is a team of one, but whenever the work needed a different specialist, ie: designer, product manager, finance quant, legal counsel, Codex became that role for exactly as long as the task required. Legal made the point sharpest: even funded startups rarely embed counsel in the build loop, yet a terms-of-service review that would normally mean an outside lawyer and a week's turnaround happened inside development in minutes and changed what we shipped. Team composition can now be assembled per task.

What's next for Personal Portfolio Analyst

  • More brokerages. Charles Schwab, Robinhood, and other popular retail platforms.
  • Fully automated imports. Work with Fidelity to get approval to automate the data ingestion process.
  • From target allocation to exact orders. Today the app shows where your portfolio should land; next it turns a chosen target, ie:Black-Litterman allocation into the precise share order list, so executing at your brokerage takes minutes instead of hours.
  • Simpler Keep subtracting from the interface. Fewer screens, clearer paths without giving up functionality that delivers value.

Built With

Share this project:

Updates