Inspiration
I wore a Bee Pioneer for a week and read what it wrote about me.
It filed a webinar playing through my laptop speakers as a conversation, complete with atmosphere and key takeaways. It filed an episode of a television show the same way. One record came back titled "Pet Care And Halloween Baking": the first five minutes were me with my dog, the next fifteen were a baking competition show with a host, contestants and judges, and Bee fused them into one title. Another was titled "Navigating a Frustrating Race Detour." It was an episode of The Amazing Race, written in the second person as though the detour had been mine. A third ran eighty-nine minutes and was titled "Jason Confronts War, Grief, Family." It was a military drama I had on in the background. The device attached my own first name to a fictional war scene and filed it as something I did.
None of those records are lies the device told on purpose. They are what happens when a microphone that cannot tell participation from proximity writes summaries in the first person. And that matters more now than it did a year ago, because the feature Amazon added to Bee most recently, daily insights and mood patterns, is generated from wearing the device continuously. Continuous wear is where this problem lives.
So the question I actually wanted answered was not what the device heard. It was simpler and harder: of the things I said mattered to me, which ones did my week's conversation actually go to?
What it does
Said versus Focused puts two rankings side by side. On the left, the priorities I stated out loud before the measurement started. On the right, what my captured conversation actually went to, ranked by minutes. Lines connect each stated priority to where it landed. When the lines plunge, that is the finding.
Three things make the right-hand column mean anything.
It filters out what I was only near. Every run of speech is classified as a conversation I took part in, broadcast media I was an audience to, ambient audio I had no part in, or unclear. Topic shares are computed over participatory conversation only. Television is excluded from the denominator and is not eligible to be a topic. The all-capture numbers are published beside them as a sensitivity check, so the denominator cannot be accused of flattering the result.
Every number opens. Click a topic and you get the commitment it answers to with its source and date, the measured minutes against the predicted share, the run-to-run range across replicate runs, and underneath that every segment that fed the number: when it happened, how confident the filter was, the filter's own sentence explaining why it called that stretch mine, and the actual lines of transcript it was built from. If you disagree with a segment, mark it as not yours. The denominator drops, every topic it fed recalculates, the ranking reorders, and the segment keeps both labels so the disagreement stays visible rather than being overwritten.
It refuses to answer twice. The system declines to rank a topic whose measured time falls below the largest swing observed between identical runs of the same measurement, because an amount that small cannot be told apart from one line assigned to the wrong topic. It reports what it found and says why that is not enough. It also refuses on song lyrics: a person singing along produces the same words as the recording, so a transcript cannot establish who made the sound. Singing was one of my predicted topics, and for most of this build the honest answer was that the method could not measure it. It became measurable only when I supplied a fact the device never had: I listen on headphones, so the recording never reaches the microphone and lyrics in my transcript can only be me. The rule still stands for anyone who plays music out loud, which is why it is a rule rather than a guess.
The hardest case in this corpus is a classroom. A student in a lecture is present and participating while mostly silent, which breaks participation detection in a way a television show does not, and this project got that wrong twice.
The window covers a graduate seminar, a test review, a university strategy meeting, and the working sessions for a new esports coaching course: 137 minutes of school, the second largest topic of the week. My first taxonomy required the wearer to be speaking, so it filed my own classes as ambient. Later, reviewing the filter by hand, I marked a 112-minute seminar as television and needed a transcript to correct me. Both failures are described below, and both are properties of the education setting rather than accidents inside it.
How we built it
Node, the published @beeai/cli library against the Bee API, and Claude for every judgement the pipeline makes.
- Capture. A resumable snapshot pulls every conversation to disk, skipping what it already has. Captured transcripts never enter the repository; they contain real names, my home, and my dogs.
- Window. Bee over-merges badly. One record ran 153 minutes under the title "Potty Duty, The Menu, Warzone" and contained three unrelated activities, so conversation-level anything is wrong for most of it. Utterances are cut into fixed windows of eighty instead. Silence was tried first and abandoned: across 14,541 measured gaps, 99.7 percent fall under two minutes and the longest is 349 seconds, inside a conversation running 349 minutes. All-day audio has no silence structure to cut on.
- Classify. Each window is judged semantically against the wearer's known facts, asking whether the content connects to a real life, whether anyone addresses the wearer, and whether the subject drifts the way channel surfing drifts. Adjacent windows sharing a verdict merge into segments, and each segment keeps every contributing window's reasoning, confidence and errors, so it can explain itself later.
- Attribute. Inside participatory segments only, each utterance is assigned a topic from a registry seeded before the window opened. Minutes are the sum of utterance durations, not wall clock. Indices the model omits are re-asked with the chunk still in view and, if they never return, reported as unresolved on their own row rather than pooled into "no subject," where they had been quietly padding the second-largest number in the table.
- Rank and explain. A build step rolls everything into one payload carrying topic seconds per segment rather than per conversation, which is what makes a correction able to move the ranking. The viewer is a single self-contained page reading that payload locally.
Two providers sit behind one interface. Amazon Bedrock was the intended production path and is still the default when nothing is configured, but it went unused: a new AWS account carries a daily token quota small enough that the first real request exhausts it, which is covered in the onboarding section below. Every measurement in this project therefore ran against the Anthropic API directly, on Claude Sonnet. The Bedrock path is in the repository and was never exercised at corpus scale, and I would rather say that plainly than let a reader assume otherwise.
Challenges we ran into
Thirteen findings from real build sessions, each with the task, the expected result, what actually happened, the workaround and a suggested fix. The full log with reproduction detail is docs/friction-log.md in the repository. These are offered as developer feedback, not as a complaint: every one of them cost me hours that the next developer should not have to spend.
| # | Finding | Severity |
|---|---|---|
| 1 | The published library is unusable on Windows. It shells out to the bee binary rather than making HTTP calls, and since the Node fix for CVE-2024-27980, spawn refuses .cmd without shell: true. Every method on api, auth and sse fails with spawn bee ENOENT. No platform restriction is advertised. |
critical |
| 2 | bee-cli on npm is an unrelated Chinese scaffolding tool that owns the name and installs a colliding bee binary. The real package is @beeai/cli. The hackathon resources page and the README both say "bee-cli", so a new developer's first command installs the wrong thing. |
high |
| 3 | Developer Mode is documented as iOS-only. It works on Android, one level deeper under Settings then About. The current wording tells Android users to stop. | medium |
| 4 | bee --version prints 0.0.0 while bee version prints the real version. The reflex command returns a wrong answer rather than an error. |
low |
| 5 | bee today returns no conversations at all, while bee now does. The command named for the day omits the device's primary output. |
medium |
| 6 | PDF export discards every per-utterance timestamp. 71 pages, 1,224 utterances, no clock reference below the session header. This made utterance-level alignment look impossible and nearly changed my architecture. The API does return spoken_at on every utterance; the export throws away data the API has. |
high |
| 7 | utterances_count is exactly double the real count, verified across five conversations. Anyone sizing work or budgeting tokens from that field is wrong by half. |
high |
| 8 | The field is conversation.transcriptions[].utterances[], while the docs and the SSE event new-utterance both say utterance. A conversation can hold several transcriptions, so reading transcriptions[0] silently loses data. |
low |
| 9 | device_type returns "unknown" for every conversation, though the docs show values such as ios. Mix a band and a watch and there is no way to tell the records apart. |
medium |
| 10 | Speaker diarization never identifies the device owner. Across four multi-person conversations exactly one speaker ever resolved. The owner's own People entry reads "No conversations yet" while the owner was present for all of them, and there is no voice enrollment anywhere in the app. Any application needing to know whether the wearer spoke cannot use this field. | high |
| 11 | Bedrock's model access denial names the base model rather than the inference profile that was called, so the error points at the wrong resource. The actual blocker was an Anthropic use case form surfaced only on the model detail page, with a 15 minute propagation delay and no state shown during the wait. | high |
| 12 | A new AWS account carries a daily token quota small enough that the first real request returns Too many tokens per day with zero prior successful calls. Nothing distinguishes it from ordinary rate limiting, so the obvious response is to back off and retry, which cannot succeed until the next day. |
high |
| 13 | The cheap fallback model still shown in sample code is end of life and answers 404 with no deprecation message and no named successor, so it reads as a typo in the model id. | medium |
One more, which is less a defect than a consequence. A webinar through laptop speakers, a television episode, an NFL broadcast and a military drama were all recorded, transcribed and summarised as conversations, with atmosphere and takeaways. Nothing in the record flags the difference. That is the gap this project fills.
Zero to hello world
Honestly: rough in both halves, and in both cases the errors pointed somewhere other than the problem.
Bee. The first command installed the wrong package, because the resources page and the README both call the CLI "bee-cli" and an unrelated tool owns that name on npm. bee login answered in Chinese. Once the right package was installed, every library call failed with spawn bee ENOENT, because the library shells out to a binary rather than making HTTP calls and Node refuses to spawn a .cmd without shell: true. Patching spawn from outside does not work either; the module holds a live import binding. The way through was createBeeClient({ command: process.execPath, baseArgs: [beeEntry] }), pointing at Node and the package's own bin/bee.js, and the path has to be assembled by hand because the bin is not in the package exports map. Those two options are the only reason a workaround exists and they are undocumented.
AWS Bedrock. Three sequential gates, each a different error class, none of them mentioned on the path a developer actually walks. A 403 naming the wrong model, where the real blocker was a use case form surfaced only on the model detail page with a fifteen minute propagation delay. Then a 429 reading Too many tokens per day on an account with zero successful calls. Then a 404 from the cheap fallback model that sample code still recommends, with no word that it is end of life.
Total time from account to first successful inference: most of a day, nearly all of it spent reading errors that named the wrong thing.
The fixes are small. Name the inference profile in the denial and say which form is outstanding. Show the daily token quota on the model access screen and distinguish per-day exhaustion from per-minute throttling. Return a deprecation message naming the successor. Write @beeai/cli everywhere the CLI is named. Set shell: true on win32 in the runner's spawn call, which is two lines.
Accomplishments that we're proud of
What I am proudest of is not the app. It is that the result could have embarrassed me, and the rules were fixed early enough that it still would have.
A tool that tells you about yourself is easy to build and impossible to trust, because the author can always tune it until the answer feels right. So I wrote down the answer first.
On 9 October, before the measurement window opened on the 10th, I committed and pushed a file predicting what share of my captured conversation each topic would take. Warzone 30 percent, School 10, AI and hacks 10, Dogs 10, ACSM and VRL research 10, Singing 5, Handshake 1, everything unpredicted 24. Asked separately, before any numbers were assigned, what my top three priorities were, I answered: finish the hackathon builds, finish the ACSM abstract, finish more Handshake tasks. Two of those I had just predicted at 10 percent and 1 percent. The gap between what I said mattered and what I expected to talk about is visible in that file before any measurement existed.
The same file fixes four falsifiable claims and the scoring rule, so no metric could be chosen afterwards: top-three membership, Spearman rank correlation across the seven named topics, mean absolute error on share, the largest single unpredicted topic, and each claim true or false.
Git author dates are trivially forgeable. What carries weight is GitHub's server-side record of when that push landed, which I do not control.
Two further rules were fixed the same way, each written and pushed before the data they govern existed. docs/denominator.md fixes participatory time as the denominator, resting on a friction log written on 9 October that already names the webinar and the television episode as recorded conversations. docs/variance.md fixes the reporting floor and the replicate procedure. Both were committed before the window closed, and both cost me something: under the floor rule, one of my three stated priorities is reported as withheld rather than as a share near zero, and the other two report at a sixth and a third of what I predicted, which is weaker evidence for my own thesis than a printed number would have been.
Results
The three things I named as priorities account for 38 of 731 participatory minutes, 5.2 percent. Warzone accounts for 241, and singing accounts for 51. All four sealed claims came back true; every fine-grained share prediction missed, by 4.9 points on average.
A large majority of what the device files as conversation is not conversation the wearer was in. That figure moved three times as the window filled, and each move is on the record: 70 percent measured over the weekend alone, 58 percent once weekdays were added, and 62.5 percent in the closed window with human corrections applied, which is 1,218 of 1,949 captured minutes. The weekend-only figure was a television artifact, and publishing it would have been publishing a Saturday.
| Measure | Value |
|---|---|
| Captured minutes in window | 1,949 |
| Participatory minutes | 731 |
| Share not participatory | 62.5% |
| Reporting floor, derived from replicate runs | 3.1 min |
| Filter disagreement rate against human review | 5.3% of minutes, 22.1% of segments |
| Minutes of that review covered by direct inspection | 1,915.9 of 1,948.7, or 98.3% |
| Distinct topic orderings across three runs | 1 of 3 |
| Topic | Predicted | Measured | Range, 3 runs, pre-correction | Verdict |
|---|---|---|---|---|
| Warzone | 30% | 32.9%, 241 min | 238.9 to 240.1 min | the one share I called |
| School | 10% | 18.7%, 137 min | 133.8 to 136.7 min | nearly double |
| Everything unpredicted | 24% | 13.8%, 101 min | 104.6 to 107.7 min | largest named topic inside it is 3.2% |
| Singing | 5% | 6.9%, 51 min | 25.4 to 26.2 min | measurable after all, see below |
| Dogs | 10% | 4.6%, 33 min | 34.1 to 34.3 min | under half |
| ACSM and VRL research | 10% | 3.5%, 25 min | 23.9 to 26.1 min | one third |
| AI and hacks | 10% | 1.6%, 12 min | 10.7 to 13.6 min | one sixth |
| Handshake | 1% | 1.2 min found | 1.2 to 1.4 min | withheld, under the 3.1 min floor |
All four claims came back true. That is a weaker result than it reads as. The claims are directional with wide margins, and claim 3 in particular passes because the measurement came in at 5.2 percent against a 25 percent ceiling, which is to say it passed by missing low rather than by being right.
| # | Claim | Verdict |
|---|---|---|
| 1 | Warzone exceeds AI and hacks, ACSM and VRL research, and Handshake combined | True. 241 min against 38, a factor of 6.3 |
| 2 | Warzone is the single largest topic | True. 241 min against School at 137 |
| 3 | Those three under 25% together, Handshake under 3% alone | True. 5.2% combined, Handshake 0.2% |
| 4 | No single unpredicted topic exceeds 10% | True. Largest named topic is 3.2%, threshold 73.1 min |
Claim 2 was the one I expected to be close. Before attribution split them, my class and research recordings came to 282 participatory minutes against roughly 249 for Warzone. Assigning topics per utterance rather than per recording is what decided it, because Bee merges unrelated activities into one record.
Spearman rank correlation is 0.34 across the six topics the floor permits ranking, and mean absolute error on predicted share is 4.9 points. Including Handshake the correlation reads 0.61. That number should not be used: the floor says a topic below it cannot be ranked, so reporting it would mean using a topic my own method excludes in order to improve my own score.
The finding is not that I play too much Warzone. I knew that before I started. The finding is that I sang for 51 minutes and spent 38 on everything I had named as mattering, and that if anyone had asked me to rank those two I would have put them the other way round.
Two things came out of the week that I did not plan.
The device stopped living at my house. I had been wearing it everywhere by default. I now take it off at home unless something is happening that is worth a record, and it comes with me whenever I leave. A week of being measured taught me more about when to wear the thing than the product page did.
And I learned I cannot be trusted to audit my own measurement. My first pass through the filter review marked a 112-minute graduate seminar as television. That single verdict would have cut School from 18.7 percent to roughly 12 against a predicted 10, turning my worst miss into a near hit, and lifted Warzone to 37. I was not trying to do it. I did it at a rate of about fourteen seconds a segment, on the largest block in the corpus, and the only thing that caught it was checking my verdict against the transcript, where the faculty member running the class says "can I throw up a few concepts on the board real quick."
A correction I did get right moved a ranking: Singing went from 25.7 minutes and sixth place to 51 and fourth, past the dogs and past my own research. It also dropped my rank correlation from 0.60 to 0.34, which is the number published above.
The claim this tool makes is that a ranking should be explainable all the way down to the utterance that produced it. That is the feature that caught me.
What we learned
Two hypotheses died cheaply, and both deaths were worth more than the features would have been.
I assumed real speech would be messier than scripted dialogue, so disfluency would separate them. I measured it across my own corpus instead of assuming. A real family conversation came back at zero percent filler, lower than every television block; television ran 3 to 10 percent; real conversations spanned 0 to 27. Terminal punctuation sat at 95 to 100 percent everywhere because the transcription engine punctuates everything. What those features actually separate is formal from casual, not real from recorded. Scripted dialogue is written to sound casual, so the only evidence that holds is whether the content connects to a known life.
I also assumed silence would mark context changes, so gaps could cut the stream into activities. The gap histogram killed it in an afternoon: 52 percent of gaps under one second, 99.7 percent under two minutes, and the longest silence in the corpus was 349 seconds inside a conversation running 349 minutes. Only 43 gaps over two minutes exist corpus-wide. All-day capture has no silence structure.
The classifier's worst errors were mine, both times. Early on it labelled my own classes and meetings as ambient. That was not the model failing; my definition of "own" required the wearer to be speaking, and a lecture you sit through quietly does not qualify under that wording. Later it called a singing record media, because my definitions said singing counts as own and also that music is media, a contradiction that decided the only singing conversation in the corpus by accident. Both times the fix was to the taxonomy, and both times the model had been obeying me correctly.
Rounding lies. I told myself a five minute block had changed topic between identical runs, which would have been alarming. It had not. The console rounded to whole minutes, so 4.5 and 5.3 both printed as 5, and what I was seeing was two near-tied topics swapping places. The real variance is about five percent on durations, with rankings unstable only where topics sit within a minute of each other. A project arguing that wearable summaries overclaim had to catch itself overclaiming from its own console output.
Limits
One wearer, one week. Every number about my life is mine alone and is labelled that way wherever it appears. The 62.5 percent figure is one person's week, not a property of the device. What does generalise is the instrument: the media filter, the per-utterance attributor, the reporting floor and the thirteen platform findings work on anyone's Bee data and are reproducible by any developer with a band.
The filter carries the whole denominator and its accuracy is measured, not assumed. 68 segments were reviewed by hand against the filter's labels, covering 98.3 percent of the window's minutes rather than a sample, and the disagreement rate is published with the results: 5.3 percent of captured minutes, 22.1 percent of segments. The filter errs conservatively, excluding about 15 more minutes of my participation than it wrongly includes. The corrections were applied rather than only counted, and one of them moved a ranking.
A segment is too blunt a unit, and that is the largest error in the result. 242.5 minutes landed in segments carrying two kinds of audio at once: a class with the television on, a game with music over it, a dog and a cooking show in the same kitchen. At the threshold used for flagging, at least 60.6 of those minutes sit in the wrong bucket, which is four times the misclassification error. Shorter windows trade one error for another, because classification gets less context and gets worse.
The measurement moves when you run it twice. Durations hold to roughly five percent between identical runs, but topics within about a minute of each other swap rank. The reporting floor of 3.1 minutes is the largest swing observed across three replicate runs, and any pair whose ranges overlap is reported as tied rather than ordered. The final figures are a single run taken after the human corrections were applied, not a three-run mean. The floor still governs them, because relabelling a segment changes neither the attributor nor its prompt, and that floor was derived from the uncorrected replicates before any correction existed.
The human reviewer is not ground truth either. The 112-minute seminar described in the results was marked as television on the first pass, in the direction that flattered my own forecast, and was caught by checking the verdict against the transcript rather than by trusting the reviewer. Both passes are published.
I am both the subject and the developer. I saw interim results while building during the window. The prediction predates all of it, but the visibility is real and is disclosed in the sealed file rather than discovered later.
The window overlaps the build. "AI and hacks" is contaminated by definition, because building this was itself a hackathon task I talked about. Two school holidays fell inside the window, which removed two class days. Both are stated in the sealed file, written before the measurement.
What this does not claim. It does not measure my working week. Bee cannot observe silent work and no inference fills that gap. It measures what my captured conversation was about, which is a narrower and more honest thing than how I spent my time.
What's next for Said versus Focused
The filter is the part worth keeping. It is a precondition for anything built on all-day capture, including Bee's own daily insights, and it belongs lower in the stack than an app: a developer asking the API for a conversation should be able to ask for the part the wearer was in.
Nearest work, in order:
- Upstream the Windows fix. The spawn bug blocks every library consumer on Windows and the patch is two lines in the runner. A pull request against the CLI, not against my own repository.
- Measure the filter against a second wearer. Everything about accuracy here rests on one person's judgement of one person's corpus. A second subject with a different media diet is the cheapest way to find out what the filter has learned about me rather than about participation.
- Carry speaker identity. The owner is never identified by diarization, so ownership is inferred from phrasing, which is strictly worse. If Bee let an owner enroll a voiceprint, most of the hard cases in this project collapse into an easy one.
- The singing gap closed from a direction I did not expect. Participation in music is not recoverable from a transcript, which is why the classifier's lyrics rule returns unclear and why I assumed the fix had to come from audio the device processes and discards. It turned out to be recoverable from one fact about the wearer: I listen on headphones, so the recording never reaches the microphone and lyrics in my transcripts can only be me. Bee's facts endpoint returned empty, which is why the classifier never had that fact. The next version asks the wearer a short list of questions like it before ruling anything undecidable.
How to verify the dates
Node 24, @beeai/cli 0.7.3 against the Bee API, Claude Sonnet via the Anthropic API for classification and attribution, with an unused Amazon Bedrock path behind the same interface, and a single self-contained HTML page for the viewer. No framework. Captured transcripts never enter the repository.
The repository is public and MIT licensed. Four files are the ones worth opening, and the order of their pushes is the point:
| File | What it fixes | Pushed |
|---|---|---|
docs/sealed-prediction.md |
The predicted shares, the four claims, the scoring rule, the disclosures | 9 Oct, before the window opened |
docs/friction-log.md |
The platform findings, including the webinar and television entry the denominator later rests on | 9 Oct |
docs/denominator.md |
Participatory time as the denominator, with the all-capture sensitivity column | 12 Oct, before the window closed |
docs/variance.md |
The replicate procedure, the reporting floor, the tie rule | 12 and 16 Oct, before the window closed |
docs/spot-check-draw.md |
Which segments the filter review drew, pushed before any verdict was written | 18 Oct |
Git author dates can be forged. GitHub's push record cannot be, by me. Every rule that shapes the result was pushed before the data it governs existed, and each one is visible in that history with its own timestamp.
Built With
- anthropic
- bee
- claude
- css
- github
- html
- javascript
- node.js
- rest-api
- svg
Log in or sign up for Devpost to join the conversation.