PsychoSight

A private, evidence-cited look at how you communicate. Experimental research prototype for self-reflection, not a clinical tool.

Inspiration

Years of text messages hold a record of how we talk to the people closest to us, but there is no private way to look at it. Research suggests everyday language carries signals about well-being. Absolutist words ("always", "never", "completely") are one example, and they were found at much higher rates in anxiety, depression and suicidal-ideation forums than in control forums. The literature is just as clear about the limits. Those findings are group-level, and a popular idea like "I-talk means narcissism" fell apart in a large replication (r = .01 across 4,811 people). Most "AI personality" apps ignore this and hand you a confident label. We wanted the opposite: a tool that reports only observable behavior, quotes its evidence, and says how unsure it is.

What it does

PsychoSight Lite analyzes one 1:1 iMessage conversation that you choose, on your own Mac, and produces a communication-patterns report:

• Weekly me-versus-them timelines for absolutist-word rate, "I" and "we" pronoun rates, who starts conversations, approximate reply time, message length, follow-up turns, and late-night messaging.
• An anchored 1-to-5 rubric of six observable behaviors in your own messages, such as acknowledging the other person's feelings, making repair attempts after friction, and shifting the topic to yourself.
• 3 to 5 observations, each scoped to "this conversation", with verbatim quotes and a confidence label.
• A "What this cannot tell you" box. There are no diagnoses, no labels, and no scores against norms or cutoffs. When there is too little text, it shows "insufficient data" instead of a number.

The AI step is strictly opt-in. Without it, you still get the full deterministic report, and nothing leaves your Mac.

How we built it

• Importer: a loader built on our own msgexport.py reads a read-only snapshot of the Messages database, filters out tapbacks, attachments and duplicates, and enforces 1:1 chats.
• Sessionizer: consecutive messages from one sender under 30 seconds apart form a turn. A gap over 20 minutes starts a new episode. The time math is DST-safe.
• Deterministic features: a 19-word absolutist list from published research, pronoun rates and initiation ratio, computed per ISO week. Weeks with fewer than 300 words per side return null instead of zero.
• LLM scoring (opt-in): up to 40 episodes, mixing the strongest signals with a seeded random sample. The rubric uses strict JSON output. Each episode is scored twice, and an item is kept only if the two runs agree within 1 point. Quotes must appear verbatim in your own messages, and the model's reasoning is discarded.
• Synthesis: observations are checked after generation for valid episode ids, verbatim quotes, and banned diagnostic wording. Confidence comes from how many episodes support an observation and how well the runs agreed.
• Report: assembled and validated against a JSON Schema. A schema failure is a hard error, never a silently bad report.
• Interface: a Python CLI plus a FastAPI server on 127.0.0.1, with SQLite storage and a DeepSeek model for the LLM step. A React and Vite web interface is in progress on its own branch. A synthetic, seeded demo conversation (634 messages, 59 episodes, 8 weeks) lets anyone try the pipeline without personal data.
• Privacy by design: the LLM cache stores scores and quote offsets with no message text, and logs contain only ids and counts. Tests enforce this.

Challenges we ran into

• Overclaiming: our first design, a 5-stage pipeline mapping messages to clinical-style T-scores, was found by an audit to be unsupported. We cut it down to a descriptive, non-clinical, evidence-gated tool.
• Unreliable LLM scores: single runs are noisy, so we added score-twice agreement, anchored rubrics, and a rule that quotes must be verbatim and from the right speaker.
• Untrusted text: message content can contain instructions, so it is treated as data and never as commands to the model.
• Time and thin data: daylight-saving boundaries, weeks with almost no messages, and noisy reply times all needed explicit handling.
• Safety wording: we wrote automated tests that lint prompts, the schema and the output for diagnostic language.
• Time limit: we built this in a one-hour sprint with parallel agents. Real-world testing on a Mac with Full Disk Access is the part still to do.

Accomplishments that we're proud of

• A full local-first pipeline from messages to a cited, schema-validated report, with about 4,000 lines of Python and 94 passing offline tests.
• Safety built into the code instead of added as a disclaimer: opt-in enforcement, no message bodies in logs or cache, and wording lints.
• A design grounded in the research and in an independent audit, with limitations shown inside the report.
• A clean contract (JSON Schema, API docs, sample reports) so the interface can be built separately from the backend.

What we learned

• A lot of "obvious" language-and-psychology claims do not hold up. Absolutist words are a real signal at the group level, but first-person pronouns are not a marker of narcissism.
• Showing evidence, uncertainty and "insufficient data" builds more trust than a single score.
• Checking an LLM against itself (two runs) and against the source text (verbatim quotes) removes a lot of nonsense cheaply.
• Privacy is mostly about what you decline to store.

What's next for PsychoSight

1. Real-world checks: run on real conversations with user consent
2. Validate the rubric against public-domain questionnaires, starting with the IPIP Big Five, before any construct is called a measurement.
3. Separate stable traits from temporary states using a latent state-trait model, so a bad week isn't read as a personality.
4. Fairness audit across demographic groups using differential-item-functioning methods.
5. Multi-conversation timelines, a polished macOS interface, and an on-device model option so nothing needs to leave the Mac.

Built With

• deepseek
• python