-
-
A small pilot comparison shows the fine-tuned LoRA model improving valid JSON output and scam-type prediction over the base Qwen model.
-
Trustfall starts with a privacy-first Terac collection task where workers submit one suspicious and one legitimate message.
-
Workers can upload a .eml file or paste text for SMS, DMs, marketplace posts, and other message types.
-
The workflow dashboard tracks real Terac-collected messages, human labels, and confirms that synthetic QA fixes are not used.
-
Other Terac workers label redacted messages with risk level, scam type, red flags, confidence, and practical safety advice.
-
Trustfall sanitizes links, emails, phone numbers, codes, and private identifiers before anything is stored.
-
Admins can review sanitized submissions and download deterministic training and holdout evaluation JSONL exports.
Inspiration
Suspicious messages are one of the most common ways people encounter scams, phishing, impersonation, and social engineering. Not to mention, there is a huge vunerability group: old people using outdated email handlers like Yahoo, in which there is a lot of evidence showing that it cannot reliably filter scams itself. But building better classifiers for these messages is hard because the best examples often come from real emails, texts, DMs, and marketplace posts that may contain sensitive personal information.
What it does
Trustfall helps collect privacy-protected, real human judgments on suspicious and legitimate messages without storing raw private message text, and trains models based on this collected data to improve its general judgement.
How we built it
Next.js App Router, Prisma, SQLite locally, browser-side .eml parsing, server-side redaction, Terac callback integration, train/eval exports, and a Qwen baseline workflow.
Challenges we ran into
The hardest part was balancing usefulness and privacy. Scam and phishing messages are valuable because they are realistic, but realism can also mean they contain names, phone numbers, links, codes, addresses, or other sensitive details. We had to design the workflow so the browser, the API, and the participant review step all reduce privacy risk before data is stored.
Another challenge was separating collection from labeling. We did not want workers to label their own submissions, and we wanted Wave 2 to depend on messages collected from other participants. That required thinking carefully about task ordering, label counts, and how to prioritize messages with fewer labels.
We also had to keep the project lightweight enough for a hackathon while still making the data pipeline auditable. Instead of adding a large external dataset or synthetic labels, we focused on making a clean Terac-only workflow with deterministic exports and a baseline evaluation path.
Accomplishments that we're proud of
We are proud that Trustfall treats privacy as a core product feature, not an afterthought. Raw message text is only held while the participant is filling out the form, and stored records are redacted before they can be used downstream.
We are also proud of the two-wave workflow. It creates a cleaner separation between message collection and human labeling, which makes the resulting dataset more trustworthy. The app includes practical details like .eml parsing, attachment stripping, redacted previews, participant attestations, label-count prioritization, admin visibility, and train/eval exports.
Most importantly, the system is designed around real Terac-collected human labels. That makes the project more realistic than a demo built on public or synthetic data, and it gives us a foundation for measuring whether fine-tuning actually improves over a base model on held-out examples.
What we learned
We learned that data collection UX matters just as much as model training. If the contribution flow feels unsafe or confusing, people will not provide high-quality examples. Showing a redacted preview before submission gives participants more control and makes the privacy model easier to understand.
We also learned that “human-labeled data” is not automatically trustworthy. You need clear task separation, duplicate labels, export rules, and held-out evaluation to make the data useful for training. Building the workflow forced us to think about provenance, consent, privacy, and evaluation as one connected system.
Finally, we learned that a small, focused dataset collected under the right constraints can be more valuable than a large dataset with unclear provenance.
What's next for Trustfall
Next, we want to complete the full fine-tuning loop: collect more Terac-labeled examples, train on the sanitized human-labeled training split, and compare the fine-tuned model against the Qwen baseline on held-out evaluation messages.
We also want to improve the labeling interface with clearer rubrics, disagreement tracking, confidence scores, and adjudication for messages where human labels conflict. On the privacy side, we want to add stronger redaction checks, better preview highlighting, and support for more message formats.
Longer term, Trustfall could become a general-purpose human-data collection layer for safety-focused classifiers: phishing, scams, marketplace fraud, impersonation, spam, and other high-risk communication patterns where realistic examples matter but privacy cannot be compromised.
Log in or sign up for Devpost to join the conversation.