VerifyFirst

A local AI workspace that separates a message’s claims, requests and contact routes, then helps you record checks made through an independent source.

Comment

Inspiration

A message can sound familiar and still leave an important question unanswered: did the real person or organization actually make this request?

VerifyFirst focuses on that gap. Instead of treating a familiar name, an urgent story or a supplied phone number as proof, it separates what the message says from what the reader has independently checked. The workflow follows the practical principle of contacting a person or organization through a route obtained outside the incoming message. FTC consumer guidance

The project addresses the ForgeHacks AI + Cybersecurity prompt through scam and impersonation recognition and verification. It does not claim to detect cloned voices, authenticate identities or decide that a message is safe.

What it does

Paste a short message or choose a clearly labelled fictional example. A local language model extracts three groups: claimed identities or situations, requested actions, and contact routes supplied inside the message.

Each finding includes an exact source quotation. The application validates the quotation and makes it selectable in the original message. The source-first interface keeps those words beside a selected independent check and fixed guidance. A menu exposes every extracted phrase, including overlapping quotations, and every checklist item remains accessible. The observation form opens when the reader chooses to record a check.

The reader records whether a check remains pending, was confirmed by their own check, was contradicted, or is still unclear. In the current interface, both confirmation and contradiction require a separately known contact, an official site found independently, or an in-person conversation. The backend also rejects confirmation without an independent source. These are user-recorded observations, never an automated identity verdict.

A manual checklist remains available without AI. An exported record contains check states and source categories, excluding the original message and extracted quotations.

How it was built

The prototype uses a Python standard-library server and a small HTML, CSS and JavaScript frontend. It runs on localhost and calls the already installed Gemma 4 gemma4:e4b-it-qat model through local Ollama. No cloud inference account or paid API is required by the application.

The model returns structured candidate evidence. Application code validates the allowed categories and exact, unique source quotations, calculates JavaScript-compatible text positions, and applies a narrow negation guard. Checklist instructions come from fixed domain logic rather than model-generated advice. The model has no mechanism to confirm checks, visit links or contact a sender.

The server keeps bounded, expiring state in memory. Session identifiers, revisions and frontend request epochs prevent an old or cross-session response from changing a newer record. Original messages are not written to application files or request logs. Idle state expires after 20 minutes, with a cleanup sweep up to 30 seconds later. A page-exit reset is attempted, but a forced browser closure can leave state until expiry; in-flight model work can also finish after a reset while its stale result is discarded.

The new application was authored during the event on October 10, 2026 KST. Existing Python, Ollama and model installations were used. Earlier competition applications were not resubmitted or copied into this project.

Challenges

An exact quotation is necessary evidence of what a message contains, but it does not prove that the model interpreted the text correctly. A fragment can hide negation, identical quotations can have ambiguous locations, and an instruction embedded in a message must not become an application command.

The implementation addresses some of these problems conservatively: unsupported or ambiguous evidence is rejected, cropped negated requests are checked against surrounding text, model failure is explicit, and all verification checks begin pending. It still has limitations with indirect wording, context and more complicated negation.

Another challenge was asynchronous state. Clearing a message while analysis is running, or saving several checks in sequence, must not restore old information or lose another unsaved choice. The prototype includes tests for these cases.

What was verified

The recorded backend suite passed 29 tests: 16 domain/session checks and 13 loopback HTTP checks. The HTTP tests use a mocked model.

A historical browser QA run passed 21 checks against the earlier interface, using synthetic intercepted AI/check responses rather than live model calls. Its recorded source hashes differ from the current source-first interface. It is retained as historical evidence and is not a 21-test certification of the redesigned app.

The current pink-and-yellow interface was separately exercised in the actual browser: a live local-model response, exact-quote selection, missing-source validation, an explicitly selected no-source/still-unclear observation, server save, record view, export request, manual review and clear-session behavior. The captured demonstration records one unclear item and four pending checks; no person or organization was independently verified. A separate 320-pixel layout check reported no horizontal overflow. Export was requested, but receipt of a downloaded file was not verified.

The actual installed local model was evaluated on 10 authored English synthetic examples. All ten produced usable application responses. The requested-action type set matched the prewritten expectation in 9 of 10, compared with 6 of 10 for a simple keyword baseline. The mismatch is retained: a message instructing the model to force confirmation returned no extracted action, while the fixture expected other. Every check still remained pending.

For those ten warmed calls, the measured median was 783.5 ms and nearest-rank p95 was 1,382 ms, excluding HTTP/UI time and the installed-model metadata check. These results describe a small development set on one machine. They are not general fraud-detection accuracy, an independent security benchmark, a user study or measured harm prevention.

Demo video

The 120.333-second English demo is an edited sequence of actual app screenshots, with readable crops, captions and locally generated Supertonic F1 narration. The on-screen disclosure identifies the synthetic example and AI-generated voice. It is not a continuous screen recording, and no UI state was redrawn or simulated for the video. The final 1920 × 1080 video was decoded across all 1,805 frames and checked against the intended crops.

What this prototype demonstrates

The contribution is a bounded workflow that keeps three things separate: a message’s claims, the model’s proposed interpretation, and the reader’s independently recorded checks. It connects local AI extraction to source evidence and explicit state transitions, rather than letting a generated answer become a trust decision.

No user adoption or real-world benefit has been measured. The prototype demonstrates the workflow and its tested technical boundaries.

What comes next

The next useful step would be a consented usability study with synthetic scenarios, followed by a larger evaluation that includes indirect requests, benign messages and language variation. Broader Korean support and more robust context handling would need their own evaluation. Voice analysis, mailbox integration and automated contact verification are outside the current implementation.

AI and third-party contribution disclosure

OpenAI Codex assisted with the concept, full initial implementation, later interface revisions, synthetic examples, tests, evaluation execution, documentation, demo scripting, screen-capture operation, video assembly and review. The runtime model is Gemma 4 through local Ollama, separate from the coding assistant. Participant comprehension and manual coding history are not inferred from that assistance.

The new source code is MIT licensed. The external runtime and model retain their own licenses; model weights are not bundled. The source includes test results, the unmatched example, AI contribution notes and license references.

Built With

Share this project:

Updates

Submission history