Proofwork

Inspiration

A table can contain every expected number and still be wrong. An extraction update can move a value into the wrong column, while a keyword search reports success. Document-pipeline maintainers need a small, repeatable way to protect the relationships that matter in their documents, without manually reviewing every new output.

What it does

Proofwork turns human-reviewed PDF table facts into reusable regression checks. A reviewer opens a source PDF, confirms a table, row key, named column and exact expected value, and links the check to a page region. They compare baseline and candidate Markdown or HTML tables and inspect pass, failure and unresolved results. A failure can take the reviewer back to the source evidence.

The browser processes documents locally. Exported project JSON contains checks, evidence coordinates, outputs and source hash metadata, but not PDF bytes. A Node.js CLI uses the same checking engine so reviewed checks can also act as a CI gate.

How we built it

The React and TypeScript interface uses PDF.js for source review. Structured Markdown and HTML parsing identifies table rows and columns. Exact comparisons normalize Unicode and whitespace while retaining differences in signs, units, punctuation and precision. Ambiguous matches remain unresolved rather than silently passing. SHA-256 binds the saved evidence to the reviewed source PDF; it does not certify the provenance of arbitrary candidate text. Static hosting serves the browser app, and CI runs the shared engine locally through Node.js.

Challenges we ran into

Duplicate labels, malformed tables, headerless data and compound row identities expose the boundary of a simple exact-check workflow. We chose explicit unresolved states and blocking gates instead of guessed matches. We also separated source facts from controlled extraction mutations so a demonstration would not imply that synthetic faults were naturally occurring failures.

Accomplishments

The demo uses a real TableFormer research-paper table. A controlled candidate swaps values across two columns: the expected number is still present elsewhere, but the source-linked check correctly flags the wrong relationship.

Independent real-world validation covered 13 real PDFs and two external fixtures. The native authoring workflow was usable on 6 of 8 evaluated cases, below the preset 80% target, so the overall verdict is PARTIAL. Eight frozen correct-fact checks passed. In 88 controlled variants built from real inputs, 48 faults were rejected, 16 ambiguous cases remained unresolved and 24 controls passed. These counts describe this test set, not general extraction accuracy. The public report retains failures, limitations and a separately corrected reference error.

What we learned

Reliable document checks need a trusted human-reviewed reference and a precise relationship, not just a plausible number. A useful failure report must explain what changed and show the evidence needed to decide the next step. Reporting an unsupported case honestly is more valuable than making it look successful.

Community impact and sustainability

The community addressed is developers and QA teams maintaining PDF extraction pipelines. Misplaced table values can undermine downstream datasets and research workflows. Proofwork offers this community a source-linked review and regression workflow; wider social impact has not been measured. The current product is free and requires no hosted processing service. A possible future business model is paid onboarding, support and team workflow services around reusable check libraries. This is an unvalidated proposal: we have no paying customers or proven willingness-to-pay results to claim.

Static delivery avoids a central document-processing bottleneck; each user's browser performs its own work. The CLI fits independently managed CI jobs. This is an architectural property, not a measured throughput claim. Large-document memory limits still require further evaluation.

What's next

Proposed work includes support for headerless tables, compound row keys and reviewed table matching. These features are not implemented. Proofwork does not extract tables, perform OCR, infer source truth, or use an AI model. It complements extraction tools by checking selected facts after extraction.

Built With

• github
• indexeddb
• node.js
• parse5
• pdf.js
• react
• typescript
• vite