Inspiration
I didn’t know that evaluation was “a thing,” or that what I did for fun was already a serious methodology. My specialty is visual culture and communication, not engineering. I wasn’t trying to solve AI. I wanted to understand how human experience could translate into binary reasoning.
It doesn’t matter how large the stack becomes. At the end of every corridor, machines are binary. We expect them to perceive without discernment, even though they’re incapable of either. When context is missing, models fill the gaps through inference. As those inferences accumulate, the output can become detached from both the prompt and reality, placing the responsibility on the human to adjust to the machine.
Polinko began with a simple question: what happens if human judgement is translated into the machine’s native structure?
The goal isn’t anthropomorphism. Machines don’t experience existence. The goal is to make behaviour observable, preserve accountability, and examine the space where the human remains autonomous while the machine remains automated.
What it does
Polinko uses binary evaluation gates determined by human judgement. There’s either PASS or FAIL. There’s no weighting, confidence score, or partial credit that asks the system to interpret the space between them.
Failure is the important signal. Polinko preserves failures, evaluates them in context, and traces the behavioural patterns that emerge across repeated cases. The question isn’t simply whether one response was correct. The question is whether the same behavioural boundary remains reliable as the context changes.
OCR is one mature evaluation lane because its source is externally checkable. Polinko evaluates difficult handwriting and sketches, instructing the model to transcribe only what it can reliably decipher and use explicit placeholders when the source is unreadable. The data is inconsistent on purpose. Instead of rewarding inference, repeated evaluation across changing cases allows patterns to form from evidence.
How we built it
“We” is important because Polinko was built collaboratively. I designed the research method, system architecture, and evaluation gates. Codex engineers my design while I work alongside it to maintain alignment, meaning, and coherence.
Before each task, we use documentation to establish a working contract. This gives Codex a clear boundary while allowing me to follow the implementation, ask questions, and understand what’s being built.
Polinko uses local glue code to remain confidential, lightweight, and traceable. Scripts automate build hygiene, validation, runtime maintenance, evidence reporting, and repeated evaluation. Transcripts are indexed by concept, and curated records preserve their verbatim source alongside structured insights. Automated checks enforce that evidence contract.
Polinko predates Build Week, and is developed entirely with Codex. During the Build Week period, Codex and GPT-5.6 were used to audit, refactor, validate, document, and present the active system. This included repository cleanup, runtime and architecture review, the public website surface, UI and UX, metadata, evidence documentation, visual QA, and submission preparation.
GPT-5.6 was especially effective at gathering context from Polinko’s dense documentation and long development history. It could generate meaningfully different design options, interpret detailed feedback, and move between engineering, research, documentation, and visual work without losing the project’s underlying logic.
Challenges we ran into
The most difficult aspect of Polinko has been learning how my theory is engineered into architecture and runtime while working alongside Codex.
AI reflects human design and limitations. It isn’t the creator of the project, so it’s my responsibility to understand what the agent is doing. Because Codex makes its process visible, I can follow along, ask questions, and learn how each decision affects the system.
Long-running context also creates continuity challenges. Polinko has evolved across many sessions, models, hypotheses, and refactors. I audit the build every day to ensure that technical improvements don’t separate the architecture from the theory. Documentation, source evidence, and working contracts allow the collaboration to continue without asking the model to rely on stale memory.
Accomplishments that we’re proud of
Polinko began as try.py because I was trying things and learning whether I could engineer the theory I already had. The early glue code got Polinko moving. The refactor is turning that exploration into durable architecture without erasing how it formed.
I’m proud that Polinko preserves its own development as evidence. Its transcripts, evals, early scripts, decisions, corrections, and retained failures remain traceable.
The Build Week demo now runs through a single command:
make build-week-demo
The command steps through the live evaluation path in the terminal: repo preflight, API smoke, OCR source metadata, a binary OCR eval, result counts, evidence artifacts, cleanup, and a final PASS/FAIL/ERROR summary. Codex provides the working narration while the command prints the source, gate, result, and saved report paths, so the demo shows both the collaboration process and the evidence trail.
Polinko doesn’t merely produce a result. I can follow how the result was formed, inspect its evidence, and trace how my theory becomes architecture and runtime.
What we learned
I learned that one of the biggest mistakes in agentic coding is assuming that “works” means “good.” An API can be connected and a model can produce a response while the result still lacks meaning, coherence, or method. Capability alone doesn’t make an output useful. Human judgement gives the system its boundaries.
I also learned that refactor, revise, redo, restart, rework, restore, and reset are human concepts. I don’t consider the earlier versions failures. Each version records part of the journey. Refactoring doesn’t erase those marks. It determines what belongs in the active build, what belongs in the evidence archive, and what was scaffolding that helped me learn.
Reliability isn’t a single correct answer. It’s continuity of behaviour across changing contexts. That’s why hygiene, documentation, process, and method matter. They allow the engineering to change while the research remains coherent.
What’s next for Polinko
First, I’m completing Polinko’s script, architecture, and runtime refactor. The early glue code got Polinko moving. Now I’m separating durable architecture from the scaffolding that helped me learn, while preserving the evidence that explains how the method formed.
After that, I want to test whether the same evaluation method holds across different models rather than remaining tied to one provider. gpt-oss is a future test within Polinko, not a separate project. The same binary gates and manual evals can establish which behaviours come from Polinko’s structure and which come from the underlying model.
Once the operator surface is stable, I can return to the smaller Toy Factory experiments and continue Polinkofolio, where Polinko’s transcripts, theory, observations, and visual research can become publicly readable.
Log in or sign up for Devpost to join the conversation.