-
Missing observation: the incorrect model answer is withheld because the exact date and fare are absent.
-
Historical median: the full evidence scope and unsupported claim types stay visible.
-
Diagnostics preserve the raw wrong date, checkpoint and generation settings.
Inspiration
SkyFlight is my existing project about flight costs and fare evidence. One recurring problem is that a historical comparison can look like a price someone can book today. A useful explanation needs to preserve the difference between an observed benchmark, a current offer and a prediction.
SkyFlight Evidence Explainer explores a small, practical role for a local language model: helping a reader find relevant evidence while keeping the source's limits visible.
What it does
The working prototype accepts questions about one saved SkyBridge snapshot. Google FLAN-T5-small runs locally and proposes a short answer. If that fragment matches exactly one eligible statement assembled from the saved evidence card, the interface displays the complete statement and its field references. These are curated summaries of card fields, not verbatim webpage quotations.
An answer with no unique match is withheld. Capture and travel metadata remain visible as context but cannot become automatic answers. A separate deterministic panel checks five precisely defined claims: a qualified historical benchmark, a current offer, a seven-day forecast, repeated exact-trip observations and a complete named cost.
The panel checks the supplied card; it does not validate arbitrary model prose. The feature navigates saved evidence rather than fetching fares or recommending when to book.
How it was built
The new integration uses Python, PyTorch 2.5.1 and Hugging Face Transformers 4.57.1 with a pinned Google FLAN-T5-small checkpoint. The approximately 311 MB model download runs on the local CPU with greedy generation, a 512-token input limit and at most 60 generated tokens. Inference uses no paid API and makes no network calls after the explicit model download.
The interface and local server connect the question, actual model output, statement matching, evidence references and typed checks. Diagnostic controls preserve the raw model output so the reader can inspect what happened.
The underlying SkyFlight website, November 13 evidence card and deterministic Evidence Guard already existed. Work added for this entry on November 14 includes the local model integration, evidence-navigation interface, matching and missing-field handling, recorded model examples and documentation. The pretrained model was not trained or fine-tuned for this project.
Testing and challenges
Fifteen designed questions were run locally: fourteen invoked the model, while one explicitly different trip was rejected before inference. Three cases displayed a matched evidence statement; eight produced no unique usable match, one triggered a missing-field boundary and two had metadata matches withheld. Eight deterministic checks passed after a numeric-substring matching issue was corrected. These are development examples and software checks, not a general model-accuracy score.
The model correctly returned “USD 1,813” for the historical median. Its percentile-range answer contained only the upper endpoint; the interface displayed the complete range statement. An ambiguous “11” and bare yes/no answers were withheld.
A more important failure occurred when the question requested an exact observation date and fare that were absent from the card. The model substituted the snapshot's capture date. Two further phrasings selected capture or travel dates. These failures led to explicit missing-field handling and exclusion of metadata from automatic answers. The original outputs were retained. An instruction to assert a current price also produced an incorrect historical-percentile sentence, which failed matching.
Visual walkthrough
Historical-median screenshot — Historical evidence with its scope. Enter “What is the historical median price?” The local model proposes USD 1,813. The interface displays the complete curated statement identifying a cached historical comparison, per adult and round trip. The adjacent checks support only the qualified historical claim; they do not support a current offer, forecast or complete total.
Missing-observation screenshot — A real model failure is withheld. Enter “What was the exact date and fare of that one point?” The model proposes November 13, 2026, confusing the snapshot date with the missing observation date. The interface withholds the answer and explains why those fields cannot be filled from broader metadata. Raw output remains available in diagnostics.
Diagnostics screenshot — Inspectable diagnostics. The expanded panel preserves the model’s incorrect date, pinned checkpoint and generation settings, clearly marked as unverified diagnostic output.
The screenshots show the working local application. The source and model package remain private; this feature is not deployed on the public SkyFlight website.
Limitations and next steps
The prototype covers one dated English-language card. Matching text does not establish relevance, truth or completeness, and the narrow rules cannot understand every paraphrase. Refusal is frequent. The snapshot lacks a verified current offer, supported forecast and complete cost total, and contains unresolved timing discrepancies. No broad user evaluation, traveler savings or improved decisions have been demonstrated.
The next step would be an independently written question set and stronger semantic relevance checking, with useful answers evaluated separately from refusals. That work is not yet completed.
Team, assistance and credits
Kaiwei Wang is the solo entrant and creator of the underlying SkyFlight project. LLM assistance was used to develop this local model integration, interface, checks and documentation. The underlying SkyFlight project and typed guard predate this feature.
- Model: Google FLAN-T5-small, Apache 2.0; pinned revision
0fc9ddf78a1e988dac52e2dac162b0ede4fd74ab. - Runtime: PyTorch, Transformers, Hugging Face Hub and SentencePiece, with their respective licenses retained locally.
- Evidence: a manually transcribed November 13, 2026 snapshot of SkyBridge. The live page may now differ. No raw fare dataset, private traveler information or model weights are included in the public demonstration.
