-
-
Three layers aligned in time: NWS forecast, NOAA/NDBC station, and a privacy-safe local gauge record.
-
Source-traceable comparison showing temperature and wind-speed differences with aligned wind direction.
-
The tool verifies comparable timing before interpreting discrepancies and preserves valid timezone labels.
Inspiration
Local weather forecasts can be technically correct for a broad area while still failing to describe what a person is actually experiencing at a specific location.
The idea for Local Reality Check of Weather came from repeatedly seeing differences between forecasts, alerts, weather-station readings, photographs, and direct local observations. Rain alerts sometimes arrived after the rain had already started, repeated warnings created fatigue, and localized conditions such as smoke, fog, cloud cover, wind, or temperature changes were not always fully explained.
This project began as a small branch of a larger weather-research concept. For the OpenAI Build Week submission, the scope was deliberately reduced to one practical question:
When local weather information conflicts, what is missing, and what should be checked next?
What it does
Local Reality Check of Weather is an investigation tool for comparing different layers of local weather evidence.
A user can enter:
- Forecast information
- Weather-station observations
- Hyperlocal observations
- Source identifiers
- Timing and location details
- Visible conditions such as cloud cover, smoke, haze, fog, rain, wind, and reduced visibility
The tool compares the available evidence and produces:
- A discrepancy level
- Factor-by-factor classifications
- A confidence assessment
- Missing information
- A timeline of the submitted evidence
- Questions that should be investigated next
- Suggested follow-up observations
The purpose is not to declare that a forecast or weather source is simply “wrong.” The purpose is to identify where the evidence agrees, where it conflicts, and what additional information may explain the difference.
How we built it
The prototype was built in Python with a Streamlit interface.
The project uses structured evidence models for forecasts, weather-station records, and hyperlocal observations. Each record can preserve its source identity so the information remains traceable during analysis.
The analysis system combines deterministic comparison rules with OpenAI-assisted interpretation and question generation. Deterministic thresholds handle measurable conflicts such as temperature, humidity, wind, visibility, and timing differences. The model then helps organize the evidence, identify missing context, and generate useful follow-up questions.
The submission was intentionally limited to local evidence sources at the town, county, and state levels. Broader regional, national, and experimental weather sources remain outside the initial MVP.
The repository also includes sample data, documentation, evidence models, source tracking, and automated tests.
Challenges we faced
One major challenge was preventing the tool from making conclusions that the evidence could not support.
Weather information may differ because of distance, observation time, sensor placement, elevation, localized cloud cover, smoke, terrain, coastal influence, or rapidly changing conditions. A disagreement does not automatically prove that one source failed.
Another challenge was keeping the submission small enough to complete while preserving the larger purpose of the idea. The broader research concept contains many possible atmospheric, environmental, and infrastructure inputs. For this MVP, the focus remained on observable local discrepancies and the next useful question.
Source tracking was also important. Evidence without a clear source, timestamp, or location can create more confusion instead of improving understanding.
Accomplishments that we are proud of
The project now provides a working structure for turning scattered local weather observations into an organized investigation.
Important accomplishments include:
- Preserving source IDs throughout the analysis
- Separating raw evidence from research questions
- Supporting forecast, station, and hyperlocal observations
- Identifying missing fields instead of silently inventing information
- Producing follow-up questions based on the submitted evidence
- Supporting multiple weather hazards, including rain, heat, wind, smoke, air quality, snow, ice, and fog
- Creating automated tests for the core analysis behavior
The project also demonstrates that observations rejected as incomplete or conflicting may still have value when they are properly labeled and placed into context.
What we learned
The largest lesson was that local weather disagreement is often an information problem rather than a simple accuracy problem.
A forecast, station reading, photograph, and human observation may each describe a different part of the same event. The useful work is determining how those pieces relate.
We also learned that asking better questions can be more valuable than forcing an immediate answer. Missing timestamps, source identity, direction, duration, sensor location, or visibility details can completely change the interpretation of an event.
The prototype reinforced the importance of traceability, uncertainty, and honest limits when using AI to evaluate real-world evidence.
What's next
The next step is to test the tool with more real local weather events and seek feedback from meteorologists, forecasters, and weather observers.
Future development may include:
- Improved terminology and confidence scoring
- Better handling of delayed or repeated weather alerts
- Photo and observation validation
- Additional sensor and environmental inputs
- Comparison across multiple local sources
- Exportable investigation reports
- A structured destination for incomplete, rejected, or underused weather data
The long-term goal is not to replace professional meteorology. It is to help people and researchers organize local evidence, expose missing information, and ask better questions about why actual conditions may differ from the information they received.
Log in or sign up for Devpost to join the conversation.