OpenPolicy AI

• 

  Principle/Confidence Scoring Pipeline

2026 Duke University Society-Centered AI Hackathon Concept Theme: Evaluation Benchmarks for Society-Centered AI Juhyun Nam & Grady McCarter Abstract The exponential expansion and implementation of artificial intelligence in economic and social domains has been closely followed by a sharp increase in political and regulatory activity in the United States. Between 2016 and 2024, the number of AI-related bills active at the state and federal levels grew by more than an order of magnitude, reflecting rising concerns about AI alignment, safety, and fairness. Despite this, AI governance remains fragmented across jurisdictions, agencies, and policy practices, making it difficult to accurately evaluate how political actions and individuals collectively shape responsible AI development. While benchmarking has become a central practice for measuring progress in technical AI systems, the ability to evaluate AI policy actions remains limited. We present OpenPolicy AI, a benchmark system designed to aggregate, classify, and quantitatively evaluate political actions in relation to AI governance. The system files through public legislative records, voting histories, bill sponsorships, and official statements through its pipeline. A keyword-based filter reduces irrelevant content, after which LLMs classify actions according to predefined AI alignment principles. These signals are combined using deterministic scoring functions that account for strength, recency, and confidence, producing normalized and comparable scores across political actors. By transforming dispersed policy into standardized, public-facing benchmarks, OpenPolicy enables transparent, nonpartisan comparisons of AI-related political behavior. This project demonstrates how benchmarking methodologies traditionally applied to AI performance can be extended to governance contexts, offering a scalable approach for tracking and evaluating fair and responsible AI growth in an increasingly complex regulatory landscape.

Link To Presentation: https://prodduke-my.sharepoint.com/:p:/g/personal/gtm18_duke_edu/IQDCf70on6npTo-PShFKXu8WAWCiZdAM2tI2i2G19HfsXeI?e=5pu92X

Built With

• claude
• node.js
• react
• redis
• sql
• typescript