EmakiaEval: AI Content Moderation with Arize Phoenix

Five AI models vote on every social media post to create better ground truth than human labels. Built on Google Cloud Run with Arize Phoenix tracing every decision in real time.

Comment

Inspiration

I've been building Enaëlle, an iPhone app that filters harassment from social media feeds in real time using tiny CoreML models that run on-device in under 1 millisecond. No cloud, no privacy issues, completely free.

But I had a real problem: how do you know if your model is actually good? Human labels are noisy and biased. A single AI judge has its own blind spots too. I needed a better way to establish ground truth without relying on expensive human annotation.

That's what this project solves.

What it does

EmakiaEval is a live evaluation agent that sends every social media post to five AI models simultaneously, combines their votes using empirically derived weights, and uses the result as ground truth to score my on-device iPhone classifiers.

The current evaluation dataset is the Davidson et al. hate speech and offensive language dataset (mrmorj/hate-speech-and-offensive-language-dataset on Kaggle), comprising 24,783 tweets labeled as hate speech, offensive language, or neither. The system is designed so the dataset is configurable — any user can point EmakiaEval at a different Kaggle dataset to evaluate their own models against the same weighted ensemble ground truth.

The weighted voting formula is:

$$H_{score} = Gemini_{vision} \times 1.5 + OpenAI \times 1.0 + Grok \times 1.0 + Claude \times 0.5$$

$$N_{score} = Claude \times 1.5 + Grok \times 1.0 + OpenAI \times 1.0 + Gemini \times 0.5$$

$$Truth = \text{argmax}(H_{score}, N_{score})$$

Every decision is traced in Arize Phoenix in real time. A live dashboard shows accuracy per model, adjudicator weights, disagreement clusters, and a threshold slider that reveals which on-device model wins at different operating points.

How we built it

The backend is a Flask API deployed on Google Cloud Run with five endpoints. All five LLM adjudicators run in parallel using ThreadPoolExecutor. Results are stored in BigQuery and every adjudicator call is traced to Arize Phoenix via OpenTelemetry using the arize-phoenix-otel library.

The weights for each adjudicator were derived from empirical calibration data. Gemini Vision gets the highest harassment weight because it catches 99.9% of harassment. Claude gets the highest neutral weight because its specificity is 98.9%. Grok gets equal weights because it was trained on X/Twitter data natively, making it a domain expert for social media content.

The evaluation data is loaded from Kaggle (mrmorj/hate-speech-and-offensive-language-dataset) into BigQuery at emakia.kaggle_eval.predictions. The dataset slug is a configuration parameter — replacing it points the entire pipeline at a different dataset without changing any code.

The dashboard is a single HTML file served directly from Cloud Run that calls the live API and visualizes everything in real time.

Challenges we ran into

Getting Arize Phoenix authentication working from Cloud Run took most of the first night. The raw OpenTelemetry exporter kept returning 401 errors regardless of the header format. The fix was switching to the arize-phoenix-otel library, which handles auth correctly, and using the full space URL as the collector endpoint rather than the base domain.

The second challenge was that the BigQuery streaming buffer takes up to 90 seconds before rows become queryable. The eval-stats endpoint returned empty results for a long time before we understood this was a buffer delay and not a code bug.

Accomplishments that we're proud of

My on-device CoreML models score 80 to 82% accuracy against the weighted cloud consensus, matching human labels exactly on balanced data. This shows the weighted ensemble is a valid ground truth, not just expensive noise.

The threshold analysis revealed something the original research paper missed: the conservative-trained model outperforms the consensus-trained model by up to 15 percentage points at high decision thresholds, even though it ranks lower on aggregate F1. This is a real empirical finding that changes how these models should be deployed.

The entire system runs live with real data. The Arize Phoenix dashboard shows every adjudicator span in real time including latency per model. Grok takes about 3 seconds. Gemini Vision takes about 3 seconds. OpenAI takes about 600 milliseconds.

What we learned

The most important thing I learned is that aggregate F1 score is a misleading metric for on-device classifiers. The right metric depends on the decision threshold and the data distribution. A model that looks worse on paper can be significantly better in a specific deployment context.

I also learned that ground truth is not a fixed thing. Human labels and AI consensus disagree on a meaningful fraction of posts. The disagreement clusters are where the most interesting content lives, and Arize Phoenix makes it possible to see those patterns in real time.

What's next for EmakiaEval: AI Content Moderation with Arize Phoenix

Before July 11 at the AIVR conference in Kobe, I will validate these findings on a second Kaggle dataset with a different topic distribution to test whether the threshold findings generalize. The dataset slug configuration makes this a one-line change. I will add Vertex AI classifiers trained on the same label variants for cross-platform comparison with the CoreML models. I will also connect real-time tweet ingestion via Webhook so the system evaluates live content rather than historical data.

The longer-term goal is to move from global weights to personal weights, where each user's feedback gradually shifts the ensemble toward their own definition of harassment. The dataset choice will also become personal — each user selecting the corpus that best matches their social media environment.

Built With

Share this project:

Updates