Trinity

We came to HackMTY 2026 from Guadalajara with a simple question inspired by Altur’s challenge:

Can we really tell whether we are talking to a human or a machine?

At first, we thought the problem was straightforward: given an audio recording, classify it as human or synthetic.

But the more we explored the problem, the more we realized that a conversation is much richer than the sound of a voice. We identified three dimensions:

Acoustics: How does the voice sound?
Semantics: What is being said?
Behavior: How does the speaker interact?

This led us to a deeper question:

What actually makes a conversation feel human?

Exploring the Problem

We started by treating the project as a machine learning experiment. We explored the provided dataset, analyzed its variables, performed clustering, engineered features, and experimented with different predictive approaches, including XGBoost, hierarchical structures, ensembles, and different feature combinations.

Our performance improved quickly.

But instead of simply celebrating the higher accuracy, we asked a more important question:

Was our model learning what makes a voice synthetic, or was it simply learning this particular dataset?

That question changed the direction of the project.

Building Our Own Laboratory

Instead of continuing to optimize against the same data, we decided to challenge our own assumptions.

We built a controlled experimental environment using ElevenLabs, generating synthetic conversations and creating scenarios that our original models had never seen. We also recorded real human interactions and experimented with cloned voices.

This led to one of our most interesting discoveries:

Voice and behavior do not necessarily have the same origin.

A human can speak through a cloned voice. A synthetic agent can use a voice that sounds completely human. A human voice can even be controlled by synthetic conversational behavior.

Suddenly, human vs. synthetic was no longer a simple binary classification problem.

Breaking Our Own Model

When we tested our models on these new scenarios, they broke.

Models that performed extremely well on the original dataset dropped significantly on data generated under different conditions. In our experiments, performance did not consistently exceed 80%.

Initially, this looked like a failure.

It turned out to be one of the most valuable results of the project.

It showed us that benchmark performance is not the same as generalization. A model that performs perfectly on familiar data is not necessarily a detector that can be trusted in the real world.

So instead of hiding this weakness, we designed the system around it.

From Classification to Evidence

Our approach evolved from asking:

“Which model has the highest accuracy?”

to asking:

“What evidence do we have that this interaction is suspicious?”

Our final system combines multiple sources of evidence.

Acoustic Evidence

We analyze characteristics of the audio signal, including spectral information and speech patterns. We extract MFCC-based acoustic features and other temporal properties from the conversation.

Behavioral Evidence

We found conversational timing to be particularly interesting.

For every agent-to-caller response, we measure the latency

$$ \lambda = t_{\text{caller,start}} - t_{\text{agent,end}} $$

This allows us to capture response delays, interruptions, silence, and other aspects of conversational dynamics.

Rather than only asking how a voice sounds, we ask:

Does the interaction behave like a human conversation?

Our behavioral features include latency statistics, speech duration, silence, overlap, barge-ins, and conversational timing patterns.

Probabilistic Fusion

We process acoustic and behavioral features through separate machine learning branches and combine their outputs into a final probability:

$$ P(\text{synthetic}\mid x) $$

This gives us a system that does not depend on a single “magic” feature. Instead, it combines multiple imperfect signals.

What We Learned

The biggest lesson was not a specific model or feature.

It was that detecting synthetic voices is fundamentally a generalization problem.

As generative voice technology becomes better, obvious acoustic artifacts become increasingly difficult to rely on. A synthetic voice can sound human, but reproducing every aspect of human interaction is a much harder problem.

At the same time, we learned that even behavioral signals are not inherently impossible to imitate. Pauses, hesitation, filler words, and other characteristics that initially seemed uniquely human can be reproduced by increasingly capable systems.

That means the goal cannot be to find one permanent characteristic that separates humans from machines.

Instead, we need systems that: