Project FraudProofed

Inspiration

Context

In today’s financial landscape, investing and securities trading have become central to individual wealth accumulation. As participation in capital markets expands globally, the consequences of securities fraud have grown more severe for retail investors, employees, and institutions alike. Compounding this challenge is the fact that securities fraud takes many forms – from insider trading to financial misreporting and pump-and-dump schemes – making single-metric detection approaches increasingly inadequate.

The 2020 accounting scandal at Luckin Coffee illustrates the scale and persistence of this problem. After the company admitted to fabricating approximately USD 310 million in sales, its share price plunged by more than 80% and it was ultimately delisted from U.S. exchanges. The episode inflicted substantial financial losses on retail investors and underscored a critical weakness in existing monitoring systems: traditional financial indicators often detect fraud only after significant damage has already occurred.

Our Solution

FraudProofed addresses this gap through a preventive, early-warning framework that leverages publicly available earnings call data. Building on prior research demonstrating links between managerial language, vocal stress patterns, and financial misconduct, the system performs multimodal analysis of transcript semantics and executive vocal characteristics. Through evaluating tone, inflection, and linguistic features from quarterly earnings calls, FraudProofed generates a forward-looking fraud risk signal for the subsequent two quarters, trained on historical earnings communications matched with documented fraud events.

By moving beyond purely numerical screening toward behavioral and communication-based signals, FraudProofed enhances risk awareness and equips investors, auditors, and regulators with earlier, more actionable insights into potential corporate misconduct.

What it does

Target Audience

The target audience for FraudProofed can be anyone, as the only inputs required are a text/audio file of a company’s earnings call – data that is usually publicly available. Our product thus democratizes risk assessment and knowledge so that everyone, regardless of investing experience or institution size, can make more informed investment and trading decisions.

The types of users for our products can include:

Individual investors looking to reduce overall risk in their portfolios
Asset managers looking to enhance their due diligence efforts when constructing portfolios or investment products
Regulators and auditors looking for additional information sources to identify and investigate fraud incidents

How to use FraudProofed

Our software takes the following as inputs:

Company's quarterly report in PDF (e.g. Apple's Q1 2026 report)
Audio file (In mp3/csv) and a transcript (txt) of the company's earnings call (Apple's Q1 2026 Earnings Call)

Based on this, the following will be generated:

Probabilities of a fraud taking place in the next 4, 8, 16 and 20 quarters
Performance metrics of the model being used.

How we built it

Data sources and Models

We used data from the Multi-Modal Aligned Earnings Conference (MAEC) call dataset, containing 3,443 earnings calls from 578 companies spanning 2015–2018. Our labels of fraud come from SEC enforcement actions (AAERs). If the SEC later caught a company committing fraud during a period, we label calls from that period as fraudulent.
The training data was split into 3 separate groups for training, validation and testing.
The model we used to analyze text and audio is LightGBM.

Feature Engineering

7 summary statistics are computed across all sentences: Mean, standard deviation, maximum, minimum, first, last, slope.

For earnings call transcripts, each text file is already converted to sentence-level embeddings with 384 dimensions to be evaluated.
For audio files, 29 acoustic features are evaluated in each sentence, including: Pitch, intensity, voice quality, fluency, timing.
For quarterly report PDFs, text from the Management Discussion & Analysis is extracted to do similar text-level analysis with 384 dimensions.

Late Fusion

The three models for transcript, audio and report are kept separate, their probability outputs only combined at the end. Bootstrap stability selection was used to select the best relative weights of each model. We engaged in 2-way fusion (combining transcript and corresponding audio file), and 3-way fusion (all 3 models).

Challenges

Dealing with highly imbalanced data: Less than 5% of data were true cases of fraud, causing us to search for the most appropriate classification models.
Multimodal alignment: It was challenging to merge sentence-level text embeddings, acoustic features, as well as filing-level MDNA embeddings where each has to align to the same 3443 row index.