Sexy Bai Cai

Inspiration

Our inspiration came from the growing use of AI-generated media in scams and misinformation. In Singapore, deepfake videos impersonating public figures such as Lawrence Wong have been used in scam campaigns, demonstrating how convincing synthetic media can be weaponized to deceive people. This showed us that AI-generated content is no longer just a technical or research problem — it has real implications for cybersecurity and public trust. We wanted to explore whether forensic signals hidden within AI-generated images could be used to build a more reliable way of identifying synthetic content and reducing the risk of deception.

What it does

My team is developing a deep learning-based model that detects whether an image is AI-generated or real. The main challenge is that modern generative models can produce highly realistic images, making it increasingly difficult to distinguish synthetic images from real ones based purely on visual appearance. More importantly, a detector can learn the wrong features during training. Instead of learning general characteristics of AI-generated images, it may memorize artifacts associated with a particular generator, dataset, or image quality. Our solution therefore focuses not only on achieving high accuracy on clean test data, but on building a detector that can generalize across different generators, identify subtle forensic traces, remain reliable under real-world image transformations, and avoid being biased toward particular classes or sources.

Firstly, our system addresses generator-specific overfitting through multi-source training. A detector trained primarily on one generator may learn the specific visual fingerprint of that generator rather than learning general characteristics of synthetic images. By exposing the model to multiple synthetic sources and generators, our system encourages it to identify patterns that are shared across different types of AI-generated images. In simple terms, we want the model to learn “what makes an image AI-generated” rather than “what makes an image generated by one particular AI model.” This improves the model’s ability to generalize across different generators, including generators that may not be strongly represented in the training data.

Secondly, our system uses hybrid forensic feature extraction to look beyond the overall appearance of an image. Conventional image-classification models are good at understanding high-level information such as objects, shapes, textures, composition, and other semantic features. However, AI-generated images can look highly realistic while still containing subtle pixel-level inconsistencies. Our hybrid architectures therefore combine high-level semantic information with localized forensic features, allowing the model to examine both the overall image and fine-grained patterns such as unusual textures, residual signals, high-frequency details, and subtle boundaries. In simple terms, the model does not only ask “Does this image look realistic?”; it also looks more closely at “Are there tiny patterns that suggest this image was synthetically generated?”

Thirdly, we focus on robustness to real-world image transformations. Images encountered online may be compressed, resized, blurred, or affected by noise, and these transformations can alter or remove the subtle artifacts that a detector relies on. Our solution therefore incorporates distortion-based augmentation and robustness testing so that we can both improve and measure how well the detector performs under degraded image conditions. By exposing the model to different types of corrupted images during training, we encourage it to rely less on fragile artifacts and more on stable forensic features. We then stress-test the model under different corruption types and severity levels to identify how much its performance changes when image quality decreases.

Fourthly, we evaluate cross-domain generalization through external validation. A model can achieve high accuracy on its own test set while still failing on completely different data if it has learned dataset-specific or generator-specific characteristics. To address this, we evaluate the detector on an independent external benchmark representing an unseen data distribution. This provides an out-of-distribution evaluation that asks whether the model can still distinguish real and AI-generated images when it encounters data outside the environment in which it was trained. Strong performance on this external benchmark provides stronger evidence that the model has learned generalizable forensic representations rather than simply memorizing the training data.

Finally, our training strategy is designed to provide balanced and stable learning. We use class-balanced and source-balanced sampling so that neither the real or synthetic class nor any individual generator has a disproportionate influence on the learning process. This is particularly important in a multi-source dataset, where one generator may contribute significantly more images than others. By giving different classes and sources a more balanced representation during training, we reduce the risk that the model becomes overly specialized to a particular source. Combined with staged and iterative optimization, this allows us to progressively refine the model and address weaknesses identified during evaluation. Overall, our solution goes beyond building a classifier that simply recognizes what the training data’s fake images look like. We aim to build a detector that learns more general evidence of synthetic generation: generalizable across generators, sensitive to subtle forensic traces, robust to real-world image transformations, and capable of transferring to previously unseen data distributions.

How we built it

We built the solution as a deep learning pipeline for binary image classification, where the model learns to distinguish between real images and AI-generated images. The development process consists of several stages, including dataset preparation, balanced multi-source training, hybrid model development, robustness training and testing, and external validation. We used PyTorch and Torchvision as the main deep learning frameworks, with Python-based data processing and evaluation pipelines. Rather than developing a single model and evaluating it only on a clean test set, we iteratively developed and evaluated different architectures while deliberately introducing challenges such as multiple image generators, imbalanced data sources, and real-world image distortions.

We first built the training dataset by combining real and synthetic images from multiple sources. One of our initial benchmarks is CIFAKE, which contains real images derived from CIFAR-10 together with AI-generated counterparts produced using Stable Diffusion. While this provides a useful benchmark for synthetic-image detection, training exclusively on one generator can cause the model to learn generator-specific artifacts rather than general synthetic characteristics. To address this, we extended the training data using supplementary multi-source synthetic datasets and collections from different generative pipelines. The data ingestion process is handled through components such as multisource_dataset.py and source_balanced.py, which allow us to organize and sample data from different sources in a controlled way. This gives the model exposure to a wider variety of synthetic images and reduces the likelihood that it will simply memorize the visual fingerprint of one particular generator.

We also paid particular attention to how the training data is sampled. Having multiple sources is not sufficient if one source or class is significantly larger than the others. For example, if one generator contributes most of the synthetic images, the model may receive far more training signals from that generator and become overly specialized to its characteristics. Similarly, if the dataset contains many more real images than synthetic images, the model may become biased toward the majority class. To reduce these effects, we implemented class-balanced and source-balanced sampling. During training, the batches are constructed so that different classes and synthetic sources have a more balanced representation. This is important because each training batch contributes gradients that are used to update the model's parameters. If one class or generator dominates the batches, it will also have a disproportionately large influence on these parameter updates. By balancing the training data, we make the optimization process less dominated by any particular source and encourage the model to learn features that are useful across different types of images.

After establishing the data pipeline, we developed and compared several model architectures. We first implemented standard CNN-based image classification backbones as baseline models in model.py. These baseline models provide a reference point for determining whether the additional forensic components in our custom architectures actually improve performance. We then developed a series of hybrid architectures, including HybridModel, HybridV2Model, HybridV3Model, and HybridV31Model. These models were developed iteratively, with each version experimenting with different approaches to feature extraction and feature fusion. Instead of relying entirely on a conventional image-classification backbone, the hybrid architectures combine high-level spatial and semantic representations with more localized residual and frequency-related information. The purpose is to allow the model to understand both the overall content of an image and the subtle low-level patterns that may reveal synthetic generation.

The intuition behind this architecture is that AI-generated images can look highly realistic at a semantic level while still containing small inconsistencies at the pixel or texture level. A conventional backbone might recognize that an image contains a realistic face, object, or scene, but this does not necessarily tell us whether the image was generated by an AI system. Our forensic feature extraction therefore focuses on more localized information, such as unusual textures, residual patterns, high-frequency details, and subtle boundaries that may be difficult for humans to notice. The hybrid architecture effectively gives the model two complementary perspectives: one that asks what the image looks like overall, and another that examines the image more closely for potential traces of synthetic generation. The different model versions allow us to experiment with how these features should be combined and whether more advanced feature fusion or attention mechanisms can improve fine-grained detection.

We then incorporated robustness into the development and evaluation pipeline because real-world images are rarely preserved in exactly the same form as their original files. Images shared through social media or messaging platforms can undergo JPEG compression, resizing, blurring, noise, and other forms of quality degradation. These transformations can alter the low-level artifacts that a detector may rely on, meaning that a model performing well on clean images may experience a significant performance drop when the same images are degraded. To simulate these conditions, we developed an automated distortion pipeline using components such as distort_dataset.py and robustness.py. These tools generate degraded versions of images under different corruption types and severity levels, allowing us to systematically measure how the detector's performance changes as image quality decreases.

Importantly, robustness testing and robustness improvement are treated as two related but separate processes. The robustness testing pipeline allows us to measure how much the model's performance drops under different distortions, while distortion-based data augmentation can be incorporated into training to improve the model's resilience. For example, we can expose the model to training images that have been randomly compressed, resized, blurred, or affected by noise. This encourages the model to rely less on fragile artifacts that disappear after a transformation and more on stable synthetic patterns that remain informative even when image quality changes. We can then stress-test the resulting model again to determine whether its performance under these corruptions has improved. This creates an iterative feedback loop where robustness evaluation can identify weaknesses that guide further training and model development.

Once the models have been trained and evaluated under controlled and corrupted conditions, we perform external validation to determine whether their learned representations transfer beyond the original training distribution. A model can achieve very high accuracy on an internal test set while still failing on images from a different dataset or generation pipeline. This is especially concerning for synthetic-image detection because a model may memorize dataset-specific or generator-specific artifacts. To test whether our model has learned more general forensic features, we use an independent external validation suite through bytedance_validation.py. This provides an out-of-distribution evaluation in which the model is exposed to an unseen data distribution. In simple terms, instead of only asking whether the model can recognize images similar to those it trained on, we take it outside of its familiar environment and ask whether it can still distinguish real from AI-generated images. Strong performance on this external benchmark provides stronger evidence that the model's learned representations can transfer across domains rather than simply memorizing the characteristics of the training data.

Throughout the development process, we also use staged and iterative optimization rather than treating model development as a single training run. The training pipeline in train.py, together with the staged-training tests, allows us to organize the optimization process into controlled stages and evaluate how different training strategies affect convergence and performance. At the same time, the progression from HybridModel to HybridV2Model, HybridV3Model, and HybridV31Model represents an iterative architecture-development process. We train a version, evaluate its performance, identify weaknesses, modify the architecture or training strategy, and then test the next version. This allows the model to be improved based on observed failure cases rather than relying solely on assumptions about which architecture should work best.

To support the development workflow, we use VS Code and Cursor for local development, scripting, and test-suite management, while Jupyter and Google Colab provide interactive environments for rapid experimentation and GPU-based model training. The project uses Python 3.12 together with package-management tools such as Astral uv, using pyproject.toml and a deterministic uv.lock environment to keep dependencies consistent. We also use pip and requirements.txt where appropriate. For quality assurance, we use pytest to test important components of the pipeline, including model forward passes, dataset integrity, multi-source command-line interfaces, and data-staging functionality. This helps ensure that changes to the training pipeline do not unintentionally break other parts of the system.

The main deep learning components are implemented using PyTorch and Torchvision. PyTorch provides the neural network modules, training loops, optimization, loss functions, and tensor operations, while Torchvision provides computer-vision models and transformation utilities. Pillow and OpenCV are used for image loading, preprocessing, augmentation, and distortion generation, while NumPy supports numerical operations and pandas is used for metadata and dataset-related data handling. For evaluation, we use scikit-learn to calculate metrics such as accuracy, precision, recall, ROC-AUC, and confusion matrices. The results of robustness experiments and other evaluations are also recorded in structured outputs such as results/robustness.json, allowing us to compare model performance across different experimental conditions.

Overall, we built the system as more than a conventional image classifier. The pipeline begins with diverse real and synthetic datasets, uses multi-source and balanced sampling to reduce generator and class bias, and then feeds the data into both standard baseline models and progressively developed hybrid forensic architectures. The models are trained and optimized through iterative and staged processes, while robustness experiments expose them to realistic image corruptions such as compression, resizing, blur, and noise. Finally, external out-of-distribution validation tests whether the features learned by the model can transfer to an unseen synthetic distribution. This development process allows us to evaluate the detector from multiple perspectives: whether it can distinguish real from synthetic images, whether it can identify subtle forensic traces, whether it can generalize across generators, whether it can withstand real-world image degradation, and whether its learned representations remain useful when it encounters data outside its original training distribution.

Challenges we ran into

One of the biggest challenges we encountered was generalization. A model can achieve strong performance on a familiar dataset while still failing when the source of the images changes. This was particularly important for our task because AI-generated images from different generators can have very different characteristics. We found that simply increasing the model’s complexity does not necessarily solve this problem; in some cases, a more powerful model can actually become better at memorizing dataset- or generator-specific artifacts. This motivated us to look more carefully at the composition of our training data, introduce multiple synthetic sources, and use source-balanced sampling rather than relying on a single dominant distribution.

Another challenge was distinguishing genuine forensic signals from dataset-specific shortcuts. Some visual patterns that appear predictive during training may not actually be evidence of AI generation. For example, differences in image resolution, compression, preprocessing, or the way images are stored can unintentionally become signals that the model uses for classification. This creates a shortcut-learning problem: the model may obtain good accuracy without learning the forensic features we actually care about. We therefore had to consider not only whether the model was accurate, but also whether the features it relied on were likely to remain useful when the data source changed.

We also encountered challenges with model architecture selection. There is no guarantee that a more complicated architecture will outperform a simpler CNN. Adding additional feature-extraction or fusion components can introduce more parameters and make optimization more difficult, while some features that appear useful theoretically may not provide meaningful improvements in practice. This led us to develop multiple hybrid variants rather than assuming that the first architecture would be optimal. Comparing different versions allowed us to identify whether improvements came from genuinely useful forensic representations or simply from increasing model complexity.

A further challenge was robustness under image degradation. The low-level artifacts that are useful for detecting synthetic images can also be among the first features affected by compression, resizing, blur, and noise. This creates a trade-off: the model needs to be sensitive enough to detect subtle forensic traces, but not so dependent on fragile pixel-level artifacts that its predictions collapse after an image is compressed. We therefore had to evaluate performance under different distortion levels and consider distortion-based augmentation as part of training. This made robustness a much more difficult problem than simply maximizing accuracy on clean images.

We also had to deal with class and source imbalance during optimization. Even when the overall number of real and synthetic images appears balanced, the synthetic class may contain very uneven contributions from different generators. This means that a model can still become highly specialized to whichever generator contributes the most images. Since the model learns through repeated parameter updates from its training batches, these imbalances can influence the direction of optimization. Addressing this required us to think about balance at multiple levels rather than only checking whether the total number of real and fake images was equal.

Another challenge was choosing an appropriate decision threshold. The model does not directly output “real” or “fake”; it produces a probability or confidence score. Converting that score into a final binary prediction requires selecting a threshold. We observed that the default threshold could make the detector somewhat more likely to classify an image as fake than real. This is important because accuracy alone does not reveal whether the model is systematically favoring one class. We therefore had to examine metrics such as precision, recall, the confusion matrix, and the distribution of prediction probabilities rather than treating the default 0.5 threshold as automatically optimal. This highlighted the importance of calibrating the decision boundary according to the intended use of the detector.

Finally, evaluating whether an improvement was genuinely meaningful was itself a challenge. When we changed the architecture, training strategy, sampling method, or augmentation, an improvement on one test set did not necessarily mean that the model had become better overall. A change could improve clean-set accuracy while hurting robustness, or improve performance on one generator while reducing performance on another. This meant that we could not rely on a single metric or a single benchmark to decide which version was best. We instead had to evaluate models from multiple perspectives, including clean performance, class-level behavior, robustness under distortions, and performance on external unseen data.

Overall, one of our main challenges was realizing that AI-generated image detection is not simply an accuracy-maximization problem. A model can achieve a high score while relying on shortcuts, generator-specific fingerprints, or fragile image artifacts. The difficult part was therefore finding a balance between accuracy, generalization, robustness, and reliable decision-making. These challenges shaped our iterative development process and motivated the use of multi-source training, balanced sampling, hybrid architectures, distortion-based augmentation, robustness testing, threshold analysis, and external validation.

Accomplishments that we're proud of

One accomplishment we are particularly proud of is the way we iteratively improved both the scale of our training and the way the model was optimized. Rather than settling on our initial configuration, we experimented with different dataset sizes, training durations, architectures, and fine-tuning strategies. This allowed us to progressively refine the system and understand how different choices affected both its performance and its ability to generalize.

One of the first major changes we made was moving from full fine-tuning to staged partial fine-tuning. In our initial experiments, we allowed almost the entire pretrained backbone to be updated from the beginning. This gave the model considerable freedom to adapt, but it also increased the risk of the model adapting too strongly to the characteristics of our training data. We therefore changed to a more controlled fine-tuning strategy for our subsequent experiments.

With staged partial fine-tuning, we first freeze the pretrained backbone and train the newly added classification components. We then selectively unfreeze deeper backbone blocks while keeping earlier layers frozen, allowing the model to gradually adapt its higher-level representations while preserving more general visual features from pretraining. We also use different learning rates for the classifier and backbone, with a smaller learning rate for the pretrained layers. This approach became part of our training methodology for both the 25,000- and 50,000-sample configurations, rather than being a change made only when moving from 25k to 50k.

We are also proud of our systematic increase in training data scale. We experimented with 25,000, 50,000, and eventually 100,000 samples, allowing us to investigate how additional training data affected the detector. The 50,000-sample configuration represented an important milestone because it substantially increased the amount of information available to the model while remaining practical for iterative experimentation. We also trained the model for 7 epochs, giving it more opportunities to learn from the expanded dataset. Rather than assuming that more data or more epochs would automatically improve the detector, we used these experiments to observe how training scale affected both internal performance and generalization.

Another accomplishment we are proud of is the progressive development of our model architectures. We did not treat the first model we built as the final solution. Instead, we developed multiple hybrid variants, moving from the initial HybridModel through Hybrid V2, V3, and V3.1. Each iteration allowed us to experiment with different approaches to feature extraction and fusion while using the results of previous experiments to guide subsequent development. This iterative process helped us move from a baseline image classifier toward a model specifically designed for fine-grained synthetic-image detection.

The strongest evidence of our progress came from our evaluation results. Our baseline achieved 82.85% internal validation accuracy and 66.49% accuracy on the external OOD benchmark, while our newer Hybrid V3.1 configuration achieved 90.65% internal validation accuracy and 83.61% OOD accuracy. This represents an improvement of 7.80 percentage points on internal validation and 17.12 percentage points on the external benchmark. We are particularly proud of the OOD improvement because it shows that our progress was not limited to performing better on familiar validation data. The model also became substantially more effective when evaluated on a different distribution.

We also observed a significant reduction in classification errors. False negatives decreased from 3,304 to 853, meaning that the number of synthetic images incorrectly classified as real was reduced by 74.2%. At the same time, false positives decreased from 1,334 to 838, representing a 37.2% reduction in genuine images incorrectly classified as synthetic. Improving both types of errors was important to us because a useful detection system needs to identify synthetic content without simply becoming overly aggressive and flagging genuine content.

Overall, what we are most proud of is not any single experiment, but the progression of the project as a whole. We started by exploring a conventional full fine-tuning approach, identified limitations in how the model was adapting, and subsequently adopted staged partial fine-tuning as part of our training methodology. We then systematically experimented with increasing the training scale from 25k to 50k and 100k samples, trained for up to 7 epochs, and iteratively developed our hybrid architectures. These changes were evaluated not only on internal validation data but also on an external OOD benchmark. The resulting improvement from 66.49% to 83.61% OOD accuracy, together with substantial reductions in both false negatives and false positives, represents one of the clearest demonstrations of how our iterative engineering process improved the system.

What we learned

One of the biggest things we learned is that high accuracy does not necessarily mean a model has truly learned the problem. In AI-generated image detection, a model can perform extremely well on a familiar dataset while relying on shortcuts or artifacts that do not generalize to other generators or datasets. This taught us to look beyond a single accuracy number and ask whether the model is learning meaningful forensic features that remain useful when the data distribution changes. As a result, generalization became just as important to us as raw classification performance.

We also learned that the diversity and composition of the training data can be just as important as the architecture of the model. Initially, it is tempting to think that using a more powerful neural network will automatically produce a better detector. However, if the training data is dominated by one generator or one type of image, increasing model capacity may simply allow the model to memorize those characteristics more effectively. Working with multiple sources and balancing them showed us that giving the model diverse examples is critical for encouraging it to learn more general patterns.

Another important lesson was that the model can learn signals that we did not intend it to learn. Image datasets contain many characteristics beyond the actual content of the image, including resolution, compression, preprocessing, and source-specific properties. If these characteristics correlate with the labels, the model may use them as shortcuts. This taught us to think carefully about dataset construction and experimental design, rather than assuming that every feature the model uses is a meaningful forensic feature. In other words, a model finding a correlation does not necessarily mean that it has learned the underlying concept we wanted it to learn.

We also learned that fine-grained forensic detection requires a different perspective from ordinary image classification. A real image and an AI-generated image may look almost identical at a semantic level, so simply understanding what is depicted in the image is not always sufficient. Small texture inconsistencies, residual patterns, high-frequency signals, and other subtle details can contain useful information. This motivated our exploration of hybrid architectures that combine high-level semantic representations with localized forensic features. Through this process, we learned that effective synthetic-image detection requires the model to examine both the “big picture” and the tiny details.

Another key lesson was that robustness cannot be assumed from clean test-set performance. A detector that works well on an original high-quality image may behave very differently after JPEG compression, resizing, blur, or noise. This showed us that robustness needs to be treated as a separate dimension of model performance. We learned to distinguish between simply measuring robustness and actively improving it through techniques such as distortion-based data augmentation. This changed the way we evaluated our models: instead of asking only “How accurate is it?”, we also ask, “How much does its performance change when the conditions become less ideal?”

We also learned that the default probability threshold is not necessarily the best decision boundary. The model produces a probability or confidence score, but converting that score into a final real-or-fake prediction requires a threshold. Our experiments showed that the model could have a tendency to favor one class depending on the threshold and the underlying probability distribution. This taught us that model evaluation should include precision, recall, confusion matrices, and threshold analysis rather than relying exclusively on accuracy at a fixed threshold. The threshold should ultimately reflect the trade-off between false positives and false negatives that matters for the intended application.

Perhaps most importantly, we learned that external validation is essential for understanding whether a model actually generalizes. A model can appear successful when evaluated on data that closely resembles its training distribution, but its behavior can change significantly when it encounters an unseen generator or dataset. Testing on an external out-of-distribution benchmark gave us a more realistic indication of whether the model had learned transferable representations. This reinforced the idea that a detector should be evaluated not only on how well it performs in the environment where it was developed, but also on how well it handles situations it has never encountered before.

Finally, we learned that model development is an iterative process rather than a straight path toward a single “best” architecture. Each experiment can reveal a different weakness: one model may perform better on clean data, another may be more robust to distortions, while another may generalize better to an unseen source. Instead of assuming that the most complicated model is automatically the best, we learned to use experiments and failure cases to guide subsequent changes. This created a cycle of training, evaluation, analysis, and refinement, where every experiment provided information that helped us make the next version of the system more reliable.

Overall, the most important lesson we took away is that building a reliable AI-generated image detector is fundamentally a generalization problem, not just a classification problem. We learned that diverse data, careful sampling, appropriate architecture design, robustness testing, threshold analysis, and external validation all matter when trying to build a detector that works beyond a controlled benchmark. More broadly, the project taught us to be more critical of seemingly strong model results: rather than asking only whether a model performs well, we need to ask why it performs well, what features it is relying on, and whether those features will still work when the data changes.

What's next for Sexy Bai Cai

Moving forward, we see several potential applications for this technology, particularly in cybersecurity and combating digital misrepresentation. As generative AI becomes increasingly capable of producing realistic images and videos, synthetic media could be used for impersonation, social engineering, scams, misinformation, and other forms of digital deception. A more generalizable and robust detection system could potentially serve as an additional layer of defense by flagging suspicious synthetic content before it is used to deceive users.

Our next step would be to extend the system beyond controlled image benchmarks and evaluate it on a broader range of real-world content, including images from different platforms, generators, compression pipelines, and emerging generative models. We could also explore extending the approach from image detection to video and multimodal deepfake detection, where both visual and temporal inconsistencies can be analyzed. Another important direction would be improving calibration and interpretability, so that the system does not simply output “real” or “fake,” but can provide a confidence score or forensic evidence explaining why content was flagged.

Ultimately, we envision this technology as part of a broader cybersecurity and digital-authenticity pipeline rather than as a standalone solution. The goal is not to assume that every AI-generated image is malicious, but to provide tools that help people and organizations identify potentially manipulated or synthetic content and make more informed decisions about what they see online.