About the Project
Motivation
The SOTA-AI Monthly Challenge is explicitly designed to reward versatility, not specialization. Unlike typical hackathons that allow narrow optimization, this challenge required solving three fundamentally different problems—Classical Machine Learning, Natural Language Processing, and Computer Vision—under a single evaluation framework.
What motivated this project was the idea that strong AI engineering is not about excelling in one modality, but about adapting reasoning across data types, constraints, and evaluation criteria.
What I Learned
This project reinforced several core lessons:
- Strong performance often comes from understanding failure modes, not chasing leaderboard metrics.
- Small, high-quality supervision signals can outperform large noisy datasets.
- Semantic evaluation requires fundamentally different strategies than surface-level optimization.
- Engineering under constrained resources forces better architectural decisions.
Perhaps most importantly, I learned to treat debugging, evaluation, and iteration as first-class components of model design.
How the Project Was Built
The solution was developed as a phase-wise pipeline, tailored to each task:
Task 1 — Classical Machine Learning
A tabular dataset with unstable patterns and distribution shifts required extensive exploratory analysis, feature reasoning, and careful validation. Given the compressed leaderboard, the emphasis was on signal extraction and robustness rather than overfitting.
Task 2 — Computer Vision
A detection model was trained on clear-weather images and evaluated on fog-heavy scenes. This required aggressive data augmentation, careful anchor handling, and architectural choices that prioritized generalization under domain shift.
Task 3 — Natural Language Processing
The NLP task was an inverse question–answering problem: given an answer (A), infer the most plausible question (Q) such that:
$$ Q \rightarrow A $$
Since no ground-truth questions were provided, a synthetic supervision strategy was employed. Candidate questions were generated and filtered using cycle consistency, approximated via cosine similarity between the original answer and a regenerated answer:
$$ \text{score}(A, Q) = \cos(\text{embed}(A), \text{embed}(A')) $$
This enabled:
- parameter-efficient fine-tuning using QLoRA,
- deterministic test-time generation,
- and a final tail-focused semantic repair phase that improved robustness without regression.
Challenges Faced
- Extremely noisy or adversarial data in the Classical ML task.
- Severe domain shift in Computer Vision due to fog conditions.
- Ill-posed supervision and semantic evaluation in NLP.
- Limited compute, requiring careful batching and monotonic improvement strategies.
Each challenge required different reasoning, reinforcing the importance of adaptability over brute force.
Final Outcome
The final system demonstrates that careful engineering, clear reasoning, and disciplined iteration can produce competitive, explainable results across all three AI domains—exactly the skill set this challenge aims to identify.
Log in or sign up for Devpost to join the conversation.