About TsukyNLP - NeuroLogic '26 Project

💡 Inspiration

The spread of misinformation has become one of the most pressing challenges of our digital age. From disaster response being delayed by false tweets to public trust eroding due to fake news, we saw firsthand how language can be weaponized. When we discovered NeuroLogic '26, we were inspired by the opportunity to build AI systems that could help combat these problems at scale.

What really drove us was the realization that these aren't just technical challenges—they're human challenges. Behind every toxic comment is a person who might be hurt. Behind every fake news article is someone who might make a wrong decision. Behind every disaster tweet is a community that needs help. We wanted to build something that matters.

📚 What We Learned

This hackathon was an incredible learning journey for our team:

Technical Skills

• Multilingual NLP: Working with English and Hindi text taught us about cross-lingual transfer learning and the challenges of handling multiple scripts and cultural contexts
• Model Selection: We learned that bigger isn't always better—DeBERTa-v3-small gave us 89% F1 while being 4x faster than the base model
• Threshold Optimization: Discovered that the default 0.5 threshold isn't optimal—our disaster classifier performed best at 0.53
• Feature Engineering: Combining multiple text features (source, type, content) with special tokens significantly improved performance

Practical Insights

• Time Management: With only 12 hours, we had to make smart tradeoffs between model complexity and training time
• Cross-Validation: 3-fold CV gave us confidence in our models' generalization without taking too long
• Early Stopping: Patience=1 saved us hours while maintaining performance

Team Collaboration

• Division of Labor: Each team member tackled one challenge, allowing us to work in parallel
• Code Reusability: We built a shared template that all three challenges could adapt
• Documentation: Writing clear READMEs helped us stay organized and made our work reproducible

🔨 How We Built Our Project

Challenge 1: Disaster Tweet Classification

Goal: Identify informative disaster-related tweets (F1-Score: 0.8966)

Approach:

1. Data Exploration: Analyzed 9,000 tweets, found balanced classes (50/50 split)
2. Feature Engineering: Combined Tweet Text + Information Source + Information Type using special tokens [SRC], [TYPE], [TWEET]
3. Model: Fine-tuned DeBERTa-v3-small with 3-fold stratified CV
4. Optimization: Layer-wise learning rate decay (LLRD) with backbone at 0.8× and head at 5.0× base LR
5. Threshold Tuning: Found optimal threshold of 0.53 through OOF predictions

Key Innovation: The structured input format helped the model understand feature roles, improving classification accuracy.

Challenge 2: Fake News Detection

Goal: Detect fake news articles (Accuracy: 99.96%)

Approach:

1. Data Processing: Combined title and article text with [SEP] token
2. Model: DistilBERT-base-uncased (40% smaller, 60% faster than BERT)
3. Training: 3 epochs with 85/15 train/val split, FP16 mixed precision
4. Validation: Achieved near-perfect metrics across both classes

Key Innovation: Truncating text to 512 characters before tokenization balanced context and speed.

Challenge 3: Multilingual Toxic Comment Classification

Goal: Detect toxic comments in English and Hindi (ROC-AUC: 0.9879)

Approach:

1. Data Analysis: 9,000 samples with balanced toxic/non-toxic distribution
2. Model: XLM-RoBERTa-base (pre-trained on 100 languages)
3. Training: 3 epochs with stratified split, max_length=128 tokens
4. Metrics: Tracked ROC-AUC, accuracy, F1, precision, and recall

Key Innovation: Using a multilingual model eliminated the need for separate models per language.

Technical Stack

• Framework: Hugging Face Transformers + PyTorch
• Platform: Google Colab with T4 GPU
• Training: Mixed precision (FP16), gradient accumulation, early stopping
• Validation: Stratified K-fold cross-validation for robust evaluation

🚧 Challenges We Faced

1. Time Constraints

Problem: 12 hours to build three complete NLP systems

Solution:

• Chose smaller, faster models (DeBERTa-small, DistilBERT)
• Reduced folds from 5 to 3
• Used early stopping to avoid unnecessary epochs
• Parallelized work across team members

2. Multilingual Complexity

Problem: Handling both English and Hindi in the toxic comment classifier

Solution:

• Selected XLM-RoBERTa, specifically designed for multilingual tasks
• Kept preprocessing minimal to preserve language-specific features
• Used Unicode normalization to handle different scripts

3. Class Imbalance Concerns

Problem: Worried about potential bias in predictions

Solution:

• Used stratified splits to maintain class distribution
• Monitored per-class metrics (precision, recall) not just overall accuracy
• Applied threshold optimization on OOF predictions

4. Reproducibility

Problem: Ensuring others could run our code and get the same results

Solution:

• Set random seeds everywhere (Python, NumPy, PyTorch)
• Created detailed requirements.txt files
• Wrote step-by-step reproduction instructions
• Included all hyperparameters in config dictionaries

5. Model Selection Trade-offs

Problem: Balancing performance vs. training time

Initial Attempt: Started with DeBERTa-v3-base (183M parameters)

• Training time: ~3 hours per challenge
• Would have missed the deadline

Final Solution: Switched to smaller models

• DeBERTa-v3-small (44M parameters): 4× faster, only 1% accuracy drop
• DistilBERT: 60% faster than BERT, 97% of performance retained
• Total training time: ~1 hour for all three challenges

6. Threshold Optimization

Problem: Default 0.5 threshold wasn't optimal for disaster classification

Solution:

• Implemented threshold search from 0.30 to 0.70
• Used out-of-fold predictions to avoid overfitting
• Found optimal threshold of 0.53, improving F1 by 0.4%

7. Feature Engineering

Problem: How to effectively combine multiple text features?

Experiments:

• Concatenation: source + type + tweet → Okay results
• Special tokens: [SRC] source [TYPE] type [TWEET] tweet → Best results

Learning: Structured input helps transformers understand feature roles.

🎯 What's Next

If we had more time, we'd explore:

1. Ensemble Methods: Combine predictions from multiple models for higher accuracy
2. Data Augmentation: Back-translation, paraphrasing, and synonym replacement
3. Hyperparameter Tuning: Use Optuna for systematic optimization
4. Error Analysis: Deep dive into misclassified examples to find patterns
5. Deployment: Build a web API for real-time predictions
6. Explainability: Add LIME or SHAP to explain model decisions

🏆 Final Thoughts

This hackathon pushed us to our limits and taught us that great AI isn't just about the biggest models—it's about smart engineering, efficient training, and understanding the problem deeply. We're proud of what we built in 12 hours and excited to see how these models perform in the final evaluation.

Most importantly, we learned that NLP can be a force for good. Whether it's helping first responders find real disaster information, protecting people from fake news, or making online spaces safer, language AI has the power to make a real difference.

---

Team TsukyNLP

• Dachatorn Chitnatharn
• Chakorn Silalai
  
• AlizionS

"Building the future of language AI, one challenge at a time."

Built With

• google-colab
• numpy
• pandas
• python
• scikit-learn