VERDANT

Prototype

Inspiration- The endless cycle of tutorials that stopped at “train a model” left me craving a complete, reproducible end‑to‑end pipeline. I wanted a single repository where anyone could clone, run a notebook, train, and instantly serve a model with minimal friction. Proving that a tiny Transformer (≈ 2 M parameters) can still hit > 90 % accuracy on the IMDb sentiment dataset sparked the idea to build a lightweight yet powerful demo. What it does- Ingests the dataset, tokenizes it with a byte‑level BPE tokenizer, and streams the data to keep memory usage low. Trains a custom Transformer encoder (built in PyTorch) using mixed‑precision and gradient accumulation for efficient GPU usage. Evaluates the model with accuracy, macro‑averaged F1, and detailed classification reports. Exports the trained model to ONNX and serves it via a FastAPI endpoint (POST /predict) that returns sentiment predictions in sub‑millisecond latency on a modest CPU. Docker‑izes the entire stack with a multi‑stage build, producing a compact image (< 250 MB) ready for deployment. Includes MkDocs documentation, CI pipelines (GitHub Actions), and a full test suite to ensure reproducibility. How we built it- Phase Highlights Scaffold Generated with cookieccutter → src/, tests/, docs/, CI files. Data pipeline Used HuggingFace datasets (streaming mode) + tokenizers for efficient tokenization. Model Implemented a lightweight Transformer encoder in PyTorch (≈ 2 M parameters). Training Leveraged PyTorch Lightning for clean loops, early stopping, and checkpointing; added torch.cuda.amp mixed‑precision and gradient accumulation (8 steps). Evaluation Produced classification_report (sklearn) and confusion‑matrix visualisations (matplotlib). Export & Serve Exported with torch.onnx.export → ONNX Runtime; wrapped in FastAPI + Uvicorn. Containerisation Multi‑stage Dockerfile (builder → python:3.11-slim). CI/CD GitHub Actions for linting, testing, Docker image build/push. Docs MkDocs site with usage tutorials, API reference, and design rationales. Challenges we ran into Memory spikes while tokenizing the full dataset → solved with streaming mode and batch‑wise processing. Long training times on a single GPU → introduced gradient accumulation and mixed‑precision to halve epoch duration. ONNX export errors caused by unsupported ops → simplified the classification head to a plain nn.Linear and used dynamic_axes. Bloated Docker image (> 1 GB) breaking CI limits → switched to a multi‑stage build, drastically shrinking the final image. Cross‑platform reproducibility → fixed all random seeds (numpy, torch, random) and pinned exact dependency versions in requirements.txt. Accomplishments that we're proud of- Achieved > 90 % accuracy on IMDb sentiment with a 2 M‑parameter model. Delivered sub‑millisecond inference on a CPU‑only setup via ONNX Runtime. Produced a compact Docker image (< 250 MB) that builds and runs within CI limits. Provided full documentation (MkDocs) and a single‑command deployment experience. Established a reproducible pipeline (fixed seeds, version‑locked dependencies) that works on Windows, macOS, and Linux. What we learned- Streaming large text corpora is essential for memory‑efficient pipelines. Mixed‑precision training combined with gradient accumulation can reduce GPU memory by ≈ 40 % without hurting model quality. ONNX export is a powerful way to decouple training from inference, but you must stay within the operator set supported by the runtime. Multi‑stage Docker builds are indispensable for keeping CI/CD pipelines fast and cost‑effective. Clear docstrings, type hints, and a CHANGELOG.md dramatically improve onboarding for external contributors. What's next for VERDANT- Multi‑language support – add tokenizers and pretrained embeddings for Spanish, French, and German reviews. Active learning loop – integrate a small UI for users to label uncertain predictions and retrain the model on‑the‑fly. Model zoo – expose a simple plugin system so others can swap in BERT, RoBERTa, or DistilBERT variants. Edge deployment – compile the ONNX model to TensorRT / OpenVINO for ultra‑low‑latency inference on edge devices. Community challenges – host a hackathon where participants improve accuracy, reduce model size, or add new features, with a leaderboard integrated into the FastAPI service.