Inspiration
Modern developers often rely on massive, resource-heavy LLMs that are over-engineered for basic syntax checking, refactoring, and local code completion. For the Global Innovation Build Challenge V2 (Track 01: Foundational LLM Development), we wanted to build a hyper-efficient, domain-specific alternative: CodeHelper-Lite.
What it does
CodeHelper-Lite is an ultra-lightweight 41.6M parameter Causal Transformer model designed specifically for syntax assistance, bug fixing, and multi-language code synthesis. It features an optimized 8-layer architecture with RoPE positional encodings, SwiGLU FFN, RMSNorm, and bfloat16 mixed-precision support, running seamlessly with low latency and a high token-per-second throughput.
How we built it
- Architecture & Training: Built using PyTorch and Hugging Face Transformers concepts, optimizing parameters strictly below the 50M limit.
- UI & Experience: Designed a premium, minimalist Dark Mode interface with a sophisticated Sky Blue (
#0ea5e9) accent palette for developer consoles and offline evaluation. - Quantization & Edge Readiness: Engineered with SmoothQuant/INT8 capabilities to ensure zero-latency execution on local environments and browsers.
Challenges we ran into
Balancing model depth and hidden dimensions to keep the total parameter count tightly under the strict 50M constraint while maintaining high code-synthesis accuracy required meticulous architectural adjustments and custom token mapping.
What we learned
We gained deep insights into foundational LLM optimization, parameter budgeting, and crafting high-performance developer tools optimized for edge constraints.
Built With
- css3
- google-ai-studio
- html5
- javascript
- python
- pytorch
- transformers
