fpGPT

Inspiration

Our group consists of 3 ECE majors who switched from computer science. With an interest in embedded systems and inspiration from Taalas/ChatJimmy, which which bakes LLM weights into ASIC silicon, we wanted to bridge the gap between machine learning and hardware engineering.

What it does

fpGPT is a standalone, transistor level GPT engine running on an Intel Cyclone V FPGA (DE1-SoC). Machine Learning models are running directly on a chip without an OS, CPU, or GPU, prompts are sent through an ESP32 microcontroller over UART.

Zero External Memory: Weights, activations, and KV cache fit entirely inside on-chip Block RAM (M10K BRAM).
Autonomous Silicon Execution: Runs without an OS, CPU, or GPU receives prompts and streams generated text over UART in real time.

How we built it

Model & Training: Designed and trained a character-level GPT in PyTorch.
Model Compiler (compile.py): Symmetrically quantizes FP32 weights to INT8, calculates power-of-two bit-shifts, and exports synthesizable Verilog headers and ROM hex images.
Silicon Architecture (Verilog): Built the core processor from scratch—including transformer_engine.v, attention.v with BRAM KV caching, DSP-backed dense_layer.v, integer layer_norm.v, and generation_controller.v.
Hardware Bridge: Created custom ESP32-S2 firmware to provide a USB-C to FPGA UART bridge.
Verification: Verified numerical accuracy with Python and simulated execution in Questa/ModelSim (Quartus Prime).

Challenges we ran into

Our biggest hurdle was figuring out how to get UART to get all 16 bytes from the FPGA to the server. Also centering the project around a propritery hardware device (Intel Cyclone V FPGA (DE1-SoC)) provided a unique challenge.