Real-Time Raytraced Pac-Man

• 

  Python script render, which the RTL is checked against

• 

  Real-life LCD screen image!

Inspiration

Ray-tracing is a computer graphics technique used to produce realistic lighting effects. It's commonly used to render images or videos with realistic lighting ahead of time.

This project is inspired by NVIDIA RTX GPUs, which have specialised ray-tracing units to enable real-time ray-tracing, meaning you can have those cool lighting effects while playing video games! I've never had an RTX GPU but the concept was interesting enough that I wanted to make a real-time raytracing processor.

What it does

This is a miniature version of Pac-Man running at a 400x240 pixel resolution, so the rules should be pretty familiar: eat all the pellets and don't run into ghosts.

The map and all the characters are represented as 3D objects, ray-traced on an FPGA and shown on the built-in LCD screen. Despite the pixelation, you can still see Pac-Man and friends cast shadows and reflections on the surrounding surfaces. The Super-Pellets even glow! To add more fun, Pac-Man's movement is determined by tilt controls!

Tricks to make it run smoothly in real-time

• Fixed-point arithmetic: For fast calculations compared to floats
• Multi-core parallelism: Calculations for multiple pixels run on separate cores in parallel to increase throughput
• Double frame buffering: The LCD only starts drawing frames when they have been fully rendered, eliminating screen tearing

Development process

All hardware is my own, including the Tang Mega 138K FPGA, so most of the work was writing a design to run on the FPGA.

My usual approach (as a student) doing digital design projects is to carefully plan out the architecture before slowly implementing it in SystemVerilog. This hackathon is my first time using agentic AI to do hardware design; it certainly made this project possible by cutting down the time spent writing RTL and doing timing analysis, although perhaps at the cost of human-readability.

Challenges we ran into

One big challenge overall was that synthesizing and place-and-routing a design to check for timing problems is a very time-consuming process, especially when designs are scaled up and made larger. By the end of this project, the time to create a bitstream almost reached an hour.

Another challenge was that when scaling up, it was easy to run out of resources like block RAM, DSP multipliers and so on. It may be possible to share some resources (like dividers and multipliers) between sequential operations, but this comes at the cost of added muxes and hence timing pressure.

What I learned

How to use AI agents for the first time!

What's next

Right now this raytracer can only draw spheres, planar and half-sphere primitives. Fun additions would include triangle meshes (like stl files) and texture support!

It would also be good to work towards more parallelism; right now, calculations are split between 3 cores, but there may be an architecture which allows for more cores.

Built With

• claude
• gowin
• python
• systemverilog