GlassBox

• 

  GlassBox - Re-Imagined: an interactive learning lab for exploring how a tiny language model thinks, learns, and generates text in real time.

Inspiration

Language models are everywhere, but for most people, what happens inside them still feels like a black box. We wanted to make those concepts easier to understand by turning them into something people can actually interact with.

That inspired GlassBox: an interactive learning lab where users can experiment with a small language model and see how changes to the model affect its behavior in real time.

What it does

GlassBox lets users explore important concepts behind language models instead of only reading about them.

Users can:

• Change the temperature and see how the next-character probability distribution changes.
• Inspect attention weights to understand what parts of the input the model is focusing on.
• Train the model and watch its training and held-out loss change over time.
• Compare the model's predictions before and after training.
• Generate text using a repeatable sampling seed.
• Explore visual representations of the model and its internal behavior.

Our goal is to make concepts like attention, probability, sampling, embeddings, and training more visual and approachable.

How we built it

We built the learning model from scratch in Python and NumPy using a small character-level, transformer-style architecture. The backend implements tokenization, embeddings, positional embeddings, causal self-attention, next-character prediction, temperature-based sampling, cross-entropy loss, gradient-based output-layer training, and seeded text generation.

We connected the model to a FastAPI server that creates isolated learner sessions and exposes endpoints for prediction, training, generation, probabilities, attention data, and loss data.

The frontend uses React and TypeScript to turn this information into an interactive learning experience, with visualizations for concepts such as probabilities, attention, training, and model architecture.

Challenges we ran into

One of our biggest challenges was deciding how much of a language model to build ourselves while keeping everything understandable and interactive within a hackathon timeframe.

We also had to make sure experiments were meaningful. For example, changing temperature should change the probability distribution without changing the model's weights, while training should actually update weights and affect future predictions.

Another challenge was translating numerical model data, such as attention weights and embeddings, into visualizations that make sense to someone learning these concepts for the first time.

What we learned

Building the model ourselves gave us a much deeper understanding of what happens between giving a language model an input and receiving a prediction.

We learned how tokenization, embeddings, positional information, attention, logits, softmax, temperature, loss, gradients, and sampling connect together as parts of one system. We also learned a lot about connecting a machine-learning backend to an interactive web interface.

What's next for GlassBox

We would like to expand GlassBox with more interactive model visualizations, larger models and datasets, additional learning experiments, and AI-powered explanations that adapt to what each learner is exploring.

Ultimately, we want GlassBox to make learning about AI feel less like looking at a black box and more like opening one.

Built With

• fastapi
• numpy
• python
• react
• typescript