Introduction
Team
Tech Stack
Appeal
Features
Game Menu
Gaze Calibration
Main Gameplay (4 Camera + No Dispatch)
Main Gameplay (6 Camera + Active Dispatch)
Eye Tracking
Attention Lost
Debrief
Figma
Figma Make

Inspiration

After hearing about the category "Serious Game", I came up with this project idea with our team after remembering about the psychology principle of sustained attention. Combining the elements of computer vision, job training, and the job at the pinnacle of paying attention, we chose to do a game that is a Night Shift Security Camera Training Simulator. - Lilly

Why?

"Psychologists discovered the human attention span lasts roughly 20 minutes. After these 20 minutes, security guards will miss up to 95% of occurrences. " Citation

CCTV operator training is straight forward, but maintaining peak operational performance isn’t. What if CCTV teams could assess and train their observation concentration through a game?

Many ways to play:

This game can be used as a rapid scanning and crisis trainer like demonstrated in the video or a slow endurance concentration game.

Ex) rapid paced game = short time (<10 minutes), many events, fast reaction time required.

Ex) slow endurance game = long time (1 hour+), few infrequent events, paying attention consistently required.

What it does

Knight's Eye is a browser game where you're the night operator at a security desk. There are anomalies that you must let your dispatcher know to alert knights to investigate. This is a "Serious Game" that can serve as a job trainer or skills practicer.

Watch the feeds: 2,4, or 6 live camera feeds show a building full of indiscernible NPCs. Most are ordinary. A few are suspicious: loitering, revisiting doors, entering restricted rooms, or avoiding the cameras. Either track them by their movement patterns, or by observing suspicious behavior (green glows).

Gaze tracking: A 13-point calibration maps your webcam gaze to the screen, so the game always knows which feed you're watching. You can also use your mouse to stand in as a camera.

Report and radio: Report a sighting on a feed, or hold Space and say "Camera 3, sighting." A simulated (AI) dispatcher answers over the radio in a realistic or robotic voice and sends a knight through the building to investigate. Your speed in scanning and reporting determines if the suspect runs free.

Distractions: Pop-ups cover your feeds, feeds glitch, and dispatch checks in with questions like "How many people are on camera 2?" that you answer by voice.

Debrief: When the shift ends, you get a grade, your catches and false reports, a heatmap of where your attention went on each feed, a per-camera timeline, and coaching. The dispatcher reads a summary aloud.

How we built it:

Frontend: TypeScript and Vite

Gaze Tracking: MediaPipe: runs in the browser calculating eye tracking from your iris position, eyelid opening, head yaw, pitch and position, which is then fit on a ridge regression on a 13-point calibration, and a One Euro filter to smooth the output.

World Simulation: A tile-based building with rooms, doors, and six cameras. The office is larger than what the cameras can see creating blinds spots. NPCS follow scripted plans and move with A*path finding.

Simulated (AI) Dispatcher: Google Gemma via Hugging Face Inference Providers. The game is brought to life by a dispatch team that follows your command, investigates your dispatches, and requests your understanding of the scene. Choose either from dynamic or preset lines generated by Gemma or templates, and realistic and robotic voices made with Eleven Labs or Web Speech API. The dispatch system still works even after support for Hugging Face and Eleven Labs run out of credits. Every line has a template fallback if the AI takes more than 2 seconds.

Voice: ElevenLabs text-to speech create pre-generated MP3s to save credits and Web Speech API as a fallback.

Architecture: We split up the project into 4 parts: (vision, world, game, analytics), each part was owned by one teammate letting us work in parallel. Together we shared design decisions about gameplay and interactions!

Design: The concept of this project was inspired from Windows 98. The design was first drawn on paper, formed as bare boxes, and then remade in Figma Design and Figma Make.

Hosting: The game is hosted on Github Pages at a custom domain: ([knightseye.us]. This was registered through GoDaddy, and the API keys to connect to ElevenLabs and Google Gemma via Hugging Face were hosted through Netlify which connected back to GitHub.

Challenges we ran into

Team of first-time hackers and having to start with teaching what is git, and how to use it, how to set up a project to everyone being able to contribute to the project and have an impressive result!
Getting accurate calibrated eye tracking that was intuitive to use. Needed to consider camera placement, head moving for scanning, posture, head tilt.
Hosting the game site. Our setup up to host was quite complicated since we needed to integrated our API keys!
```
Code -> Github Pages  -----> Pork Bun ----> Go Daddy

  L --> Netlify ----^
```
Making sure the game is reactive and interactable with NPC pathfinding techniques (A*)

Accomplishments that we're proud of

Eye tracking actually working and being properly calibrated.
Integrated an AI system into our Simulated Dispatcher that will work even after we stop having the free trials.
- Properly hosting a website with a custom Domain (first time for everyone!)
- Game User interface style and aesthetics!

What we learned

Working with a team
New technologies (Typescript, Vite, Netlify, GitHub Pages, Pork Chop, GoDaddy Domains, Hugging Face: Google Gemma, Eleven Labs: Text to Speech, Web Speech API)
Web Hosting
Github (Pulling, Pushing, Merging/Committing)