Game Controller for ALS

• 

  Hit a Villager!!

• 

  Hit a Golem(idk)!!

• 

  Gameplay_SS01

• 

  Gameplay_ss02

• 

  Calibration Success!!

Inspiration

Stephen Hawking could not play Minecraft.

What it does

Tracks your eye movement and plays your Minecraft character accordingly. Blink = attack/break Up/Left/Right/Down = change FOV Middle = walk forward

How I built it

Used existing AI software WebGazer.js for eye tracking. HTML5 canvas for the control overlay. Python script for executing keystrokes and mouse movements. Websockets for communication between the control page and the Python script.

Challenges I ran into

Calibration issues, conversion of values, background processes/hang ups, networking.

Accomplishments that I'm proud of

Made the thing.

Categories

• Rated E

What I learned

Asyncio

What's next for Game Controller for ALS

• Add right click/use
• Add inventory selecting
• Expand to other games

Built With

• dlib
• eye-tracking
• javascript
• python
• webgazer
• websockets