PBKsurfers: Body-Controlled Subway Surfer Game

Inspiration

We were inspired by the desire to make gaming more interactive and engaging. The idea of controlling a game with your body seemed exciting, and we wanted to push the boundaries of traditional gameplay. By using computer vision and body movements, we aimed to create a more immersive and competitive experience that adds a fun twist to Subway Surfer.

What it does

PBKsurfers transforms the popular Subway Surfer game into a physical experience. Players use their body movements to control the character in the game. Whether it’s swiping left, right, jumping, or ducking, all actions are mapped to real-time movements detected using OpenCV, Tesseract, and other tools. Players can compete with friends to see who can score the highest while controlling the game solely with their bodies.

How we built it

We used the following technologies to bring PBKsurfers to life:

• Tesseract: Leveraged Tesseract for text recognition to handle input commands during gameplay.
• Python: Python was the core language for development, enabling us to easily integrate computer vision libraries and control the game.
• OpenCV: OpenCV was used for real-time image processing to detect and track players' movements using the webcam.
• PyAutoGUI: PyAutoGUI was used to simulate keypresses and map detected body movements to Subway Surfer’s in-game actions.
• Pillow (PIL): The Pillow library was utilized for image manipulation and processing, allowing us to enhance the game's visuals and integrate feedback during gameplay.
• Firebase: Firebase was integrated to provide real-time multiplayer functionality and store user data, scores, and progress in the cloud.

Challenges we ran into

• Movement Detection Accuracy: Ensuring that the system accurately detected and mapped body movements to the game controls posed a challenge, especially in varying lighting conditions.
• Real-Time Processing: Processing real-time video input while maintaining smooth game performance was a major hurdle, as it required optimizing the code for faster detection without delays.
• User Interface (UI) Design: Designing an interface that was intuitive for players, allowing them to easily calibrate the system and start playing without confusion.

Accomplishments that we're proud of

• Seamless Body-Control Integration: We successfully mapped physical gestures to in-game controls, creating a unique and fun gaming experience.
• Multiplayer Support: We implemented multiplayer functionality using Firebase, allowing friends to compete against each other in the same game environment.
• Customizable Controls: The system is flexible, allowing users to customize body gestures for specific game actions.
• Real-Time Feedback: Integrated feedback mechanisms using Firebase to track player progress and show results in real-time.

What we learned

• Computer Vision Techniques: We gained valuable experience using OpenCV for motion detection and Tesseract for text recognition in an interactive application.
• Performance Optimization: We learned the importance of optimizing real-time video processing for smoother gameplay.
• Game Interaction Design: We learned how to design an interface that translates physical interaction into a digital experience in a way that is intuitive and fun for users.
• Cloud Integration: We gained hands-on experience using Firebase to manage real-time data and user interactions in a multiplayer setup.

What's next for PBKsurfers

• Mobile App Integration: We plan to extend the app to mobile devices, allowing users to control the game using their phone cameras.
• Enhanced Multiplayer Features: We aim to add more competitive features, such as leaderboards and multiplayer modes, where players can track their scores.
• AI-Assisted Training Mode: Implementing an AI that can simulate opponent moves for training or practice sessions, so users can compete against an AI before playing with others.
• Additional Game Modes: Adding new game modes and challenges to keep the experience fresh and exciting for players.## What it does

Built With

• firebase
• opencv
• pil
• python
• tesseract