GAPAS — Game Aesthetic Preference Analysis System

Inspiration

Game visuals strongly influence how players experience and perceive a game, but studying these aesthetic preferences at scale is difficult. Researchers need large, consistent, well-structured gameplay image datasets, while manually capturing, labeling, validating, and organizing screenshots is time-consuming.

GAPAS (Game Aesthetic Preference Analysis System) was developed to make this process more systematic. The goal is to turn raw gameplay screenshots into structured, research-ready aesthetic data that can be analyzed by researchers and used for future machine-learning studies.

What it does

GAPAS provides an end-to-end workflow for building gameplay aesthetics datasets:

• Capture gameplay screenshots directly through browser screen sharing.
• Upload existing collections of gameplay screenshots in bulk.
• Detect and remove near-duplicate images.
• Analyze screenshots using Vision Language Models (VLMs).
• Convert visual observations into structured aesthetic annotations.
• Validate AI-generated responses against a defined schema.
• Export the resulting dataset for further research and analysis.

The system is designed around a modular VLM architecture, allowing different AI providers to be integrated without redesigning the complete application.

How we built it

GAPAS is built as a web-based application using Next.js, React, TypeScript, MongoDB Atlas, and Cloudinary. The application contains separate modules for screenshot capture, storage, duplicate detection, VLM analysis, validation, and dataset export.

The VLM layer uses a provider-based architecture so that models such as Google Gemini and other VLM providers can be integrated through a common interface. This makes the system flexible for experimentation with different AI models.

Gameplay screenshots are processed through the pipeline, analyzed by the selected VLM, and converted into structured data. Schema-based validation is then used to ensure that generated responses follow the expected dataset structure before export.

Challenges we faced

One of the main challenges was making AI-generated visual analysis consistent enough for research use. Vision-language models can produce useful observations, but their responses may vary in format and detail.

To address this, GAPAS uses structured prompts, a defined annotation schema, validation, and a modular VLM interface. Another challenge was building a reliable workflow for handling large numbers of gameplay screenshots while avoiding duplicate data.

What we learned

Through this project, we learned how to combine a modern web application with AI-based visual analysis and research-oriented data processing. We also learned that using AI for research is not only about generating predictions—the surrounding pipeline for data collection, validation, organization, and reproducibility is equally important.

GAPAS helped us understand how VLMs can be integrated into practical research workflows and how structured datasets can transform unorganized visual information into data that can support further analysis and machine-learning research.

Future Scope

GAPAS can be extended with additional VLM providers, larger-scale automated dataset generation, improved aesthetic trait models, researcher collaboration features, and more advanced statistical and machine-learning analysis of the generated dataset.

Built With

• ai
• cloudinary
• dataset
• google-gemini
• mongodb-atlas
• next.js
• node.js
• react
• typescript
• vision-language-models