Inspiration
In the digital era where cash is seldom used, money feels more abstract and intangible which makes it easy to overspend. We've noticed that people tend to spend their physical cash more carefully because they can see and feel it. Physically seeing money being used really gives the purchase weight and we realized that showing the user a physical representation of their spending is a powerful way to understand how much they actually spend.
What it does
Financial Reality is a VR application with a web interface that shows the user a physical representation of their spending so they can better understand their spending habits.
Web Interface
The web interface allows users to upload bank statements into our system. The user can see a list of all their statements and list of transactions. This lets us get the user's information without having to do this in VR. The statements are stored in our PostgreSQL database that can then be retrieved in the VR application via an API call.
VR Application
The VR application can load in user statements that they've entered into the system via the web interface. They can then visit different rooms that have various visualizations of their statement information. These visualizations all work towards showing the scale of their spending. For example, there is a visualization that shows piles of cash representing different categories of user spending. Seeing the size difference of different spending categories immediately shows the difference in how much you're spending between the two.
How we built it
We built Financial Reality's VR application using Unity game engine. Unity has powerful native support for integrating with VR technology and has many online resources to help us when we get stuck. Unity divides up areas into scenes so each visualization can get its own area without needing to worry about space. A lobby area was created for users to load in their data and acts as a hub for accessing the various visualizations we've built. To bring our scenes to life, we used Unity's native scripting components to add functionality to the application. Unity scripts are built in C# and interact directly with the GameObjects they are attached to. The API calls to the backend are done via a script as well since Unity has support for accessing the web. All testing was done through a Meta Quest 3S and a Meta Quest 2.
The web interface is built using vite + React framework with a Node.js backend. The backend uses Express.js to create endpoints for interfacing with the frontend. The web interface allows users to upload their bank statements as a CSV file which can then be accessed in the VR game. The backend stores this in a PostgreSQL database and exposes this data and aggregations on its API. The app is fully Dockerized, with separate containers for the frontend, backend, and database. The app is hosted on AWS EC2 at https://financialreality.photo The app was built using the assistance of generative AI.
Challenges we ran into
Unity
Most of our team are first time Unity users so setting up the Unity environment proved challenging. We also had some trouble setting up Github to work with Unity initially but we managed to resolve that issue early in development. We also had to learn how to use Unity itself. We found Unity intuitive to use but we had some trouble dealing with niche issues specific to VR development.
VR Technology
We used the Meta Quest 3S when starting the project. This was a great choice to use since Unity has great support for this device. We ran into issues late into development when the headset stopped working due to overuse. This was a critical issue. We had to change the Unity environment to support testing without a headset and had to quickly source a new device to use.
Web Interface
The biggest challenge when it came to the web interface was the design process. We had a hard time figuring out what tech stack to use. We initially went for a FastAPI backend for simplicity but moved away from that design due to our team being unfamiliar with the technology. We also changed our database a few times. We eventually settled on a combination of Express.js, React + Vite, and PostgreSQL, as we were most familiar with those technologies. Designing the flow of the application and structuring the data from the ground up was also difficult, as we had no experience creating a web app from scratch.
Accomplishments that we're proud of
We are extremely proud to have worked on such an ambitious project for this Hackathon! A majority of the technologies we used to created Financial Reality were new to us and everyone in the team did their best to learn and utilize these new tools. We are also proud of the concept of Financial Reality. We believe this is a great way to understand your personal finances on a more concrete scale. Finally, we are very proud of how little generative AI we used through the project. We tried to do as much of this project as we could by ourselves.
What we learned
Our team learned a lot throughout the process of creating Financial Reality. We learned a lot of new technologies spanning different areas. We touched on using game engines, building around VR, web design, API design, and 3D modeling. We also learned a lot about project management. This was an ambitious project with many different parts and we had to learn how to efficiently delegate tasks and integrate changes. In addition, we learned a lot about the challenges of deploying a web app by deploying a fully Dockerized full-stack app to AWS EC2 with HTTPS/SSL. Finally we learned a lot more about troubleshooting and problem solving than ever before. Because we had so little time to work on this project, we had to efficiently develop solutions and quickly think through problems.
What's next for Financial Reality
Due to time constraints, we weren't able to add as many visualizations as we wanted. In addition to seeing your money in physical dollars, we also wanted to allow users to use other objects as units, such as food or household items. We also want to implement interactive, animated visualizations that could allow you to replay your monthly spending in a time lapse or a projection of it. There were also many basic features we had to skip to focus on iterating on the core functionality, such as users, more tools to manage data, bank integrations, etc. Lastly, we want to improve the VR environment. We want to add a proper environment for each visualization as well as make the lobby look better.
Log in or sign up for Devpost to join the conversation.