VRM Real Estate Bot

Inspiration

The inspiration behind the VRM Real Estate Bot was to create a helpful tool for real estate agents, enabling them to practice and improve their communication and problem-solving skills with clients in various situations.

What it does

Our bot is designed to simulate common scenarios that real estate agents often encounter while interacting with clients. By engaging with the bot, agents can role-play different situations, such as handling client inquiries, negotiating deals, providing property information, and addressing client concerns.

How we built it

Developing the VRM Real Estate Bot required a comprehensive blend of cutting-edge technologies. We leveraged OpenAI's powerful API with Langchain to implement the core logic and conversational capabilities, enabling dynamic and contextually relevant interactions. For the client-side, we chose NextJS 13, React, and Tailwind to ensure a smooth and responsive user experience. The bot's dynamic character rendering was achieved through the use of three.js and @pixiv/three-vrm libraries, coupled with the browser's Web speech API for voice interactions.

Challenges we ran into

One of the hardest technical hurdles we overcame was rendering character models and adding animations and expressions to those models since we had never touched anything similar to that before. There was also the issue of figuring out a format to send and receive messages from OpenAI's API that we iterated on to arrive at a good format.

Accomplishments that we're proud of

Finally getting the models to be animated with expressions was something we are proud of since this was uncharted waters for use and we were unsure if it was possible in the given timeline.

What we learned

We learned a lot about how LLM's work and how best to prompt them to integrate them into a product that can fulfill a niche. We also learned how to use the browsers Web Speech API and working with three.js to render characters to an html canvas.

What's next for VRM Real Estate Bot

Some potential next steps that we could explore could be training the bot with conversations that real estate agents have had to make simulations more realistic. We could also take conversations that real estate agents have had and give feedback on what they could do better. An area that would be interesting is more specific scenarios regarding real estate and maybe introducing difficulty to test how good a real estate agent is with dealing with clients.