Inspiration

We were inspired by the idea of bringing AI to life. Instead of just talking to AI on a screen, we wanted to build a smart system that can move, understand, and work in the physical world.

What It Does

Our project is an AI-powered receptionist designed for hotels, offices, hospitals, and other customer-service spaces.

The system can:

Talk with users through voice
Convert speech into text using speech-to-text
Understand what the user wants
Reply naturally using text-to-speech
Verify hotel bookings
Check bookings using a guest name, phone number, or booking ID
Detect invalid, cancelled, or mismatched bookings
Recommend nearby restaurants, hospitals, tourist spots, and other useful places
Create a mock taxi-booking request
Send housekeeping requests through a webhook
Control hardware such as LEDs, buttons, microphones, speakers, and sensors
Connect the ESP32 with the desktop application using Serial communication or Wi-Fi
Handle errors without breaking the whole conversation

The main idea is to make the experience feel simple for the user.

A guest can walk up, speak normally, and get help without needing to use a complicated app or website.

How We Built It

We divided the project into different modules and then connected everything into one complete system.

AI Receptionist

We developed the main AI receptionist and designed prompts that guide its behaviour.

We also created conversation flows for different situations, such as:

Greeting a guest
Checking a booking
Handling a failed verification
Recommending nearby places
Booking a taxi
Sending a housekeeping request
Responding to unclear questions
Recovering from errors

The AI uses speech-to-text to understand the user and text-to-speech to reply out loud.

We also added intent detection so the system can understand whether the user wants to verify a booking, request a service, ask for directions, or do something else.

Booking System

We created a booking database that stores guest information and reservation details.

The receptionist can verify a booking using:

Guest name
Phone number
Booking ID

The system also checks for different error cases, including:

Booking not found
Incorrect booking ID
Cancelled booking
Guest details not matching
Missing information
Duplicate or unclear results

Instead of showing a confusing error, the AI explains the issue in a simple and polite way.

Nearby Recommendations

We developed a recommendation feature that helps guests discover nearby places.

The AI can suggest:

Restaurants
Cafes
Hospitals
Pharmacies
Shopping areas
Tourist attractions
Transport locations

The recommendation system is designed to give useful and easy-to-understand options based on the guest's request.

Housekeeping Requests

We connected housekeeping requests to a webhook.

This means the AI can take a guest's request, such as asking for towels or room cleaning, and send it to another system.

The feature also handles errors, such as:

Missing room number
Failed request
Invalid service type
No response from the webhook

ESP32 and Hardware

We programmed the ESP32 and connected it to the desktop application using Serial communication or Wi-Fi.

The ESP32 can work with physical components such as:

Buttons
LEDs
Microphones
Speakers
Sensors

For example, an LED can show whether the system is listening, thinking, speaking, or facing an error.

The hardware makes the project feel more like a real embodied AI system instead of a normal computer program.

Challenges We Ran Into

Building the project was exciting, but it definitely came with challenges.

Understanding Different Users

People do not always speak in the same way.

One person might say:

“Can you check my booking?”

Another person might say:

“I have a reservation under my name.”

Both users want the same thing, but the sentences are different.

We had to design the AI so it could understand different wording, accents, speaking styles, and incomplete questions.

Handling Booking Errors

Booking verification was more complicated than just searching for a name.

The system had to understand the difference between:

No booking found
Wrong phone number
Cancelled booking
Correct name but incorrect booking ID
Multiple guests with similar names
Missing guest information

We had to make sure the receptionist gave the correct response for each case.

Connecting Hardware and Software

Connecting the ESP32 to the desktop application was another major challenge.

Sometimes the Serial connection failed, messages arrived late, or the hardware did not respond as expected.

We had to test the communication carefully and make sure both the software and hardware understood the same commands.

Making the Conversation Feel Natural

We did not want the receptionist to sound robotic or repeat the same sentence every time.

We improved the prompts and conversation flow so the AI could sound friendly, clear, and helpful.

Managing Multiple Features

The project included many different parts, so integrating everything was difficult.

A small problem in one module could affect the entire system.

We learned that testing each feature separately before combining everything saved a lot of time.

Accomplishments That We're Proud Of

We are proud that we created more than just a chatbot.

Our project combines AI, voice interaction, databases, webhooks, and physical hardware into one complete experience.

Some of our biggest accomplishments include:

Building a working AI receptionist
Creating natural voice conversations
Detecting user intentions
Verifying bookings using multiple details
Handling invalid and cancelled bookings
Creating nearby-place recommendations
Building a mock taxi-booking system
Sending housekeeping requests through a webhook
Connecting an ESP32 with the desktop application
Controlling real hardware components
Integrating all the modules into one system

The part we are most proud of is that the project actually connects AI decisions with physical actions.

That is what makes it an embodied AI system.

What We Learned

This project taught us that building a useful AI system is not only about getting the AI to answer questions.

A real system also needs to:

Understand the user
Ask for missing information
Handle mistakes
Protect user data
Recover from errors
Communicate clearly
Connect with other software
Control physical devices

We learned about:

Prompt engineering
Conversation design
Speech-to-text
Text-to-speech
Intent detection
Databases
Booking verification
APIs
Webhooks
ESP32 programming
Serial communication
Wi-Fi communication
Sensors and hardware
System integration
Testing and debugging

We also learned the importance of teamwork.

Different parts of the project needed different skills, and the final system only worked when everything was connected properly.

What's Next for Physical AI for Companies

Physical AI has a lot of potential for companies.

In the future, systems like ours could be used in:

Hotels
Hospitals
Schools
Airports
Shopping malls
Offices
Banks
Restaurants
Reception areas
Customer-support centres

Our next steps would be to make the system more advanced and ready for real-world use.

Possible future features include:

Multilingual conversations
Real hotel-management-system integration
Live room availability
Real taxi-service APIs
Digital payment support
Smart room control
Face recognition with proper privacy protection
Better sensors and cameras
More natural voice responses
Personalised guest recommendations
Emergency assistance
Cloud-based monitoring
Mobile app integration
Better security and data protection

For companies, Physical AI could reduce waiting times, automate repetitive work, improve customer service, and provide support 24/7.

The future of AI is not only about smarter software.

It is about building systems that can understand people, interact with the world, and take useful action.

Team Responsibilities

AI and Conversation System

The AI module focuses on:

Developing the AI receptionist
Designing prompts
Creating conversation flows
Integrating speech-to-text
Integrating text-to-speech
Detecting user intentions
Making sure responses are accurate and helpful

Booking Database

The booking module focuses on:

Creating the booking database
Searching bookings using guest name, phone number, or booking ID
Verifying guest information
Detecting invalid bookings
Handling cancelled bookings
Managing mismatched details
Returning clear error messages

Services and Recommendations

The service module focuses on:

Recommending nearby places
Creating the mock taxi-booking feature
Sending housekeeping requests through a webhook
Managing service data
Handling API and webhook errors

ESP32 and Hardware Integration

The hardware module focuses on:

Programming the ESP32
Connecting it to the desktop application
Using Serial communication or Wi-Fi
Handling buttons
Handling microphones
Controlling LEDs
Connecting speakers
Reading sensors
Integrating all software and hardware modules

Final Thoughts

The AI Embodied System shows how artificial intelligence can move beyond the screen.

By combining natural conversation, voice technology, booking management, service requests, recommendations, and real hardware, we created a system that can understand users and interact with the physical world.

This project is an early step toward a future where AI is not only something we talk to, but something that can actively help us in real spaces. """