Inspiration
I was inspired to build CANDY AI because I wanted to make computer usage easier and smarter. Many everyday tasks, like opening applications, finding files, and searching the internet, take time when done manually. I wanted to create my own AI desktop assistant that could understand voice commands and help users complete these tasks easily. I also wanted to explore how AI could make computers more natural and user-friendly, including for people who speak Telugu and English.
What it does
CANDY AI is a smart desktop assistant built with Python and Generative AI. It is designed to help users perform everyday computer tasks through simple commands.
- Open applications such as Chrome and VS Code.
- Find and open files and folders.
- Search the web for information and tutorials.
- Assist with everyday desktop tasks and automation.
- Interact with AI for helpful responses.
- Support voice commands in English and Telugu.
My goal is to make computer interactions easier and help users save time.
How we built it
I built CANDY AI using Python as the main programming language. I explored and integrated different tools to develop its AI and desktop automation capabilities.
- Python: Core application logic.
- Google Gemini API: Generative AI integration.
- Ollama: Support for local AI models.
- Speech Recognition: Processing voice commands.
- PyAutoGUI: Desktop automation.
- CustomTkinter: User interface.
- OS and Webbrowser modules: File operations and website access.
The main workflow is Voice Input → Speech Recognition → AI Processing → Action Execution → Response.
Challenges we ran into
One of my biggest challenges was connecting different Python modules and making them work together correctly. I also faced issues with dependencies, environment setup, and getting the assistant to respond and execute commands as expected.
Debugging these problems taught me the importance of testing each component, understanding errors, and improving the application step by step. Although some features still need improvement, these challenges motivated me to keep learning and developing CANDY AI.
Accomplishments that we're proud of
I am proud of turning my idea into a real AI desktop assistant project. I explored how Generative AI, Python, voice interaction, and desktop automation can work together in one application.
I am especially proud of choosing a practical problem and working toward a solution that can make everyday computer tasks easier. Building CANDY AI has strengthened my confidence in developing AI-powered applications and solving technical problems.
What we learned
While building CANDY AI, I learned about Python development, Generative AI integration, speech recognition, local AI models, and desktop automation.
I also learned that creating an AI assistant involves more than generating answers. It requires understanding user intent, selecting the right action, executing it correctly, and handling errors. This project improved my problem-solving, debugging, and project-building skills.
What's next for CANDY AI – Your Smart Desktop Assistant
My next goal is to make CANDY AI more reliable, intelligent, and useful.
Future improvements include:
- Better voice conversations and Telugu language understanding.
- More accurate file and application management.
- Improved offline functionality using local AI models.
- Screen understanding to help users based on what is displayed.
- More advanced task automation.
- Confirmation before performing sensitive actions.
- Better error handling and a smoother user interface.
My vision is to grow CANDY AI into a dependable personal desktop assistant that helps people interact with their computers naturally and complete everyday tasks more efficiently.
Just Say It, CANDY Does It! 🍬
Built With
- customtkinter
- generative-ai
- google-gemini-api
- ollama
- os
- pyautogui
- python
- python-dotenv
- speech-recognition
- webbrowser
