llm-switchbot

A comprehensive smart home automation platform integrating gpt-oss with SwitchBot Cloud API v1.1. Features natural language device control. Enables seamless voice-to-device automation.

Comment

📝 About the project

🎯 What inspired us

We were inspired by the everyday need to control smart homes using natural language. Current smart home systems require complex app operations or memorizing specific commands, making them difficult to use, especially for elderly people or those not familiar with technology.

We thought that if we could control smart homes through natural conversations like "Please turn on the air conditioner if the room is hot around 6 AM", anyone could easily create a comfortable living environment.

Additionally, the emergence of gpt-oss, an open-source large language model, opened up possibilities to achieve high-performance AI functionality in local environments. This allowed us to build smart home AI while maintaining privacy and avoiding expensive API costs.

🧠 What we learned

1. LLM and Hardware Integration Technology

  • Harmony Format: Learned standardized tool calling formats and understood how LLMs control actual devices
  • SwitchBot API: Implementation techniques for IoT device integration, webhooks, signature authentication, etc.
  • Real-time Processing: Built low-latency pipelines from sensor data acquisition to AI decision-making and device control

2. Natural Language Processing Practical Implementation

  • Prompt Engineering: Technology to convert ambiguous natural language into specific device operations
  • Fallback Processing: Alternative processing through pattern matching when LLMs fail
  • Context Understanding: Decision logic considering environmental information like time, temperature, and humidity

3. Monorepo Development and CI/CD

  • pnpm workspace: Efficient management of multiple packages
  • TypeScript: Building complex systems while maintaining type safety
  • Test-Driven Development: Quality assurance and regression prevention during feature additions

🏗️ How we built our project

Architecture Design

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Next.js Web   │    │  Fastify API    │    │  SwitchBot API  │
│   (React 18)    │◄──►│   (TypeScript)  │◄──►│   (Cloud API)   │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │              ┌─────────────────┐              │
         └──────────────►│   gpt-oss LLM   │◄─────────────┘
                        │   (20B/120B)    │
                        └─────────────────┘

Technology Stack

  • Frontend: Next.js 14, React 18, Tailwind CSS
  • Backend: Fastify, TypeScript, Prisma
  • AI/LLM: gpt-oss (20B/120B), OpenAI API (fallback)
  • IoT Integration: SwitchBot Cloud API v1.1
  • Development Environment: pnpm workspace, Jest, ESLint

Key Feature Implementation

1. Natural Language Workflow Parsing

// Generate automation rules from natural language
const workflow = await parser.parseWorkflow(
  "Please turn on the air conditioner if the room is hot around 6 AM",
  userId
);

2. Real-time Sensor Monitoring

// Evaluate conditions based on temperature/humidity sensor data
const result = await evaluator.evaluateConditions([
  { type: 'temperature', operator: 'greater_than', value: 26 }
]);

3. Scene Learning Functionality

// Learn user operation patterns and suggest automation
const suggestions = await learner.getSceneSuggestions(context);

🚧 Challenges we faced

1. Complexity of LLM and Hardware Integration

Challenge: gpt-oss's tool calling functionality was immature, making integration with actual device control difficult.

Solution:

  • Custom implementation of Harmony Format
  • Improved reliability through fallback processing
  • Incremental feature implementation (basic operations → automation → learning)

2. Balancing Real-time Performance and Privacy

Challenge: Local LLMs are slow to respond, while cloud APIs pose privacy risks.

Solution:

  • Hybrid architecture (local-first, cloud fallback)
  • Improved response speed through caching
  • Flexible configuration through environment variables

3. Processing Natural Language Ambiguity

Challenge: Converting ambiguous expressions like "around 6 AM" or "hot" into specific operations was difficult.

Solution:

  • Fallback processing through pattern matching
  • Utilizing context information (time, sensor data)
  • Incremental accuracy improvement (basic patterns → LLM parsing → learning features)

4. Development Environment and CI/CD Complexity

Challenge: Testing, type checking, and CI/CD configuration in a monorepo structure was complex.

Solution:

  • Efficient dependency management through pnpm workspace
  • Strict TypeScript type definitions
  • Incremental test implementation (unit → integration → E2E)

🎉 Achievements and Future Prospects

Implemented Features

  • ✅ Natural language smart home control
  • ✅ Creation and execution of automation workflows
  • ✅ Conditional decision-making based on sensor data
  • ✅ Learning user operation patterns
  • ✅ Privacy-focused local AI processing

Technical Achievements

  • 73/73 tests passed: 100% test coverage
  • Low-latency processing: Sensor → AI decision → control within seconds
  • High reliability: 99%+ uptime through fallback processing
  • Scalability: Easy addition of new features through modular design

Future Prospects

  • gpt-oss 120B model integration for improved accuracy
  • Matter standard support for expanded device compatibility
  • Voice recognition feature addition for completely hands-free operation
  • Community features for sharing automation rules

Through this project, we experienced the potential brought by AI and IoT integration and created a future of smart homes that anyone can easily use. We learned that overcoming technical challenges enables us to provide better user experiences.

Built With

Share this project:

Updates