Smart-Energy-Copilot-v2.0-Multimodal-Edge-AI-Dashboard

Smart Energy Copilot: multimodal AI dashboard using PaddleOCR-VL, ERNIE+Unsloth, CAMEL-AI agents, deployed on RDK X5 for private edge insights with sleek ERNIE web visualization.

Comment

Inspiration

Energy bills and IoT logs are often messy, fragmented, and hard to interpret. We wanted to empower households and enterprises with a private, edge‑deployed AI dashboard that makes energy usage transparent, actionable, and visually engaging—without relying on cloud latency or exposing sensitive data.

What it does

  • Parses bills and IoT logs with PaddleOCR-VL for accurate multimodal text extraction.
  • Interprets consumption patterns using ERNIE fine‑tuned via Unsloth, uncovering inefficiencies and trends.
  • Employs CAMEL-AI agents to generate actionable insights, such as cost‑saving recommendations or anomaly alerts.
  • Runs on RDK X5 for low‑latency, private edge deployment.
  • Presents results in a sleek, interactive ERNIE‑built web visualization.

How we built it

  • Integrated PaddleOCR-VL for multimodal parsing of structured and unstructured energy data.
  • Fine‑tuned ERNIE models with Unsloth for domain‑specific energy pattern recognition.
  • Designed CAMEL-AI agent workflows to simulate collaborative reasoning and generate practical recommendations.
  • Deployed the system on RDK X5 hardware to ensure edge privacy and responsiveness.
  • Built a modern web dashboard powered by ERNIE’s visualization capabilities for intuitive user interaction.

Challenges we ran into

  • Ensuring OCR accuracy across diverse bill formats and noisy IoT logs.
  • Balancing model complexity with edge hardware constraints.
  • Navigating IP rights and compliance for AI model fine‑tuning.
  • Achieving seamless integration between multimodal parsing, reasoning agents, and visualization layers.

Accomplishments that we're proud of

  • Delivering a fully functional multimodal AI dashboard on edge hardware.
  • Achieving real‑time insights without cloud dependency.
  • Creating a unified workflow that bridges OCR, fine‑tuned models, and agent‑based reasoning.
  • Designing a sleek, user‑friendly visualization that makes energy data approachable.

What we learned

  • Multimodal AI pipelines require careful orchestration to balance accuracy and efficiency.
  • Edge deployment demands optimization strategies that differ from cloud setups.
  • Collaborative agent frameworks (like CAMEL-AI) can transform raw data into meaningful, actionable narratives.
  • Visualization is not just a “front‑end”—it’s critical for user trust and adoption.

What’s next for Smart-Energy-Copilot-v2.0-Multimodal-Edge-AI-Dashboard

  • Expand support for more IoT devices and regional bill formats.
  • Integrate predictive analytics for forecasting energy costs and usage.
  • Add gamified features to encourage sustainable energy habits.
  • Explore partnerships with smart home platforms for seamless integration.
  • Optimize further for ultra‑low power edge hardware to broaden accessibility.

Built With

  • camel-ai-agents
  • ernie
  • ernie-(fine?tuned-via-unsloth)
  • paddleocr-vl
  • rdk-x5
  • visualization
  • web
Share this project:

Updates