coralflow

Inspiration

Machine learning on edge devices often breaks at the last mile: models are trained in the cloud or on a dev machine, but deployment, monitoring, and retraining live in separate tools and teams. For resource-constrained teams, GPUs aren't always available, and model drift is hard to catch in time.

CoralFlow started from a simple idea: collapse the full TinyML lifecycle into one CLI pipeline—from a CSV dataset to a TFLite model on an edge device, plus drift detection and auto-retraining, mostly on ordinary CPUs. An LLM agent lowers the barrier so developers can drive the whole workflow in natural language.

What it does

CoralFlow is an AI-driven TinyML continuous training platform. It provides:

• Train — Local TensorFlow/Keras text classification (no GPU), or Vertex AI via --cloud (AutoML, Gemini fine-tuning)
• Validate — SavedModel → TFLite conversion with size (<10MB) and latency checks
• Deploy — Push .tflite to edge gateways over HTTP (/health, /api/v1/model, /api/v1/checksum)
• Predict & monitor — Local inference, Arize Phoenix OTEL tracing, prediction logs (JSONL)
• Auto-retrain — Add ground_truth to logs, then monitor --retrain triggers retraining
• LLM agent — coralflow agent REPL orchestrates train / validate / deploy / predict via function calling
• One-command pipeline — coralflow-edge-pipeline ./data/urgent.csv runs train → validate → deploy

Built-in datasets (urgent for message urgency, expense for spending categories) and a demo retrain-loop show drift and recovery end to end.

How we built it

Architecture: Python 3.10+ monorepo (edge_train package) with Click as the unified CLI entry point (coralflow).

Dev UX includes scripts/dev for automatic venv setup, make lint, and the coralflow-edge-pipeline script for the full edge workflow.

Challenges we ran into

1. No standard edge deploy protocol — Most TFLite REST servers only do inference, not model push. We defined a CoralFlow HTTP contract and shipped a Flask reference gateway.

2. TextVectorization vs TFLite — Local inference uses SavedModel (vectorization in-graph); edge deploy uses TFLite. Validate had to handle INT32 inputs correctly for latency estimation.

3. Disconnected train and deploy — train doesn't auto-deploy. We wired steps together with coralflow-edge-pipeline and the agent toolchain, and moved device config from a JSON registry to a .env array.

4. Closing the retrain loop — Prediction logs, ground-truth labeling, accuracy thresholds, and retrain triggers had to work together; Phoenix visualizes drift in production, not just offline metrics.

5. Dual cloud/local paths — Vertex AI routing for text/image/table/video plus local Keras meant many env vars; we centralized config via load_config() and .env.

Accomplishments that we're proud of

• Zero-GPU local training — Full train → validate → deploy on WSL2 / a laptop
• Agent that actually runs tools — Function calling executes real CLI operations, not chat-only demos
• One-command edge pipeline — coralflow-edge-pipeline ./data/urgent.csv from CSV to TFLite on device
• Observable retrain demo — demo retrain-loop shows baseline failure → label → retrain → accuracy recovery
• Open and extensible — MIT license; edge transport extensible to MQTT/SSH/BLE; cloud path supports Vertex endpoint inference
• Solid test coverage — Gateway endpoints, .env device parsing, and pipeline integration tests in CI

What we learned

• TinyML's value is in ops, not training — Models are small and train fast; the hard part is deployment, versioning, drift, and automated retraining.
• LLMs are great for orchestration, not replacement — The agent's job is intent, command choice, and parameters; deterministic CLI still does the work.
• Edge needs push + verify — Checksum validation makes OTA updates auditable, better than raw scp/FTP.
• Observability belongs in the predict path — Phoenix spans plus prediction logs connect live behavior to "when to retrain."
• Simpler config wins — Moving from a JSON registry to EDGE_DEVICES in .env cut friction for multi-device setups.

What's next for CoralFlow

• Edge inference API — Extend the gateway with /predict so devices run TFLite locally, not just receive models
• More transports — MQTT, SSH, BLE for industrial and IoT deployments
• Auto-deploy hooks — Optional validate + deploy after train (train --auto-deploy)
• Multimodal edge — TFLite deploy and validation for image/table models
• Richer agent — Deeper dataset QA, cloud vs local routing advice, pre-deploy resource checks
• Fleet management — Status rollup and rollback for batch -d all deploys
• Docs & examples — Pi/Jetson one-click guides and Docker images for the edge gateway

Built With

• aiohttp
• arize-phoenix-otel
• click
• csv/json-storage
• flask
• gcp-vertex-ai
• http-edge-deploy
• openai-compatible-llm-api
• python
• tensorflow/tflite