Edge Computing LLM

I have built an open, Go-first platform for running and observing private language models on resource-constrained Linux edge systems. The current reference path combines Ubuntu, single-node k3s, NVIDIA GPU acceleration, Ollama/GGUF models, Open WebUI, OpenTelemetry, Prometheus, and Grafana—without treating a laptop or small server like an unlimited cloud cluster.

Built With

• gguf
• grafana
• helm
• k3s
• linux
• nvidia
• ollama
• openwebui