pondok-ai

Formerly published as local-ai-stack. Pondok is Indonesian for a small lodge or shelter.

English · Bahasa Indonesia · 한국어 · العربية · Español · 中文 · Français · 日本語 · Deutsch · Português · Русский · हिन्दी · Türkçe · Italiano · Tiếng Việt

Translations are machine-generated and may lag behind the English README.

Run your own private AI stack locally with one command.

Quickstart • Features • Presets • CLI Commands • API Gateway • Examples • Documentation

Overview

pondok-ai is a developer-friendly, zero-friction CLI and container orchestration toolkit that boots a complete, private AI environment directly on your local workstation.

Powered by Docker Compose v2, pondok-ai gives you local LLMs, chat UI, vector search, database persistence, and an OpenAI-compatible API gateway in seconds. Prompts, chats, and embeddings are processed on your machine; pulling container images and model weights requires an internet connection.

npx pondok-ai init
npx pondok-ai up

Features

Local Processing: Prompts, chats, embeddings, and inference run on your hardware. Initial container image pulls and model downloads require internet access.
⚡ Zero Setup Friction: Automatically detects OS, architecture, RAM, and GPU (Apple Metal / NVIDIA CUDA).
Modular Presets: Scale from a lightweight chat interface (minimal) to a RAG stack (standard) or an autonomous automation pipeline (full).
OpenAI-Compatible Gateway: Point standard OpenAI SDKs or LangChain directly to http://localhost:8000/v1 with zero code rewrites.
Built-in Model Manager: Pull, list, inspect, and remove models with built-in RAM/disk safety warnings.
Built-in Diagnostics: Built-in doctor command diagnoses port collisions, Docker daemon status, and resource limits before you run into issues.

Architecture

┌───────────────────────────────────────────────────────────────┐
│                      Host Machine (CLI)                       │
│                                                               │
│   Port 3000                 Port 8000           Port 5678     │
│       │                         │                   │         │
└───────┼─────────────────────────┼───────────────────┼─────────┘
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Open      │             │ API       │       │ n8n       │
  │ WebUI     │             │ Gateway   │       │ (Full)    │
  └─────┬─────┘             └─────┬─────┘       └─────┬─────┘
        │                         │                   │
        ├─────────────────────────┼───────────────────┤
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Ollama    │             │ Qdrant    │       │ Postgres  │
  │ (LLMs)    │             │ (Vector)  │       │ (State)   │
  └───────────┘             └───────────┘       └───────────┘

Presets

Preset	Services Included	Min RAM	Recommended For
`minimal`	Ollama, Open WebUI	8 GB	Quick local chat, lightweight experimentation
`standard` (default)	Ollama, Open WebUI, PostgreSQL, Qdrant, API Gateway	16 GB	RAG pipelines, app development, OpenAI replacement
`full`	Standard + n8n automation engine	16+ GB	Autonomous AI agents, visual workflow automation

Quickstart

Quickstart via npx

Prerequisites: Node.js 22 or later, Docker, and Docker Compose v2.

1. Initialize the stack

# Interactive setup wizard with hardware auto-detection
npx pondok-ai init

# Or non-interactive with a chosen preset
npx pondok-ai init --preset standard --yes

2. Start services with automated health checks

npx pondok-ai up

Once running, access your stack:

Open WebUI: http://localhost:3000
API Gateway: http://localhost:8000
Ollama: http://localhost:11434
Qdrant: http://localhost:6333/dashboard (when mapped)

3. Smart Hardware Detection & Model Recommendations

# Get smart hardware-aware recommendations based on your RAM/VRAM
npx pondok-ai model recommend

# Install model with pre-flight memory and disk safety check
npx pondok-ai model install llama3.2:3b

Installer scripts (experimental, not covered by CI)

macOS & Linux

curl -fsSL https://raw.githubusercontent.com/rasidi3112/pondok-ai/main/scripts/install.sh | bash

Windows (PowerShell)

iwr -useb https://raw.githubusercontent.com/rasidi3112/pondok-ai/main/scripts/install.ps1 | iex

CLI Commands

Command	Description
`pondok-ai init`	Run interactive configuration wizard with hardware detection
`pondok-ai up`	Validate Compose config, boot containers, and wait for health checks
`pondok-ai down`	Stop and remove stack containers
`pondok-ai start`	Start existing stopped containers
`pondok-ai stop`	Stop running containers
`pondok-ai restart`	Restart stack containers
`pondok-ai status`	Show container health and URLs
`pondok-ai logs [service]`	View logs (`-f` to stream)
`pondok-ai doctor`	Diagnose Docker, RAM, GPU/VRAM, ports, and model recommendations
`pondok-ai update`	Re-pull configured pinned images with automated backup and health checks
`pondok-ai backup`	Backup configs and data volumes with SHA-256 integrity checks (not encrypted)
`pondok-ai restore <path>`	Restore from a SHA-256 integrity-checked backup (not encrypted)
`pondok-ai reset`	Reset stack (preserves volumes unless `--delete-data` is specified)
`pondok-ai config`	Inspect or edit active configuration
`pondok-ai model list`	List downloaded models
`pondok-ai model recommend`	Hardware-tailored model recommendations (RAM, VRAM, Disk)
`pondok-ai model install <name>`	Pull a new model with pre-flight resource safety checks
`pondok-ai model remove <name>`	Delete an installed model
`pondok-ai model status`	Inspect model resource footprints

Images are pinned to explicit versions. pondok-ai update refreshes the configured versions; it does not select newer releases. After installing a newer CLI, run pondok-ai update --upgrade-images to migrate only the CLI's legacy default image tags. Custom image values are preserved.

API Gateway

The API Gateway provides drop-in OpenAI REST endpoints:

Set PONDOK_AI_API_KEY to the generated API_SECRET_KEY before calling a /v1/* endpoint.

The gateway limits authenticated /v1/* requests per client (source IP and API key) to 120 requests per minute by default. Configure RATE_LIMIT_MAX_REQUESTS and RATE_LIMIT_WINDOW_MS in .env if your local client needs different limits.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PONDOK_AI_API_KEY" \
  -d '{
    "model": "llama3.2:3b",
    "messages": [
      { "role": "user", "content": "Hello from pondok-ai!" }
    ]
  }'

Using with the Official OpenAI Node.js SDK

import OpenAI from 'openai';

const apiKey = process.env.PONDOK_AI_API_KEY;
if (!apiKey) {
  throw new Error('Set PONDOK_AI_API_KEY to your generated API_SECRET_KEY.');
}

const openai = new OpenAI({
  baseURL: 'http://localhost:8000/v1',
  apiKey,
});

const completion = await openai.chat.completions.create({
  model: 'llama3.2:3b',
  messages: [{ role: 'user', content: 'Explain quantum computing in one sentence.' }],
});

console.log(completion.choices[0].message.content);

Examples

examples/openai-compat/: Drop-in OpenAI SDK integration script.
examples/rag-example/: End-to-end RAG pipeline using Qdrant vector database and Ollama embeddings.

Tested in CI

Every pull request runs an end-to-end test on GitHub Actions (amd64 and arm64): init, up, pull a very small model (smollm:135m, about 90 MB), list it via /v1/models, a non-streaming /v1/chat/completions request, rate limiting (401 and 429), and a check that Open WebUI's start page responds. Not covered yet: streaming responses, embeddings, GPU acceleration, and macOS/Windows hosts.

Known limitations

Not tested on real macOS or Windows hosts yet. On Apple Silicon, Docker containers cannot use the GPU and pondok-ai runs Ollama inside Docker, so inference there is CPU-only.
GPU acceleration is not covered by CI.
Streaming responses and embeddings are not covered by CI yet.
The images are large (several GB). Plan for substantial free disk space; a 32 GB cloud workspace with 19 GB free ran out of space while extracting the Ollama image.

Documentation

Contributing

Contributions are welcomed! Check out CONTRIBUTING.md to get started.

License

Built With

dockerfile
javascript
powershell
shell
typescript

Updates

Ahmad Rasidi started this project — Oct 12, 2026 06:32 PM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.