DDAG — Dynamic Database API Gateway

Turn PostgreSQL, MySQL, Oracle, and SQL Server into secure, governed REST APIs in minutes—with OAuth2, RBAC, caching, audit trails, observability, and automatic OpenAPI documentation.

Comment

Inspiration

Many teams need to expose database data to internal applications, automation tools, partners, and reporting systems. However, even a simple database query often requires a dedicated backend service, authentication logic, authorization rules, logging, documentation, deployment, and ongoing maintenance.

This creates repeated engineering work. Teams end up building many small services that solve nearly the same problem, while database credentials, SQL queries, access policies, and audit records become scattered across different applications.

We created DDAG — Dynamic Database API Gateway to provide a consistent and governed way to publish database operations as secure APIs, without building and maintaining a separate backend service for every endpoint.

What it does

DDAG is a self-hosted database API gateway that turns parameterized SQL into managed REST APIs.

Administrators can:

  • Register database connections
  • Define HTTP routes and methods
  • Write parameterized SQL templates
  • Configure typed request parameters
  • Apply scopes, roles, client grants, IP allowlists, and rate limits
  • Validate and publish an API
  • Generate OpenAPI documentation automatically
  • Monitor requests, metrics, and audit records from a web dashboard

DDAG supports PostgreSQL as its primary end-to-end path and provides a shared connector architecture for MySQL/MariaDB, Oracle, and SQL Server.

The gateway handles the common infrastructure surrounding database-backed APIs, including:

  • OAuth 2.0 client credentials
  • RS256 JWT validation
  • Role-based access control
  • Query timeouts and row limits
  • Safe parameter binding
  • Read-response caching
  • Connection pooling and circuit breakers
  • Structured logs and request IDs
  • Audit trails
  • Prometheus metrics
  • OpenAPI 3 and Swagger UI

Instead of giving applications direct access to a database, DDAG provides a controlled API boundary where every operation can be validated, authorized, monitored, and audited.

How we built it

We designed DDAG using separate control-plane and data-plane responsibilities.

The control plane manages database connections, API definitions, access policies, clients, scopes, publishing workflows, and operational configuration through an administrative dashboard.

The data plane receives API requests, validates access tokens, matches published routes, evaluates policies, safely binds parameters, executes queries through the appropriate database connector, and returns structured responses.

The backend is written in Go to provide strong concurrency, predictable deployment, and efficient request processing. The administration dashboard is built with Vue 3 and Vite.

PostgreSQL stores DDAG's metadata and configuration, while Redis supports caching and runtime coordination. The project is containerized and can be launched locally using Docker Compose. Deployment resources are also provided for server and container-based environments.

To reduce database-specific logic in the gateway, we created a shared connector architecture. Each connector translates named parameters into the placeholder format required by its database driver while keeping untrusted values separate from the SQL template.

We also implemented a complete API lifecycle:

  1. Create an API definition
  2. Configure its SQL and parameters
  3. Validate the definition
  4. Apply security and operational policies
  5. Publish the route
  6. Access it using OAuth credentials
  7. Observe its behavior through logs, metrics, and audit records
  8. Deprecate or retire it when it is no longer needed

Challenges we faced

One of the biggest challenges was supporting multiple SQL dialects without sacrificing query safety. PostgreSQL, MySQL, Oracle, and SQL Server use different drivers, placeholder formats, and database behaviors. We needed a connector boundary that could normalize these differences while still using bound parameters rather than unsafe string interpolation.

Another challenge was designing a platform that is flexible without allowing unrestricted database access. DDAG must let administrators publish useful queries while enforcing limits around operations, execution time, returned rows, client permissions, and network access.

Caching also required careful handling. Read operations can benefit from caching, but write operations must never be treated as safely cacheable or automatically retried. The gateway therefore considers the operation type before applying caching and reliability behavior.

Security was another major area of work. We implemented encrypted secrets, JWT verification, OAuth scopes, role-based permissions, client grants, internal service authentication, replay protection, rate limiting, and audit records. These controls needed to work together without making the API publishing process unnecessarily complicated.

Finally, observability had to be built into the platform rather than added as an afterthought. Request IDs, structured logs, metrics, and audit events must remain correlated across the gateway, connector, cache, and source database execution path.

Accomplishments that we're proud of

We are proud that DDAG brings the full lifecycle of a database-backed API into one platform.

A user can go from a database connection and SQL query to a secured, documented, observable API without creating an entirely new backend project.

We are also proud of:

  • The separation between control-plane management and data-plane execution
  • The shared multi-database connector architecture
  • Safe typed parameter binding
  • Fine-grained access policies
  • Automatic OpenAPI generation
  • Built-in caching and reliability controls
  • Detailed request and audit visibility
  • A self-hosted deployment model
  • A dashboard that makes API governance accessible without editing configuration files manually

What we learned

We learned that converting SQL into an HTTP endpoint is the easy part. The real challenge is everything required to operate that endpoint safely: identity, authorization, query limits, secret management, lifecycle controls, monitoring, documentation, and recovery procedures.

We also learned that multi-database support should be designed around explicit connector boundaries. Trying to hide all database differences inside a single execution layer quickly becomes difficult to test and maintain.

Another important lesson was that governance should be part of the publishing workflow. Security and operational policies are more effective when they are configured before an API becomes publicly available, rather than added later.

Finally, we learned that a platform like DDAG should complement application services rather than replace them. Simple and governed database operations are a strong fit, while complex business workflows and multi-step transactions may still belong in dedicated services.

What's next for DDAG

Our next priorities are:

  • Expanding live integration testing for MySQL, Oracle, and SQL Server
  • Improving policy simulation before an API is published
  • Adding richer usage analytics and performance insights
  • Supporting safer versioning and rollback of API definitions
  • Expanding automated security and compatibility testing
  • Improving developer onboarding and guided API creation
  • Adding more deployment and high-availability options
  • Building a broader ecosystem around reusable connectors and policies

Our long-term goal is to make DDAG a practical, open, and self-hosted foundation for teams that need to expose database capabilities without repeatedly rebuilding the same API infrastructure.

Built With

Share this project:

Updates