Echolocate

Inspiration

Every developer has run into the same problem: you need to integrate with an internal service or a third-party partner API, but the documentation is missing, outdated, or completely wrong. Developers typically waste hours running manual curl commands and parsing cryptic error responses. When you ask standard AI assistants for help, they guess parameters, assume schemas, and generate code that fails at runtime. We built Echolocate to test a different approach: what if an AI agent did not guess, but empirically explored? By sending real HTTP probes into an unknown API and analyzing the responses, Echolocate maps the target service and writes integration code that is verified against the live endpoint.

What It Does

Given only an API base URL, Echolocate autonomously figures out how to use it. It runs a grounded Reason, Act, and Observe loop where it formulates hypotheses, executes live HTTP probes in a network sandbox, and inspects real status codes and response bodies. Once it understands the endpoints, it generates a production-ready Python SDK with typed Pydantic models. It runs the code against the target in an isolated subprocess, and if tests fail, it feeds the traceback into an autonomous repair loop that patches the code until verified. It also runs active boundary fuzzing to discover required fields and numeric constraints, audits discovered routes for common OWASP security vulnerabilities like broken authorization or missing rate limits, and exports both an OpenAPI 3.1 specification and a standalone FastAPI mock server for offline testing. A web dashboard provides live streaming logs, token cost metrics, and an interactive request playground.

How We Built It

Echolocate is written in Python using FastAPI, httpx, and Pydantic, running on Nebius Token Factory with the NVIDIA Nemotron 3 model family. Instead of running a large, expensive model for every step, we split the workflow across three tiers. Nemotron 3 Nano handles high-frequency probing and next-step hypotheses because it is fast and lightweight. Nemotron 3 Super interprets error messages, deduces authentication tokens, and diagnoses compiler tracebacks during code repair. Nemotron 3 Ultra runs once at the end to synthesize the final typed Python SDK.

By delegating repetitive probing tasks to Nano and reserving Ultra for final code generation, total exploration cost stays low:

$$C_{\text{total}} = N_{\text{probe}} \cdot C_{\text{nano}} + N_{\text{repair}} \cdot C_{\text{super}} + 1 \cdot C_{\text{ultra}}$$

Compared to running a large frontier model across all iterations, this tiered architecture reduces exploration costs by over 85 percent:

$$\text{Savings} = 1 - \frac{N_{\text{probe}} \cdot C_{\text{nano}} + N_{\text{repair}} \cdot C_{\text{super}} + C_{\text{ultra}}}{N_{\text{total}} \cdot C_{\text{ultra}}} \approx 88\%$$

A complete multi-step exploration run on a live service finishes in under 60 seconds for less than two cents.

Challenges We Faced

Early prompts caused models to assume standard REST conventions, such as guessing standard path layouts. We resolved this by strictly grounding the agent state in observed HTTP responses: if a behavior is not backed by an actual status code and payload, the agent cannot record it as fact. Another challenge was subprocess code verification, as generated SDKs often failed on their first run due to minor syntax or import issues. We solved this with a reflection loop where stderr logs are sent back to Nemotron 3 Super to analyze the error and patch the code until tests pass. Finally, we wanted to avoid the generic look of many AI dashboards, so we rebuilt the web interface with clean typography, vector icons, and a unified telemetry strip inspired by modern developer platforms like Linear.

Accomplishments

Every generated function, header, and Pydantic field is verified against real HTTP responses, eliminating hallucinated methods. We successfully mapped real endpoints, including the public GitHub API, converging on working code in under a minute. The project maintains a solid test suite with 50 out of 50 tests passing in pytest across all agent, sandbox, and dashboard modules, and it can generate a zero-dependency FastAPI replica server in seconds.

What We Learned

We learned that model tiering works remarkably well. Smaller models like Nemotron 3 Nano handle structured exploration tasks quickly and cheaply when given concise system prompts and clear tools. We also saw that execution feedback beats single-shot prompting: giving an LLM runtime feedback from a compiler produces far more reliable code than trying to generate a perfect script in a single prompt.

What's Next

Next, we plan to expand probe support to GraphQL and gRPC endpoints. We also want to add automatic pull request generation that pushes the SDK and unit tests straight to GitHub, as well as background drift detection to alert engineering teams whenever an undocumented internal API changes.

Built With

• ai-agents
• api-security
• autonomous-agents
• fastapi
• fuzzing
• httpx
• javascript
• nebius
• nemotron
• nemotron-3
• nvidia
• openai-api
• openapi
• owasp
• pydantic
• pytest
• python
• react-framework
• rest-api
• rich
• sse
• typer
• uv
• uvicorn