NDAi | Devpost

Our agentic system at work

Inspiration

Small language models are becoming a more desired industry as AI has become more prevalent in our world. Combating the growing concerns over privacy and ethics in data collection, SLMs provide a balance between function and security. From their natural privacy, to the complete circumvention of the need for a data center, SLMs hold a very promising future.

What it does

NDAi is an orchestrated system of agents that can build any lightweight, privacy-focused AI applications that strictly use local language models for their core functionality. Acting like an end-to-end software development team, the agents are designated with specialized roles such as a planner and verifier in order to successfully produce the secure local language model.

How we built it

We constructed a multi-agent collaborated system of agents with designated roles. By paring Gemini as a strategic engine with Codex as our main execution agent, we were able to seamlessly allow Codex to accurately generate and iterate through our codebase.

Challenges we ran into

Since our project's outputs hinged on our team of AI agents designing and implementing our small language model, it was particularly difficult to debug our output applications traditionally. This fundamental limitation of vibe-coding caused us to spend a majority of our time manually reverse engineering the generated codebase in order to isolate the faulty prompts. Because there are currently no standard developer tools to address this particular issue, we had to rely on our own intuition and rigorous manual testing. This demonstrated a large trade-off when utilizing AI agent workflows, although they give you great initial results, they make complex debugging incredibly difficult for definitive solutions.

Accomplishments that we're proud of

Our product has successfully created SLM based apps which is exactly what we sought to do. Although, we are the most proud of the idea by far. While simply creating an SLM would by itself be a useful and notable tool by itself, the use of a system to generate the SLM allowed us to bypass one of the biggest problems with SLMs, that being their limited scope and capabilities, since they don't have the same databases to rely on compared to LLMs.

What we learned

Our biggest takeaway from this hackathon was the shift in mindset from traditional code writers to prompt engineers and systems architects. Because we had little say in the agent's workflow, the prompts themselves became our source of code. When bugs emerged, it wasn't a simple one line switch, and instead understanding and reverse engineering how our LLMs interpreted our prompts and how we structured those inputs. We learned to be rigorous with our context and define strict boundary rules in order to get the deterministic SLM results we wanted. We came out realizing that in the future of software development, engineering the prompt is just as critical as engineering the logic itself.

What's next for NDAi

We plan to implement support for other specialized SLMs in order to have further applications in more specialized areas. With more dedicated funding, we could also expand and enhance our system of agents in order to optimize, implement and polish the outputted SLM that our agent system generates, while maintaining the same data integrity of the resulting local run model. We intend to develop an internal tooling framework that would map and trace our prompts, which would visually link every generated code block back to its original prompt in order to eliminate the tedious back-searching we faced during the hackathon.