Bastion

Inspiration

We don’t have a standard way to evaluate and measure the security risk of an AI agent’s actions over long contexts. \ To measure security risks in an agent, we have to look at the whole context of an agent's actions and inputs \ New prompt-injection attacks and security risks for agents appear all the time. \ So users shouldn’t have to rely on the security of each specific model and agent they use.

What it does

We built a standardised way for users to measure and track the risk of their agents over time. \ Bastion works by abstracting the agent’s input/output tokens. It represents the agent as a linear sequence of actions over time. (for example, one action might be the agent using the search-tool or editing a certain file, etc.). \ This lets us analyse the flow of the agent at different levels of abstraction so we can intelligently see trends in its actions over time.

How we built it

Monolithic Python repository Custom-built toxicity model built with Takara AI's DS1 embedding, runs concurrently with chosen LLM \ MLP Classifier \ Grounding to ensure that the agent doesn't act on misinformation \ Crusoe's Managed Inference API is used for inference \ Tool classifier \ Front-end dashboard built with Lovable in TypeScript

How to locally run the project

Clone the given repository and run the following in terminal: \ npm run dev

The backend is hosted by ourselves and is called via our custom API to this program. The backend logic can still be analysed in the antislopfactory repository.

Challenges we ran into, and solved

Ensuring that the risk is appropriately tuned was an issue. Our risk Making sure that the conversations streamed in 'real-time'

Accomplishments that we're proud of

Significantly improved prompt injection resilience Runs directly on CPU with reasonable performance

What we learned

Became more proficient with semantics and developing models in Python How to engineer prompts to gain the desired outputs from Lovable, and be able to apply this to other AI tools.

What's next for Bastion

Implement execution limits so that users can ensure agent is stopped before it carries out anything deemed too risky \ Implement reprompting to allow users to change prompts if answers given result in a high risk score

Built With

• crusoe
• lovable
• python
• react
• typescript