ChangePilot

Evidence-driven change management with AI-assisted release risk assessment

Comment

Inspiration

While working on the project, I wanted to build something more practical than another basic CRUD application. I was interested in how AI could be used in a real software-development workflow.

Software changes can sometimes cause unexpected problems, and reviewing a change usually requires looking at different pieces of information before making a decision. I thought it would be useful to have a system that brings the change request and its supporting evidence together and uses AI to give the reviewer an initial risk assessment.

That idea became ChangePilot.

I also wanted to keep the final decision with a human instead of letting AI automatically approve or reject a change.

What it does

ChangePilot is a backend platform for managing software change requests and assessing their risk.

A change follows this workflow:

Draft → Under Review → Approved / Rejected → Closed

Users can create change requests, add supporting evidence, submit them for review, and run an AI analysis.

The AI returns things like:

  • Risk score and risk level
  • Confidence
  • Risk factors
  • Affected areas
  • Missing evidence
  • Recommendation

The AI only provides an assessment. It cannot approve, reject, submit, or close a change.

The system also includes authentication, roles, ownership checks, and audit events so that actions are controlled and traceable.

How we built it

I built the project mainly as a REST API using Node.js and Express.

For the database, I used PostgreSQL with Prisma. Authentication is handled using JWT, and different roles have different permissions.

For the AI part, I created a provider layer so that the application is not tightly connected to one AI service. There is a deterministic fake provider for development and testing, along with an OpenAI provider foundation.

One part I spent a lot of time on was connecting evidence to the AI analysis. The AI receives the change request and its attached evidence, but the evidence is treated as untrusted input. It should provide context, not instructions.

I also added validation, security headers, CORS handling, error handling, transactions, audit events, and tests for important security cases.

Challenges we ran into

One of the hardest parts was deciding how much control the AI should have.

It would have been easy to make the AI automatically approve or reject changes, but I didn't want to build the system that way. I wanted the AI to assist the reviewer rather than replace the reviewer.

Another challenge was handling evidence safely. Since evidence can contain arbitrary user-provided text, I had to make sure it couldn't simply be treated as an instruction to the AI.

During security testing, I also found an ownership issue where a requester could potentially access another user's change request. I fixed that and added regression tests for it.

There were also plenty of smaller issues around database testing, authentication, API consistency, and getting all the different parts of the backend to work together.

Accomplishments that we're proud of

I'm most proud that the project became more than a basic CRUD API.

Some of the things I was able to implement are:

  • Complete change-request lifecycle
  • JWT authentication and role-based access
  • Ownership checks and IDOR protection
  • Evidence management
  • AI-based risk analysis
  • AI provider abstraction
  • Audit events
  • Structured AI analysis storage
  • Input validation and error handling
  • Security-focused tests
  • PostgreSQL persistence
  • GitHub Actions CI

I also liked the idea of keeping AI advisory throughout the system. The reviewer still makes the actual decision.

What we learned

The biggest thing I learned is that building an AI feature is not just about sending a prompt to an API.

A lot of the work is around deciding what information the AI should receive, how to handle untrusted input, how to structure its response, and what the AI should not be allowed to do.

I also learned much more about backend security while working on authentication, authorization, ownership checks, and testing cases where users try to access resources they don't own.

This project also gave me a better understanding of how different backend pieces—API routes, services, database models, authentication, AI, and tests—fit together in a larger application.

What's next for ChangePilot

The current submission mainly focuses on the backend and API.

The next major step is building a proper web interface around it so that someone can use ChangePilot without interacting directly with the API.

After that, I'd like to deploy it, connect it with development workflows such as GitHub/CI, add more evidence-management features, and improve the AI analysis with more real-world scenarios.

The longer-term idea is to make ChangePilot useful during the actual software release process rather than just as a demonstration project.

Built With

Share this project:

Updates

posted an update —

AI is advisory — not autonomous const analysis = await aiProvider.analyze({ changeRequest, evidence, });

// AI produces an assessment only. // It never changes the ChangeRequest lifecycle. return saveAIAnalysis({ changeRequestId: changeRequest.id, ...analysis, }); Caption: AI evaluates release risk, but lifecycle decisions remain under human control.

Log in or sign up for Devpost to join the conversation.

posted an update —

IDOR / ownership protection const change = await prisma.changeRequest.findUnique({ where: { id }, });

if (!change) { throw new AppError("Change request not found", 404); }

if ( user.role === "REQUESTER" && change.requesterId !== user.id ) { throw new AppError("Forbidden", 403); }

Caption: Requesters can only access their own change requests, preventing unauthorized object access.

Log in or sign up for Devpost to join the conversation.

Submission history