ShadowWork – AI Workflow Discovery Platform

Inspiration

A lot of employees do the tasks every day. They handle customer tickets check orders verify policies and process refunds. These tasks are done manually even when experienced employees follow the process.

ShadowWork was inspired by the idea of a system that can watch how an employee does a task understand the workflow and turn that workflow into something that can be reused.

We were inspired by the idea of an AI system that can observe how an employee performs a task understand the workflow of the employee and turn that workflow of the employee into a skill.

This idea led us to build ShadowWork – AI Workflow Discovery Platform.

What it does

ShadowWork. Records what an employee does during a workflow.

For example in our customer refund workflow the employee can do these things:

1. Open a customer ticket

2. Check the customer order

3. Check the refund policy

4. Process the refund

ShadowWork records these actions of the employee. Analyzes them to find the workflow pattern of the employee.

It then generates a skill from the observed workflow of the employee.

The learned skill can be executed automatically for another ticket or order.

The system also shows decision-based execution: refunds within the approval limit can be approved automatically while refunds above the limit are sent for manager approval.

The core idea is:

Employee performs task → ShadowWork observes the task of the employee → Workflow of the employee is analyzed → Reusable skill is generated → Skill executes automatically.

How we built it

We built the prototype using a React frontend and a Python FastAPI backend.

The frontend provides an employee workspace where users can do customer-support actions such as opening tickets, checking orders checking refund policies and processing refunds.

The backend records each action as a workflow event of the employee.

A workflow analyzer processes these events. Identifies the workflow type and reusable steps of the employee.

We then created a skill executor that takes the generated workflow skill and executes its steps automatically using the provided ticket order and refund information of the employee.

The prototype also includes a simulated approval decision.

A refund of ₹5000 or less can be automatically approved, while a refund above ₹5000 requires manager approval.

Challenges we ran into

One of the challenges was connecting the employee actions, workflow recording, workflow analysis and automatic skill execution into a single flow of the employee.

We also had to make sure that the generated skill was not just displayed as a result but could actually be executed with new ticket IDs order IDs and refund amounts of the employee.

Another challenge was handling refund scenarios, especially distinguishing between automatically approved refunds and refunds that require manager approval of the employee.

Accomplishments that we're proud of

We successfully built a working prototype that demonstrates the workflow-learning concept of ShadowWork.

The system can observe employee actions record them analyze the workflow of the employee generate a skill and execute that skill automatically.

We are particularly proud of demonstrating that the same learned workflow of the employee can be applied to input values instead of requiring the employee to manually repeat every step of the workflow.

The manager approval condition also demonstrates how automation can be combined with oversight instead of completely removing humans from the process of the employee.

What we learned

Through this project we learned how workflow observation of the employee can be converted into data and then transformed into a reusable automation of ShadowWork.

We also learned how frontend actions can communicate with a backend API, how workflow events can be recorded and analyzed and how a generated skill can be executed programmatically of the employee.

Importantly we learned that useful AI automation is not only about performing actions automatically.

It should also understand when human approval is required and keep humans in control of decisions of the employee.

What's next for ShadowWork – AI Workflow Discovery Platform

The current prototype demonstrates the core concept using a customer-support environment of ShadowWork.

Our next step is to connect ShadowWork with enterprise applications through tools and APIs of ShadowWork.

Of using simulated ticket order and policy data the platform could interact with systems such, as CRM, helpdesk order-management and knowledge-base applications of ShadowWork.

We also plan to improve workflow understanding so that ShadowWork can discover complex workflows handle different workflow variations and generate more reliable reusable skills of ShadowWork.

The long-term vision is to create an AI workflow discovery platform where employees demonstrate how work is done once ShadowWork learns the process. Approved AI skills can safely automate repetitive work while keeping managers involved when human approval is required of ShadowWork.

Built With

• ai
• css
• fastapi
• javascript
• python
• react
• rest-api
• workflow
• workflow-automation