Inspiration

The growing need for data privacy in corporate environments highlighted a significant problem: relying on external cloud APIs to analyze sensitive business data is often a security risk. We were inspired to build a powerful, fully on-premise solution that could handle complex reasoning for real-time risk assessment without data ever leaving the local network.

What it does

CC is an enterprise risk warning application that continuously monitors and analyzes business data streams. It uses advanced natural language processing to identify potential anomalies, compliance issues, and operational risks, providing detailed alerts and logical breakdowns to help teams mitigate threats before they escalate.

How we built it

We developed the asynchronous backend using a combination of Python and Node.js. The core intelligence runs on a dedicated, local AI compute server equipped with dual 24GB RTX 4090 GPUs. We utilized the vLLM framework to deploy a ~30B parameter Qwen 3.5 model, optimizing it for high-throughput inference. The development environment was primarily on macOS Tahoe 26.4.1, leveraging the OpenCode desktop client for codebase management and the Cursor CLI for terminal-assisted development. API endpoints and asynchronous tasks were thoroughly tested using the Hoppscotch desktop client.

Challenges we ran into

One of the primary hurdles was managing GPU memory effectively across the dual 4090s to comfortably host the 30B Qwen model. Tuning the vLLM deployment to minimize latency while maintaining high concurrency for our backend requests required several iterations of configuration and debugging.

Accomplishments that we're proud of

We are incredibly proud of achieving low-latency, high-accuracy inference on a fully local hardware deployment. Seamlessly connecting our asynchronous backend frameworks with the vLLM reasoning engine to process complex enterprise business logic reliably was a major milestone for the team.

What we learned

We gained deep, practical insights into LLM deployment infrastructure, specifically regarding tensor parallelism and memory allocation with vLLM on multi-GPU setups. We also significantly improved our workflows for building, debugging, and testing AI-integrated asynchronous backend systems.

What's next for CC

We plan to expand CC's data ingestion capabilities to handle a wider variety of structured and unstructured business documents. We are also looking into further pipeline optimizations on our compute server to potentially run multi-agent workflows for deeper risk analysis.