Inspiration
As quantitative researchers, we deal with massive, highly sensitive financial datasets (tick-data, order books). Uploading proprietary, alpha-generating data to cloud-based AI tools poses a critical security and intellectual property risk. We wanted the power of AI-driven data pipelines without the cloud dependency. The UGREEN AI NAS, with its robust local compute and UGOS ecosystem, presented the perfect opportunity to build a 100% private, on-premise solution. Furthermore, manual data structuring is prone to human error, which often leads to the Probability of Backtest Overfitting (PBO). We were inspired to build a tool that autonomously enforces rigorous data hygiene right at the storage layer.
What it does
Q-Data Agent transforms the UGREEN AI NAS from a passive storage drive into an autonomous quantitative data pipeline. Users simply drop raw financial CSV or JSON files into a designated NAS folder. The agent instantly detects the new file, cleans data anomalies (like NaN values and mismatched timestamps), and automatically structures the data into in-sample and out-of-sample directories for walk-forward optimization. Finally, it uses local AI to analyze the dataset's metadata and generates a comprehensive Markdown report assessing its statistical viability (e.g., checking data depth for Deflated Sharpe Ratio evaluation).
How we built it
We utilized the UGREEN Skill Developer Package and UGOS CLI capabilities to create a multi-agent orchestration pattern.
Coordinator Agent: Hooks into the NAS file system to monitor specific directories for file-drop events.
Data-Processing Agent: Runs optimized Python logic (Pandas/NumPy) to sanitize and split the data.
LLM Analysis Agent: Leverages local AI capabilities to read the sanitized headers, row counts, and date ranges to generate a statistical viability report. Before adapting it to the UGOS ecosystem, we rigorously tested the core data-processing logic and memory allocation on a local Proxmox/k3s homelab environment to ensure maximum efficiency.
Challenges we ran into
The biggest technical hurdle was memory management. Financial tick-data files can be massive, and loading them entirely into memory for processing can cause severe bottlenecks or crash local NAS hardware. We had to optimize the Data-Processing Agent to utilize efficient chunking techniques and memory-mapped file operations to ensure it runs smoothly within the hardware limits of a consumer/prosumer NAS, without interrupting other background tasks.
Accomplishments that we're proud of
We are incredibly proud of proving that advanced, enterprise-grade quantitative workflows do not require expensive cloud infrastructure or third-party API subscriptions. We successfully built a fully autonomous, zero-touch AI pipeline that respects 100% data privacy and runs natively on the UGREEN AI NAS. By integrating complex data-splitting logic with local LLM summarization, we bridged the gap between raw storage and actionable quantitative insights.
What we learned
We gained deep insights into the architecture of the UGOS file system and how to seamlessly bridge local storage events with AI tool calling. We also learned how to highly optimize Python-based data operations for NAS environments, balancing system RAM constraints with the heavy compute demands of dataset sanitization.
What's next for Q-Data Agent
We plan to expand Q-Data Agent to support highly compressed, specialized quantitative formats like Parquet and HDF5. We also aim to integrate a Zero-Knowledge (ZK) cryptographic layer that can generate local proofs of data integrity—ensuring that a dataset hasn't been tampered with before a backtest runs, all without exposing the underlying alpha. Ultimately, we want to connect Q-Data Agent directly to local backtesting engines for a seamless end-to-end quant workflow.
Log in or sign up for Devpost to join the conversation.