Inspiration
Engineering changes often look simple on the surface, but their impact can extend across multiple documents, specifications, and system components. For example, renaming a database field from customer_id to account_id might seem straightforward, but the old name could still appear in API specifications or other engineering documents.
I wanted to explore how AI could help engineering teams identify these inconsistencies before they become implementation problems. This inspired me to build SystemLens, an AI-powered engineering consistency review tool designed for UGREEN NAS.
What it does
SystemLens analyzes engineering documentation stored on a UGREEN NAS to identify conflicting information, missing details, and potential inconsistencies across sources.
Rather than functioning as another document-based chatbot, SystemLens follows a structured review workflow. It discovers relevant files, extracts information, reconciles findings across documents, and generates evidence-backed reports with references to original sources.
It also distinguishes between confirmed inconsistencies and missing information, helping engineers investigate potential issues without treating every difference as an error.
How we built it
I built SystemLens as a modular AI skill using the available UGREEN NAS file and document operations.
The workflow consists of several capabilities:
- File Discovery: Finding and listing relevant engineering documents.
- Information Extraction: Extracting relevant content and answering questions using available documentation.
- Reconciliation: Comparing information across sources to identify conflicting claims and missing details.
- Evidence Collection: Connecting findings to their supporting source material.
- Report Generation: Producing structured reports that engineers can review and use to investigate potential issues.
I focused on making each stage of the workflow structured and traceable so that the final output would be more than just an AI-generated summary.
Challenges we ran into
One of my biggest challenges was moving beyond a generic RAG-based document question-answering system. Simply retrieving relevant documents and generating an answer was not enough. I needed to design a workflow that could actually compare information across sources and identify why a particular inconsistency mattered.
Another challenge was ensuring that AI-generated findings remained grounded in actual documentation. Engineering decisions require accuracy, so I wanted every finding to have supporting evidence rather than relying on unsupported AI-generated conclusions.
I also had to work within the available NAS operations and design the skill around those capabilities.
Accomplishments that we're proud of
I am particularly proud of turning the idea of an AI documentation assistant into a structured engineering consistency review workflow.
Some accomplishments include:
- Developing a modular AI skill specifically designed for engineering documentation review.
- Building a reconciliation workflow that identifies conflicting information across documents.
- Incorporating evidence collection so findings can be traced back to their original sources.
- Designing the reporting process to distinguish confirmed inconsistencies from missing information.
- Creating a solution that supports engineering decision-making rather than simply generating answers.
Most importantly, I was able to approach the problem from an engineering perspective instead of building another general-purpose chatbot.
What we learned
This project taught me that building useful AI applications requires much more than connecting an LLM to a collection of documents.
I learned the importance of structured workflows, evidence-based reasoning, and clearly defining what an AI system can and cannot conclude. I also gained a better understanding of designing modular AI skills around specific platform capabilities.
One of my biggest takeaways was that AI should help people make better-informed decisions, not replace the reasoning and judgment behind those decisions.
What's next for SystemLens
I would like to continue developing SystemLens with deeper UGREEN NAS integration, broader document format support, and more advanced engineering consistency checks.
Future improvements could include detecting dependencies between system components, tracking inconsistencies across documentation updates, and expanding the types of engineering changes the tool can review.
Ultimately, I want SystemLens to help engineering teams spend less time tracking down documentation conflicts and more time building reliable systems.
Built With
- ai
- discovery
- document
- evidence-based
- extraction
- information
- llm
- nas
- processing
- python
- reconciliation
- reporting
- structured
- ugreen