DLYogCognifyAI™: Multi-Agent Responsible AI RAG Guard System

Allows any company to upload product docs. AI-powered, Responsible Novel RAG generates Q&A for seamless product discovery. Multi-agent system handles docs, monitoring, search, agile and analysis tasks

Comment 2

Inspiration

The inspiration for DLYogCognifyAI™ stems from our mission of promoting AI for Good—developing ethical, responsible AI solutions that benefit businesses and society. We envisioned a multi-agent platform that not only streamlines product documentation processing but also ensures compliance with Responsible AI standards. Our commitment to Responsible AI led to the creation of the Prompt Guard Agent, which ensures all AI operations are carried out ethically and transparently.

Additionally, a previous issue with our Dell RTX 3090 Ti overheating during AI model training, resulting in hardware damage, inspired the development of a GPU Monitoring Agent. This agent safeguards the system by monitoring temperatures and shutting it down when necessary to prevent damage.

Finally, challenges with traditional Retrieval Augmented Generation (RAG) systems inspired us to innovate. We created a novel RAG approach that preprocesses documents using Large Language Models (LLMs) to provide comprehensive, contextually accurate insights that outperform traditional methods.

What it does

DLYogCognifyAI™ is a multi-agent platform designed to enhance product discovery, requirement analysis, and agile development. It enables companies to upload product documentation and preprocesses them using a novel RAG technique, powered by LLMs like Google T5 and LLAMA3. The platform delivers intelligent Q&A and holistic insights that traditional RAG systems cannot match.

Key agents include:

  • Document Agent: Handles document uploads and indexes them for quick, efficient retrieval.
  • Search Agent: Accelerates product discovery based on uploaded documents.
  • GPU Monitoring Agent: Monitors GPU temperatures, automatically shutting down the system if they exceed safe limits to prevent damage.
  • Requirement Analyst/Agile Agent: Automates requirement generation, simulating agile processes to deliver structured epics, features, and stories, which is invaluable for enterprise projects.
  • Responsible AI Agent: Ensures Responsible AI principles are adhered to in every AI interaction, enforcing ethical AI usage with the Prompt Guard Agent.

How we built it

We utilized NVIDIA AI Workbench for the development of DLYogCognifyAI™, leveraging Dell RTX 3090 Ti servers for AI model hosting and a MacBook for remote access. The development process included the following key steps:

  • NVIDIA AI Workbench was installed on a Dell machines where we deployed Google T5, Chroma DB, and Nomic Embedding as part of the container within the AI Workbench Project. On a separate Dell machine, we deployed Meta's LLAMA model, accelerated by the NVIDIA TRT Engine for AI inference. Additionally, we tested the integration with the NVIDIA NeMo microservice API (https://integrate.api.nvidia.com/v1/chat/completions). We designed the application with a generic API interface that dynamically switches between different LLM API endpoints based on environment configuration values. This flexibility allowed us to seamlessly toggle between a local LLAMA endpoint and the NVIDIA NeMo Microservice endpoint without requiring any code modifications. This demonstrates the true power of the NVIDIA AI Workbench and its environment configuration capabilities, enabling effortless adaptability in real-world AI deployments.

  • Through the workbench’s Traefik reverse proxy (https://docs.nvidia.com/ai-workbench/user-guide/latest/reference/proxy.html) feature, we connected to the servers remotely from a MacBook, enabling seamless workflow transitions between machines.

  • We automated deployment processes using nvwb CLI and GitHub Actions, streamlining CI/CD practices.

  • ChromaDB was implemented to handle document embeddings, ensuring efficient and fast retrieval for the novel RAG system.

Challenges we ran into

  • Overheating Hardware: Overheating issues with our Dell RTX 3090 Ti prompted us to develop a GPU Monitoring Agent that prevents hardware damage during AI model training.
  • RAG Complexity: Developing the novel RAG technique that worked with small model like Flan T5 required extensive testing to fine-tune document preprocessing and intelligent Q&A generation.
  • User Experience: Improving the UX for product discovery and agile analysis was a challenge. We are focused on further enhancing this aspect for intuitive user interactions.

Accomplishments that we're proud of

  • Innovative RAG Technique: We developed a novel RAG approach that delivers more contextually accurate answers than traditional methods, while utilizing more efficient models like LLAMA3 and Google Flan T5.
  • GPU Monitoring Agent: This agent prevents hardware damage by monitoring and managing GPU temperatures, offering a crucial safeguard for systems running intensive AI workloads.
  • Responsible AI at the Core: We successfully integrated a Prompt Guard Agent to ensure all AI interactions comply with Responsible AI principles, enhancing trustworthiness in AI-driven solutions.
  • Seamless Development: Leveraging NVIDIA AI Workbench streamlined our remote development workflow, Log Monitoring, accelerating testing and deployment across different environments. Also Automation Deployment using nvwb CLI and GitHub Action

What we learned

  • NVIDIA AI Workbench significantly simplifies managing complex AI environments, allowing seamless access to remote servers and rapid iterations.
  • Responsible AI should be deeply embedded into AI systems from the outset, ensuring all interactions follow ethical guidelines.
  • Multi-agent architectures offer scalability and efficiency, making it possible to handle complex workflows like document management, product discovery, and agile development in a distributed, agent-based system. -Automation of Deployment for NVIDIA AI Workbench Apps using CICD We developed GitHub Action workflow and use nvwb CLI to deploy deployment quickly. We plan to follow same pattern for production deployment in future.

What's next for DLYogCognifyAI™: Multi-Agent Responsible AI RAG Guard System

  • Integration with 3rd Party Systems: We're planning to integrate with platforms like Azure DevOps, enabling seamless transitions from requirement analysis to automated feature tracking within enterprise development workflows.
  • Improved Security: We will enhance security features for cloud-based deployments, ensuring data integrity and compliance with industry standards.
  • Scalability Enhancements: We'll further optimize the platform to handle larger datasets and more complex AI models, enhancing its utility for a wider range of industry use cases.
  • Real-Time Collaboration: Future updates will include real-time collaboration features to boost teamwork and productivity.
  • Predictive Monitoring: We aim to enhance the GPU Monitoring Agent with predictive capabilities, providing early warnings to prevent potential hardware failures before they occur.

By focusing on these future improvements, DLYogCognifyAI™ will continue to evolve into a robust, scalable solution for industries across different sectors.

Built With

  • chromadb
  • dell-rtx-3090ti-pcs
  • flask
  • llm
  • llm-agents
  • nvidia-ai-workbench
  • nvidia-nemo-microservice-api
  • nvidia-trt-engine-llama3-accelerated-model-local
  • nvwb-cli
  • python
Share this project:

Updates