Knowledge-Augmented Table Question Answering with Hierarchical Multi-Agent Systems

A production-ready multi-agent system for answering questions over incomplete tables by systematically augmenting them with knowledge from Wikidata. This project achieves 92.9% success rate (vs 28.6% baseline) through intelligent knowledge graph integration.

📊 Key Results

• 225% improvement in success rate (28.6% → 92.9%)
• 183% increase in confidence scores (0.314 → 0.889)
• 100% accuracy on knowledge-intensive queries
• 193 Wikidata facts retrieved across augmented queries

🎯 Overview

Traditional TableQA systems fail when queries require information not present in table columns. This system solves that through a hierarchical multi-agent architecture that:

1. Detects information gaps in tables using schema analysis
2. Links table entities to Wikidata identifiers (98% confidence)
3. Retrieves relevant facts via ensemble property prediction
4. Constructs hybrid graphs combining table structure with knowledge
5. Performs multi-path reasoning with consensus detection

Research Paper: Full technical details in the B.Tech project report (see repository)

Evaluation results are available in benchmarks/evaluation_results/.

Installation

git clone https://github.com/ADIITJ/kg-table-qa.git
cd kg-table-qa
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Configure API key
cp .env.example .env
# Edit .env and add GEMINI_API_KEY

Usage

from kg_table_qa.main import TableQASystem
from kg_table_qa.core.schemas import TableData
import asyncio

async def main():
    system = TableQASystem()

    table = TableData(
        columns=["Name", "Institution", "Field"],
        data=[["Alice", "MIT", "AI"], ["Bob", "Stanford", "ML"]]
    )

    result = await system.answer_question("Where did Alice study?", table)
    print(result["answer"])

asyncio.run(main())

Running Benchmarks

# Baseline (table-only)
python run_baseline.py

# Augmented (with knowledge graph)
python run_benchmark.py

Results are saved to benchmarks/evaluation_results/.

🏗️ Architecture

Five specialized agents coordinated by an intelligent orchestrator:

1. Gap Detection Agent: Identifies missing attributes through LLM-based schema analysis (95% confidence)
2. Entity Linking Agent: Maps table cells to Wikidata QIDs with query-aware filtering (55% efficiency gain)
3. KG Retrieval Agent: Four-strategy ensemble voting for property prediction (P17, P463, P30, etc.)
4. Graph Builder Agent: Constructs hybrid NetworkX graphs (table + KG topology)
5. Reasoning Agent: Multi-path BFS traversal with consensus detection
6. Orchestrator: Intelligent routing with 5-row sample optimization

Optimizations:

• Sample-based gap detection (95% token reduction)
• Entity filtering (55% fewer API calls)
• SPARQL caching (70% hit rate)
• Table-independent retrieval

📁 Project Structure

kg-table-qa/
├── kg_table_qa/                  # Core implementation
│   ├── agents/                   # Multi-agent system
│   │   ├── base.py              # BaseAgent abstract class
│   │   ├── orchestrator.py      # Agent coordinator
│   │   ├── gap_detector.py      # Gap detection
│   │   ├── entity_linker.py     # Entity linking
│   │   ├── kg_retriever.py      # Knowledge retrieval
│   │   ├── graph_builder.py     # Hybrid graph construction
│   │   ├── reasoning_agent.py   # Multi-path reasoning
│   │   └── pandas_executor.py   # Table operations
│   ├── core/                     # Core utilities
│   │   ├── llm_client.py        # Gemini API wrapper
│   │   ├── prompts.py           # Prompt templates
│   │   └── schemas.py           # Pydantic models
│   ├── kg/                       # Knowledge Graph
│   │   ├── wikidata.py          # SPARQL client
│   │   ├── entity_linking.py    # Entity resolution
│   │   └── cache.py             # Redis caching
│   ├── graph/                    # Graph operations
│   │   ├── builder.py           # Graph construction
│   │   └── operations.py        # Graph traversal
│   ├── data/                     # Data processing
│   │   └── processor.py         # Table loaders
│   ├── utils/                    # Utilities
│   │   ├── config.py            # Configuration
│   │   └── logging.py           # Structured logging
│   └── main.py                   # System entry point
├── benchmarks/
│   ├── evaluation_results/       # Evaluation data
│   │   ├── tables/              # Test tables (CSV)
│   │   ├── questions/           # Test questions (JSON)
│   │   └── results/             # Performance metrics
│   └── scripts/                  # Evaluation scripts
├── data/                         # Sample datasets
├── evaluation/                   # Evaluation framework
├── .github/                      # GitHub workflows & docs
│   └── copilot-instructions.md  # AI coding guidelines
├── run_baseline.py               # Baseline evaluation
├── run_benchmark.py              # Augmented evaluation
├── requirements.txt              # Dependencies
├── .env.example                  # Environment template
└── README.md                     # This file

🔬 Evaluation Results

Performance on 14 real-world questions across 4 tables:

Metric	Baseline	Augmented	Improvement
Success Rate	28.6% (4/14)	92.9% (13/14)	+225%
Avg Confidence	0.314	0.889	+183%
Failed Questions	8	0	-100%

By Question Type:

• Table-only: 100% (both systems)
• KG Required: 0% → 100% (baseline → augmented)
• Hybrid: 0% → 80% (baseline → augmented)

Performance Characteristics:

• Entity linking: 14.2s avg (60% of total time)
• Gap detection: 1.9s (optimized with 5-row sampling)
• Graph building: 1.4s (efficient NetworkX operations)
• Total latency: 11× baseline (23.6s vs 2.1s)

🛠️ Technology Stack

Component	Technology
Language	Python 3.10+
LLM API	Google Gemini
Knowledge Graph	Wikidata (SPARQL)
Graph Library	NetworkX 3.2.1
Data Processing	Pandas 2.2.0
Async Framework	asyncio, httpx
Validation	Pydantic 2.6.0
Logging	structlog 24.1.0
Caching	Redis 5.0.1
Testing	pytest, pytest-asyncio

🎓 Academic Context

Project: B.Tech Final Year Project
Author: Atharva Date ([email protected])
Mentor: Dr. Suman Kundu, Assistant Professor
Institution: Indian Institute of Technology Jodhpur
Department: Computer Science and Engineering
Year: 2025

Full Report: Complete technical documentation available in LaTeX format

📄 License

MIT License - see LICENSE file for details

📧 Contact

Atharva Date
Email: [email protected]
GitHub: @ADIITJ
Repository: kg-table-qa

🙏 Acknowledgments

• Dr. Suman Kundu for project guidance
• IIT Jodhpur Computer Science Department
• Wikidata community for knowledge graph infrastructure
• Google Gemini for LLM API access

📚 Citation

If you use this work in your research, please cite:

@misc{date2025kgtableqa,
  title={Knowledge-Augmented Table Question Answering with Hierarchical Multi-Agent Systems},
  author={Date, Atharva},
  year={2025},
  institution={Indian Institute of Technology Jodhpur},
  note={B.Tech Final Year Project}
}

Built With

• python