landing-page
dashboard
ai-chat
ai-chat-2
keywords-ai-traces-dashboard-for-logging

Canvas Agent: Building an Autonomous AI Teaching Assistant

💡 The Problem

Educators are drowning in admin work.

While teaching assistants help, they're expensive and scarce. Most professors spend hours every week clicking through Canvas LMS just to update due dates, create assignments, or post announcements.

We asked ourselves: What if an AI could actually do the work?

Not a chatbot that answers questions—an agentic system that navigates APIs, makes decisions, and executes complex workflows autonomously.

🏗️ Architecture

We built Canvas Agent as a Multi-Agent Workflow using LangGraph, with specialized agents handling different responsibilities.

The Core Design

┌─────────────────────────────────────────┐
│            Supervisor Node              │
│         (Routes & Orchestrates)         │
└──────────────┬──────────────────────────┘
               │
       ┌───────┴───────┐
       ▼               ▼
┌─────────────┐  ┌─────────────┐
│   Canvas    │  │   Content   │
│  Executor   │  │  Specialist │
│  (Actions)  │  │  (Drafting) │
└─────────────┘  └─────────────┘

Supervisor Node: Routes tasks to the right worker and decides when the job is done
Canvas Executor: The "hands"—15+ Canvas API tools to create assignments, list courses, grade submissions
Content Specialist: The "brain"—drafts pedagogical content, rubrics, and communications

Intelligent Model Routing

To optimize cost without sacrificing quality, we implemented a Gateway Pattern using Keywords AI:

# Simplified routing logic
def route_request(task_complexity):
    if task_complexity == "simple":  # "List my courses"
        return "gpt-4o-mini"  # Fast, cheap
    else:  # "Draft a rubric for an ethics essay"
        return "gpt-4o"  # Deep reasoning

Result: ~90% cost reduction on routine tasks with sub-second latency.

Real-Time Observability

keywords ai organization id: 4654fa89-ab48-4a8b-8b36-da87dcfef3d1

Every agent action logs a custom business event through Keywords AI. Instead of just tracking tokens, we track outcomes: assignments created, announcements posted, rubrics drafted.

This transforms technical logs into a productivity dashboard.

🧠 Lessons Learned

1. Agents Need Guardrails

Early on, the agent confused "Announcements" with "Assignments." Same API, different endpoints, very different consequences.

Fix: Strict system prompts in prompt_templates.py that force explicit disambiguation before any write operation.

2. Latency is UX

Waiting 10 seconds for "list my courses" feels broken. Users expect instant feedback for simple queries.

Fix: Semantic caching via Keywords AI headers reduced repeated queries from ~2s to <100ms.

3. Tool Schema Bloat Kills Performance

The Canvas API is massive. Passing all tool definitions to the LLM overwhelms the context window and confuses the model.

Fix: Dynamic tool filtering—only load schemas relevant to the current task category.

🚧 Technical Challenges

The Infinite Loop Problem

Issue: The Supervisor would sometimes loop endlessly, unsure when to terminate.

Root Cause: Worker agents returned ambiguous success signals.

Solution: Explicit "FINISH" detection in the Supervisor node:

def should_terminate(worker_response):
    success_signals = ["completed", "created successfully", "posted"]
    return any(signal in worker_response.lower() for signal in success_signals)

State Management Across Agents

Issue: Context getting lost between agent handoffs.

Solution: LangGraph's built-in state management with explicit checkpointing after each worker completes.

Markdown Preservation

Issue: React frontend was stripping markdown tables from rubric outputs.

Solution: Refactored server.py to pass raw markdown through without sanitization, letting the frontend renderer handle formatting.

📊 Results

Metric	Before	After
Avg. response time (simple)	2.1s	0.09s
Cost per 1K requests	$4.20	$0.85
Admin tasks automated	0%	78%

🔧 Tech Stack

Orchestration: LangGraph
LLM Gateway: Keywords AI
API Integration: Canvas LMS REST API
Frontend: React
Backend: Python/FastAPI

🚀 What's Next

Canvas Agent proves AI can be more than a conversation partner—it can be a reliable worker. The multi-agent pattern with intelligent routing opens up possibilities beyond education: any domain with repetitive API-driven workflows is fair game.

Key insight: The magic isn't in any single model. It's in the orchestration layer that knows when to use which tool.

Built with LangGraph, Keywords AI, and too many redbulls.

Built With

canvas-lms
fastapi
keywords-ai
langgraph
llm
openai
python
react
trae

CLAI