Inspiration
Despite the rapid progress in AI assistants, most interactions still happen through dense blocks of text inside chat windows. While this format works for quick answers, it is far from ideal for learning complex ideas.
People naturally understand concepts better through visual explanation, narration, and progressive storytelling. This is why explainer videos, diagrams, and classroom whiteboards are such powerful teaching tools.
We started wondering:
What if AI explained ideas the way great teachers do — visually, step-by-step, and interactively?
Instead of returning paragraphs, an AI system could construct a guided explanation using visuals, narration, and structured slides.
This idea led to Knowal, a canvas-based AI system where explanations unfold like a multimodal story.
What it does
Knowal transforms AI responses into interactive visual explanations.
When a user asks a question, Knowal generates a live canvas session that combines narration, diagrams, images, and animations to walk through the concept step by step.
Rather than scrolling through paragraphs, users explore an AI-generated explainer experience that unfolds visually.
Key capabilities include:
Multimodal explanations
Knowal blends different forms of media to communicate ideas clearly:
- narrative text explanations
- generated diagrams and visuals
- SVG-based illustrations
- transitions and visual emphasis
- synchronized voice narration
The result feels closer to an interactive explainer video than a typical chatbot answer.
Canvas-based learning interface
At the center of Knowal is a dynamic canvas where the system builds structured visual slides for each explanation.
The AI can:
- generate diagrams and conceptual visuals
- highlight key relationships
- animate transitions between ideas
- organize explanations into progressive slides
This creates a visual-first AI interaction model, replacing static chat responses.
Adjustable explanation depth
Knowal includes a depth control slider that lets users decide how detailed the explanation should be.
Depending on the selected depth, the system can produce:
- a quick high-level overview
- a conceptual walkthrough
- a detailed deep dive
This allows the same system to support both beginners and advanced learners.
Interactive concept exploration
Users can click on any word, phrase, or concept in the explanation to explore it further.
When this happens, Knowal opens a secondary canvas layer dedicated to that concept. After exploring it, users can easily return to the original explanation.
This creates a branching learning experience, similar to how people naturally explore ideas.
Image-based problem solving
Users can also upload images such as:
- homework questions
- diagrams
- screenshots
Knowal analyzes the visual input and builds a step-by-step explanation directly on the canvas, helping users understand both the reasoning and the final answer.
How we built it
Knowal is designed as a multimodal AI system that converts user inputs into structured visual learning experiences.
AI models and capabilities
Knowal is powered by the Amazon Nova family of foundation models, which provide the reasoning and multimodal capabilities behind the system.
At the core of the experience is Nova 2 Lite, a fast and cost-efficient reasoning model that drives most of the workflow. Nova 2 Lite is used to understand user queries, extract the most relevant explanation material, and structure the information into a coherent narrative that can be translated into visual learning steps on the canvas.
During development we also explored other capabilities from the Amazon Nova ecosystem. Experimented with Nova 2 Sonic to evaluate conversational speech interaction for voice-driven explanations, while Nova multimodal embeddings were tested for interpreting image inputs such as diagrams or screenshots.
Power-packed Nova intelligence allows Knowal to process multimodal inputs and transform them into coherent visual explanations.
Frontend
The user interface is built using:
- React.js for the application framework
- HeroUI for interface components
- HTML Canvas for rendering diagrams, slides, and animations
The canvas acts as the primary storytelling surface where Knowal constructs its explanations.
Backend
The backend system manages agent workflows and user sessions using:
- Hono.js for server infrastructure and API routing
- Drizzle ORM for type-safe database operations
- PostgreSQL / SQLite to store sessions, canvas states, and interaction history
This setup allows Knowal to stream visual explanations dynamically as they are generated.
Challenges we ran into
Moving beyond chat-based AI responses
Most language models are designed to produce text. Our challenge was enabling the system to describe visual layouts and graphical elements that could be rendered on a canvas.
We addressed this by designing a structured representation for visual storytelling instructions.
Coordinating narration with visuals
For explanations to feel natural, narration needed to align with the visual elements appearing on the canvas.
This required careful synchronization between the AI output and the rendering pipeline.
Managing multimodal inputs
Handling combinations of voice, text, and images while maintaining coherent explanations required thoughtful prompt design and structured processing of inputs.
What we learned
Building Knowal reinforced an important insight:
The future of AI interfaces will extend beyond chat.
While text-based responses are powerful, they are not always the most effective way to communicate complex ideas.
We found that combining multimodal AI capabilities with a structured visual interface dramatically improves how explanations are delivered.
Another key takeaway was that AI systems can function not only as information providers but also as creative directors that orchestrate text, visuals, and narration into cohesive experiences.
What's next for Knowal
We see Knowal evolving into a fully interactive AI learning platform.
Future improvements include:
Collaborative sessions
Enable multiple users to join the same canvas session and explore explanations together.
More advanced visual simulations
Introduce richer animations, simulations, and interactive visualizations to explain complex systems.
Expanded use cases
Beyond education, Knowal could also support:
- product walkthroughs
- technical documentation explanations
- onboarding experiences
- marketing storytelling
- knowledge base exploration
Knowal reimagines how people interact with AI by turning static answers into interactive visual explanations.
Instead of simply reading responses, users can watch ideas unfold and explore them visually.
Log in or sign up for Devpost to join the conversation.