Clay

AI LMS that makes current system accessible for Neurodivergents

Comment

Inspiration and Problem Identified

I have ADHD, and I have always struggled with deadlines, keeping track of too many things, and knowing how long an assignment will actually take. Traditional LMS platforms often make that harder. To find one assignment, I may need to click through several pages, open different files, and read a large block of instructions before I can even begin.Time perception has also been a major problem for me. I have started essays in the evening thinking I had enough time, then realized they were due at midnight and kept writing until 3 a.m. I always wished there were something that could help me understand the assignment, break it down, and tell me what actually matters next.

When I saw the Education track for, I knew I wanted to build something I would personally use during my senior year. I also wanted to know whether this was only my problem. I used ChatGPT Deep Research to look through Reddit posts, student discussions, accessibility research, and neurodivergent communities. I found many people describing the same struggles: not knowing how to start, missing small assignments, losing track of deadlines, getting overwhelmed by dense interfaces, and forgetting where they stopped.

That made me confident that Clay was worth building.

Clay is based on one idea:

Every student processes, plans, and experiences education differently, so the learning system should be able to adapt("mold") to them.

Clay started with neurodivergent students, but the same features can help anyone who wants a calmer interface, clearer planning, voice-first access, or an easier way to resume work.

What it does / Solution

Clay is an AI-native LMS designed to make the current education system more accessible for neurodivergent students.

It brings academic work, planning, memory, accessibility, and AI support into one system.

Clay can connect to an existing LMS(like Canvas, Blackboard, Brightspace and more) and organize courses, deadlines, assignments, and updates inside a more personalized interface. It helps the student understand what matters, start the work, and return later without rebuilding their entire thought process.

Core features include:

  • Clay Plan: Creates a visual plan for each assignment, with milestones, time estimates, calendar blocks, progress, and replanning when work is missed.
  • One Next Step: Recommends one useful action based on deadlines, dependencies, workload, available time, and the student’s pace.
  • Assignment Mold: Turns dense assignment instructions into a clearer structure while preserving the original requirements and rubric.
  • Sensory modes: Includes Complete Calm, Focus Tunnel, Motion Safe, Visual Quiet, Reading Ease, Predictable Mode, and Voice-First Mode.
  • Academic Memory: Remembers notes, decisions, preferences, working context, and the exact place where the student stopped, helps to learn new material by connecting examples from prevous learnings.
  • Memory Map: Shows how courses, assignments, milestones, notes, conversations, and preferences connect.
  • Grounded companion: Uses actual course, assignment, plan, calendar, and memory context. Through chat, students can ask what to work on, explain an assignment, change the interface, create or revise a plan, resume where they stopped, search personalizations, and understand why Clay made a recommendation.
  • Voice-First Mode: Lets students use Clay through voice with a minimal interface. They can ask questions, navigate the LMS, plan work, change settings, and continue the same conversation across text and voice.
  • Personalizations: Lets students change fonts, themes, layouts, navigation, sounds, and interface behavior and more, they can full mold the "clay"
  • Calendar: Combines deadlines, milestones, study blocks, and connected calendar events. Merge your Outlook, Google Calender, Sports, and Academic in one place so your don't ahve to open mutiple tabs.

How I built it

I built Clay with Next.js, TypeScript, Vercel, Neon Postgres, and Drizzle.

The AI companion uses a hybrid system to max out token efficicy:

  • Deterministic code handles deadlines, milestone dependencies, progress, priority scoring, permissions, preference changes, and other academic facts.
  • GPT OSS 20b(via Groq) handles open-ended conversation when language reasoning is useful.
  • Typed tools control what the model can access and prevent unrestricted database or system actions.
  • Clay selects only the relevant course data, conversation history, working context, and Memory records for each request.
  • Common actions such as “Why this task?”, “Where did I stop?”, or “Use dark mode” bypass the model entirely, to save tokens.

For voice, I used LiveKit for persistent realtime sessions, Silero VAD for turn detection, Groq Whisper for transcription, and Amazon Polly or ElevenLabs for speech output when configured.

Clay Plan combines AI-generated task breakdown with deterministic dates, milestone states, calendar availability, time estimates, approval, and replanning.

Personalizations use validated settings and design tokens instead of allowing the model to freely rewrite the app.

Challenges I ran into

The biggest challenge was deciding what Clay should actually be. I did not want to build something only for myself, so I spent a lot of time researching the problems neurodivergent students face and making sure Clay solved a real need.

Another challenge was avoiding an LLM wrapper. I knew Clay needed AI, but I did not want the model deciding deadlines, permissions, progress, or academic facts. That is why Clay uses deterministic systems for important product state and AI only where language and reasoning are useful.

Customization was also difficult. I originally wanted Clay to generate any feature a student requested, but arbitrary code inside an LMS is hard to secure, test, and isolate. I narrowed that into safe personalizations, reviewed /build requests, and a future controlled component system.

Voice also took a lot of work. I did not want to add text-to-speech to a chatbot and call it voice mode. The system needed proper turn detection, interruption, routing, shared memory, and continuity between text and voice, which is why I built it around LiveKit.

Accomplishments that I'm proud of

I am proud that Clay has many features but still feels light and calm. That matters because an LMS designed for neurodivergent students should not become another overwhelming system.

I am especially proud of:

  • Keeping instructor requirements separate from AI-generated interpretation.
  • Building priority, milestone, timing, and permission logic deterministically.
  • Making Memory useful to the agent instead of only displaying it as a graph.
  • Creating sensory modes that produce real changes instead of acting like cosmetic theme switches.
  • Supporting temporary and permanent personalizations.
  • Keeping text and voice inside the same conversation and memory system.
  • Routing predictable requests away from the LLM to reduce cost and latency.
  • Designing fallbacks so the main LMS still works when an AI or voice provider fails.

Most importantly, Clay feels like one connected product instead of AI Slop.

What I learned

I learned that accessibility cannot be reduced to a settings page. Fonts, contrast, and reduced motion help, but many of the real problems are structural: unclear instructions, too many choices, distant deadlines, lost context, unpredictable layouts, and difficulty restarting.

I also learned that personalization should not mean guessing someone’s diagnosis. Clay works better when the student directly says what helps, previews the change, and stays in control.

Another major lesson was that using more AI does not automatically make a product better. Clay became more reliable when routine actions bypassed the model and important academic information came from real tools and deterministic systems.

Finally, I learned that planning should live next to the work. Students should not need another dashboard just to understand how an assignment will get finished.

What's next for Clay

The next step is real student feedback.

Connect stronger OpenAI models(like 5.6) via API through the provider route that is already in place, and test Clay with neurodivergent students and educators.

I have already shown Clay to a professor and several friends, and their response made me want to continue beyond the hackathon.

I plan to use Clay during my own upcoming semester, run beta tests, see where the product still creates friction, and improve it before a wider release.

How I used ChatGPT 5.6 and Codex

Clay was built through a human-directed loop between me, ChatGPT, and Codex.

  • My role: I chose the problem, product direction, interface, scope, tradeoffs, priorities, and final approvals.
  • ChatGPT 5.6: I used it for brainstorming, product research, ChatGPT Deep Research, reviewing Reddit and accessibility discussions, challenging ideas, writing requirements, and breaking the work into phases.
  • Codex: I used it as the engineering partner. It created the repository, implemented features, wrote migrations, reviewed code, ran security checks, created tests, verified browser workflows, and produced architecture diagrams.

The process was iterative:

  1. I discussed the product and requirements with ChatGPT.
  2. I gave the plan to Codex.
  3. Codex implemented and tested it.
  4. I reviewed the UI and behavior.
  5. I brought the results back to ChatGPT for critique and the next phase.
  6. I sent the revised plan back to Codex.

I did not hand-write every line of the repository. My role was closer to product owner, system designer, and reviewer.

I remained responsible for the decisions, visual approval, credentials, scope, and what was accepted or rejected. AI-generated work was reviewed against the repository and checked with automated tests and browser verification.

Built With

Share this project:

Updates