Memory Castle

An educational app that generates interactive memory palaces, quizzes, and characters for people trying to learn large amounts of information, including adult students and professional learners.

Comment

Inspiration

Memory Castle was inspired by what I saw when studying for exams as a student: Active recall is powerful, but conventional flashcards can feel repetitive and disconnected from context.

The memory-palace method is an ancient memorization technique that assigns content that the user wants to memorize to an object on a path. The memory-palace method offers vivid spatial cues, but building a useful palace manually takes time, imagination, and practice.

I wanted to combine the strengths of both flashcards and the memory palace. Instead of treating a memory as an isolated card, Memory Castle gives it a place by creating a room with visual cues, navigable paths, characters, and opportunities for retrieval practice.

A crucial design principle emerged early: AI-generated mnemonic content should not become the learner’s source of truth. The learner’s verified response remains separate, authoritative, and unchanged unless the learner explicitly edits it. AI may propose a memorable interpretation around that response, but it cannot rewrite it.

Memory Castle is a tool beyond “flashcards with an AI theme.” It's a local-first learning game where verified knowledge, spatial exploration, active recall, deterministic game systems, and constrained generative assistance work together.

What it does

Memory Castle turns learner-approved study material into an explorable 2D pixel-art castle.

A learner can:

  1. Create, name, save, load, rename, and delete a memory castle.
  2. Add question-and-response memories manually or draft them from bounded text, document, PDF, image, or spreadsheet inputs.
  3. Review and edit generated drafts before saving anything.
  4. Navigate connected rooms using keyboard or on-screen controls.
  5. Enter rooms through doors and paths while the game handles walls, furniture, NPCs, and memory-object collision.
  6. Practice active recall through character dialogue.
  7. Receive a bounded semantic recall score and a concise tip when appropriate.
  8. Earn in-game prizes based on recall performance and that can be given to characters as gifts.
  9. Request a structured AI-generated mnemonic room.
  10. Review that proposal before deciding whether to activate it or retain the deterministic room.
  11. View recall and exploration progress on a castle map.
  12. use synchronized text navigation and reduced-motion settings.
  13. Save the castle locally and resume it later.

Every room has a deterministic fallback, so the core experience remains usable without an API key or when an AI request fails.

AI-generated rooms use only controlled memory-object assets and bounded properties such as approved positions, sizes, colors, motifs, and mnemonic cues. Model output cannot inject HTML, CSS, JavaScript, SVG, or arbitrary URLs. Objects are normalized into safe arrangements that keep doors and navigation routes reachable.

This creates a complete learning loop:

Add knowledge → review it → enter its room → recall it → explore its mnemonic cues → track progress → return later.

How I built it

Memory Castle is a working Next.js and React application written in strict TypeScript. Its main technologies include:

  • Next.js 16 and React 19
  • TypeScript with strict type checking
  • Zod runtime validation
  • OpenAI Responses API with structured output
  • Deterministic seeded generation
  • Versioned browser-local persistence
  • Vitest unit and component tests
  • Playwright end-to-end journeys
  • A responsive, keyboard-accessible pixel-art interface

Building with Codex

I used GPT-5.6 through Codex as an iterative engineering and design collaborator.

The workflow was not a one-shot prompt that generated the application. I repeatedly:

  1. Ran and reviewed the product.
  2. Identified a bounded problem or product goal.
  3. Supplied Codex with screenshots and observable acceptance criteria.
  4. Had Codex inspect the existing repository and trace the relevant rendering, state, schema, or collision path.
  5. Reviewed the implementation in the running app.
  6. Used linting, strict type checks, tests, and production builds to validate it.
  7. Continued the design loop based on what I observed.

Codex contributed across the full stack:

  • React interaction and state management
  • deterministic castle topology
  • seeded room and character generation
  • movement and collision geometry
  • object placement and safe navigation corridors
  • responsive pixel-art rendering
  • local-storage schemas and recovery
  • server-side OpenAI routes
  • structured-output prompts and schemas
  • validation and fallback behavior
  • keyboard and reduced-motion support
  • generated visual assets
  • unit tests and browser journeys

This required genuine cross-system reasoning. For example, changing the visible size of a memory object also meant adjusting its collision contour, placement constraints, depth ordering, and journey tests. Adding AI room generation required much more than calling a model: it required schemas, moderation, request limits, output validation, safe normalization, learner review, persistence rules, and deterministic fallback behavior.

Responsible AI architecture

The runtime AI features are deliberately constrained:

Learner-controlled input
        ↓
Bounded client request
        ↓
Server-only API route
        ↓
Payload limits, timeout, and moderation
        ↓
OpenAI Responses API with strict structured output
        ↓
Server-side Zod validation
        ↓
Safe normalization or deterministic fallback
        ↓
Reviewable proposal
        ↓
Learner acceptance

The learner controls the verified response and whether generated material is saved. AI may draft memories, propose mnemonic arrangements, help sort content, or evaluate semantic recall, but it cannot replace verified knowledge.

The local MVP has no advertising, analytics, tracking, payments, social features, or accounts. Learning records remain in browser storage, and the interface discloses that clearing browser data may remove them.

Challenges I ran into

Making generated rooms playable

Placing attractive objects is much easier than producing a playable room. Early layouts could put objects too close together or near doors, paths, NPCs, and wall boundaries.

I developed bounded placement rules, reserved navigation corridors, size-aware collision geometry, and deterministic normalization. AI can choose relevant objects, colors, cues, and an intended arrangement, while the application moves unsafe placements to the nearest valid location rather than trusting arbitrary coordinates.

Preserving learner-controlled truth

Mnemonic generation creates a subtle integrity risk: an imaginative model response could be mistaken for the verified definition.

I addressed this structurally. Verified responses and mnemonic content are stored separately. Model instructions declare the verified response authoritative, output schemas prevent silent replacement, and generated room content requires learner review. If a verified response changes, stale generated content is not silently treated as current.

Making a game rather than a visual mockup

A coherent castle required much more than attractive room art. I implemented movement, reciprocal room entrances, collision, door transitions, NPC interaction, scoring, gifts, progress, persistence, menus, and safe modal behavior.

Small visual changes often exposed deeper systems problems. A wall that looked shorter needed a matching collision boundary. A rug needed to be visually below the hero, shadow-free, and non-colliding. A memory object needed to maintain its intended scale while still keeping paths clear.

Keeping AI optional

Network, moderation, parsing, and model failures are normal conditions, not impossible edge cases. The application therefore includes deterministic room generation and recall scoring fallbacks. Invalid or partial AI output is rejected rather than persisted.

Accessibility inside a spatial interface

A memory palace is inherently visual and spatial, but essential use should not depend exclusively on vision, precise pointer input, animation, or remembering a map.

The app includes keyboard controls, visible focus states, semantic dialogs, a synchronized text-navigation representation, reduced-motion support, scrollable long-form dialogue, and textual recall-status labels. Accessibility still needs continuing manual evaluation across screen readers, zoom levels, and devices, but it is part of the product architecture rather than a final cosmetic pass.

Accomplishments that I'm proud of

A non-trivial, working Codex-built implementation

Memory Castle is not an API wrapper or static demonstration. It combines domain logic, rendering, persistence, collision mathematics, constrained AI workflows, and automated validation in a runnable product.

The core experience works deterministically, while AI adds optional, reviewable creative assistance. That separation makes the implementation more resilient and demonstrates thoughtful use of Codex across engineering, product design, debugging, testing, and visual iteration.

A coherent product experience

The project supports a complete journey from an empty castle to saved memories, navigable rooms, recall practice, mnemonic generation, progress tracking, and later resumption.

Its pixel-art architecture, hardwood floors, memory objects, professional NPCs, map, dialogue system, and floating-orb hero form one consistent visual and interaction language rather than a collection of unrelated prototypes.

A specific and credible use case

Memory Castle is designed for adult students, certification candidates, professional learners, and others studying knowledge-heavy material.

It addresses a concrete problem: turning factual material into reusable retrieval practice can be tedious, while constructing a traditional memory palace can require substantial setup. Memory Castle reduces that setup burden while preserving learner review and active participation.

The project does not claim guaranteed learning outcomes. Its current demonstrated value is that it makes structured recall practice more contextual, explorable, and personally meaningful. Measuring changes in retention requires future user research rather than a product claim.

A distinctive idea

Memory Castle combines four elements that are usually separated:

  1. Learner-approved, verified knowledge
  2. Structured and reviewable AI mnemonic proposals
  3. A deterministic game world with real navigation and collision
  4. Active recall integrated directly into exploration

The room is not merely a generated illustration, and AI does not generate an unconstrained world. Each room is functional memory architecture: doors connect knowledge, objects serve as mnemonic cues, NPCs create retrieval opportunities, and the map makes the learner’s growing knowledge structure visible.

Strong safety and quality boundaries

Safety constraints strengthened the product instead of diminishing it:

  • API keys remain server-side.
  • Inputs and model output are treated as untrusted.
  • Output must pass strict schemas.
  • Unknown fields and unbounded values are rejected.
  • Generated content cannot contain executable markup or arbitrary URLs.
  • AI room proposals require review.
  • Verified responses remain authoritative.
  • Failures resolve to safe deterministic behavior.
  • Synthetic, general-audience content is used for demos and tests.

What I learned

I learned that effective use of Codex depends on a disciplined feedback loop. Precise observations, screenshots, bounded requests, repository inspection, and executable tests produced much stronger results than asking for an entire product in one step.

I also learned that responsible AI design is largely systems design. A responsible prompt is helpful, but it is not enough. Trustworthy behavior comes from server boundaries, strict schemas, explicit ownership rules, validation, cancellation, safe errors, review steps, and deterministic fallbacks.

Another lesson was that visual polish and game correctness are inseparable. Depth ordering, asset proportions, collision boxes, wall boundaries, door visibility, and navigation clearance jointly determine whether a room feels coherent.

Finally, I learned that generative features are most useful when they expand learner agency rather than replace it. Memory Castle’s AI works best as a constrained creative collaborator: it can suggest an evocative arrangement, but the learner decides what knowledge is correct and which mnemonic becomes part of the castle.

What's next for Memory Castle

The next stage would be to evaluate the experience with a narrowly defined group, such as adult certification candidates using synthetic or non-sensitive study material.

Planned work includes:

  • Conducting structured usability sessions and accessibility reviews
  • Comparing first-use comprehension with a conventional flashcard workflow
  • Measuring completion, voluntary return, and subjective cognitive load
  • Evaluating recall performance without claiming causation prematurely
  • Improving responsive behavior across more mobile and desktop devices
  • Testing with multiple screen readers, zoom settings, and input methods
  • Adding learner-controlled export and backup
  • Expanding the approved memory-object catalog
  • Improving mnemonic editing without weakening verified-response boundaries
  • Exploring privacy-conscious cross-device persistence only after the local MVP is thoroughly validated

The long-term vision is a learning environment where knowledge does not feel like an endless list. It becomes a place the learner can build, navigate, revisit, and remember—while remaining firmly in control of what is true.

Built With

Share this project:

Updates