Handwritten Notes Intelligence

Elevator Pitch: Turn handwritten notes into a searchable, analyzable, and visual knowledge graph with one upload.

Comment

The Story (US-style write-up)

Inspiration

  • Real-world pain: whiteboard snapshots, classroom chalk notes, and paper excerpts become unsearchable images.
  • Vision: convert fragmented handwritten content into structured knowledge assets; enable semantic search, reveal connections, and visualize a personal graph.
  • Everyday scenarios: exam review (recalling “that page’s formula”), project retros (finding “that meeting’s conclusion”), reading (linking “that chapter’s ideas”).

What it does

  • Drag-and-drop image upload for OCR and AI semantic analysis.
  • Auto-extract keywords, categories, and related concepts; tag notes and generate a knowledge graph.
  • Interactive graph: mini-map, controls, multi-select alignment, and node style customization.
  • Semantic search, tag filtering, note management, and local persistence.

How we built it

  • React + Vite + TypeScript for a fast SPA; react-router-dom for routing; zustand for state (persisted); reactflow for graph UI.
  • IndexedDB via dexie; OCR and AI analysis are abstracted behind service modules to swap real APIs easily.
  • Key challenges & solutions:
    • Consistent note–tag–concept modeling and sync (src/store/index.ts:171-238).
    • Balancing graph interactivity and performance with mini-map, controls, batch alignment, and style editing (src/pages/KnowledgeGraph.tsx).

Challenges we ran into

  • Multilingual handwritten OCR accuracy vs latency; on-device vs cloud tradeoffs.
  • Domain generalization for semantic analysis across learning notes, meetings, and book excerpts.
  • Privacy/compliance for local storage and potential cloud sync (GDPR, data minimization).

Accomplishments we’re proud of

  • Clear modular pipeline from “raw image” to “structured knowledge”, ideal for learning modern FE engineering and visualization.
  • Rich graph interactions and node styling mechanics lower the barrier to hands-on practice.

What we learned

  • Productizing AI requires balancing data structures, UX, and performance.
  • Knowledge management values a loop of “searchable, connectable, reusable”.

What’s next for Shouxiebiji (Business Plan)

  • Pro Edition: high-accuracy OCR and multi-model semantic analysis; on-prem, team collaboration; knowledge asset management, roles, audit.
  • Education: platform for schools/training with auto-grading, mastery graphs, personalized learning paths.
  • Enterprise: auto-structuring meeting minutes, archiving whiteboard content, cross-team knowledge discovery; integrations with existing KBs.
  • Open Ecosystem: plugin-based models and export/sync APIs for a “knowledge apps” marketplace.

LaTeX math support examples:

Inline: $E=mc^2$; Block:

$$ f(x) = ax^2 + bx + c \ \nabla_\theta .\, \mathcal{L}(\theta) = \frac{\partial \mathcal{L}}{\partial \theta} $$

Tech Stack (Detailed)

  • Core: React 18, Vite 6, TypeScript 5.8
  • Routing & State: react-router-dom 7, zustand 5 (persist)
  • UI & Styling: TailwindCSS 3, @radix-ui/*, lucide-react, sonner
  • Forms & Validation: react-hook-form, zod
  • Graph & Visualization: reactflow 11, d3
  • Data & Local Storage: dexie 4 (IndexedDB)
  • Tooling: eslint 9, typescript-eslint 8, postcss, vite-tsconfig-paths
  • Swappable AI services: src/lib/aiService.ts for OCR and analysis abstraction

Project Structure

src/
  assets/
  components/
    CustomEdge.tsx
    Layout.tsx
    NoteNode.tsx
  hooks/
    useTheme.ts
  lib/
    aiService.ts
    database.ts
    utils.ts
  pages/
    Home.tsx
    Upload.tsx
    Notes.tsx
    NoteDetail.tsx
    KnowledgeGraph.tsx
    Profile.tsx
  router/
    index.tsx
  store/
    index.ts
  App.tsx
  main.tsx
  index.css

Getting Started

  • Install deps: pnpm install
  • Start dev: pnpm run dev (default http://localhost:5173)
  • Build: pnpm run build
  • Preview: pnpm run preview
  • LAN access: pnpm run dev -- --host

Runtime Mechanics (Data Flow & Logic Graph)

  • Routing & pages: src/router/index.tsx:10-41; global toast in src/App.tsx:7-9.
  • Upload pipeline: processFiles orchestrates OCR and AI (src/pages/Upload.tsx:56-131); OCR via OCRService.processImage (src/lib/aiService.ts:26-47), AI via AIAnalysisService.analyzeText (src/lib/aiService.ts:72-81).
  • State & graph: useAppStore.updateKnowledgeGraph maps notes/tags/concepts to nodes/edges (src/store/index.ts:171-238).
  • Graph UI: ReactFlowProvider wraps the page (src/pages/KnowledgeGraph.tsx:1041-1045); connections via onConnect (src/pages/KnowledgeGraph.tsx:136-149); drag-stop persistence via onNodeDragStop (src/pages/KnowledgeGraph.tsx:156-164); mini-map and controls toggles (src/pages/KnowledgeGraph.tsx:958-969).
  • Persistence: IndexedDB utilities in src/lib/database.ts (notes/files/settings I/O and stats).

Built With

Share this project:

Updates