herakoi-web: Instructions file for Claude Code

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

Node and pnpm versions are pinned in mise.toml; run mise install once to get them.

pnpm dev          # Vite dev server (http://localhost:5173)
pnpm dev:https    # HTTPS dev server (required for camera on mobile)
pnpm build        # Production build → dist/
pnpm lint         # biome check (including a11y rules) + typecheck
pnpm lint:biome:fix  # Auto-fix Biome issues (run before committing manually)
pnpm typecheck    # TypeScript only (tsc --noEmit)
pnpm test         # Full Vitest suite (--pool=threads)
pnpm commit       # Commitizen prompt for conventional commits

Run a single test file: pnpm vitest run src/plugins/detection/mediapipe/plugin.test.tsx

Architecture Overview

Herakoi is a webcam hand-tracking sonification app: MediaPipe detects hand landmarks → pixel colors at those points are sampled → colors are converted to audio via Web Audio API.

Plugin Pipeline

The app is built around a compile-time plugin system with four pipeline slots:

Slot	Interface	Current implementation
detection	PointDetector	mediapipe/ — MediaPipe Hands
sampling	ImageSampler	hsv/ — HSV color space sampler
sonification	Sonifier	oscillator/ — Web Audio oscillators
visualization	(optional)	debugHud/ — Canvas overlay

Data flow: Video → MediaPipe → normalized (0–1) points → pixel color samples ({h,s,v,...}) → oscillator tones → speakers.

Plugin discovery (src/engineConfig.ts): Uses import.meta.glob() to find all src/plugins/<kind>/<name>/plugin.ts(x) files at build time. The path determines the plugin's kind and id (e.g. detection/mediapipe).

Plugin isolation: Biome enforces that plugins only import from #src/core/*, #src/shared/*, or other plugins — never from shell/app code. Adding a new plugin means creating a directory under src/plugins/<kind>/<name>/ with a plugin.ts(x) entry exporting a typed plugin object.