humla: Instructions file for Claude Code

Humla — project notes

What this app is

Humla is a personal macOS meeting-notes app inspired by Granola. You take freeform notes during a meeting; in parallel the app records mic + system audio, transcribes it, and produces an AI summary that fuses your notes with the transcript. Personal/small-team, not SaaS — your data, your API keys, local SQLite, no backend.

The name is Norwegian for "bumblebee".

Core capabilities

• Hybrid capture (parallel streams) — mic + macOS system audio recorded simultaneously via a Swift sidecar, kept as two separate streams end-to-end (no mixdown). Each gets its own VAD-bounded chunk WAVs, its own full.wav, its own Whisper invocations with its own prior_context trail. In-person meetings produce only mic chunks (system stays silent → no chunks emitted) and the diarizer runs on the mic stream so multiple humans in the same room get distinct labels. Remote calls produce both, and both get diarized — the mic is never assumed to hold a single person just because the system stream has content. You is earned, not assigned: it survives only when the mic diarize resolves to exactly one voice.
• Four STT providers — OpenAI (whisper-1 / gpt-4o-transcribe / mini / diarize), on-device Whisper via Metal, Deepgram (nova-3, nova-2, base), and Groq (whisper-large-v3-turbo). All slot into the stt::BatchSttAdapter trait so the dispatch path is provider-agnostic.
• Per-language routing — transcribe_config (typed JSON, single source of truth) is { default: ProviderConfig, per_language: BTreeMap<String, ProviderConfig> }. Resolution at chunk time: per-note language → per-language override → default. E.g. Norwegian → local NB Whisper, English → Deepgram Nova-3, default → OpenAI whisper-1.
• Whisper quality preset — Fast (greedy) / Balanced (beam=3) / Quality (beam=5, low no_speech threshold) for the local provider; bundles sampling strategy + confidence thresholds together so the user picks one knob.
• Per-note transcription language — global Settings → Language is the default; each note has its own language chip that overrides for that note.
• Offline diarization on stop — speaker-diarize Swift sidecar runs after recording_stop. Two engines selectable via the diarize_model setting: Community-1 (FluidAudio's OfflineDiarizerManager — community-1 segmentation + VBx clustering with PLDA) and Sortformer (NVIDIA end-to-end, 4-speaker cap). Branches on which streams produced content: mic-only diarizes mic_full.wav and emits Speaker 1: / Speaker 2:; when both streams have content, both are diarized and numbered off one shared counter so a mic speaker and a system speaker can never collide on a number (build_hybrid_labels). The old shortcut — hard-label every mic chunk You: and diarize only sys_full.wav — meant a single stray system chunk (a notification chime, a few seconds of video) collapsed an entire in-person meeting onto one You: line, since presence of any sys chunk routed the recording into remote-call mode. You: is now applied only when the mic diarize resolves to exactly one voice.
• Speaker rename + colour-coded pills — each unique speaker gets one of four design-token colours (interactive blue, success green, warning gold, accent red, cycling for 5+). A chip strip above the transcript lets the user click any speaker to rename inline; rename is a regex line-anchored rewrite of the transcript text — no separate metadata table.
• Two-source summaries — model gets [Notater] (typed notes) and [Transkripsjon] (transcript) as two labelled blocks in one user message. Preset prompts are deliberately minimal (kind of summary + language only); the only source-trust signal is the parenthetical tags [Notater] (user-written) / [Transkripsjon] (auto) — the old explicit "favour notes for intent, transcript for facts" rule was dropped. An empty side is sent as (ingen) rather than omitted, so thinking models don't second-guess a missing block.
• Per-note presets — Meeting / 1:1 / Lecture / Interview / Brainstorm / Voice memo, each with its own summary prompt. Custom prompts also supported (rows in summary_prompts table, referenced as custom:<id>).
• Custom vocabulary — per-user list of names and tech terms biasing decoding. Threaded through Whisper-shaped providers as initial_prompt, Deepgram as keyterm (Nova-3) or keywords (other models) query params.
• Trailing transcript context — every chunk's transcription receives the last ~150 committed words as prior_context (Whisper's initial_prompt slot for OpenAI/Local/Groq; Deepgram ignores it because its keywords is a per-token boost, not a continuation primer). Single biggest mitigation against silence-driven hallucinations and proper-noun drift.
• VAD-bounded chunks — sidecar rotates each chunk at natural speech pauses (min 1.0 s / max 15 s / 500 ms silence trigger) instead of a fixed timer.
• Reasoning-model temperature handling — gpt-5.x / o-series reject temperature; openai::summarize detects via is_reasoning_model() and omits.
• Local-LLM summary tuning — Ollama path goes through the native /api/chat endpoint (not the OpenAI-compat shim at /v1) so think is a real toggle instead of being template-stripped. num_ctx is sized adaptively from the actual prompt length + num_predict budget, rounded up to powers of two, bounded [8192, 65536] — a flat 65K KV cache was OOMing the model runner on tighter Macs (the cache is multi-GB on top of model weights). keep_alive: 0 releases the model immediately after each summary so RAM frees up. Sampling is model-family-aware (sampling_profile(model, think)): Qwen models get the loop-breaker profile (presence_penalty=1.5, frequency_penalty=0.5, repeat_penalty=1.0) that breaks Qwen 3.5's thinking-phase loop class; every other family (e.g. the recommended gemma4:12b-mlx) gets neutral defaults (presence/frequency=0, repeat_penalty=1.1) rather than penalties tuned for Qwen. Only ollama_chat_stream (the summary path) carries this; the agentic tool-calling step (ollama_chat_step) uses neutral temperature=0. Cloud OpenAI summary streams and shares post_chat with the chat path, which retries transient Kind::Request errors twice (500 ms / 1.5 s backoff) — covering the case where reqwest tries to reuse a pooled HTTP/2 connection that OpenAI's edge has half-closed. Retries can't rescue a connection reaped while the model reasons, which is why the path streams at all; see the openai.rs row.
• Your notes over MCP (#172) — Humla ships its own Model Context Protocol server, so Claude Code, Codex or any other MCP client can search and read the library from wherever the user works. Six read-only tools (search_notes, get_note, get_transcript, list_notes, list_folders, list_clients), off until explicitly enabled in Settings → General → Integrations, which also offers a ready-to-paste config snippet per client. Read-only in this version, and no tool returns or references an audio file — keep_audio stays the single absolute gate on audio (#24) and this is not an exception above it. The workspace is resolved, never accepted as an argument, so Personal and workspace notes can't reach each other however a client asks. Search is keyword-only on purpose: an agentic client substitutes repeated queries for vector recall, which buys no API key, no per-query cost, no Keychain prompt and no network. The gap that does bite is cross-language — the index is lexical, so a query in the wrong language returns nothing, and the fix chosen is to carry each note's language as data (lang: on every hit and row, language as a filter) rather than to name any language in a tool description. Shipping it first-party is a maintenance decision as much as a feature: Granola never did, so its users wrote nine servers against its local store, and the schema became a public API maintained by strangers.
• A Note names itself (#90) — a Note whose Title Humla owns gets a short phrase derived from the head of its own content. menubar::is_replaceable_title is the single answer to "may this Title be replaced?": empty, a Recording 19 Aug 14:32 timestamp, or the Imported audio import fallback. A real filename stem is the name the user gave the file and is never touched. Two triggers, both content-settled: the post-stop recording chain, spawned concurrently with the diarize pass rather than after it — the transcript is already final by then (recording_stop drains every in-flight transcribe before it snapshots), and what diarization adds is Speaker N: prefixes a title has no use for, so waiting cost minutes on a long meeting for nothing — and a debounced body-settled checkpoint in the Note view for a Note that was typed and never recorded. It goes after record_detected_language, which resolves auto to what was actually spoken, because the title's language directive reads it. Deliberately not on summary completion — summarising is a button the user presses, so that trigger would leave the motivating case (a menu-bar recorder who never opens the Note) on its timestamp forever — and deliberately not during a recording, where it would spend GPU exactly when local Whisper needs it. The call reuses the summary path's resolve_provider, forces think: false, and substitutes a small non-reasoning model when the configured summary model reasons; it does not stream, so the summary panel never sees it. Failure is silent on the automatic path (a user who never asked for a Title must not be told one failed) and loud on ⋯ → Regenerate title, which bypasses eligibility because the user asked. One-shot by construction: once written, a Title is no longer replaceable, so a Note recorded in several takes keeps the first take's Title and regenerate is the escape hatch. The call is bracketed by a title_status event (per-note, like summary_status), which the open Note view answers by standing a shimmer in for the title box — the text there is about to be replaced, so dimming it would leave the wrong answer on screen while the right one loads. is_replaceable_title is mirrored in TS as isReplaceableTitle (src/lib/noteSync.ts) — change both.
• Transcribe on your terms (#146) — Transcribe manually (Settings → Recording → Audio retention) trades the live transcript for a quiet Mac: the meeting is captured and its per-source streams retained, and nothing reaches a provider until the user presses Transcribe on the note. Gated twice, deliberately. keep_audio is the outer gate and stays so — deferring on a device that stores no audio would discard the meeting, and force-retaining behind the user's back is the exact class of exception #24 exists to remove (sessions::defer_transcription owns the rule; Settings shows the toggle only once retention is on, so the UI and the backend agree). And it applies only to on-device Whisper, resolved through the note's own language: what the setting buys is a Mac that isn't running Metal inference for an hour, and a cloud provider costs the machine nothing, so a per-language override routing one note to Deepgram keeps its live transcript while a Norwegian note defers. The decision is read once, at recording_start — flipping the setting mid-meeting leaves the running capture alone, since half a transcript is worse than either regime. Each take carries a persisted transcribed flag in sessions.json (defaulting true, so every take in every existing library reads back as transcribed rather than offering to re-transcribe the world), and it is independent of the setting's current value: turning Transcribe manually off later still leaves old takes transcribable. Pressing Transcribe replays each pending take's retained audio through the same decode → VAD chunk → transcribe → diarize pipeline a file import uses, oldest first — because each take's timeline is numbered off the takes before it. Three things make it not-a-recording, each load-bearing: it never takes the single live-capture slot, so replaying an hour of audio doesn't stop the user recording their next meeting; it reports on its own per-note transcribe_status channel rather than recording_status, so it can't blank the bar of a recording running on another note (DiarizePolicy::report_phase is what keeps the shared diarize path quiet); and its chunks never stream into note.transcript — the text arrives once, from rebuild_note_transcript over every session's timeline, so a replay that dies half-way leaves nothing behind that no timeline accounts for (ADR-0004), and a newly transcribed take lands at its own position in the note even when a later take was already transcribed (which combine_with_snapshot, appending to the end, could not). A diarize that can't run is not a failure here (LabelFallback::Unlabelled): the text lands without Speaker N: prefixes rather than being thrown away with the labels. A take that transcribed nothing is deliberately not marked transcribed — silence and "every chunk failed" are indistinguishable from there, and the second has to stay retryable. note_sessions answers both halves of "would Transcribe do anything" as canTranscribe (untranscribed and the raw streams still on disk — session_has_replayable_audio, not session_has_audio, since a take reduced to its mixed playback.wav still has audio and still can't be replayed), so a take swept away by "Delete stored audio" never offers a button that can only fail.
  • The same machinery re-transcribes, from a RefreshCw icon button in the Transcript panel beside Copy — mirroring the Summary panel's regenerate, because the two panels' actions shouldn't read as different kinds of control. sessions::TranscribeScope is the difference and the whole of it (mirrored by the "pending" | "all" union on ipc.transcribeNote — change both): Pending finishes what a deferred capture left waiting, All re-runs every take whose raw streams survive (canRetranscribe), which is the fix for a recording that came back off the wrong language or the wrong model. takes_to_transcribe is the single answer both surfaces mirror, and it filters on replayable audio under both scopes so a run can never be offered that would silently skip every take. All reaches the legacy flat take that Pending can't (it reads back transcribed: true) — deliberately, since a note recorded before per-session storage is exactly the kind whose language needs correcting. And because All replaces text the note already had while db::set_transcript keeps no history, the command snapshots a note revision first, so the run is undoable from the note's existing revision list; Pending needs none, since it only adds takes that had no text. A take that fails mid-run stops the run without discarding what the takes before it committed — their timelines are written and their text is in the note, so the checkpoints and the asset push still happen, and the failure becomes the outcome only after.
  • A session's scalar metadata stays immutable, duration_ms included. cloud-sync's push_session derives a session's last-write-wins key from its own started_at on the stated grounds that "index / started_at / duration / streams never change", so a duration_ms corrected after the fact re-pushes under a byte-identical key and converges only because the server's hook compares < rather than <=. That is why a deferred take's length is taken from the sidecar's full_recording event at capture time (CaptureSink::captured_duration_ms → finalize_session, which prefers the timeline and falls back to this) rather than backfilled once its timeline exists. record_transcribed_take therefore writes the flag and nothing else.
• The note views are a card grid — All notes and Folder render NoteCard in a .nd-notegrid, with no date headings: the card carries its own full date (formatNoteDate) because there is nothing above it to say when. Each card is date, title, a four-line excerpt, then state · client · folder. The excerpt prefers the summary's opening line (noteExcerpt) — the distilled version of everything else on the note — falling back to typed body, then to the transcript with speaker labels stripped. "Opening line" is gated, not literal: horizontal rules, model preamble ("Her er …", anything ending in :) and fragments under 20 chars are stepped over — the first of them kept only when the summary holds nothing stronger — and a fragment body likewise yields to the transcript. Multi-select is unchanged from the old list row (#19): the same Cmd/Shift-click, Space, Esc and checkbox rules, with the checkbox moved to the card's top-right corner.
  • A borderless card needs a recessed ground. .nd-well darkens the scroll region a few percent, because the card's edge is entirely tone against what it sits on — and graphite declares --color-canvas and --color-surface both as #ffffff, which leaves a white card on a white page with nothing but a shadow. Darkening is the one move that works in every theme and both modes; in dark the surface is already the lighter of the pair, so pushing the ground down widens the same gap. For the same reason selection cannot be a fill alone — --color-accent-soft is tuned against the canvas, and graphite's #efefef lands darker than the well, so a picked card would recede. The accent ring is what reads everywhere. Everything else the card owns comes from the theme — radius from --radius-card, shadow from --shadow-card — so it is the third inset card in fact and not only in description. Under prefers-contrast: more the card takes a real border and drops the shadow: tone-as-edge is exactly what that preference asks us to stop doing.
  • .nd-notegrid's track floor is minmax(min(var(--card-min, 300px), 100%), 1fr). The min() is load-bearing — a bare minmax(300px, 1fr) cannot shrink past its floor and overflows the view sideways once the body column is narrower than one card, which it is at the layout's own minimum width.
• Folders — flat folder list, per-note assignment; sidebar search matches note titles/bodies/transcripts and renders a flat result list annotated with each note's folder name.
• Chat over your notes — an agentic retrieval loop (chat/), surfaced as the Chat tab of the Note right panel. The assistant searches and reads notes with three tools (search_notes over FTS5 keyword + semantic embeddings, get_note, list_notes), streams its answer, and cites the notes it drew from as chips that navigate to the source. Two per-conversation retrieval filters live on the conversation row and bind every turn regardless of what the model asks for: breadth (note | folder | all — what is in reach) and the authorship pin (owner_filter, a user id — whose notes are in reach, workspace-only, surfaced as the "Created by me" toggle beside the breadth picker). Multiple conversations per note, with history; a conversation can be renamed or deleted (hard, with a confirm — there is no Trash for chat) from the sidebar row's right-click menu, the /chat app bar's ⋯, or the note pane's history popover. Runs on a chat provider configured separately from transcription and summary (Settings → Chat; OpenAI or a local server). The local provider is still stored as "ollama" but is not Ollama-only (#179): the adapter picks Ollama's native /api/chat on :11434 and plain OpenAI-compat everywhere else, the same call summarize_with_base makes — mirrored in TS as isOllamaUrl, which decides only what advice to print. The embedder for semantic retrieval has its own address and name (embed_base_url / embed_model, both blank = the chat server and embeddinggemma), because the chat server is frequently not the one that can embed: mlx_lm.server serves no /v1/embeddings route at all, llama-server wants its own process, and LM Studio names the same model differently. Settings probes it with a real one-word embedding rather than reading the chat server's model listing — a listing cannot tell you a route is missing. A reachable embedder and an embedded corpus are two facts, since chunk_embeddings is keyed (text_hash, model) and search filters on the model: naming a different embedder leaves every existing vector unreachable, so chat_unembedded_note_count says how many notes that is and chat_embed_missing runs the same backfill the next launch would. localChatHint is the one readiness ladder behind both the Settings tab and the Note pane's setup prompt — the two answered the same question in two copies, and only one of them got fixed the first time. In a workspace, turns route to humla-cloud and retrieval happens server-side. Reachable inside a note, at its own library-wide /chat route, and — scoped to one folder — at /folder/:id/chat ("Chat about this folder" on the folder row's context menu, #110).
  • A conversation's scope is (tenant, scope, scope_id) — note (the note id), folder (the folder id, #110) or global (a fixed sentinel). The three populations never leak into each other, locally or server-side. Breadth is a live filter within a thread, not its identity: a Note's pane can widen to its folder or the library, but a note-less target is pinned to its own reach (all for /chat, folder for a folder chat) because with no anchor the target's identity is its reach — pinned_breadth / targetPinsScope own that rule on each side. Deleting a folder hard-deletes its conversations (they have no Trash and their whole reach was that folder); its notes only reparent, so the Sidebar confirms only when threads would actually be lost. ChatTarget::from_ids takes the note and folder ids as alternatives — both at once is an error, and an empty id is an error under either.
  • Opening a pane drafts or resumes, by target (resumes_on_open in Rust, targetResumesOnOpen in TS — mirrored, change both). /chat drafts: it opens on an unsaved conversation with the prompt cards showing, persisting nothing until the first turn, so an abandoned thread leaves no row and "+" is a local reset that costs no IPC. A note's Chat tab resumes that note's most-recent thread, deliberately — a note is an anchor, so continuing the same line of thinking is the plausible default there. A drafting pane's breadth and authorship pin live in the pane and ride in on chat_send (DraftSettings), which is what preserves the #61/#103 guarantee without the lazy row that used to be hidden from the list yet still resolved to by the next send. Conversation lists hide zero-message rows for drafting targets only (in a note, an empty thread is the draft being resumed) — in SQL for Personal so paging stays honest, and off the server's message_count for a workspace, whose messages never land in the local table.
• Every word of the transcript has a timeline behind it (#169, ADR-0004) — the styled reader renders from the merged timelines, so text note.transcript carries and no timeline accounts for is invisible there, and the first rebuild (rebuild_note_transcript, reached by cycling a speaker label, deleting a chunk, re-diarizing or unifying) deletes it outright — out of the summary, chat retrieval and embeddings with it. It arose because combine_with_snapshot prepends the prior take's transcript while serialize_timeline sees the current session's chunks alone, so any recording that landed text but wrote no session assets left an orphaned prefix. Three defences, in order:
  • Prevention. diarize_and_apply no longer returns early when the diarize model is missing; it serializes the chunks with an empty label — and the label alone, since word timings are orthogonal to who spoke — so the take still writes a session — which also gives a later re-diarize somewhere to attach to.
  • Repair on open. note_timeline_repair compares comparable_words of the transcript against the timelines' projection; orphaned_prefix recovers the leading lines the projection misses and synthesize_orphan_timeline writes them as a session at manifest index 0 (named "Earlier transcript", never "Recording 0" — it was never a take). Idempotent, and it runs after the workspace session pull so a still-arriving timeline can't be duplicated. A note with no timeline at all is left alone — it renders the textarea and hides nothing, and a synthesized session would take its free-text editing away.
  • Render-time guard. When coversTranscript comes back false — a timeline present but short because malformed lines were skipped, an asset that never downloaded — the note falls back to TranscriptView over the whole string. The comparison is the backend's; the client never re-derives the grouping rule (Rust groups on label alone, groupTimeline on label and session, deliberately), and comparisons normalize word sequences, never line counts. One more mirrored pair to change together: split_label (Rust) and parseTranscriptLines (Note.tsx) are the same "Label: prefix" rule, and the repair splits lines by the first while the reader draws them by the second. Separately, commit_rebuilt_transcript refuses to commit an empty projection over a non-empty transcript, which protects pre-sessions notes that have a transcript and no timeline at all from being blanked by one click.
• Transcript editing routes through the timeline (#170) — on a note that has a timeline, the styled reader renders from it and note.transcript is a projection of it, so editing is per turn: a hover pencil opens that turn in place, and committing calls note_timeline_set_chunk_text with the whole run of chunk indices the turn spans (TimelineGroup.indices), which writes the text into the lowest index, empties the rest, and re-derives the transcript from every session's timeline — the same path note_timeline_set_chunk_label and note_timeline_delete_chunk already take. Word timings on an edited turn are dropped (they describe words that are gone); its bounds are kept, so it still highlights during playback and only per-word karaoke is lost, on that turn. The old panel-wide textarea wrote the derived copy and touched no timeline, so the edit was invisible in the reader — permanently — while summary, chat and embeddings read the edited string. A note with no timeline keeps the whole-transcript textarea (TranscriptEditor): with no timeline there is no second copy to orphan. Both are locked while a recording is in flight.