godot-whisper copilot-instructions.md

Copilot Instructions — Godot Whisper

Project Overview

Godot Whisper is a GDExtension plugin that integrates whisper.cpp into the Godot Engine for real-time and offline speech-to-text transcription. It targets Godot 4.2+ via godot-cpp.

Architecture

• Build system: SCons (SConstruct at root). No CMake for the plugin itself.
• Language: C++ (godot-cpp style). Source files live in src/.
• Third-party deps are git submodules under thirdparty/:
  • whisper.cpp (v1.8.4, ggml-org/whisper.cpp) — the ML inference engine
  • godot-cpp (branch 4.2) — Godot C++ bindings
  • opencl_headers, opencl_icd_loader — OpenCL support (headers & ICD loader)
  • libsamplerate (0.1.9) — audio resampling
• After cloning, run git submodule update --init --recursive.

Key Source Files

File	Purpose
src/speech_to_text.h/.cpp	Main SpeechToText node — transcription, VAD, resampling
src/resource_whisper.h/.cpp	ResourceWhisper — wraps a whisper model binary
src/resource_loader_whisper.h/.cpp	Custom ResourceLoader for .bin model files
src/register_types.h/.cpp	GDExtension entry point — class registration
SConstruct	Build script (all platforms)
include/config.h, include/build_config.h	Build-time configuration

whisper.cpp v1.8.4 Layout

The submodule has a new directory structure (changed from the old monolithic layout):

• include/whisper.h — public API header
• src/whisper.cpp — whisper implementation
• ggml/include/ — ggml public headers (ggml.h, ggml-cpu.h, ggml-metal.h, ggml-backend.h, …)
• ggml/src/ — ggml core (ggml.c, ggml.cpp, ggml-backend.cpp, ggml-alloc.c, …)
• ggml/src/ggml-cpu/ — CPU backend (+ arch/ subdirs for arm, x86, wasm, …)
• ggml/src/ggml-metal/ — Metal backend (multiple .cpp/.m files)
• ggml/src/ggml-opencl/ — OpenCL backend (self-contained, embedded kernels)
• ggml/src/ggml-vulkan/ — Vulkan backend (auto-detected when glslc available)
• ggml/src/ggml-webgpu/ — WebGPU backend (opt-in, requires Emscripten 4.0.10+)