rasa-integrating-asr

Integrating a Custom ASR Provider

The provider documentation is: $documentation

Use the documentation as the source of truth for the provider protocol. If it is missing, ask for the provider's streaming ASR API documentation before implementing. Follow links from it only when needed to resolve authentication, WebSocket messages, audio formats, endpointing, or limits.

Workflow

1. Inspect the assistant's installed Rasa version and the actual ASREngine, ASREngineConfig, ASREvent, and RasaAudioBytes APIs. Follow those signatures when they differ from examples.
2. Review existing custom speech components and project conventions before choosing a module path. Default to addons/custom_asr.py only when no convention exists.
3. Read the provider documentation and complete the feasibility checks in references/integration-framework.md.
4. If a hard requirement is unsupported or undocumented, report every incompatibility found and stop. Do not invent protocol messages, transcript semantics, or client-side endpointing.
5. Design the configuration and event mapping, then implement the custom ASREngine.
6. Configure its fully qualified class path under the voice channel's asr key in credentials.yml. Keep secrets in environment variables.
7. Add focused tests for provider-message mapping, transcript accumulation, audio conversion, and malformed/error events.
8. Run the project's relevant tests, lint/type checks, and Rasa configuration validation. Report any validation that could not be run.

Feasibility gate

A custom ASR integration must have:

• a documented real-time streaming API compatible with the ASREngine connection lifecycle;
• a documented end-of-utterance or end-of-turn signal that can produce exactly one NewTranscript;
• an audio encoding and sample rate that the provider accepts directly or that can be converted safely from RasaAudioBytes;
• documented authentication suitable for a server process;
• clear partial, final, error, and connection-close message semantics.