rasa-integrating-tts

Integrating a Custom TTS Provider

The provider documentation is: $documentation

Use the documentation as the source of truth for the provider protocol. If it is missing, ask for the provider's TTS API documentation before implementing. Follow links from it only when needed to resolve authentication, request/response messages, audio formats, streaming behavior, interruption, or limits.

Workflow

1. Inspect the assistant's installed Rasa version and the actual TTSEngine, TTSEngineConfig, and RasaAudioBytes APIs. Follow those signatures when they differ from examples.
2. Review existing custom speech components and project conventions before choosing a module path. Default to addons/custom_tts.py only when no convention exists.
3. Read the provider documentation and complete the protocol analysis in references/integration-framework.md.
4. Choose streaming or non-streaming input from documented provider behavior. Do not mark an engine as streaming merely because its response audio arrives in chunks.
5. Design the configuration and audio conversion, then implement the custom TTSEngine.
6. Configure its fully qualified class path under the voice channel's tts key in credentials.yml. Keep secrets in environment variables.
7. Add focused tests for configuration, request messages, audio conversion, lifecycle, and provider errors.
8. Run the project's relevant tests, lint/type checks, and Rasa configuration validation. Report any validation that could not be run.

Select the synthesis mode

Use non-streaming input when the provider requires the complete text before synthesis:

• keep streaming_input false;
• implement synthesize() as the complete text-to-audio lifecycle;
• yield RasaAudioBytes as provider audio becomes available.

Use streaming input only when the provider accepts multiple incremental text chunks in one synthesis session:

• set streaming_input = True;
• implement the installed contract's connection, text-chunk, text-done, and audio streaming methods;
• preserve ordering and flush semantics documented by the provider.