Voice AI Agents use three core components to support voice interactions:

• Large Language Model (LLM): Understands user requests and generates responses.
• Automatic Speech Recognition (ASR): Converts spoken input into text.
• Text-to-Speech (TTS): Converts generated responses into spoken audio.

While the LLM can inherently understand and generate content in multiple languages, ASR and TTS require language-specific configurations. To support dynamic language switching during a conversation, all three components must work together.

For example, when a user requests a switch from English to German, the system must:

• Transcribe speech using the appropriate ASR language configuration.
• Generate responses in the selected language.
• Use a matching TTS voice to produce spoken output in that language.

This behaviour is managed through:

• Speech Profiles, which define the available ASR and TTS configurations.
• Language Tasks, which detect and process language change requests.
• Context Capture Tools, which store the selected language and apply it throughout the conversation.

Prerequisites

Before enabling dynamic language switching, ensure that:

• ASR and TTS configurations are available for all supported languages.
• Each supported language includes both speech recognition and speech synthesis capabilities.

Configure Dynamic Language Switching

Follow these steps to enable real-time language switching during a conversation.

Step 1: Configure ASR and TTS Languages

Configure the required ASR and TTS providers for every language that your Voice AI Agent will support.

Step 2: Create a Speech Profile

Create a Speech Profile that includes all supported ASR and TTS language configurations.

Steps

1. Open Speech Profile settings.
2. Add ASR models for each supported language.
3. Add corresponding TTS voices for each language.
4. Assign the Speech Profile to your Voice AI Agent.

This configuration allows the agent to switch between supported languages during runtime.

Step 3: Create a Language Task

Create a dedicated task to detect and process language change requests, such as:

• "Switch to German"
• "Speak Spanish"
• "Can we continue in French?"

Recommended Configuration

Suggested Instructions

Instructions:Detect if the user wants to change the language.If no specific language is mentioned, ask the user to choose from <list of available languages>.If a language is mentioned, verify it exists in <list of available languages>.If not valid or unclear, ask the user to select from the available options.Once confirmed, map the selected language to its corresponding language code (e.g., "en", "de").Call <insert tool here> to update the language code.Confirm the language change briefly in the selected language

Step 4: Create a Context Capture Tool

Create a Context Capture Tool to store the selected language and maintain language consistency throughout the conversation.

Recommended Configuration

How It Works

When a user requests a language change:

1. The Language Task detects the request.
2. The selected language is validated against the supported language list.
3. The Context Capture Tool updates the Language variable.
4. ASR begins transcribing speech using the newly selected language configuration.
5. The LLM generates responses in the selected language.
6. TTS uses the corresponding voice and language settings for spoken output.

The language preference remains active for the rest of the conversation, ensuring a consistent multilingual experience.