> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deepslate.eu/llms.txt
> Use this file to discover all available pages before exploring further.

# LiveKit Plugin (Python)

> Integrate Deepslate voice AI with LiveKit Agents (Python) for real-time voice applications

The `deepslate-livekit` package provides a `RealtimeModel` implementation for the [LiveKit Agents](https://github.com/livekit/agents) framework, enabling seamless integration with Deepslate's unified voice AI infrastructure.

> [!NOTE]
>
> Using the **Node.js / TypeScript** LiveKit framework instead? See the [LiveKit Plugin (Node.js)](https://docs.deepslate.eu/livekit-node.md) page for the `@deepslate-labs/livekit` package.

> [!NOTE]
>
> This plugin lives in the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks). We welcome contributions — feel free to open issues or pull requests there.

## Prerequisites

* A Deepslate account with API credentials
* Python 3.11+
* LiveKit server and API credentials
* (Optional) ElevenLabs API key for server-side TTS

## Installation

```bash theme={null}
pip install deepslate-livekit
```

## Environment Variables

Set up your credentials as environment variables:

| Variable | Required | Description |
| - | - | - |
| `DEEPSLATE_VENDOR_ID` | Yes | Your Deepslate vendor ID |
| `DEEPSLATE_ORGANIZATION_ID` | Yes | Your Deepslate organization ID |
| `DEEPSLATE_API_KEY` | Yes | Your Deepslate API key |
| `DEEPSLATE_MODEL` | No | Opal model to run on. Defaults to the platform default model |
| `ELEVENLABS_API_KEY` | No | ElevenLabs API key for server-side TTS |
| `ELEVENLABS_VOICE_ID` | No | ElevenLabs voice ID |
| `ELEVENLABS_MODEL_ID` | No | ElevenLabs model (e.g., `eleven_turbo_v2`) |

> [!WARNING]
>
> Never expose your Deepslate or ElevenLabs API keys to clients. This plugin is for **server-side use** with LiveKit Agents.

## Quick Start

```python theme={null}
from typing import cast

from livekit import agents
from livekit.agents import AgentServer, AgentSession, Agent, room_io

import deepslate.livekit
from deepslate.livekit import DeepslateRealtimeSession, ElevenLabsTtsConfig

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful voice AI assistant.")

    async def on_enter(self) -> None:
        session = cast(DeepslateRealtimeSession, self.realtime_llm_session)
        await session.speak_direct("Hello! How can I help you today?")

server = AgentServer()

@server.rtc_session()
async def my_agent(ctx: agents.JobContext):
    session = AgentSession(
        llm=deepslate.livekit.RealtimeModel(
            tts_config=ElevenLabsTtsConfig.from_env()
        ),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_options=room_io.RoomOptions(),
    )

if __name__ == "__main__":
    agents.cli.run_app(server)
```

## Configuration Reference

> [accordion] RealtimeModel Parameters
>
> | Parameter | Type | Default | Description |
> | - | - | - | - |
> | `vendor_id` | `str \| None` | env: `DEEPSLATE_VENDOR_ID` | Deepslate vendor ID |
> | `organization_id` | `str \| None` | env: `DEEPSLATE_ORGANIZATION_ID` | Deepslate organization ID |
> | `api_key` | `str \| None` | env: `DEEPSLATE_API_KEY` | Deepslate API key |
> | `base_url` | `str` | `https://app.deepslate.eu` | Base URL for the Deepslate API |
> | `model` | `DeepslateModel \| str \| None` | env: `DEEPSLATE_MODEL`, else platform default | Opal model to run on, see [Model Selection](#model-selection) |
> | `system_prompt` | `str` | `"You are a helpful assistant."` | Default system prompt |
> | `temperature` | `float` | `0.3` | Sampling temperature (0.0–2.0) |
> | `generate_reply_timeout` | `float` | `30.0` | Timeout in seconds for `generate_reply` (0 = no timeout) |
> | `ws_url` | `str \| None` | `None` | Direct WebSocket URL override — useful for local development |
> | `tts_config` | `ElevenLabsTtsConfig \| HostedTtsConfig \| HostedVoiceCloneConfig \| None` | `None` | TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis, `HostedTtsConfig` for a pre-cloned Deepslate-hosted voice, or `HostedVoiceCloneConfig` to clone a voice on the fly from an audio sample. |
> | `http_session` | `aiohttp.ClientSession \| None` | `None` | Shared aiohttp session |
> | `vad_confidence_threshold` | `float` | `0.4` | Minimum confidence to consider audio as speech (0.0–1.0) |
> | `vad_min_volume` | `float` | `0.0` | Minimum volume threshold (0.0–1.0) |
> | `vad_start_duration_ms` | `int` | `150` | Duration of speech to detect start (ms) |
> | `vad_stop_duration_ms` | `int` | `390` | Duration of silence to detect speech end (ms) |
> | `vad_backbuffer_duration_ms` | `int` | `1000` | Audio buffered before speech detection (ms) |

> [accordion] VAD Configuration
>
> Voice Activity Detection is handled **server-side** by Deepslate. You tune it via the `vad_*` parameters on `RealtimeModel` — no client-side VAD pipeline is needed.
>
> | Parameter | Default | Description |
> | - | - | - |
> | `vad_confidence_threshold` | `0.4` | Minimum confidence score to classify audio as speech (0.0–1.0) |
> | `vad_min_volume` | `0.0` | Minimum audio volume to consider (0.0–1.0) |
> | `vad_start_duration_ms` | `150` | Consecutive speech duration required to start a turn (ms) |
> | `vad_stop_duration_ms` | `390` | Silence duration required to end a turn (ms) |
> | `vad_backbuffer_duration_ms` | `1000` | Audio buffered before the detection window (ms) |

> [accordion] HostedTtsConfig
>
> Use a voice cloned and hosted within Deepslate — no external TTS provider credentials required. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output.
>
> | Parameter | Type | Default | Description |
> | - | - | - | - |
> | `voice_id` | `str` | required | The ID of the hosted (cloned) voice to use for synthesis |
> | `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |
>
> **`HostedTtsMode` values:**
>
> | Value | Description |
> | - | - |
> | `HIGH_QUALITY` | Best output quality with still relatively low latency. Recommended for most use cases (default). |
> | `LOW_LATENCY` | Low latency generation mode that takes next to no time to complete. Output quality may be significantly reduced. |
>
> ```python theme={null}
> from deepslate.livekit import HostedTtsConfig, HostedTtsMode
>
> # Default — high quality
> llm = deepslate.livekit.RealtimeModel(
>     tts_config=HostedTtsConfig(voice_id="de56bbd1-3ea7-4543-a17c-504664d9116b")
> )
>
> # Explicit low latency mode
> llm = deepslate.livekit.RealtimeModel(
>     tts_config=HostedTtsConfig(
>         voice_id="de56bbd1-3ea7-4543-a17c-504664d9116b",
>         mode=HostedTtsMode.LOW_LATENCY,
>     )
> )
> ```

> [accordion] HostedVoiceCloneConfig
>
> Clone a voice on the fly from a raw audio sample, with no pre-uploaded voice ID required. The sample is sent as part of the session initialization request. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output.
>
> | Parameter | Type | Default | Description |
> | - | - | - | - |
> | `audio_data` | `bytes` | required | Raw 16-bit signed little-endian PCM audio data of the voice sample (10 to 15 seconds of speech) |
> | `audio_sample_rate` | `int` | required | Sample rate of the audio sample in Hz |
> | `audio_channels` | `int` | required | Number of audio channels in the sample |
> | `ref_text` | `str` | required | Exact transcript of the voice sample, including disfluencies and false starts |
> | `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |
>
> The `mode` parameter accepts the same `HostedTtsMode` values as `HostedTtsConfig` above.
>
> ```python theme={null}
> from pathlib import Path
> from deepslate.livekit import HostedVoiceCloneConfig
>
> llm = deepslate.livekit.RealtimeModel(
>     tts_config=HostedVoiceCloneConfig(
>         audio_data=Path("voice_sample.pcm").read_bytes(),
>         audio_sample_rate=24000,
>         audio_channels=1,
>         ref_text="Hi there, um, thanks for calling. How can I help you today?",
>     )
> )
> ```
>
> > [!TIP]
> >
> > For best results, provide a clean 10 to 15 second sample and a `ref_text` that transcribes the audio exactly, including filler words and false starts. An accurate transcript improves the quality of the cloned voice.

> [accordion] ElevenLabsTtsConfig
>
> Configure server-side text-to-speech with ElevenLabs. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output and automatic interruption handling.
>
> | Parameter | Type | Description |
> | - | - | - |
> | `api_key` | `str` | ElevenLabs API key (env: `ELEVENLABS_API_KEY`) |
> | `voice_id` | `str` | Voice ID (env: `ELEVENLABS_VOICE_ID`) |
> | `model_id` | `str \| None` | Model ID, e.g., `eleven_turbo_v2` (env: `ELEVENLABS_MODEL_ID`) |
> | `location` | `ElevenLabsLocation` | API endpoint region — `US` (default), `EU`, or `INDIA` |
> | `voice_settings` | `ElevenLabsVoiceSettingsConfig \| None` | Fine-grained voice control (see below) |
>
> Use `ElevenLabsTtsConfig.from_env()` to create a config from environment variables.
>
> **`ElevenLabsVoiceSettingsConfig`** — fine-grained control over the synthesized voice:
>
> | Parameter | Type | Description |
> | - | - | - |
> | `stability` | `float \| None` | Voice consistency (0.0–1.0); higher = more stable |
> | `similarity_boost` | `float \| None` | Clarity and similarity to the original voice (0.0–1.0) |
> | `style` | `float \| None` | Style exaggeration (0.0–1.0) |
> | `use_speaker_boost` | `bool \| None` | Boost similarity to the original speaker |
> | `speed` | `float \| None` | Speaking speed multiplier |
>
> ```python theme={null}
> from deepslate.livekit import (
>     ElevenLabsTtsConfig,
>     ElevenLabsVoiceSettingsConfig,
>     ElevenLabsLocation,
> )
>
> tts_config = ElevenLabsTtsConfig.from_env(
>     location=ElevenLabsLocation.EU,
>     voice_settings=ElevenLabsVoiceSettingsConfig(
>         stability=0.7,
>         similarity_boost=0.85,
>         speed=1.1,
>     ),
> )
> ```
>
> > [!TIP]
> >
> > When using ElevenLabs TTS, automatic interruption handling (context truncation) is enabled. The server tracks exactly what was spoken before the interruption, keeping the model's context accurate. Without server-side TTS, you can use LiveKit's standard TTS integration, but this interruption context tracking will not be available.

## Model Selection

Connections use the platform's default Opal model unless you pin one with `model`:

```python theme={null}
from deepslate.livekit import DeepslateModel, RealtimeModel

# opal-v2.1 is the current platform default.
# See Available Models for other model ids.
llm = RealtimeModel(model=DeepslateModel.OPAL_V2_1)

# Equivalent, using the model id as a plain string:
# llm = RealtimeModel(model="opal-v2.1")
```

See [Available Models](https://docs.deepslate.eu/opal.md#available-models) for the current list and how to request access to preview models.

## Features

> [card] Real-time Voice Streaming
>
> Low-latency bidirectional audio streaming for natural conversations

> [card] Server-side VAD
>
> Voice activity detection handled server-side for reliable, configurable speech detection

> [card] Function Tools
>
> Define and use function tools with the `@function_tool()` decorator

> [card] ElevenLabs TTS
>
> Server-side TTS with regional endpoints and fine-grained voice settings

> [card] Low Latency Mode
>
> Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality

> [card] Voice Cloning
>
> Use a pre-cloned hosted voice or clone one on the fly from a raw audio sample, with no external TTS credentials

> [card] Direct Speech
>
> Speak text directly via TTS without routing through the LLM

> [card] Conversation Queries
>
> Run one-shot side-channel inference without affecting the main conversation

> [card] Chat History Export
>
> Export the full conversation history on demand

> [card] Live Configuration
>
> Update the system prompt and temperature mid-session without reconnecting

## Sending a Welcome Message

To greet the user, speak directly the moment the agent becomes active. Override `Agent.on_enter()` and call `speak_direct()` on the realtime session that the `AgentSession` created for you, reachable via `self.realtime_llm_session`. `speak_direct()` buffers the utterance until the session is ready, so no fixed delay or event handling is needed:

```python theme={null}
from typing import cast

from livekit import agents
from livekit.agents import AgentSession, Agent
import deepslate.livekit
from deepslate.livekit import DeepslateRealtimeSession, ElevenLabsTtsConfig

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful voice AI assistant.")

    async def on_enter(self) -> None:
        session = cast(DeepslateRealtimeSession, self.realtime_llm_session)
        await session.speak_direct(
            "Please note that this call is handled by an AI and may be recorded.",
            uninterruptable=True,
        )

@server.rtc_session()
async def my_agent(ctx: agents.JobContext):
    model = deepslate.livekit.RealtimeModel(
        tts_config=ElevenLabsTtsConfig.from_env()
    )
    session = AgentSession(llm=model)
    await session.start(room=ctx.room, agent=Assistant())
```

## Function Tools

Use the `@function_tool()` decorator to give your agent capabilities:

```python theme={null}
from livekit.agents import function_tool

@function_tool()
async def lookup_weather(location: str) -> str:
    """Get the current weather for a location.

    Args:
        location: The city or location to get weather for
    """
    # Your implementation here
    return f"The weather in {location} is sunny and 22°C"

class WeatherAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a helpful weather assistant.",
            tools=[lookup_weather],
        )
```

## The Deepslate Session

For Deepslate-specific capabilities (direct speech, conversation queries, history export), use the `DeepslateRealtimeSession` that the `AgentSession` created for the active agent. Don't call `session.llm.session()` for this, since it creates a new session that isn't connected to the running call.

```python theme={null}
from typing import cast

from deepslate.livekit import DeepslateRealtimeSession

# session is the AgentSession you started
rt = cast(DeepslateRealtimeSession, session.current_agent.realtime_llm_session)
```

Inside an agent method such as `on_enter()`, use `self.realtime_llm_session` instead.

## Direct Speech

`speak_direct()` lets you synthesize and play audio directly — bypassing the LLM entirely. This is useful for scripted prompts, confirmations, or fallback messages.

```python theme={null}
await rt.speak_direct(
    "Welcome back! How can I help you today?",
    include_in_history=True,  # Record as an assistant turn (default: True)
    uninterruptable=False,    # Allow the user to barge in (default: False)
)
```

Setting `include_in_history=False` speaks the text without adding it to the conversation context — ideal for system-level announcements.

Setting `uninterruptable=True` makes the utterance play to completion: overlapping user speech is ignored until playback finishes.

## Conversation Queries

`query_conversation()` runs a one-shot inference call on a side channel, separate from the main conversational turn. The result is returned as a string and does **not** affect the conversation history or trigger any audio.

```python theme={null}
summary = await rt.query_conversation(
    prompt="Summarize the conversation so far in one sentence.",
)
print(summary)  # e.g., "The user asked about weather in Berlin."
```

You can also pass `instructions` to further constrain the model's output format.

## Chat History Export

Export the full conversation history as a list of structured message dicts at any point during a session:

```python theme={null}
# Listen for the export result
@rt.on("chat_history_exported")
def on_history(messages):
    for msg in messages:
        print(msg["role"], msg["content"])

# Request the export
await rt.export_chat_history(
    await_pending=False,  # Set True to wait for any in-flight operations first
    exclude_audio=False,  # Set True to omit audio blobs (transcripts only)
)
```

Each message follows the `ChatMessageDict` structure with `role`, `delivery_status`, `ephemeral`, and a `content` list of typed content blocks (`text`, `input_audio`, `tool_call`, `tool_result`, etc.).

## Live Configuration

Update the system prompt or temperature mid-session without reconnecting:

```python theme={null}
await session.llm.update_options(
    system_prompt="You are now a concise assistant. Keep replies under two sentences.",
    temperature=0.8,
)
```

Changes take effect on the next model turn.

## Contributing

This plugin is open source. Visit the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks) to:

* Report issues
* Submit pull requests
* Request features

## Next Steps

> [card] [WebSocket API](https://docs.deepslate.eu/websocket.md)
>
> Low-level WebSocket access for custom integrations

> [card] [API Reference](https://docs.deepslate.eu/api-reference/realtime.md)
>
> Full message schemas and configuration options

> [card] [LiveKit Agents Docs](https://docs.livekit.io/agents/)
>
> LiveKit Agents framework documentation

> [card] [GitHub Repository](https://github.com/deepslate-labs/deepslate-sdks)
>
> Source code, issues, and contributions


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.