LiveKit Plugin (Python)

The deepslate-livekit package provides a RealtimeModel implementation for the LiveKit Agents framework, enabling seamless integration with Deepslate’s unified voice AI infrastructure.

Using the Node.js / TypeScript LiveKit framework instead? See the LiveKit Plugin (Node.js) page for the @deepslate-labs/livekit package.

This plugin lives in the deepslate-sdks monorepo. We welcome contributions — feel free to open issues or pull requests there.

Prerequisites

A Deepslate account with API credentials
Python 3.11+
LiveKit server and API credentials
(Optional) ElevenLabs API key for server-side TTS

Installation

pip install deepslate-livekit

Environment Variables

Set up your credentials as environment variables:

Variable	Required	Description
`DEEPSLATE_VENDOR_ID`	Yes	Your Deepslate vendor ID
`DEEPSLATE_ORGANIZATION_ID`	Yes	Your Deepslate organization ID
`DEEPSLATE_API_KEY`	Yes	Your Deepslate API key
`ELEVENLABS_API_KEY`	No	ElevenLabs API key for server-side TTS
`ELEVENLABS_VOICE_ID`	No	ElevenLabs voice ID
`ELEVENLABS_MODEL_ID`	No	ElevenLabs model (e.g., `eleven_turbo_v2`)

Never expose your Deepslate or ElevenLabs API keys to clients. This plugin is for server-side use with LiveKit Agents.

Quick Start

from livekit import agents
from livekit.agents import AgentServer, AgentSession, Agent, room_io

import deepslate.livekit
from deepslate.livekit import ElevenLabsTtsConfig

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful voice AI assistant.")

server = AgentServer()

@server.rtc_session()
async def my_agent(ctx: agents.JobContext):
    session = AgentSession(
        llm=deepslate.livekit.RealtimeModel(
            tts_config=ElevenLabsTtsConfig.from_env()
        ),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_options=room_io.RoomOptions(),
    )

    await session.generate_reply(
        instructions="Greet the user and offer your assistance."
    )

if __name__ == "__main__":
    agents.cli.run_app(server)

Configuration Reference

RealtimeModel Parameters

Parameter	Type	Default	Description
`vendor_id`	`str \| None`	env: `DEEPSLATE_VENDOR_ID`	Deepslate vendor ID
`organization_id`	`str \| None`	env: `DEEPSLATE_ORGANIZATION_ID`	Deepslate organization ID
`api_key`	`str \| None`	env: `DEEPSLATE_API_KEY`	Deepslate API key
`base_url`	`str`	`https://app.deepslate.eu`	Base URL for the Deepslate API
`system_prompt`	`str`	`"You are a helpful assistant."`	Default system prompt
`temperature`	`float`	`1.0`	Sampling temperature (0.0–2.0)
`generate_reply_timeout`	`float`	`30.0`	Timeout in seconds for `generate_reply` (0 = no timeout)
`ws_url`	`str \| None`	`None`	Direct WebSocket URL override — useful for local development
`tts_config`	`ElevenLabsTtsConfig \| HostedTtsConfig \| None`	`None`	TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis or `HostedTtsConfig` for Deepslate-hosted cloned voices.
`http_session`	`aiohttp.ClientSession \| None`	`None`	Shared aiohttp session
`vad_confidence_threshold`	`float`	`0.5`	Minimum confidence to consider audio as speech (0.0–1.0)
`vad_min_volume`	`float`	`0.01`	Minimum volume threshold (0.0–1.0)
`vad_start_duration_ms`	`int`	`200`	Duration of speech to detect start (ms)
`vad_stop_duration_ms`	`int`	`500`	Duration of silence to detect speech end (ms)
`vad_backbuffer_duration_ms`	`int`	`1000`	Audio buffered before speech detection (ms)

VAD Configuration

Voice Activity Detection is handled server-side by Deepslate. You tune it via the vad_* parameters on RealtimeModel — no client-side VAD pipeline is needed.

Parameter	Default	Description
`vad_confidence_threshold`	`0.5`	Minimum confidence score to classify audio as speech (0.0–1.0)
`vad_min_volume`	`0.01`	Minimum audio volume to consider (0.0–1.0)
`vad_start_duration_ms`	`200`	Consecutive speech duration required to start a turn (ms)
`vad_stop_duration_ms`	`500`	Silence duration required to end a turn (ms)
`vad_backbuffer_duration_ms`	`1000`	Audio buffered before the detection window (ms)

HostedTtsConfig

Use a voice cloned and hosted within Deepslate — no external TTS provider credentials required. Pass an instance to RealtimeModel(tts_config=...) to enable audio output.

Parameter	Type	Default	Description
`voice_id`	`str`	required	The ID of the hosted (cloned) voice to use for synthesis
`mode`	`HostedTtsMode`	`HostedTtsMode.HIGH_QUALITY`	Quality/latency tradeoff for synthesis

HostedTtsMode values:

Value	Description
`HIGH_QUALITY`	Best output quality with still relatively low latency. Recommended for most use cases (default).
`LOW_LATENCY`	Low latency generation mode that takes next to no time to complete. Output quality may be significantly reduced.

from deepslate.livekit import HostedTtsConfig, HostedTtsMode

# Default — high quality
llm = deepslate.livekit.RealtimeModel(
    tts_config=HostedTtsConfig(voice_id="c3dfa73f-a1ab-4aad-b48a-0e9b9fe4a69f")
)

# Explicit low latency mode
llm = deepslate.livekit.RealtimeModel(
    tts_config=HostedTtsConfig(
        voice_id="c3dfa73f-a1ab-4aad-b48a-0e9b9fe4a69f",
        mode=HostedTtsMode.LOW_LATENCY,
    )
)

ElevenLabsTtsConfig

Configure server-side text-to-speech with ElevenLabs. Pass an instance to RealtimeModel(tts_config=...) to enable audio output and automatic interruption handling.

Parameter	Type	Description
`api_key`	`str`	ElevenLabs API key (env: `ELEVENLABS_API_KEY`)
`voice_id`	`str`	Voice ID (env: `ELEVENLABS_VOICE_ID`)
`model_id`	`str \| None`	Model ID, e.g., `eleven_turbo_v2` (env: `ELEVENLABS_MODEL_ID`)
`location`	`ElevenLabsLocation`	API endpoint region — `US` (default), `EU`, or `INDIA`
`voice_settings`	`ElevenLabsVoiceSettingsConfig \| None`	Fine-grained voice control (see below)

Use ElevenLabsTtsConfig.from_env() to create a config from environment variables.ElevenLabsVoiceSettingsConfig — fine-grained control over the synthesized voice:

Parameter	Type	Description
`stability`	`float \| None`	Voice consistency (0.0–1.0); higher = more stable
`similarity_boost`	`float \| None`	Clarity and similarity to the original voice (0.0–1.0)
`style`	`float \| None`	Style exaggeration (0.0–1.0)
`use_speaker_boost`	`bool \| None`	Boost similarity to the original speaker
`speed`	`float \| None`	Speaking speed multiplier

from deepslate.livekit import (
    ElevenLabsTtsConfig,
    ElevenLabsVoiceSettingsConfig,
    ElevenLabsLocation,
)

tts_config = ElevenLabsTtsConfig.from_env(
    location=ElevenLabsLocation.EU,
    voice_settings=ElevenLabsVoiceSettingsConfig(
        stability=0.7,
        similarity_boost=0.85,
        speed=1.1,
    ),
)

When using ElevenLabs TTS, automatic interruption handling (context truncation) is enabled. The server tracks exactly what was spoken before the interruption, keeping the model’s context accurate. Without server-side TTS, you can use LiveKit’s standard TTS integration, but this interruption context tracking will not be available.

Features

Real-time Voice Streaming

Low-latency bidirectional audio streaming for natural conversations

Server-side VAD

Voice activity detection handled server-side for reliable, configurable speech detection

Function Tools

Define and use function tools with the @function_tool() decorator

ElevenLabs TTS

Server-side TTS with regional endpoints and fine-grained voice settings

Low Latency Mode

Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality

Direct Speech

Speak text directly via TTS without routing through the LLM

Conversation Queries

Run one-shot side-channel inference without affecting the main conversation

Chat History Export

Export the full conversation history on demand

Live Configuration

Update the system prompt and temperature mid-session without reconnecting

Sending a Welcome Message

To greet the user, speak directly the moment the agent becomes active. Override Agent.on_enter() and call speak_direct() on the realtime session that the AgentSession created for you, reachable via self.realtime_llm_session. speak_direct() buffers the utterance until the session is ready, so no fixed delay or event handling is needed:

from typing import cast

from livekit import agents
from livekit.agents import AgentSession, Agent
import deepslate.livekit
from deepslate.livekit import DeepslateRealtimeSession, ElevenLabsTtsConfig

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You are a helpful voice AI assistant.")

    async def on_enter(self) -> None:
        session = cast(DeepslateRealtimeSession, self.realtime_llm_session)
        await session.speak_direct(
            "Please note that this call is handled by an AI and may be recorded.",
            uninterruptable=True,
        )

@server.rtc_session()
async def my_agent(ctx: agents.JobContext):
    model = deepslate.livekit.RealtimeModel(
        tts_config=ElevenLabsTtsConfig.from_env()
    )
    session = AgentSession(llm=model)
    await session.start(room=ctx.room, agent=Assistant())

Function Tools

Use the @function_tool() decorator to give your agent capabilities:

from livekit.agents import function_tool

@function_tool()
async def lookup_weather(location: str) -> str:
    """Get the current weather for a location.

    Args:
        location: The city or location to get weather for
    """
    # Your implementation here
    return f"The weather in {location} is sunny and 22°C"

class WeatherAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a helpful weather assistant.",
            tools=[lookup_weather],
        )

Direct Speech

speak_direct() lets you synthesize and play audio directly — bypassing the LLM entirely. This is useful for scripted prompts, confirmations, or fallback messages.

# Inside a session or agent callback
await session.llm.session().speak_direct(
    "Welcome back! How can I help you today?",
    include_in_history=True,  # Record as an assistant turn (default: True)
    uninterruptable=False,    # Allow the user to barge in (default: False)
)

Setting include_in_history=False speaks the text without adding it to the conversation context — ideal for system-level announcements. Setting uninterruptable=True makes the utterance play to completion: overlapping user speech is ignored until playback finishes.

Conversation Queries

query_conversation() runs a one-shot inference call on a side channel, separate from the main conversational turn. The result is returned as a string and does not affect the conversation history or trigger any audio.

summary = await session.llm.session().query_conversation(
    prompt="Summarize the conversation so far in one sentence.",
)
print(summary)  # e.g., "The user asked about weather in Berlin."

You can also pass instructions to further constrain the model’s output format.

Chat History Export

Export the full conversation history as a list of structured message dicts at any point during a session:

from livekit.agents import EventTypes

# Listen for the export result
@session.llm.session().on("chat_history_exported")
def on_history(messages):
    for msg in messages:
        print(msg["role"], msg["content"])

# Request the export
await session.llm.session().export_chat_history(
    await_pending=False,  # Set True to wait for any in-flight operations first
    exclude_audio=False,  # Set True to omit audio blobs (transcripts only)
)

Each message follows the ChatMessageDict structure with role, delivery_status, ephemeral, and a content list of typed content blocks (text, input_audio, tool_call, tool_result, etc.).

Live Configuration

Update the system prompt or temperature mid-session without reconnecting:

await session.llm.update_options(
    system_prompt="You are now a concise assistant. Keep replies under two sentences.",
    temperature=0.8,
)

Changes take effect on the next model turn.

Contributing

This plugin is open source. Visit the deepslate-sdks monorepo to:

Report issues
Submit pull requests
Request features

Next Steps

WebSocket API

Low-level WebSocket access for custom integrations

API Reference

Full message schemas and configuration options

LiveKit Agents Docs

LiveKit Agents framework documentation

GitHub Repository

Source code, issues, and contributions

Getting started

Integration

SDK/Plugins

Voice Cloning

Prompt Engineering

LiveKit Plugin (Python)

Prerequisites

Installation

Environment Variables

Quick Start

Configuration Reference

Features

Real-time Voice Streaming

Server-side VAD

Function Tools

ElevenLabs TTS

Low Latency Mode

Direct Speech

Conversation Queries

Chat History Export

Live Configuration

Sending a Welcome Message

Function Tools

Direct Speech

Conversation Queries

Chat History Export

Live Configuration

Contributing

Next Steps

WebSocket API

API Reference

LiveKit Agents Docs

GitHub Repository

​Prerequisites

​Installation

​Environment Variables

​Quick Start

​Configuration Reference

​Features

Real-time Voice Streaming

Server-side VAD

Function Tools

ElevenLabs TTS

Low Latency Mode

Direct Speech

Conversation Queries

Chat History Export

Live Configuration

​Sending a Welcome Message

​Function Tools

​Direct Speech

​Conversation Queries

​Chat History Export

​Live Configuration

​Contributing

​Next Steps

WebSocket API

API Reference

LiveKit Agents Docs

GitHub Repository

Prerequisites

Installation

Environment Variables

Quick Start

Configuration Reference

Features

Sending a Welcome Message

Function Tools

Direct Speech

Conversation Queries

Chat History Export

Live Configuration

Contributing

Next Steps