Pipecat w/ AssemblyAI Cloud Example Project

A sample voice agent for Pipecat Cloud that demonstrates building and deploying a conversational AI agent using AssemblyAI's universal streaming for speech-to-text.

Prerequisites

• Python 3.10+
• Linux, MacOS, or Windows Subsystem for Linux (WSL)
• Docker and a Docker repository (e.g., Docker Hub)
• A Docker Hub account (or other container registry account)
• Pipecat Cloud account
• Cartesia account
• AssemblayAI account

Note: If you haven't installed Docker yet, follow the official installation guides for your platform (Linux, Mac, Windows). For Docker Hub, create a free account and log in via terminal with docker login.

Get Started

Clone this repo then:

1. Set up your Python environment

A virtual environment is recommended to manage your Python dependencies. Using (uv)[https://docs.astral.sh/uv/]:

# Create virtual environment and download project dependencies
uv sync

2. Authenticate with Pipecat Cloud

pcc auth login

3. Acquire required API keys

This starter requires the following API keys:

• AssemblyAI Api Key: Get from assemblyai.com/dashboard/api-keys
• OpenAI API Key: Get from platform.openai.com/api-keys
• Cartesia API Key: Get from play.cartesia.ai/keys
• Daily API Key: Automatically provided through your Pipecat Cloud account

Your DAILY_API_KEY can be found at https://pipecat.daily.co under the Settings in the Daily (WebRTC) tab.

5. Configure to run locally (optional)

You can test your agent locally before deploying to Pipecat Cloud:

# Set environment variables with your API keys
export ASSEMBLYAI_API_KEY="your_assemblyai_key"
export CARTESIA_API_KEY="your_cartesia_key"
export DAILY_API_KEY="your_daily_key"
export OPENAI_API_KEY="your_openai_key"

Or copy the env.example file to a .env file with:

cp env.example .env

Then fill in the API key values.

Launch the bot.py script locally:

env LOCAL_RUN=1 uv run bot.py

Deploy & Run

1. Build and push your Docker image

# Build the image (targeting ARM architecture for cloud deployment)
docker build --platform=linux/arm64 -t my-first-agent:latest .

# Tag with your Docker username and version
docker tag my-first-agent:latest your-username/my-first-agent:0.1

# Push to Docker Hub
docker push your-username/my-first-agent:0.1

2. Create a secret set for your API keys

The starter project requires API keys for OpenAI and Cartesia:

# Copy the example env file if not done earlier
cp env.example .env

# Edit .env to add your API keys:
# CARTESIA_API_KEY=your_cartesia_key
# OPENAI_API_KEY=your_openai_key
# ...

# Create a secret set from your .env file
pcc secrets set my-first-agent-secrets --file .env

Alternatively, you can create secrets directly via CLI:

pcc secrets set my-first-agent-secrets \
  CARTESIA_API_KEY=your_cartesia_key \
  OPENAI_API_KEY=your_openai_key

3. Deploy to Pipecat Cloud

pcc deploy my-first-agent your-username/my-first-agent:0.1 --secrets my-first-agent-secrets

Note (Optional): For a more maintainable approach, you can use the included pcc-deploy.toml file:

agent_name = "my-first-agent"
image = "your-username/my-first-agent:0.1"
secret_set = "my-first-agent-secrets"

[scaling]
    min_instances = 0

Then simply run pcc deploy without additional arguments.

Note: If your repository is private, you'll need to add credentials:

# Create pull secret (you’ll be prompted for credentials)
pcc secrets image-pull-secret pull-secret https://index.docker.io/v1/

# Deploy with credentials
pcc deploy my-first-agent your-username/my-first-agent:0.1 --credentials pull-secret

4. Check deployment and scaling (optional)

By default, your agent will use "scale-to-zero" configuration, which means it may have a cold start of around 10 seconds when first used. By default, idle instances are maintained for 5 minutes before being terminated when using scale-to-zero.

For more responsive testing, you can scale your deployment to keep a minimum of one instance warm:

# Ensure at least one warm instance is always available
pcc deploy my-first-agent your-username/my-first-agent:0.1 --min-instances 1

# Check the status of your deployment
pcc agent status my-first-agent

By default, idle instances are maintained for 5 minutes before being terminated when using scale-to-zero.

5. Create an API key

# Create a public API key for accessing your agent
pcc organizations keys create

# Set it as the default key to use with your agent
pcc organizations keys use

6. Start your agent

# Start a session with your agent in a Daily room
pcc agent start my-first-agent --use-daily

This will return a URL, which you can use to connect to your running agent.

Documentation & Resources

For more details on Pipecat Cloud and its capabilities:

• Pipecat Cloud Documentation
• Pipecat Project Documentation
• Pipecat Discord community for help and discussions.

For AssemblyAI:

• AssemblyAI Universal Streaming
• AssemblyAI's Developer Documentation
• AssemblyAI Discord community