conda-mirror

Mirror conda channels

🗂 Table of Contents

• Introduction
• Installation
• Usage

📖 Introduction

This tool allows you to mirror conda channels to different backends using parallelism. You can also specify custom whitelists or blacklists if you only want to include certain kinds of packages.

💿 Installation

You can install conda-mirror using pixi:

pixi global install conda-mirror

Or using cargo:

cargo install --locked --git https://github.com/conda-incubator/conda-mirror.git

Or by downloading our pre-built binaries from the releases page.

Instead of installing conda-mirror globally, you can also use pixi exec to run conda-mirror in a temporary environment:

pixi exec conda-mirror --source robostack --destination ./robostack

🎯 Usage

CLI

You can mirror conda channels using

conda-mirror --source conda-forge --destination ./conda-forge

Subdirs

If you only want to mirror certain subdirs, you can do so using the --subdir flag:

conda-mirror --source robostack --destination ./robostack --subdir linux-64 --subdir linux-aarch64

Supported backends

You can mirror from multiple source backends, namely:

• filesystem: --source ./conda-forge-local
• http(s): --source conda-forge or --source https://prefix.dev/conda-forge
• oci: --source oci://ghcr.io/channel-mirrors/conda-forge
• s3: --source s3://my-source-bucket/channel

For mirroring authenticated channel, conda-mirror uses pixi's authentication. See the official documentation for more information.

You can mirror to multiple destination backends as well, namely:

• filesystem: --destination ./conda-forge-local
• s3: --destination s3://my-destination-bucket/channel

Configuration file

For more control like including only specific packages, you can use a configuration file and pass them to conda-mirror using --config my-config.yml.

Mirror all packages except a specific blacklist:

source: conda-forge
destination: ./my-channel

exclude:
  # you can use MatchSpecs here
  - jupyter >=0.5.0
  # you can also specify licenses in the MatchSpecs
  - "*[license=AGPL-3.0-or-later]"

Only mirror whitelisted packages:

source: conda-forge
destination: ./my-channel

include:
  - name-glob: jupyter*

Exclude all packages defined in exclude, override this behavior by specifying overrides in include:

source: conda-forge
destination: ./my-channel

include:
  - jupyter-ai
exclude:
  - name-glob: jupyter*

Only mirror certain subdirs:

source: conda-forge
destination: ./my-channel
subdirs:
  - linux-64
  - osx-arm64
  - noarch
  - win-64

S3 configuration

When using S3, you need to configure the S3 endpoint by setting the region, endpoint url, and whether to use path-style addressing. You can either set these by using the appropriate CLI flags or by using a configuration file.

source: s3://my-source-channel
destination: s3://my-destination-channel

s3-config:
  source:
    endpoint-url: https://fsn1.your-objectstorage.com
    force-path-style: false
    region: US
  destination:
    endpoint-url: https://s3.eu-central-1.amazonaws.com
    force-path-style: false
    region: eu-central-1

See pixi's documentation for configuring S3-compatible storage like Cloudflare R2 or Hetzner Object Storage.