maturin-mixed-template

日本語版はこちら：https://qiita.com/K94-ishi/items/cdcac018ff19f3f2d8b5

Introduction

I often implement Python extension modules in Rust using maturin / PyO3. However, I recently needed to build the same project as a command-line tool as well. This article summarizes how to achieve that.

Goals: The project is able to support both of the following:

• A Rust command-line executable (cargo build)
• A Python module that can be installed with pip install (maturin build)

External Links:

• My template repository:
  https://github.com/k94-ishi/maturin-mixed-template
• References:
  How to Mix Rust and Python in Your Project
  Maturin User Guide: project_layout#mixed-rustpython-project

Setting Up the Python Environment

• Install maturin:

pip install maturin

Creating the Project

When creating a Python module, you would normally use maturin new --bindings pyo3 my_project. However, since we also need to build a Rust command-line executable, we should specify the --mixed option.

• Check the --mixed option with maturin new --help:

--mixed
    Use mixed Rust/Python project layout

• Create a new mixed Rust/Python project:

maturin new --bindings pyo3 --mixed my_project

Modifying Cargo.toml and pyproject.toml

Updating Cargo.toml

• Add [features] section

  Prevent errors caused by cargo build trying to compile Python-related source code.

[features]
default = [] # Disable Python-related features by default
python = ["dep:pyo3"] # Enable `pyo3` only when the `python` feature is active

• Add "rlib" to [lib] crate-type

  This allows src/lib.rs to be built for both Python and command-line use.

[lib]
name = "my_project"
#  ["cdylib"]
crate-type = ["cdylib", "rlib"]
# "cdylib" is for Python, "rlib" is for the command-line tool

• Update [dependencies] section for pyo3

  This enables selective builds.

[dependencies]
# pyo3 = "0.23.3"
pyo3 = { version = "0.23.3", features = ["extension-module"], optional = true }

Updating pyproject.toml

• Add "python" to [tool.maturin] features

  This ensures that maturin build selects the appropriate build target.

# features = ["pyo3/extension-module"]
features = ["pyo3/extension-module, python"]

Modifying the Source Code

Updating src/lib.rs

• Separate code used for the command-line tool from Python-specific code.
• Use #[cfg(feature = "python")] to exclude Python-specific code from cargo build.

#[cfg(feature = "python")]
use pyo3::prelude::*;

pub fn sum_as_string_rs(a: usize, b: usize) -> String {
    (a + b).to_string()
}

#[cfg(feature = "python")]
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok(sum_as_string_rs(a, b))
}

#[cfg(feature = "python")]
#[pymodule]
fn my_project(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
    Ok(())
}

Adding src/main.rs

• src/main.rs is not generated by default, so create it manually.

touch src/main.rs
tree -L 3  # Check the directory structure

The directory structure should look like this:

.
├── Cargo.lock
├── Cargo.toml
├── pyproject.toml
├── python
│   ├── my_project
│   │   └── __init__.py
│   └── tests
│       └── test_all.py
└── src
    ├── lib.rs
    └── main.rs

Edit src/main.rs as follows:

use my_project::sum_as_string_rs;

fn main() {
    let text = sum_as_string_rs(1, 1);
    println!("1 + 1 = {}", text)
}

Building, Running, and pip install

cargo run
maturin build
pip install wheel
pip install ./target/wheels/my_project-*.whl

Fixing Import Issues in Python Code

Updating python/my_project/__init__.py

# from .my_project import *
from my_project import my_project

Updating python/tests/test_all.py

# import my_project
from my_project import my_project

After these changes, pytest should work correctly.

pip install pytest

my_project $ pytest
=============================== test session starts ================================
platform darwin -- Python 3.12.3, pytest-8.3.5, pluggy-1.5.0
rootdir: /<directory>/my_project
configfile: pyproject.toml
collected 1 item                                                                   

python/tests/test_all.py .                                                   [100%]

================================ 1 passed in 0.04s ================================

Conclusion

• Make good use of features to control builds.
• For more details, refer to the Maturin User Guide.