diff --git a/guide/src/project_layout.md b/guide/src/project_layout.md index ae543ef3e..df44e167a 100644 --- a/guide/src/project_layout.md +++ b/guide/src/project_layout.md @@ -1,11 +1,130 @@ # Project Layout -mention how to package type stubs +Maturin expects a particular project layout depending on the contents of the +package. -## Pure Rust Project +## Pure Rust project -TODO +For a pure Rust project, the structure is as expected and what you get from `cargo new`: -## Mixed Rust/Python Project +``` +my-rust-project/ +├── Cargo.toml +├── pyproject.toml # required for maturing configuration +└── src + ├── lib.rs # default for library crates + └── main.rs # default for binary crates +``` -TODO +maturin will add a necessary `__init__.py` to the package when building the +wheel. For convenience, this file includes the following: + +```python +from .my_project import * + +__doc__ = .my_project.__doc__ +``` + +such that the module functions may be called directly with: + +```python +import my_project +my_project.foo() +``` + +rather than: + +```python +from my_project import my_project +``` + +N.B.: there is currently no way to tell maturin to include extra data (e.g. +type stubs or `package_data` in setuptools) for a pure Rust project. Instead, +consider using the layout described below for the mixed Rust/Python project. + +## Mixed Rust/Python project + +To create a mixed Rust/Python project, add a directory with your package name +(i.e. matching `lib.name` in your `Cargo.toml`) to contain the Python source: + +``` +my-rust-and-python-project +├── Cargo.toml +├── my_project # <<< add this directory and put Python code in here +│ ├── __init__.py +│ └── bar.py +├── pyproject.toml +├── Readme.md +└── src + └── lib.rs +``` + +Note that in a mixed Rust/Python project, maturin _does not_ modify the +existing `__init__.py` in the root package, so now to import the rust module in +Python you must use: + +```python +from my_project import my_project +``` + +You can modify `__init__.py` yourself (see above) if you would like to import +functions Rust functions from a higher-level namespace. + +### Alternate Python source directory (src layout) + +Having a directory with `package_name` in the root of the project can +occasionally cause confusion as Python allows importing local packages and +modules. A popular way to avoid this is with the `src`-layout, where the Python +package is nested within a `src` directory. Unfortunately this interferes with +the structure of a typical Rust project. Fortunately, Python is nor particular +about the name of the parent source directory. You tell maturin to use a +different Python source directory in `Cargo.toml` by setting +`package.metadata.maturin.python-source`. For example: + +```toml +[package.metadata.maturin] +python-source = "python" +``` + +then the project structure would look like this: + +``` +my-rust-and-python-project +├── Cargo.toml +├── python +│ └── my_project +│ ├── __init__.py +│ └── bar.py +├── pyproject.toml +├── README.md +└── src + └── lib.rs +``` + +### Adding type information + +The simplest way to add type information for distribution is to use the mixed +Rust/Python projcet layout. In this layout, additional files in the Python +source dir (but not in `.gitignore`) will be automatically included in the +build outputs (source distribution and/or wheel). + +To distribute typing information, you need to add: + +* an empty marker file called `py.typed` in the root of the Python package +* inline types in Python files and/or `.pyi` "stub" files + +``` +my-project +├── Cargo.toml +├── python +│ └── my_project +│ ├── __init__.py +│ ├── py.typed # <<< add this empty file +│ ├── my_project.pyi # <<< add type stubs for Rust functions in the my_project module here +│ ├── bar.pyi # <<< add type stubs for bar.py here OR type bar.py inline +│ └── bar.py +├── pyproject.toml +├── README.md +└── src + └── lib.rs +``` diff --git a/guide/src/tutorial.md b/guide/src/tutorial.md index 68db49fa3..c767ec28d 100644 --- a/guide/src/tutorial.md +++ b/guide/src/tutorial.md @@ -1,3 +1,266 @@ # Tutorial -Walkthrough a simple pure Rust pyo3 bindings project. +In this tutorial we will wrap a version of [the guessing game from The Rust +Book](https://doc.rust-lang.org/book/ch02-00-guessing-game-tutorial.html) to +run in Python using pyo3. + +## Create a new Rust project + +First, create a new Rust library project using `cargo new --lib --edition 2018 +guessing-game`. This will create a directory with the following structure. + +``` +guessing-game/ +├── Cargo.toml +└── src + └── lib.rs +``` + +Edit `Cargo.toml` to configure the project and module name, and add the +dependencies (`rand` and `pyo3`). Configure `pyo3` with additional features to +make an extension module compatible with multiple Python versions using the +stable ABI (`abi3`). + +```toml +[package] +name = "guessing-game" +version = "0.1.0" +edition = "2018" + +[lib] +name = "guessing_game" +# "cdylib" is necessary to produce a shared library for Python to import from. +crate-type = ["cdylib"] + +[dependencies] +rand = "0.8.4" + +[dependencies.pyo3] +version = "0.14.5" +# "extension-module" tells pyo3 we want to build an extension module (skips linking against libpython.so) +# "abi3-py36" tells pyo3 (and maturin) to build using the stable ABI with minimum Python version 3.6 +features = ["extension-module", "abi3-py36"] +``` + +## Install and configure maturin (in a virtual environment) + +Create a virtual environment and install maturin. Note maturin has minimal +dependencies! + +```shell +ferris@rustbox [~/src/rust/guessing-game] % python3 -m venv .venv +ferris@rustbox [~/src/rust/guessing-game] % source .venv/bin/activate +(.venv) ferris@rustbox [~/src/rust/guessing-game] % pip install -U pip maturin +(.venv) ferris@rustbox [~/src/rust/guessing-game] % pip freeze +maturin==0.11.5 +toml==0.10.2 +``` + +maturin is configured in `pyproject.toml` as introduced by [PEP +518](https://www.python.org/dev/peps/pep-0518/). This file lives in the root +of your project tree: + +``` +guessing-game/ +├── Cargo.toml +├── pyproject.toml # <<< add this file +└── src + └── lib.rs +``` + +Configuration in this file is quite simple for most projects. You just need to +indicate maturin as a requirement (and restrict the version) and as the +build-backend (Python supports a number of build-backends since [PEP +517](https://www.python.org/dev/peps/pep-0517/)). + +```toml +[build-system] +requires = ["maturin>=0.11,<0.12"] +build-backend = "maturin" +``` + +Various other tools may also be configured in `pyproject.toml` and the Python +community seems to be consolidating declarative configuration in this file. + +## Program the guessing game in Rust + +When you create a `lib` projectg with `cargo new` it creates a file +`src/lib.rs` with some default code. Edit that file and replace the default +code with the code below. As mentioned, we will implement a slightly +modified version of [the guessing game from The Rust +Book](https://doc.rust-lang.org/book/ch02-00-guessing-game-tutorial.html). +Instead of implemeting as a `bin` crate, we're using a `lib` and will expose +the main logic as a Python function. + +```rust +use pyo3::prelude::*; +use rand::Rng; +use std::cmp::Ordering; +use std::io; + +#[pyfunction] +fn guess_the_number() { + println!("Guess the number!"); + + let secret_number = rand::thread_rng().gen_range(1..101); + + loop { + println!("Please input your guess."); + + let mut guess = String::new(); + + io::stdin() + .read_line(&mut guess) + .expect("Failed to read line"); + + let guess: u32 = match guess.trim().parse() { + Ok(num) => num, + Err(_) => continue, + }; + + println!("You guessed: {}", guess); + + match guess.cmp(&secret_number) { + Ordering::Less => println!("Too small!"), + Ordering::Greater => println!("Too big!"), + Ordering::Equal => { + println!("You win!", guesses); + break; + } + } + } +} + +/// A Python module implemented in Rust. The name of this function must match +/// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to +/// import the module. +#[pymodule] +fn guessing_game(_py: Python, m: &PyModule) -> PyResult<()> { + m.add_function(wrap_pyfunction!(guess_the_number, m)?)?; + + Ok(()) +} +``` + +Thanks to pyo3, there's very little difference between this and the example in +The Rust Book. All we had to do was: +1. Include the pyo3 prelude +2. Add `#[pyfunction]` to our function +3. Add the `#[pymodule]` block to expose the function as part of a Python module + +Refer to the [pyo3 User Guide](https://pyo3.rs/) for more information on using +pyo3. It can do a lot more! + +## Build and install the module with `maturin develop` + +Note that *this is just a Rust project* at this point, and with few exceptions +you can build it as you'd expect using `cargo build`. maturin helps with this, +however, adding some platform-specific build configuration and ultimately +packaging the binary results as a wheel (a `.whl` file, which is an archive of +compiled components suitable for installation with `pip`, the Python package +manager). + +So let's use maturin to build and install in our current environment. + +```shell +(.venv) ferris@rustbox [~/src/rust/guessing-game] % maturin develop +🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.6 +🐍 Not using a specific python interpreter (With abi3, an interpreter is only required on windows) + Compiling libc v0.2.105 + Compiling proc-macro2 v1.0.32 + Compiling cfg-if v1.0.0 + Compiling unicode-xid v0.2.2 + Compiling syn v1.0.81 + Compiling proc-macro-hack v0.5.19 + Compiling pyo3-build-config v0.14.5 + Compiling once_cell v1.8.0 + Compiling parking_lot_core v0.8.5 + Compiling smallvec v1.7.0 + Compiling scopeguard v1.1.0 + Compiling unindent v0.1.7 + Compiling ppv-lite86 v0.2.15 + Compiling instant v0.1.12 + Compiling lock_api v0.4.5 + Compiling getrandom v0.2.3 + Compiling quote v1.0.10 + Compiling rand_core v0.6.3 + Compiling parking_lot v0.11.2 + Compiling paste-impl v0.1.18 + Compiling rand_chacha v0.3.1 + Compiling pyo3 v0.14.5 + Compiling rand v0.8.4 + Compiling paste v0.1.18 + Compiling pyo3-macros-backend v0.14.5 + Compiling indoc-impl v0.3.6 + Compiling indoc v0.3.6 + Compiling pyo3-macros v0.14.5 + Compiling guessing-game v0.1.0 (/Users/ferris/src/rust/guessing-game) + Finished dev [unoptimized + debuginfo] target(s) in 13.31s +``` + +Your `guessing_game` module should now be available in your current virtual +environment. Go ahead and play a few games! +```shell +(.venv) ferris@rustbox [~/src/rust/guessing-game] % python +Python 3.9.6 (default, Aug 25 2021, 16:04:27) +[Clang 12.0.5 (clang-1205.0.22.9)] on darwin +Type "help", "copyright", "credits" or "license" for more information. +>>> import guessing_game +>>> guessing_game.guess_the_number() +Guess the number! +Please input your guess. +42 +You guessed: 42 +Too small! +Please input your guess. +80 +You guessed: 80 +Too big! +Please input your guess. +50 +You guessed: 50 +Too small! +Please input your guess. +60 +You guessed: 60 +Too big! +Please input your guess. +55 +You guessed: 55 +You win! +``` + +## Create a wheel for distribution + +`maturin develop` actually skips the wheel generation part and installs +directly in the current environment. `maturin build` on the other hand will +produce a wheel you can distribute. Note the wheel contains "tags" in its +filename that correspond to supported Python versions, platforms, and/or +architectures, so yours might look a little different. If you want to +distribute broadly, you may need to build on multiple platforms and use a +[`manylinux`](https://github.com/pypa/manylinux) Docker container to build +wheels compatible with a wide range of Linux distros. + +```shell +(.venv) ferris@rustbox [~/src/rust/guessing-game] % maturin build +🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.6 +🐍 Not using a specific python interpreter (With abi3, an interpreter is only required on windows) +📦 Built source distribution to /Users/ferris/src/rust/guessing-game/target/wheels/guessing_game-0.1.0.tar.gz + Compiling pyo3-build-config v0.14.5 + Compiling pyo3-macros-backend v0.14.5 + Compiling pyo3 v0.14.5 + Compiling pyo3-macros v0.14.5 + Compiling guessing-game v0.1.0 (/Users/ferris/src/rust/guessing-game) + Finished dev [unoptimized + debuginfo] target(s) in 7.32s +📦 Built wheel for abi3 Python ≥ 3.6 to /Users/ferris/src/rust/guessing-game/target/wheels/guessing_game-0.1.0-cp36-abi3-macosx_10_7_x86_64.whl +``` + +maturin can even publish wheels directly to [PyPI](https://pypi.org) with +`maturin publish`! + +## Summary +Congratulations! You successfully created a Python module implemented entirely +in Rust thanks to pyo3 and maturin. + +This demonstrates how easy it is to get started with maturin, but keep reading +to learn more about all the additional features.