Skip to content

Commit 545b5ae

Browse files
committed
docs(contributing): switch to rattler-build on Contributing packages page
1 parent 1a86edf commit 545b5ae

File tree

5 files changed

+51
-77
lines changed

5 files changed

+51
-77
lines changed

blog/2020-02-04-gsoc.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,6 @@ detailed listing of the issues that need work.
3333
We have a large backlog of maintenance and refactoring issues that
3434
are great for people with a range of experience from beginners to
3535
true code ninjas.
36-
3736
- Integration Testing for the Autotick Bot
3837

3938
> Run true integration testing on a copy of the graph to better

docs/maintainer/adding_pkgs.md

Lines changed: 51 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -41,33 +41,29 @@ The staging process i.e adding a package's recipe has three steps:
4141

4242
### Generating the recipe
4343

44-
There are, currently, three ways to generate a recipe:
45-
46-
1. If it is an R package from [CRAN](https://cran.r-project.org/), kindly
47-
start by using the [conda-forge helper script for R recipes](https://github.com/bgruening/conda_r_skeleton_helper) instead.
44+
1. If it is an R package from [CRAN](https://cran.r-project.org/), a Python package from [PyPI](https://pypi.org),
45+
a Perl package from [CPAN](https://www.cpan.org), or a Lua package from [Luarocks](https://luarocks.org),
46+
generate a recipe with `rattler-build generate-recipe`.
4847
Then if necessary, you can make manual edits to the recipe.
49-
2. If it is a python package, you can generate the recipe as a starting point with `grayskull`.
5048

5149
:::note
5250

53-
[Grayskull](https://github.com/conda-incubator/grayskull) is an automatic conda recipe generator. The goal of this project is to generate concise recipes
54-
for conda-forge and eventually replace conda skeleton. Presently, Grayskull can generate recipes for Python packages available on PyPI and also those not published on PyPI and only available as GitHub repositories.
55-
56-
Installation and usage of `grayskull`:
51+
Installation and usage of `rattler-build`:
52+
- [Install Pixi](https://pixi.sh/latest/installation/).
5753

58-
- Create a new environment using : `conda create --name MY_ENV`. Replace `MY_ENV` with the environment name.
59-
- Activate this new environment : `conda activate MY_ENV`.
60-
- Run `conda install -c conda-forge grayskull` to install `grayskull`.
61-
- Followed by `grayskull pypi --strict-conda-forge YOUR_PACKAGE_NAME` to generate the recipe. Replace `YOUR_PACKAGE_NAME` with the package name.
54+
- `pixi exec rattler-build generate-recipe INDEX_NAME PACKAGE_NAME`.
55+
Replace `INDEX_NAME` with the name of the index (e.g. `cran`) and `PACKAGE_NAME` with the name of the package.
6256

6357
:::
6458

65-
You do _not_ necessarily have to use `grayskull`, and the recipes produced by `grayskull` might need to be reviewed and edited.
66-
Read more about `grayskull` and how to use it [here](https://github.com/conda-incubator/grayskull#introduction).
59+
You do _not_ have to use `rattler-build`, and the recipes produced by `rattler-build` might need to be
60+
reviewed and edited.
61+
Read more in the [rattler-build recipe generation documentation](https://rattler.build/latest/recipe_generation/).
6762

68-
3. If it's none of the above, generate a recipe with the help of [the example recipe](https://github.com/conda-forge/staged-recipes/tree/master/recipes/example) in the [staged-recipes repository](https://github.com/conda-forge/staged-recipes) and modify it as necessary.
63+
2. If it's none of the above, generate a recipe with the help of [the example recipe](https://github.com/conda-forge/staged-recipes/tree/main/recipes/example-v1/recipe.yaml) in the [staged-recipes repository](https://github.com/conda-forge/staged-recipes), and [the rattler-build documentation](https://rattler.build/latest/),
64+
including [the full recipe specification](https://rattler.build/latest/reference/recipe_file/).
6965

70-
Your final recipe should have no comments (unless they're actually relevant to the recipe, and not generic instruction comments), and follow the order in the example.
66+
Your final recipe should have no comments (unless they're actually relevant to the recipe, not just generic instruction comments), and follow the order in the example.
7167

7268
:::note
7369

@@ -90,20 +86,19 @@ In case you are building your first recipe using conda-forge, a step-by-step ins
9086
2. Fork and clone the [staged-recipes](https://github.com/conda-forge/staged-recipes)
9187
repository from GitHub.
9288
3. Checkout a new branch from the staged-recipes `main` branch.
93-
4. Through CLI, cd inside the staged-recipes/recipes' directory.
89+
4. Through CLI, `cd` inside the `staged-recipes/recipes` directory.
9490
5. Within your forked copy, create a new folder in the recipes folder for your package (i.e, `...staged-recipes/recipes/<name-of-package>`)
95-
6. Copy [meta.yaml](https://github.com/conda-forge/staged-recipes/blob/master/recipes/example/meta.yaml) from the example directory.
96-
All the changes in the following steps will happen in the COPIED meta.yaml (i.e., `...staged-recipes/recipes/<name-of-package>/meta.yaml`).
91+
6. Copy [`recipe.yaml`](https://github.com/conda-forge/staged-recipes/blob/main/recipes/example-v1/recipe.yaml) from the example directory.
92+
All the changes in the following steps will happen in the COPIED recipe.yaml (i.e., `...staged-recipes/recipes/<name-of-package>/recipe.yaml`).
9793
Please leave the example directory unchanged!
98-
7. Modify the copied recipe (meta.yml) as needed. To see how to modify meta.yaml, take a look at
99-
[The recipe meta.yaml](#id4).
94+
7. Modify the copied recipe (`recipe.yml`) as needed. To see how to modify `recipe.yaml`, take a look at
95+
[the rattler-build documentation](https://rattler.build/latest/).
10096
8. Generate the SHA256 key for your source code archive, as described in the
101-
example recipe using the `openssl` tool. As an alternative, you can also
102-
go to the package description on [PyPi](https://pypi.org) from which you
103-
can directly copy the SHA256.
97+
example recipe using the `openssl` tool. Alternatively, for packages from [PyPI](https://pypi.org),
98+
you can also go to the package description on PyPI, from which you can directly copy the SHA256.
10499
9. Be sure to fill in the `test` section. The simplest test will simply
105100
test that the module can be imported, as described in the example.
106-
10. Remove all irrelevant comments in the `meta.yaml` file.
101+
10. Remove all irrelevant comments in the `recipe.yaml` file.
107102

108103
:::tip
109104

@@ -120,10 +115,10 @@ curl -sL https://github.com/username/reponame/archive/vX.X.X.tar.gz | openssl sh
120115
#### Checklist
121116

122117
- Ensure that the license and license family descriptors (optional) have the right case and that the license is correct. Note that case sensitive inputs are required (e.g. Apache-2.0 rather than APACHE 2.0). Using SPDX identifiers for license field is recommended. (see [SPDX Identifiers and Expressions](#spdx))
123-
- Ensure that you have included a license file if your license requires one – most do. (see [here](https://github.com/conda-forge/staged-recipes/blob/a504af81c05491bf7b0b018b2fa1efe64767985c/recipes/example/meta.yaml#L52-L55))
118+
- Ensure that you have included a license file if your license requires one – most do. (see [this section of the example recipe](https://github.com/conda-forge/staged-recipes/blob/3b151310e299162760033b9a165349fc255d28b7/recipes/example-v1/recipe.yaml#L88-L92))
124119
- In case your project has tests included, you need to decide if these tests should be executed while building the conda-forge feedstock.
125120
- Make sure that all tests pass successfully at least on your development machine.
126-
- Recommended: run the test locally on your source code to ensure the recipe works locally (see [Running tests locally for staged recipes](#staging-test-locally)).
121+
- Recommended: run the tests locally on your source code to ensure the recipe works locally (see [Running tests locally for staged recipes](#staging-test-locally)).
127122
- Make sure that your changes do not interfere with other recipes that are in the `recipes` folder (e.g. the `example` recipe).
128123

129124
<a id="feedback-and-revision"></a>
@@ -150,7 +145,7 @@ If you have questions or have not heard back for a while, you can notify us by i
150145

151146
- After the PR is merged, our [CI](../glossary.md#ci) services will create a new git repo automatically. For example, the recipe for a package named `pydstool` will be moved to a new repository [https://github.com/conda-forge/pydstool-feedstock](https://github.com/conda-forge/pydstool-feedstock). This process is automated through a CI job on the `conda-forge/staged-recipes` repo. It sometimes fails due to API rate limits and will automatically retry itself. If your feedstock has not been created after a day or so, please get in touch with the `conda-forge/core` team for help.
152147
- CI services will be enabled automatically and a build will be triggered automatically which will build the conda package and upload to [https://anaconda.org/conda-forge](https://anaconda.org/conda-forge)
153-
- If this is your first contribution, you will be added to the conda-forge [team](https://github.com/orgs/conda-forge/people) and given access to the CI services so that you can stop and restart builds. You will also be given commit rights to the new git repository.
148+
- If this is your first contribution, you will be added to the conda-forge [team](https://github.com/orgs/conda-forge/people) and be given access to the CI services so that you can stop and restart builds. You will also be given commit rights to the new git repository.
154149
- If you want to make a change to the recipe, send a [PR](../glossary.md#pr) to the git repository from a fork. Branches of the main repository are used for maintaining different versions only.
155150

156151
<a id="feedstock-repository-structure"></a>
@@ -168,13 +163,13 @@ Each feedstock contains various files that are generated automatically using our
168163

169164
#### recipe
170165

171-
This folder contains the `meta.yaml` file and any other files/scripts needed to build the package.
166+
This folder contains the `recipe.yaml` file and any other files/scripts needed to build the package.
172167

173168
<a id="license-txt"></a>
174169

175170
#### LICENSE.txt
176171

177-
This file is the license for the recipe itself. This license is different from the package license, which you define while submitting the package recipe using `license_file` in the `meta.yaml` file.
172+
This file is the license for the recipe itself. This license is different from the package license, which you define while submitting the package recipe using `license_file` in the `recipe.yaml` file.
178173

179174
<a id="ci-files"></a>
180175

@@ -207,7 +202,7 @@ The maintainer's job is to:
207202

208203
### Adding multiple packages at once
209204

210-
If you would like to add more than one related packages, they can be added to
205+
If you would like to add multiple related packages, they can be added to
211206
staged-recipes in a single pull request (in separate directories). If the
212207
packages are interdependent (i.e. one package being added lists one or more of
213208
the other packages being added as a requirement), the build script will be able to
@@ -263,20 +258,19 @@ Once these steps are complete, you can continue with the steps in [Step-by-step
263258

264259
<a id="id4"></a>
265260

266-
<a id="the-recipe-meta-yaml"></a>
261+
<a id="the-recipe-yaml"></a>
267262

268-
## The recipe meta.yaml
263+
## The `recipe.yaml`
269264

270-
The `meta.yaml` file in the recipe directory is at the heart of every conda package.
265+
The `recipe.yaml` file in the recipe directory is at the heart of every conda package.
271266
It defines everything that is required to build and use the package.
272267

273-
`meta.yaml` is in [yaml](https://en.wikipedia.org/wiki/YAML) format, augmented with [Jinja](http://jinja.pocoo.org/) templating.
268+
A full reference of the structure and fields of `recipe.yaml` file can be found in the
269+
[rattler-build recipe file documentation](https://rattler.build/latest/reference/recipe_file/).
274270

275-
A full reference of the structure and fields of `meta.yaml` file can be found in the [Defining metadata (meta.yaml)](https://conda.io/projects/conda-build/en/stable/resources/define-metadata.html) section in the conda-build documentation.
271+
In the following, we highlight particularly important and conda-forge specific information and guidelines, ordered by section in `recipe.yaml`.
276272

277-
In the following, we highlight particularly important and conda-forge specific information and guidelines, ordered by section in `meta.yaml`.
278-
279-
<a id="meta-yaml-source"></a>
273+
<a id="recipe-yaml-source"></a>
280274

281275
<a id="source"></a>
282276

@@ -288,11 +282,11 @@ In the following, we highlight particularly important and conda-forge specific i
288282

289283
#### Build from tarballs, not repos
290284

291-
Packages should be built from tarballs using the `url` key, not from repositories directly by using e.g. `git_url`.
285+
Packages should be built from tarballs using the `url` key, not from repositories directly by using e.g. `git`.
292286

293287
There are several reasons behind this rule:
294288

295-
- Repositories are usually larger than tarballs, draining shared CI time and bandwidth
289+
- Repositories are usually larger than tarballs, draining shared CI time and bandwidth.
296290
- Repositories are not checksummed. Thus, using a tarball has a
297291
stronger guarantee that the download that is obtained to build from is
298292
in fact the intended package.
@@ -302,24 +296,21 @@ There are several reasons behind this rule:
302296

303297
#### Populating the `hash` field
304298

305-
If your package is on [PyPi](https://pypi.org), you can get the sha256 hash from your package's page
299+
If your package is on [PyPI](https://pypi.org), you can get the sha256 hash from your package's page
306300
on PyPI; look for the `SHA256` link next to the download link on your package's
307301
files page, e.g. `https://pypi.org/project/<your-project>/#files`.
308302

309303
You can also generate a hash from the command line on Linux (and Mac if you
310304
install the necessary tools below).
311305

312-
To generate the `sha256` hash: `openssl sha256 your_sdist.tar.gz`
313-
314-
You may need the openssl package, available on conda-forge
315-
`conda install openssl -c conda-forge`.
306+
To generate the `sha256` hash: `pixi exec openssl sha256 your_sdist.tar.gz`
316307

317308
:::tip
318309

319310
Be sure not to checksum the redirection page. Therefore use, for example,:
320311

321312
```default
322-
curl -sL https://github.com/username/reponame/archive/vX.X.X.tar.gz | openssl sha256
313+
pixi exec --with openssl curl -sL https://github.com/username/reponame/archive/vX.X.X.tar.gz | openssl sha256
323314
```
324315

325316
:::
@@ -328,7 +319,8 @@ curl -sL https://github.com/username/reponame/archive/vX.X.X.tar.gz | openssl sh
328319

329320
#### Downloading extra sources and data files
330321

331-
`conda-build 3` supports multiple sources per recipe. Examples are available [in the conda-build docs](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#source-from-multiple-sources).
322+
`rattler-build` supports multiple sources per recipe: see
323+
[the relevant documentation section](https://rattler.build/latest/reference/recipe_file/#source-from-multiple-sources).
332324

333325
<a id="build"></a>
334326

@@ -338,23 +330,16 @@ curl -sL https://github.com/username/reponame/archive/vX.X.X.tar.gz | openssl sh
338330

339331
#### Skipping builds
340332

341-
Use the `skip` key in the `build` section along with a selector:
342-
343-
You can e.g. specify not to build …
333+
Use the `skip` key in the `build` section.
344334

345-
- on specific architectures:
346-
```yaml
347-
build:
348-
skip: true # [win]
349-
```
350-
- for specific python versions:
351-
```yaml
352-
build:
353-
skip: true # [py<35]
354-
```
335+
You can e.g. specify not to build on specific architectures:
355336

356-
A full description of selectors is
357-
[in the conda docs](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#preprocessing-selectors).
337+
```yaml
338+
build:
339+
skip:
340+
- win
341+
...
342+
```
358343

359344
<a id="optional-bld-bat-and-or-build-sh"></a>
360345

@@ -364,7 +349,7 @@ In many cases, `bld.bat` and/or `build.sh` files are not required.
364349
Pure Python packages almost never need them.
365350

366351
If the build can be executed with one line, you may put this line in the
367-
`script` entry of the `build` section of the `meta.yaml` file with:
352+
`script` entry of the `build` section of the `recipe.yaml` file with:
368353
`script: "{{ PYTHON }} -m pip install . -vv"`.
369354

370355
Remember to always add `pip` to the host requirements.
@@ -382,7 +367,7 @@ build:
382367
script: "{{ PYTHON }} -m pip install . -vv"
383368
```
384369
385-
as the installation script in the `meta.yml` file or `bld.bat/build.sh` script files,
370+
as the installation script in the `recipe.yml` file or `bld.bat/build.sh` script files,
386371
while adding `pip` to the host requirements:
387372

388373
```yaml
@@ -391,10 +376,6 @@ requirements:
391376
- pip
392377
```
393378

394-
These options should be used to ensure a clean installation of the package without its
395-
dependencies. This helps make sure that we're only including this package,
396-
and not accidentally bringing any dependencies along into the conda package.
397-
398379
Usually pure-Python packages only require `python`, `setuptools` and `pip`
399380
as `host` requirements; the real package dependencies are only
400381
`run` requirements.

docs/maintainer/knowledge_base.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1953,7 +1953,6 @@ There are two options for how to proceed:
19531953
- Specify the environment variable `SETUPTOOLS_SCM_PRETEND_VERSION` with the version string.
19541954
If specified this environment variable is the principle source for `setuptools_scm`.
19551955
There are two ways how to do this:
1956-
19571956
- If you are using build scripts, in `build.sh` specify:
19581957

19591958
```bash

docs/maintainer/updating_pkgs.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ For these reasons, maintainers are asked to fork the feedstock to their personal
5252
When a new version of a package is released on PyPI/CRAN/.., we have a bot that automatically creates version updates for the feedstock. In most cases you can simply merge this PR and it should include all changes. When certain things have changed upstream, e.g. the dependencies, you will still have to do changes to the created PR. As feedstock maintainer, you don't have to create a new PR for that but can simply push to the branch the bot created. There are two alternatives to push to the branch of the bot:
5353

5454
1. Manually setting up git remotes:
55-
5655
- Clone the conda-forge feedstock repository
5756
- Add the remote of the bot: `git remote add regro-cf-autotick-bot [email protected]:regro-cf-autotick-bot/<package>-feedstock.git`
5857
:::warning[Important]
@@ -98,15 +97,13 @@ Here we assume that you would like to update the feedstock `<feedstock>`. Feedst
9897
1. Forking the feedstock
9998

10099
Before you can submit your first PR, you have to fork conda-forge's feedstock.
101-
102100
- Navigate to [https://github.com/conda-forge](https://github.com/conda-forge)/<feedstock> in your favorite web browser and click the `fork` button.
103101
- You now have a clone of the feedstock in `https://github.com/<your-github-id>/<feedstock>` under your control.
104102
- Connect to the feedstock from your computer by using `git clone https://github.com/<your-github-id>/<feedstock>`.
105103

106104
2. Syncing your fork with conda-forge's feedstock
107105

108106
This step is only required if you have forked some time ago and your fork is missing commits from the feedstock at conda-forge.
109-
110107
- Make sure you are on the main branch: `git checkout main`
111108
- Register conda-forge's feedstock with `git remote add upstream https://github.com/conda-forge/<feedstock>`
112109
- Fetch the latest updates with `git fetch upstream`
@@ -115,7 +112,6 @@ Here we assume that you would like to update the feedstock `<feedstock>`. Feedst
115112
3. Creating your changes in a new branch
116113

117114
Now you are ready to update the recipe
118-
119115
- Create and switch to a new branch: `git checkout -b <branch-name>`. `<branch-name>` can be e.g. `update_1_0_1`.
120116
- Make your changes locally
121117
- Review your changes then use `git add <changed-files>`. Where `<changed-files>` are a whitespace separated list of filenames you changed.

docs/user/transitioning_from_defaults.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,6 @@ are guidelines on how to go about this, while backing up your work and your envi
7474
to right click on the Anaconda / Miniconda installation folder. Check if you have
7575
created additional environments outside the installation root by running
7676
`conda info --envs`. Back those up too, if necessary.
77-
7877
- Alternatively, you can also export lockfiles for all your environments, if preferred:
7978

8079
```shell

0 commit comments

Comments
 (0)