Skip to content

Move CI tasks to Taskfile.yml #1610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jul 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
dabb987
Move CI tasks to Takefile.yml
JonathanOppenheimer Jun 26, 2025
d5df1ed
Force bash shell on Windows tests
JonathanOppenheimer Jun 26, 2025
01ae4a2
Instead, set shell to bash for all tasks
JonathanOppenheimer Jun 26, 2025
2bbfcff
Move bash shell directly to the tests, rather than run task
JonathanOppenheimer Jun 26, 2025
4bde723
Change to direct bash calls in the taskfile
JonathanOppenheimer Jun 26, 2025
db65042
Remove logs folder
JonathanOppenheimer Jun 26, 2025
40e7d07
Remove Windows tests, consolidate tests
JonathanOppenheimer Jun 27, 2025
11cd2c0
remove runtime import
JonathanOppenheimer Jun 27, 2025
42415e3
revert windows change, fix env
JonathanOppenheimer Jun 27, 2025
7b58e10
warp environment
JonathanOppenheimer Jun 27, 2025
af47f57
fix antithesis order from consolidtion
JonathanOppenheimer Jun 27, 2025
b116e28
Clarify default env vars
JonathanOppenheimer Jun 30, 2025
a1ed580
Consolidate testing further into tasks
JonathanOppenheimer Jul 1, 2025
f5f008b
Restore run_env to comply with written test
JonathanOppenheimer Jul 1, 2025
eb7130a
restore env variables for avalanchego release
JonathanOppenheimer Jul 1, 2025
72fe798
Add more verbose task descriptions, and move contract commands to a task
JonathanOppenheimer Jul 1, 2025
a2e21b5
make env explicitly clear for precompiles
JonathanOppenheimer Jul 1, 2025
8f671f1
globalize environment variables
JonathanOppenheimer Jul 1, 2025
7ce3b10
Further consolidate tasks and move from .sh
JonathanOppenheimer Jul 3, 2025
c4777b2
Merge branch 'master' into move-ci-tasks
ceyonur Jul 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ jobs:
go-version-file: "go.mod"
- run: go mod download
shell: bash
- run: ./scripts/build_bench_precompiles.sh
- run: ./scripts/run_task.sh build-bench-precompiles
shell: bash
2 changes: 1 addition & 1 deletion .github/workflows/publish_antithesis_images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
password: ${{ secrets.ANTITHESIS_GAR_JSON_KEY }}

- name: Build and publish images
run: bash -x ./scripts/build_antithesis_images.sh
run: ./scripts/run_task.sh build-antithesis-images
env:
IMAGE_PREFIX: ${{ env.REGISTRY }}/${{ env.REPOSITORY }}
IMAGE_TAG: latest
2 changes: 1 addition & 1 deletion .github/workflows/publish_docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,4 @@ jobs:
PLATFORMS: "linux/amd64,linux/arm64"
AVALANCHE_VERSION: ${{ inputs.avalanche_version }}
ALLOW_TAG_LATEST: ${{ github.event_name != 'workflow_dispatch' || inputs.allow_tag_latest }}
run: scripts/build_docker_image.sh
run: ./scripts/run_task.sh build-docker-image
119 changes: 21 additions & 98 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,33 +16,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./scripts/lint_allowed_eth_imports.sh
shell: bash
- uses: actions/setup-go@v5
with:
go-version-file: "go.mod"
- name: golangci-lint
uses: golangci/golangci-lint-action@3a919529898de77ec3da873e3063ca4b10e7f5cc #v3
with:
version: v1.63
working-directory: .
args: --timeout 10m
skip-pkg-cache: true
- name: Run shellcheck
shell: bash
run: scripts/shellcheck.sh
- name: Run actionlint
shell: bash
run: scripts/actionlint.sh
- name: go mod tidy
run: |
go mod tidy
git diff --exit-code
- name: Run all lint checks
run: ./scripts/run_task.sh lint-all-ci
- name: Check go.mod and go.sum are up-to-date
run: ./scripts/run_task.sh check-go-mod-tidy
- name: Ensure consistent avalanchego version
shell: bash
run: |
bash -x ./scripts/update_avalanchego_version.sh
git diff --exit-code
run: ./scripts/run_task.sh check-avalanchego-version

unit_test:
name: Golang Unit Tests (${{ matrix.os }})
Expand All @@ -57,28 +39,15 @@ jobs:
with:
go-version-file: "go.mod"
- run: go mod download
shell: bash
- name: fjl/gencodec generated files are up to date
run: |
grep -lr -E '^// Code generated by github\.com\/fjl\/gencodec\. DO NOT EDIT\.$' . | xargs -r rm
go generate -run "github.com/fjl/gencodec" ./...
git add --intent-to-add --all
git diff --exit-code
- name: Mocks are up to date
shell: bash
run: |
grep -lr -E '^// Code generated by MockGen\. DO NOT EDIT\.$' . | xargs -r rm
go generate -run "go.uber.org/mock/mockgen" ./...
git add --intent-to-add --all
git diff --exit-code
- run: ./scripts/build.sh
shell: bash
- run: ./scripts/build_test.sh
- name: Check generated codec files are up to date
run: ./scripts/run_task.sh check-generate-codec
- name: Check generated mocks are up to date
run: ./scripts/run_task.sh check-generate-mocks
- run: ./scripts/run_task.sh build
- run: ./scripts/run_task.sh build-test
env:
TIMEOUT: ${{ env.TIMEOUT }}
shell: bash
- run: ./scripts/coverage.sh
shell: bash
- run: ./scripts/run_task.sh coverage

e2e_precompile:
name: e2e precompile tests
Expand All @@ -96,24 +65,10 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: "20.13"
- name: NPM Clean Install
run: npm ci
working-directory: ./contracts
- name: Hardhat Clean
run: npx hardhat clean
working-directory: ./contracts
- name: Hardhat Compile
run: npx hardhat compile
working-directory: ./contracts
- name: Install AvalancheGo Release
shell: bash
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
- name: Build Subnet-EVM Plugin Binary
shell: bash
run: ./scripts/build.sh
- name: Setup Contracts
run: ./scripts/run_task.sh setup-contracts
- name: Run E2E Precompile Tests
shell: bash
run: AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego DATA_DIR=/tmp/e2e-test/precompile-data ./scripts/run_ginkgo_precompile.sh
run: ./scripts/run_task.sh test-e2e-precompile-ci
- name: Upload Artifact
if: always()
uses: actions/upload-artifact@v4
Expand All @@ -137,26 +92,12 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: "20.13"
- name: NPM Clean Install
run: npm ci
working-directory: ./contracts
- name: Hardhat Clean
run: npx hardhat clean
working-directory: ./contracts
- name: Hardhat Compile
run: npx hardhat compile
working-directory: ./contracts
- name: Install AvalancheGo Release
shell: bash
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
- name: Build Subnet-EVM Plugin Binary
shell: bash
run: ./scripts/build.sh
- name: Setup Contracts
run: ./scripts/run_task.sh setup-contracts
- name: Run Warp E2E Tests
uses: ava-labs/avalanchego/.github/actions/run-monitored-tmpnet-cmd@092cf182dd92b66a8ab882b657d3ae52b174c913
with:
run: ./scripts/run_ginkgo_warp.sh
run_env: AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego
run: ./scripts/run_task.sh test-e2e-warp-ci
artifact_prefix: warp
prometheus_username: ${{ secrets.PROMETHEUS_ID || '' }}
prometheus_password: ${{ secrets.PROMETHEUS_PASSWORD || '' }}
Expand All @@ -174,17 +115,10 @@ jobs:
uses: actions/setup-go@v5
with:
go-version-file: "go.mod"
- name: Install AvalancheGo Release
shell: bash
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
- name: Build Subnet-EVM Plugin Binary
shell: bash
run: ./scripts/build.sh
- name: Run E2E Load Tests
uses: ava-labs/avalanchego/.github/actions/run-monitored-tmpnet-cmd@092cf182dd92b66a8ab882b657d3ae52b174c913
with:
run: ./scripts/run_ginkgo_load.sh
run_env: AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego
run: ./scripts/run_task.sh test-e2e-load-ci
artifact_prefix: load
prometheus_username: ${{ secrets.PROMETHEUS_ID || '' }}
prometheus_password: ${{ secrets.PROMETHEUS_PASSWORD || '' }}
Expand All @@ -200,8 +134,7 @@ jobs:
sudo apt update
sudo apt -y install qemu-system qemu-user-static
- name: Check image build
shell: bash
run: bash -x scripts/tests.build_docker_image.sh
run: ./scripts/run_task.sh test-build-image
test_build_antithesis_images:
name: Build Antithesis images
runs-on: ubuntu-latest
Expand All @@ -210,15 +143,5 @@ jobs:
- uses: actions/setup-go@v5
with:
go-version-file: "go.mod"
- name: Install AvalancheGo Release
shell: bash
run: BASEDIR=/tmp/e2e-test AVALANCHEGO_BUILD_PATH=/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
- name: Build Subnet-EVM Plugin Binary
shell: bash
run: ./scripts/build.sh
- name: Check that the antithesis workload is sane
shell: bash
run: go run ./tests/antithesis --avalanchego-path=/tmp/e2e-test/avalanchego/avalanchego --duration=60s
- name: Check antithesis image build
shell: bash
run: bash -x scripts/tests.build_antithesis_images.sh
run: ./scripts/run_task.sh test-build-antithesis-images-ci
176 changes: 176 additions & 0 deletions Taskfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
# https://taskfile.dev
# To run on a system without task installed, `./scripts/run_task.sh` will execute it with `go run`.
# If in the nix dev shell, `task` is available.

version: '3'

env:
BASEDIR: '{{.BASEDIR | default "/tmp/e2e-test"}}'
AVALANCHEGO_BUILD_PATH: '{{.AVALANCHEGO_BUILD_PATH | default "/tmp/e2e-test/avalanchego"}}'
DATA_DIR: '{{.DATA_DIR | default "/tmp/e2e-test/precompile-data"}}'

tasks:
default: ./scripts/run_task.sh --list

actionlint:
desc: Validate GitHub Actions workflows and ensure scripts use run_task.sh for reproducibility
cmd: ./scripts/actionlint.sh # tests.yml

build:
desc: Compile Subnet EVM binary with git commit and static linking flags
cmd: ./scripts/build.sh # tests.yml

build-antithesis-images:
desc: Build Docker images for antithesis testing including avalanchego node and subnet-evm builder images
cmd: bash -x ./scripts/build_antithesis_images.sh # publish_antithesis_images.yml

build-bench-precompiles:
desc: Run benchmark tests for precompile contracts with timeout and coverage
cmd: ./scripts/build_bench_precompiles.sh # bench.yml

build-docker-image:
desc: Build multi-platform Docker image with avalanchego integration and optional publishing
cmd: ./scripts/build_docker_image.sh # publish_docker.yml

build-test:
desc: Run all Go tests with retry logic for flaky tests, race detection, and coverage reporting
cmd: ./scripts/build_test.sh # tests.yml

check-avalanchego-version:
desc: Ensure consistent avalanchego version by updating and checking for changes
cmds:
- task: update-avalanchego-version
- cmd: git diff --exit-code

check-clean-branch:
desc: Checks that the git working tree is clean
cmds:
- cmd: git add --all
- cmd: git update-index --really-refresh >> /dev/null
- cmd: git status --short # Show the status of the working tree.
- cmd: git diff-index --quiet HEAD # Exits if any uncommitted changes are found.

check-generate-codec:
desc: Checks that generated codec files are up-to-date (requires a clean git working tree)
cmds:
- task: generate-codec
- task: check-clean-branch

check-generate-mocks:
desc: Checks that generated mocks are up-to-date (requires a clean git working tree)
cmds:
- task: generate-mocks
- task: check-clean-branch

check-go-mod-tidy:
desc: Checks that go.mod and go.sum are up-to-date (requires a clean git working tree)
cmds:
- cmd: go mod tidy
- task: check-clean-branch

coverage:
desc: Display test coverage statistics from coverage.out file
cmd: ./scripts/coverage.sh # tests.yml

generate-codec:
desc: Generates fjl/gencodec files
cmds:
- cmd: grep -lr -E '^// Code generated by github\.com\/fjl\/gencodec\. DO NOT EDIT\.$' . | xargs -r rm
- cmd: go generate -run "github.com/fjl/gencodec" ./...

generate-mocks:
desc: Generates testing mocks
cmds:
- cmd: grep -lr -E '^// Code generated by MockGen\. DO NOT EDIT\.$' . | xargs -r rm
- cmd: go generate -run "go.uber.org/mock/mockgen" ./...

install-avalanchego-release:
desc: Download and install AvalancheGo release binary for testing, with fallback to building from source
cmd: ./scripts/install_avalanchego_release.sh # tests.yml

lint:
desc: Run golangci-lint and check for allowed Ethereum imports in Go code
cmd: ./scripts/lint.sh

lint-all:
desc: Runs all lint checks in parallel
deps:
- lint
- actionlint
- shellcheck
- check-generate-codec
- check-generate-mocks

lint-all-ci:
desc: Runs all lint checks one-by-one
cmds:
- task: lint
- task: actionlint
- task: shellcheck
- task: check-generate-codec
- task: check-generate-mocks

setup-contracts:
desc: Set up contracts by installing NPM dependencies, cleaning Hardhat cache, and compiling contracts
dir: ./contracts
cmds:
- cmd: npm ci
- cmd: npx hardhat clean
- cmd: npx hardhat compile

shellcheck:
desc: Run shellcheck static analysis on all shell scripts with version management
cmd: ./scripts/shellcheck.sh # tests.yml

test-build-antithesis-images:
desc: Test antithesis images build
cmd: bash -x scripts/tests.build_antithesis_images.sh

test-build-antithesis-images-ci: # consolidated test-build-antithesis-images
desc: Test antithesis images build with CI setup
cmds:
- task: install-avalanchego-release
- task: build
- cmd: go run ./tests/antithesis --avalanchego-path=/tmp/e2e-test/avalanchego/avalanchego --duration=60s
- task: test-build-antithesis-images

test-build-image:
desc: Test docker image build
cmd: bash -x scripts/tests.build_docker_image.sh # tests.yml

test-e2e-load:
desc: Run end-to-end load tests using Ginkgo test framework
cmd: ./scripts/run_ginkgo_load.sh # tests.yml

test-e2e-load-ci: # consolidated test-e2e-load
desc: Run E2E load tests with CI setup
cmds:
- task: install-avalanchego-release
- task: build
- task: test-e2e-load

test-e2e-precompile:
desc: Run end-to-end precompile tests using Ginkgo with parallel execution
cmd: bash ./scripts/run_ginkgo_precompile.sh # tests.yml

test-e2e-precompile-ci: # consolidated test-e2e-precompile
desc: Run E2E precompile tests with CI setup
cmds:
- task: install-avalanchego-release
- task: build
- task: test-e2e-precompile

test-e2e-warp:
desc: Run end-to-end warp tests using Ginkgo test framework
cmd: ./scripts/run_ginkgo_warp.sh # tests.yml

test-e2e-warp-ci: # consolidated test-e2e-warp
desc: Run E2E warp tests with CI setup
cmds:
- task: install-avalanchego-release
- task: build
- task: test-e2e-warp

update-avalanchego-version:
desc: Update AvalancheGo version in go.mod and sync GitHub Actions workflow custom action version
cmd: ./scripts/update_avalanchego_version.sh # tests.yml
Loading