Skip to content

Create e2e test suite#3136

Merged
nikola-jokic merged 23 commits intomasterfrom
nikola-jokic/e2e-setup-and-default-test
Nov 19, 2025
Merged

Create e2e test suite#3136
nikola-jokic merged 23 commits intomasterfrom
nikola-jokic/e2e-setup-and-default-test

Conversation

@nikola-jokic
Copy link
Collaborator

@nikola-jokic nikola-jokic commented Dec 7, 2023

This pull request introduces a major update to the end-to-end (E2E) testing infrastructure for GitHub Actions runners. The main changes add version 2 ("-v2") variants of all existing E2E test jobs in the .github/workflows/gha-e2e-tests.yaml workflow, refactor test execution into a new script (hack/e2e-test.sh), and provide dedicated shell scripts for each test scenario. These updates improve maintainability, modularity, and flexibility of the E2E test suite, making it easier to run individual tests and add new scenarios.

E2E Test Workflow Enhancements

  • Added new "-v2" job variants for all major E2E test scenarios (default-setup-v2, single-namespace-setup-v2, dind-mode-setup-v2, kubernetes-mode-setup-v2, auth-proxy-setup-v2, anonymous-proxy-setup-v2, self-signed-ca-setup-v2, update-strategy-tests-v2, and init-with-min-runners-v2). These jobs leverage the new test scripts and only run for PRs from the same repository. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Test Execution Refactoring

  • Introduced hack/e2e-test.sh, a new script that manages environment validation, test selection, and execution for all E2E scenarios, allowing easy running of individual or all tests.
  • Added a new gha-e2e target to the Makefile to invoke the new test runner script.

Scenario-Specific Test Scripts

  • Added dedicated shell scripts for each test scenario under test/actions.github.com/, including default-setup.test.sh, auth-proxy-setup.test.sh, and anonymous-proxy-setup.test.sh, each encapsulating the setup, execution, and teardown logic for its respective environment. [1] [2] [3]

Other Minor Improvements

  • Removed unused build arguments (RUNNER_VERSION, DOCKER_VERSION) from the docker-buildx target in the Makefile for cleaner builds.

These changes collectively modernize and modularize the E2E testing pipeline, making it easier to maintain and extend.

@nikola-jokic nikola-jokic requested review from a team, mumoshu and toast-gear as code owners December 7, 2023 14:09
@nikola-jokic nikola-jokic added the gha-runner-scale-set Related to the gha-runner-scale-set mode label Dec 7, 2023
rentziass
rentziass previously approved these changes Mar 18, 2024
Copy link
Member

@rentziass rentziass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 195ef3a to 3a10e3e Compare April 3, 2025 12:42
@nikola-jokic nikola-jokic marked this pull request as draft May 14, 2025 14:43
@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 512655f to 4236ff0 Compare June 9, 2025 16:56
@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 4236ff0 to 8b09e6c Compare October 28, 2025 14:59
@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 8b09e6c to 2c4bc6d Compare November 7, 2025 13:56
@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 1fba21f to c2f699e Compare November 10, 2025 13:45
@nikola-jokic nikola-jokic marked this pull request as ready for review November 10, 2025 13:59
@nikola-jokic nikola-jokic requested a review from a team as a code owner November 10, 2025 13:59
Copilot AI review requested due to automatic review settings November 10, 2025 13:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new end-to-end testing framework for GitHub Actions Runner Controller (ARC) using bash scripts and minikube. It adds multiple test scenarios covering different deployment configurations and a test runner script to execute them.

Key Changes:

  • Added 9 new test scripts covering various ARC deployment scenarios (default, single-namespace, dind-mode, kubernetes-mode, proxy configurations, self-signed CA, update strategy, and min runners)
  • Created a shared helper library (helper.sh) with common functions for cluster management, ARC installation, and workflow execution
  • Added test runner script (hack/e2e-test.sh) to orchestrate test execution
  • Integrated new tests into GitHub Actions workflow with v2 variants of existing test jobs
  • Added gha-e2e target to Makefile and removed unused Docker build arguments

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
test/actions.github.com/update-strategy.test.sh Tests eventual update strategy with listener recreation
test/actions.github.com/single-namespace-setup.test.sh Tests ARC when controller and runners are in the same namespace
test/actions.github.com/self-signed-ca-setup.test.sh Tests ARC with self-signed CA certificates using mitmproxy
test/actions.github.com/kubernetes-mode-setup.test.sh Tests ARC in Kubernetes container mode with OpenEBS storage
test/actions.github.com/init-with-min-runners.test.sh Tests ARC with minimum runner count configuration
test/actions.github.com/dind-mode-setup.test.sh Tests ARC in Docker-in-Docker container mode
test/actions.github.com/default-setup.test.sh Tests basic ARC installation and workflow execution
test/actions.github.com/auth-proxy-setup.test.sh Tests ARC with authenticated proxy configuration
test/actions.github.com/anonymous-proxy-setup.test.sh Tests ARC with anonymous proxy configuration
test/actions.github.com/helper.sh Shared library with utilities for cluster management, ARC operations, and workflow execution
test/actions.github.com/envrc.example Example environment configuration file for test execution
hack/e2e-test.sh Test runner script that orchestrates individual test execution
.github/workflows/gha-e2e-tests.yaml Adds v2 variants of existing test jobs that use the new bash-based test framework
Makefile Adds gha-e2e target and removes unused RUNNER_VERSION and DOCKER_VERSION build args

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nikola-jokic nikola-jokic marked this pull request as draft November 10, 2025 14:35
@nikola-jokic nikola-jokic force-pushed the nikola-jokic/e2e-setup-and-default-test branch from 96b49b6 to 0a288b0 Compare November 10, 2025 14:40
@nikola-jokic nikola-jokic marked this pull request as ready for review November 19, 2025 14:44
@nikola-jokic nikola-jokic changed the title Initial e2e setup with default test Create e2e test suite Nov 19, 2025
@nikola-jokic nikola-jokic merged commit 138b39b into master Nov 19, 2025
2 checks passed
@nikola-jokic nikola-jokic deleted the nikola-jokic/e2e-setup-and-default-test branch November 19, 2025 15:26
unpollito pushed a commit to DistruApp/actions-runner-controller that referenced this pull request Jan 21, 2026
Co-authored-by: Bassem Dghaidi <568794+Link-@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gha-runner-scale-set Related to the gha-runner-scale-set mode

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants