SW-8251 Port terraware-web CI workflow to Buildkite

.buildkite/README.md

@@ -0,0 +1,91 @@
+# NOTES ABOUT THIS PIPELINE
+
+## Uploading
+
+The pipeline file here doesn't do anything in and of itself; it needs to be uploaded using the
+Buildkite agent. On the Buildkite side, in the repo configuration, we define the starting point
+of the pipeline; the step defined here is the thing that uploads `pipeline.yml` to Buildkite.
+
+```yaml
+env:
+  BUILDKITE_PLUGIN_S3_CACHE_BUCKET: 'terraformation-buildkite-cache'
+  BUILDKITE_PLUGIN_S3_CACHE_ONLY_SHOW_ERRORS: 'true'
+  BUILDKITE_PLUGIN_S3_CACHE_PREFIX: 'repo/terraware-web'
+
+agents:
+  queue: 'x86-8core'
+
+steps:
+  - label: ':pipeline: Upload pipeline'
+    plugins:
+      - peakon/git-shallow-clone#v0.0.1:
+          # Depth should be large enough that it will find the commit on the main branch
+          # from which the current branch was forked, but small enough to keep the clone
+          # operation reasonably fast.
+          depth: 200
+    command: 'buildkite-agent pipeline upload'
+```
+
+## YAML anchors
+
+We define YAML anchors for elements that are used in several places in the pipeline. The "common"
+keyword isn't interpreted by Buildkite; it's just used as a place to put the anchors. Anchors can
+also be defined inline in steps, but to make the code easier to follow, only do that if you're
+not reusing the anchor in other steps.
+
+The Buildkite docs have a brief introduction to YAML anchors, if you're not familiar with them:
+https://buildkite.com/docs/pipelines/integrations/plugins/using#using-yaml-anchors-with-plugins
+
+## Scripts
+
+Most of the steps in the build have a single "command" value that runs a shell script in the
+.buildkite/scripts directory. The script then performs the actual actions (compiling, running
+tests, etc.) While it's possible to pull that logic up into the pipeline, keeping the logic in
+scripts lets us add whatever kinds of conditional logic we need. Shell scripts can also be linted
+and IDEs can do syntax highlighting and refactoring.
+
+You'll notice the scripts have a bunch of "echo" commands that output lines starting with "---".
+Those lines mark the start of collapsible sections. More information here:
+https://buildkite.com/docs/pipelines/configure/managing-log-output
+
+## Caching and artifacts
+
+We run Buildkite on ephemeral hosts that go away when they're idle. In addition, each step in a
+Buildkite pipeline runs in a clean copy of the checked-out repo (possibly on a different host).
+We therefore can't rely on the local filesystem for caching dependencies or for making build
+output available to later steps.
+
+To deal with this, we use two Buildkite features: caches and artifacts.
+
+Caches persist between jobs and are keyed by a checksum of a list of manifest files. We use caches
+for downloaded dependencies. We use our own S3 bucket to store the caches.
+
+Caches have a concept of a level hierarchy, documented here:
+https://buildkite.com/resources/plugins/buildkite-plugins/cache-buildkite-plugin/#caching-levels
+In this pipeline, all builds read dependencies from the pipeline level and write dependencies
+to the branch and file levels. Then, after a successful build, if we're building on the "main"
+branch, we "promote" the caches to the pipeline level so they'll be available in other branches.
+
+Artifacts are per-job; we use them to pass data between steps. For example, the `dist` directory
+gets bundled up in a compressed tarfile so it can be downloaded by later steps.
+
+## Build environment
+
+As currently configured, our Buildkite builds run on EC2 instances in a dedicated VPC. The
+instances are running Amazon Linux 2023, and the build steps run directly on the host, not in
+Docker containers. That means you need to make sure you've installed the tools you need for a
+given build step, and you should try to avoid creating files outside the repo directory since
+they won't necessarily be cleaned up if another build step runs on the same host.
+
+System setup is mostly encapsulated in .buildkite/scripts/install-deps.sh.
+
+## Concurrency or the lack thereof
+
+To minimize costs, we don't currently run any of the steps in this pipeline in parallel, meaning
+we don't need to spin up multiple hosts to run a build. Linearity is enforced via "depends_on"
+directives in the build steps.
+
+Note that just because the steps are linear, there's no guarantee Buildkite will run them all
+on the same host. If there are multiple hosts available, any of them can be chosen to execute
+a step. In practice, steps usually do stick to the same host, but that's not guaranteed and
+shouldn't be relied upon.

.buildkite/hooks/pre-command

@@ -0,0 +1,41 @@
+#!/bin/bash
+#
+# Set environment variables that can be consumed by commands. This is run after plugins
+# such as the AWS Secrets Manager plugin, so it can refer to any environment variables
+# set by those plugins.
+#
+# The Buildkite agent effectively sources this file in bash before launching commands,
+# so any environment variables exported here will be available to all commands. See
+# https://buildkite.com/docs/agent/hooks for more details.
+#
+
+set -euo pipefail
+
+COMMIT_SHA="${BUILDKITE_COMMIT:0:12}"
+
+# Define tier and app version based on branch/tag
+if [[ "${BUILDKITE_TAG:-}" =~ ^v[0-9]+\.[0-9.]+ ]]; then
+    TIER=PROD
+    IS_CD=true
+    APP_VERSION="${BUILDKITE_TAG}"
+elif [[ "${BUILDKITE_BRANCH}" == "main" ]]; then
+    TIER=STAGING
+    IS_CD=true
+    APP_VERSION="x${COMMIT_SHA}"
+else
+    TIER=CI
+    IS_CD=false
+    APP_VERSION="${COMMIT_SHA}"
+fi
+
+LOWER_TIER=$(echo "$TIER" | tr '[:upper:]' '[:lower:]')
+
+# If this commit's title has a suffix like (#123), it's probably a merged PR.
+MERGED_PR_NUMBER=$(git log -1 --pretty=%s | sed -En 's/.*\(#([0-9]+)\)$/\1/p')
+
+export APP_VERSION
+export COMMIT_SHA
+export IS_CD
+export LOWER_TIER
+export MERGED_PR_NUMBER
+export TIER

.buildkite/pipeline.yml

@@ -0,0 +1,158 @@
+# See .buildkite/README.md for details about this pipeline.
+
+common:
+  - &cache-common-args
+    backend: s3
+    compression: tgz
+    restore: pipeline
+
+  - &cache-node-modules-args
+    <<: *cache-common-args
+    path: 'node_modules'
+    manifest: 'yarn.lock'
+
+  - &cache-node-modules
+    cache#v1.10.0:
+      <<: *cache-node-modules-args
+
+  - &load-secrets
+    seek-oss/aws-sm#v2.4.1:
+      json-to-env:
+        - secret-id: 'buildkite_build_secrets'
+        - secret-id: 'terraware_web_build_secrets'
+
+  - &restore-dist
+    artifacts#v1.9.4:
+      download: 'dist'
+      compressed: 'dist.tgz'
+
+  - &shallow-clone
+    peakon/git-shallow-clone#v0.0.1:
+      depth: 1
+
+agents:
+  # Run all steps on the same kind of runner.
+  queue: 'x86-8core'
+
+steps:
+  - label: ':yarn: Build and test'
+    key: 'build-and-test'
+    plugins:
+      - *shallow-clone
+      - cache#v1.10.0:
+          <<: *cache-node-modules-args
+          save:
+            - file
+            - branch
+      - artifacts#v1.9.4:
+          upload: 'dist'
+          compressed: 'dist.tgz'
+    command: .buildkite/scripts/build-and-test.sh
+
+  - label: ':playwright: Run end-to-end tests'
+    key: 'e2e-tests'
+    depends_on: 'build-and-test'
+    if: "build.branch != 'main' && build.tag == null"
+    plugins:
+      - *shallow-clone
+      - *cache-node-modules
+      - cache#v1.10.0:
+          <<: *cache-common-args
+          path: '.cache/ms-playwright'
+          manifest: 'yarn.lock'
+      - *restore-dist
+      - *load-secrets
+    command: .buildkite/scripts/run-e2e-tests.sh
+    artifact_paths:
+      - 'playwright/test-results/**/*'
+      - 'playwright-report/**/*'
+    env:
+      SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_KEYCLOAK_ISSUERURI: https://auth.staging.terraware.io/realms/terraware
+      SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_KEYCLOAK_CLIENTSECRET: dummy
+      SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUERURI: https://auth.staging.terraware.io/realms/terraware
+
+  - label: ':docker: Build and push Docker image'
+    key: 'docker'
+    depends_on:
+      - step: 'build-and-test'
+      - step: 'e2e-tests'
+        allow_failure: true
+    plugins:
+      - *shallow-clone
+      - *load-secrets
+      - cache#v1.10.0:
+          <<: *cache-common-args
+          path: '.buildx-cache'
+          save:
+            - branch
+          force: true
+      - *restore-dist
+    command: .buildkite/scripts/docker-build-push.sh
+
+  - label: ':buildkite: Promote branch-level caches to pipeline-level'
+    key: 'promote-caches'
+    depends_on: 'docker'
+    branches: 'main'
+    plugins:
+      - *shallow-clone
+      - cache#v1.10.0: &promote-cache
+          <<: *cache-node-modules-args
+          save: 'pipeline'
+      - cache#v1.10.0:
+          <<: *promote-cache
+          path: '.buildx-cache'
+    command: 'echo --- :buildkite: Promoted caches to pipeline level'
+
+  - label: ':amazon-ecs: Deploy to ECS'
+    key: 'deploy'
+    depends_on:
+      - step: 'promote-caches'
+        allow_failure: true
+      - 'docker'
+    if: "build.branch == 'main' || build.tag =~ /^v[0-9]+\\.[0-9]+/"
+    plugins:
+      - *shallow-clone
+    command: .buildkite/scripts/deploy-ecs.sh
+
+  - label: ':slack: Release notes & Jira transitions'
+    key: 'release-notes'
+    depends_on: 'deploy'
+    if: "build.tag =~ /^v[0-9]+\\.[0-9]+/"
+    plugins:
+      - *shallow-clone
+      - *load-secrets
+    command: .buildkite/scripts/release-notes.sh
+
+  - label: ':speech_balloon: Notify PR of staging deploy'
+    key: 'notify-pr'
+    depends_on:
+      - 'deploy'
+      - 'release-notes'
+    branches: 'main'
+    plugins:
+      - *shallow-clone
+      - *load-secrets
+    command: .buildkite/scripts/notify-pr.sh
+
+  - label: ':books: Deploy docs'
+    key: 'deploy-docs'
+    depends_on:
+      - 'docker'
+      - 'notify-pr'
+    branches: 'main'
+    plugins:
+      - *shallow-clone
+      - *load-secrets
+      - *cache-node-modules
+    command: .buildkite/scripts/deploy-docs.sh
+
+  - label: ':vercel: Deploy Vercel preview'
+    key: 'vercel-preview'
+    depends_on: 'build-and-test'
+    if: "build.branch != 'main' && build.tag == null"
+    plugins:
+      - *shallow-clone
+      - *load-secrets
+      - *cache-node-modules
+      - *restore-dist
+    command: .buildkite/scripts/deploy-vercel.sh

.buildkite/scripts/build-and-test.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+.buildkite/scripts/install-deps.sh --node --tools
+
+echo "--- :yarn: Install dependencies"
+yarn install --frozen-lockfile --prefer-offline
+
+echo "--- :yarn: Generate strings"
+yarn generate-strings
+
+echo "--- :prettier: Check formatting"
+yarn prettier --check .
+
+echo "--- :eslint: Run linter"
+yarn lint \
+    --rule 'react/jsx-no-bind: 0' \
+    --rule 'react-hooks/immutability: 0' \
+    --rule 'react-hooks/refs: 0' \
+    --rule 'react-hooks/set-state-in-effect: 0' \
+    --rule 'react-hooks/static-components: 0'
+
+echo "--- :typescript: TypeScript check"
+yarn ts
+
+echo "--- :jest: Run unit tests"
+yarn test:ci
+
+echo "--- :package: Build"
+echo "$APP_VERSION" > public/build-version.txt
+
+PUBLIC_TERRAWARE_API='' \
+PUBLIC_TERRAWARE_TIER="$LOWER_TIER" \
+PUBLIC_TERRAWARE_FE_BUILD_VERSION="$APP_VERSION" \
+PUBLIC_DATADOG_APP_ID="${DATADOG_APP_ID:-}" \
+PUBLIC_DATADOG_CLIENT_TOKEN="${DATADOG_CLIENT_TOKEN:-}" \
+PUBLIC_MIXPANEL_TOKEN="${MIXPANEL_TOKEN:-}" \
+yarn build