Skip to content

ci: add awslcfips to nix jobs #5205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 17, 2025
Merged

ci: add awslcfips to nix jobs #5205

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b82eeec
ci: add awslcfips to nix jobs
dougch Mar 19, 2025
073a1bf
add newer awslcfips
dougch Mar 19, 2025
ce90485
more env variables
dougch Mar 19, 2025
12b52e5
reduce duplicate env vars
dougch Mar 20, 2025
5990362
better comments on tests not run
dougch Mar 20, 2025
0c3c13e
more pr feedback
dougch Mar 20, 2025
710e6b7
put S2N_NO_HEADBUILD back
dougch Mar 20, 2025
a15f9d8
add/reorder the list of integ tests
dougch Mar 20, 2025
e3b20c8
Naming consistency
dougch Mar 20, 2025
41862a2
missing copyright
dougch Mar 20, 2025
3010b16
Update codebuild/spec/buildspec_integv2_nix.yml
dougch Mar 27, 2025
e1ebc18
Update codebuild/spec/buildspec_integv2_nix.yml
dougch Mar 27, 2025
43c2884
Collapsing waves
dougch Mar 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 170 additions & 0 deletions codebuild/spec/buildspec_integv2_nix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
---
version: 0.2
env:
shell: bash
variables:
# --ignore-environment prevents existing environment variables from being carried forward into the new shell.
# This allows for a clean, sterile environment.
NIXDEV_ARGS: --max-jobs auto --ignore-environment
# Ctest is doing Regex matching of test names, with an implicit ".*" around these, so:
# renegotiate covers both renegotiate.py and renegotiate_apache.py
INTEGV2_TEST:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we match the rebalancing work that was done? So have

  1. happy path
  2. buffered send
  3. other bucket 1
  4. other bucket 2

Copy link
Contributor Author

@dougch dougch Apr 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually - let me do this in a followup when we figure out fan-out.

happy_path client_authentication sni_match buffered_send npn sslv2_client_hello
ocsp external_psk pq_handshake serialization
signature_algorithms record_padding renegotiate$ renegotiate_apache cross_compatibility early_data hello_retry_requests
fragmentation key_update session_resumption version_negotiation
# Excluded in nix:
# - dynamic_record_sizes (fails on aarch64)
# - sslyze (tooling not available on aarch64)

batch:
build-graph:
# Cache job for x86
- identifier: nixCache_x86_64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
privileged-mode: false
type: LINUX_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodx861-ehnvuoswh2yr?region=us-east-2

# Cache Job for aarch64
- identifier: nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: false
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

# AWSLC x86
- identifier: Integ_awslc_x86_0
depend-on:
- nixCache_x86_64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
privileged-mode: true
type: LINUX_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslc
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodx861-ehnvuoswh2yr?region=us-east-2

# AWSLC aarch64
- identifier: Integ_awslc_aarch64_0
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslc
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

# AWSLC-FIPS-2022
- identifier: Integ_awslcfips2022_aarch64_0
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslcfips2022
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

# AWSLC-FIPS-2024
- identifier: Integ_awslcfips2024_aarch64_0
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslcfips2024
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

# Openssl30 x86
- identifier: Integ_openssl30_x86_0
depend-on:
- nixCache_x86_64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
privileged-mode: true
type: LINUX_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodx861-ehnvuoswh2yr?region=us-east-2

# Openssl30 aarch64
- identifier: Integ_openssl30_aarch64_0
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

# Openssl111 aarch64 only
- identifier: Integ_openssl111_aarch64_0
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_XLARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#openssl111
NIX_CACHE_BUCKET: s3://codebuildnixinteg-prod-nixcachebucketintegprodaarc-rqyksjxh6wxa?region=us-east-2

phases:
install:
commands:
- |
if [[ $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
echo "Refreshing nix cache..."
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
nix build .#devShell
nix copy --to $NIX_CACHE_BUCKET .#devShell
else
echo "Downloading cache"
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
fi
pre_build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh; configure"
fi
build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh; build"
fi
post_build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh;integ $INTEGV2_TEST"
fi


166 changes: 166 additions & 0 deletions codebuild/spec/buildspec_unit_nix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
#
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
---
version: 0.2
env:
shell: bash
variables:
NIXDEV_ARGS: --max-jobs auto
S2N_NO_HEADBUILD: "1"

batch:
build-graph:
# Cache job for x86
- identifier: nixCache_x86_64
comment: identifiers can not contain dashes
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
variables:
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-x86-64?region=us-west-2

# Cache job for aarch64
- identifier: nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
type: ARM_CONTAINER
variables:
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# Openssl30 x86
- identifier: UnitOpenssl30_x86_64
depend-on:
- nixCache_x86_64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
privileged-mode: true
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-x86-64?region=us-west-2

# Openssl30 aarch64
- identifier: UnitOpenssl30_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#default
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# Openssl111 aarch64
- identifier: UnitOpenssl111_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#openssl111
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# Openssl102 aarch64
- identifier: UnitOpenssl102_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#openssl102
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# awslc x86
- identifier: UnitAwslc_x86_64
depend-on:
- nixCache_x86_64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild:latest
privileged-mode: true
variables:
NIXDEV_LIBCRYPTO: .#awslc
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-x86-64?region=us-west-2

# awslc aarch64
- identifier: UnitAwslc_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslc
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# awslcfips 2022 aarch64
- identifier: UnitAwslcFips2022_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslcfips2022
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

# awslcfips 2024 aarch64
- identifier: UnitAwslcFips2024_aarch64
depend-on:
- nixCache_aarch64
env:
compute-type: BUILD_GENERAL1_LARGE
image: public.ecr.aws/l1b2r3y5/nix-aws-codebuild-aarch64:next
privileged-mode: true
type: ARM_CONTAINER
variables:
NIXDEV_LIBCRYPTO: .#awslcfips2024
NIX_CACHE_BUCKET: s3://s2n-tls-nixcachebucket-aarch64?region=us-west-2

phases:
install:
commands:
- |
if [[ "$CODEBUILD_BATCH_BUILD_IDENTIFIER" =~ .*"nixCache".* ]]; then
echo "Refreshing nix cache..."
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
nix build $NIXDEV_ARGS .#devShell
nix copy --to $NIX_CACHE_BUCKET .#devShell;
else
echo "Downloading cache"
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
fi
pre_build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix copy --from $NIX_CACHE_BUCKET --all --no-check-sigs
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh; configure";
fi
build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh; build";
fi
post_build:
commands:
- |
if [[ ! $CODEBUILD_BATCH_BUILD_IDENTIFIER =~ .*"nixCache".* ]]; then
nix develop $NIXDEV_ARGS $NIXDEV_LIBCRYPTO --command bash -c "source ./nix/shell.sh; unit"
fi
2 changes: 1 addition & 1 deletion nix/shell.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ function build {(set -e
javac tests/integrationv2/bin/SSLSocketClient.java
cmake --build ./build -j $(nproc)
# Build s2n from HEAD
if [[ -z "${S2N_KTLS_TESTING_EXPECTED}" ]]; then
if [[ -z "${S2N_KTLS_TESTING_EXPECTED}" && -z "${S2N_NO_HEADBUILD}" ]]; then
$SRC_ROOT/codebuild/bin/install_s2n_head.sh $(mktemp -d)
fi
)}
Expand Down
1 change: 1 addition & 0 deletions tests/unit/s2n_build_test.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ S2N_RESULT s2n_check_supported_libcrypto(const char *s2n_libcrypto)
{ .libcrypto = "awslc", .is_openssl = false },
{ .libcrypto = "awslc-fips", .is_openssl = false },
{ .libcrypto = "awslc-fips-2022", .is_openssl = false },
{ .libcrypto = "awslc-fips-2024", .is_openssl = false },
{ .libcrypto = "boringssl", .is_openssl = false },
{ .libcrypto = "libressl", .is_openssl = false },
{ .libcrypto = "openssl-1.0.2", .is_openssl = true },
Expand Down
Loading