HADOOP-19588. S3A support for S3 express access points #7746

revathivijay · 2025-06-17T18:04:13Z

Description of PR

Currently, the endpoint for using S3 accesspoint is resolved for S3 and S3 on outposts: https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/ArnResource.java#L29-L30. However, for s3express, the endpoint is s3express-<zoneId>.<region>.amazonaws.com. (https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-bucket-az-networking.html). PR adds support for this.
Additionally, for accessing objects using s3express in the S3AFileSystem, we need to parse the access point ARN for an S3express access point into the access point name, since ARNs aren't supported on S3express other than in IAM policies. (https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-directory-buckets-restrictions-limitations-naming-rules.html). This PR also addresses this change.
Update the SDK version used by the changes to 2.31.12, which is the version (at minimum) needed to use s3express access points. It does not do the SDK upgrade, but only the version of the dependency.

How was this patch tested?

Make sure changes compile by running mvn clean install -DskipTests
Running newly added unit tests in TestArnResource:

$ mvn -pl hadoop-tools/hadoop-aws test -Dtest=TestArnResource`

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.s3a.TestArnResource
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.098 s - in org.apache.hadoop.fs.s3a.TestArnResource
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.590 s
[INFO] Finished at: 2025-06-16T14:53:52-04:00
[INFO] ------------------------------------------------------------------------

Tried running integ test for S3ABucket existence using the following:

➜  hadoop git:(trunk) ✗ mvn clean test -Dit.test=ITestS3ABucketExistence -Dtest=none -Dsurefire.failIfNoSpecifiedTests=false

[INFO] Reactor Summary for Apache Hadoop Main 3.5.0-SNAPSHOT:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [  0.175 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  1.683 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  0.346 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  0.509 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  0.034 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.057 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  1.460 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  0.495 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  2.244 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  0.262 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:34 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  1.452 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [  1.350 s]
[INFO] Apache Hadoop Registry ............................. SUCCESS [  1.603 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.028 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [  9.651 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [ 20.707 s]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [  0.493 s]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [  2.054 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  1.420 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [  0.023 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [  7.008 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [  6.862 s]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [  0.037 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [  5.781 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [  3.034 s]
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [  1.759 s]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [  1.493 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 13.510 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [  7.194 s]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [  2.439 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [  3.432 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [  0.650 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [  5.310 s]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [  3.321 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [  2.688 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [  3.829 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [  2.530 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [  4.421 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [  2.416 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  1.433 s]
[INFO] Apache Hadoop Federation Balance ................... SUCCESS [  1.624 s]
[INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [  7.600 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.022 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [  1.353 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [  1.326 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [  0.020 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common .... SUCCESS [  1.626 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client .... SUCCESS [  3.227 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [  0.028 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 2.5  SUCCESS [  1.450 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests ..... SUCCESS [  2.108 s]
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [  2.412 s]
[INFO] Apache Hadoop YARN TimelineService DocumentStore ... SUCCESS [  1.498 s]
[INFO] Apache Hadoop YARN GlobalPolicyGenerator ........... SUCCESS [  1.676 s]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [  0.021 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [  1.667 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [  1.003 s]
[INFO] Apache Hadoop YARN Services ........................ SUCCESS [  0.018 s]
[INFO] Apache Hadoop YARN Services Core ................... SUCCESS [  4.002 s]
[INFO] Apache Hadoop YARN Services API .................... SUCCESS [  2.181 s]
[INFO] Apache Hadoop YARN Application Catalog ............. SUCCESS [  0.022 s]
[INFO] Apache Hadoop YARN Application Catalog Webapp ...... SUCCESS [ 17.720 s]
[INFO] Apache Hadoop YARN Application Catalog Docker Image  SUCCESS [  0.042 s]
[INFO] Apache Hadoop YARN Application MaWo ................ SUCCESS [  0.022 s]
[INFO] Apache Hadoop YARN Application MaWo Core ........... SUCCESS [  1.074 s]
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [  0.021 s]
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [  0.540 s]
[INFO] Apache Hadoop YARN UI .............................. SUCCESS [  0.135 s]
[INFO] Apache Hadoop YARN CSI ............................. SUCCESS [  4.102 s]
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [  1.254 s]
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [  1.167 s]
[INFO] Apache Hadoop MapReduce NativeTask ................. SUCCESS [  1.709 s]
[INFO] Apache Hadoop MapReduce Uploader ................... SUCCESS [  1.088 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  1.816 s]
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [  1.271 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [  2.025 s]
[INFO] Apache Hadoop Client Aggregator .................... SUCCESS [  0.839 s]
[INFO] Apache Hadoop Dynamometer Workload Simulator ....... SUCCESS [  1.458 s]
[INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SUCCESS [  1.960 s]
[INFO] Apache Hadoop Dynamometer Block Listing Generator .. SUCCESS [  1.264 s]
[INFO] Apache Hadoop Dynamometer Dist ..................... SUCCESS [  1.259 s]
[INFO] Apache Hadoop Dynamometer .......................... SUCCESS [  0.019 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  1.405 s]
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [  1.731 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [  1.915 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  1.984 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  1.413 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  1.455 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [  0.019 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [  4.972 s]
[INFO] Apache Hadoop Kafka Library support ................ SUCCESS [  1.001 s]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [  4.480 s]
[INFO] Apache Hadoop Aliyun OSS support ................... SUCCESS [  1.260 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  1.571 s]
[INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [  1.254 s]
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [  1.256 s]
[INFO] Apache Hadoop Image Generation Tool ................ SUCCESS [  1.259 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  0.711 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  0.029 s]
[INFO] Apache Hadoop Common Benchmark ..................... SUCCESS [  0.619 s]
[INFO] Apache Hadoop Compatibility Benchmark .............. SUCCESS [  0.835 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.022 s]
[INFO] Apache Hadoop Client API ........................... SUCCESS [  0.844 s]
[INFO] Apache Hadoop Client Runtime ....................... SUCCESS [  0.877 s]
[INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [  0.156 s]
[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [  0.705 s]
[INFO] Apache Hadoop Client Packaging Invariants for Test . SUCCESS [  0.047 s]
[INFO] Apache Hadoop Client Packaging Integration Tests ... SUCCESS [  0.053 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [  0.528 s]
[INFO] Apache Hadoop Client Modules ....................... SUCCESS [  0.020 s]
[INFO] Apache Hadoop Tencent COS Support .................. SUCCESS [  0.875 s]
[INFO] Apache Hadoop OBS support .......................... SUCCESS [  1.395 s]
[INFO] Apache Hadoop Volcano Engine Services support ...... SUCCESS [  1.658 s]
[INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [  0.370 s]
[INFO] Apache Hadoop Cloud Storage Project ................ SUCCESS [  0.018 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  05:32 min
[INFO] Finished at: 2025-06-16T18:04:59-04:00
[INFO] ------------------------------------------------------------------------

Tried running the same test from within the hadoop-aws folder since I didn't see it run in the above:
Additional info:

I set up auth-keys.xml as explained in https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/testing.html#Running_the_tests, and used bucket name as an S3express access point to test against it. But running into some JUnit version errors (pasted some sample errors below), which I believe is related to JUnit upgrade.

➜  hadoop-aws git:(trunk) ✗ pwd
...../hadoop/hadoop-tools/hadoop-aws
➜  hadoop-aws git:(trunk) ✗ mvn clean test -Dtest=none -Dit.test=ITestS3ABucketExistence

[ERROR] /hadoop/hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFilesystem.java:[266,5] no suitable method found for assertEquals(java.lang.String,org.apache.hadoop.security.token.Token<org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier>,org.apache.hadoop.security.token.Token<capture#1 of ? extends org.apache.hadoop.security.token.TokenIdentifier>)
    method org.junit.jupiter.api.Assertions.assertEquals(short,short,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to short)
    method org.junit.jupiter.api.Assertions.assertEquals(short,java.lang.Short,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to short)
    method org.junit.jupiter.api.Assertions.assertEquals(java.lang.Short,short,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to java.lang.Short)
    method org.junit.jupiter.api.Assertions.assertEquals(java.lang.Short,java.lang.Short,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to java.lang.Short)
    method org.junit.jupiter.api.Assertions.assertEquals(short,short,java.util.function.Supplier<java.lang.String>) is not applicable
      (argument mismatch; java.lang.String cannot be converted to short)
    method org.junit.jupiter.api.Assertions.assertEquals(short,java.lang.Short,java.util.function.Supplier<java.lang.String>) is not applicable
      (argument mismatch; java.lang.String cannot be converted to short)
    method org.junit.jupiter.api.Assertions.assertEquals(java.lang.Short,short,java.util.function.Supplier<java.lang.String>) is not applicable
      (argument mismatch; java.lang.String cannot be converted to java.lang.Short)
    method org.junit.jupiter.api.Assertions.assertEquals(java.lang.Short,java.lang.Short,java.util.function.Supplier<java.lang.String>) is not applicable
      (argument mismatch; java.lang.String cannot be converted to java.lang.Short)
    method org.junit.jupiter.api.Assertions.assertEquals(byte,byte,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to byte)
    method org.junit.jupiter.api.Assertions.assertEquals(byte,java.lang.Byte,java.lang.String) is not applicable
      (argument mismatch; java.lang.String cannot be converted to byte)
    method org.junit.jupiter.api.Assertions.assertEquals(java.lang.Byte,byte,java.lang.String) is not applicable

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? PENDING integ test and docs.
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-06-17T21:40:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 47s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 35s		trunk passed
+1 💚	compile	15m 39s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	13m 51s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	4m 14s		trunk passed
+1 💚	mvnsite	1m 44s		trunk passed
+1 💚	javadoc	1m 38s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 30s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 46s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	36m 27s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	1m 5s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 21s	/patch-mvninstall-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	compile	15m 1s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 1s		the patch passed
-1 ❌	compile	13m 1s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	javac	13m 1s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 4 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	4m 9s	/results-checkstyle-root.txt	root: The patch generated 8 new + 9 unchanged - 1 fixed = 17 total (was 10)
-1 ❌	mvnsite	0m 49s	/patch-mvnsite-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	javadoc	1m 35s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 31s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 41s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	0m 45s	/patch-spotbugs-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	shadedclient	35m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 39s		hadoop-project in the patch passed.
-1 ❌	unit	0m 47s	/patch-unit-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	asflicense	1m 6s		The patch does not generate ASF License warnings.
		215m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.50 ServerAPI=1.50 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/1/artifact/out/Dockerfile
GITHUB PR	#7746
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 63fb97d2df63 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1d5f246`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/1/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ahmarsuhail · 2025-06-18T10:36:26Z

@revathivijay can you update your PR title to HADOOP-19588. S3A support for S3 express access points using that format means your JIRA will pick up this PR automatically

hadoop-yetus · 2025-06-18T18:11:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	7m 0s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 48s		trunk passed
+1 💚	compile	15m 40s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	13m 35s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	4m 20s		trunk passed
+1 💚	mvnsite	1m 42s		trunk passed
+1 💚	javadoc	1m 39s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 46s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	36m 11s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 56s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 21s	/patch-mvninstall-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	compile	15m 3s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 3s		the patch passed
-1 ❌	compile	13m 6s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	javac	13m 6s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	4m 10s	/results-checkstyle-root.txt	root: The patch generated 8 new + 9 unchanged - 1 fixed = 17 total (was 10)
-1 ❌	mvnsite	0m 48s	/patch-mvnsite-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	javadoc	1m 34s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 33s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 42s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	0m 45s	/patch-spotbugs-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	shadedclient	35m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 39s		hadoop-project in the patch passed.
-1 ❌	unit	0m 47s	/patch-unit-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	asflicense	1m 6s		The patch does not generate ASF License warnings.
		199m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.50 ServerAPI=1.50 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/2/artifact/out/Dockerfile
GITHUB PR	#7746
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux a5ee7ada7412 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `8257459`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/2/testReport/
Max. process+thread count	607 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-06-18T23:24:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 17s		Maven dependency ordering for branch
-1 ❌	mvninstall	32m 2s	/branch-mvninstall-root.txt	root in trunk failed.
-1 ❌	compile	15m 9s	/branch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in trunk failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	13m 12s	/branch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in trunk failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
+1 💚	checkstyle	4m 13s		trunk passed
-1 ❌	mvnsite	0m 54s	/branch-mvnsite-hadoop-tools_hadoop-aws.txt	hadoop-aws in trunk failed.
+1 💚	javadoc	1m 38s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 46s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	0m 52s	/branch-spotbugs-hadoop-tools_hadoop-aws.txt	hadoop-aws in trunk failed.
+1 💚	shadedclient	35m 38s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 58s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 27s	/patch-mvninstall-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
-1 ❌	compile	14m 24s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	javac	14m 24s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	13m 21s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	javac	13m 22s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	4m 22s	/results-checkstyle-root.txt	root: The patch generated 8 new + 9 unchanged - 1 fixed = 17 total (was 10)
-1 ❌	mvnsite	0m 53s	/patch-mvnsite-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	javadoc	1m 32s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 30s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+0 🆗	spotbugs	0m 40s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	0m 54s	/patch-spotbugs-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	shadedclient	35m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 38s		hadoop-project in the patch passed.
-1 ❌	unit	0m 52s	/patch-unit-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
+1 💚	asflicense	1m 4s		The patch does not generate ASF License warnings.
		197m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.50 ServerAPI=1.50 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/3/artifact/out/Dockerfile
GITHUB PR	#7746
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux e601c29ed5af 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b6d27f0`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/3/testReport/
Max. process+thread count	700 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7746/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Revathi Vijayaraghavan added 2 commits June 16, 2025 19:30

Initial commit for S3a support for S3 express access points

0b42306

Remove commented out assertions

1d5f246

github-actions bot added build trunk TOOLS AWS labels Jun 17, 2025

revathivijay marked this pull request as ready for review June 17, 2025 18:36

revathivijay changed the title ~~[HADOOP-19588] Initial commit for S3A support for S3 express access points~~ [HADOOP-19588] S3A support for S3 express access points Jun 17, 2025

revathivijay changed the title ~~[HADOOP-19588] S3A support for S3 express access points~~ HADOOP-19588. S3A support for S3 express access points Jun 18, 2025

Fix trailing whitespaces

8257459

Fix isEmpty() not found compilation error due to jdk version mismatch

b6d27f0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-19588. S3A support for S3 express access points #7746

HADOOP-19588. S3A support for S3 express access points #7746

revathivijay commented Jun 17, 2025 •

edited

Loading

Uh oh!

hadoop-yetus commented Jun 17, 2025

Uh oh!

ahmarsuhail commented Jun 18, 2025

Uh oh!

hadoop-yetus commented Jun 18, 2025

Uh oh!

hadoop-yetus commented Jun 18, 2025

Uh oh!

Uh oh!

HADOOP-19588. S3A support for S3 express access points #7746

Are you sure you want to change the base?

HADOOP-19588. S3A support for S3 express access points #7746

Conversation

revathivijay commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Jun 17, 2025

Uh oh!

ahmarsuhail commented Jun 18, 2025

Uh oh!

hadoop-yetus commented Jun 18, 2025

Uh oh!

hadoop-yetus commented Jun 18, 2025

Uh oh!

Uh oh!

revathivijay commented Jun 17, 2025 •

edited

Loading