HADOOP-19612 Rpc auth header #7844

mccormickt12 · 2025-07-30T18:50:20Z

Add a new auth header to the rpc header proto for access token support. This should support different access tokens within the same connection.

Contributed-by: Tom McCormick [email protected]

Description of PR

Backport of #7803

How was this patch tested?

Testing details in #7803

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

Add a new auth header to the rpc header proto for access token support. This should support different access tokens within the same connection. Contributed-by: Tom McCormick <[email protected]>

hadoop-yetus · 2025-07-30T21:30:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 7s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	17m 47s		Maven dependency ordering for branch
+1 💚	mvninstall	38m 41s		branch-3.3 passed
+1 💚	compile	19m 58s		branch-3.3 passed
+1 💚	checkstyle	2m 58s		branch-3.3 passed
+1 💚	mvnsite	3m 10s		branch-3.3 passed
+1 💚	javadoc	2m 34s		branch-3.3 passed
+1 💚	spotbugs	5m 55s		branch-3.3 passed
+1 💚	shadedclient	42m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 31s	/patch-mvninstall-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvninstall	1m 2s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	compile	0m 55s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	cc	0m 55s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	0m 55s	/patch-compile-root.txt	root in the patch failed.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 31s	/results-checkstyle-root.txt	root: The patch generated 14 new + 171 unchanged - 0 fixed = 185 total (was 171)
-1 ❌	mvnsite	0m 36s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvnsite	1m 5s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	javadoc	1m 44s		the patch passed
-1 ❌	spotbugs	0m 33s	/patch-spotbugs-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	spotbugs	1m 2s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	shadedclient	13m 2s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 33s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	unit	1m 5s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		158m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/1/artifact/out/Dockerfile
GITHUB PR	#7844
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat
uname	Linux 529b4c69fed7 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `86e1950`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/1/testReport/
Max. process+thread count	557 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-07-31T05:53:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	17m 56s		Maven dependency ordering for branch
+1 💚	mvninstall	38m 17s		branch-3.3 passed
+1 💚	compile	19m 0s		branch-3.3 passed
+1 💚	checkstyle	2m 57s		branch-3.3 passed
+1 💚	mvnsite	3m 12s		branch-3.3 passed
+1 💚	javadoc	2m 36s		branch-3.3 passed
+1 💚	spotbugs	5m 50s		branch-3.3 passed
+1 💚	shadedclient	41m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	1m 0s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s		the patch passed
+1 💚	compile	18m 12s		the patch passed
+1 💚	cc	18m 12s		the patch passed
+1 💚	javac	18m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 55s	/results-checkstyle-root.txt	root: The patch generated 14 new + 171 unchanged - 0 fixed = 185 total (was 171)
+1 💚	mvnsite	3m 9s		the patch passed
+1 💚	javadoc	2m 28s		the patch passed
+1 💚	spotbugs	6m 9s		the patch passed
+1 💚	shadedclient	41m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 48s		hadoop-common in the patch passed.
-1 ❌	unit	232m 51s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 12s		The patch does not generate ASF License warnings.
		461m 55s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestLargeBlockReport
	hadoop.hdfs.protocol.TestBlockListAsLongs
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/2/artifact/out/Dockerfile
GITHUB PR	#7844
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat
uname	Linux ab5835a8a704 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `e5f42e5`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/2/testReport/
Max. process+thread count	2186 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/2/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

mccormickt12 · 2025-07-31T14:15:55Z

failures seem to be flaky tests, unrelated to my changes

[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   TestBlockListAsLongs.testFuzz:139->checkReport:156 Â» NoSuchMethod java.nio.Byt...
[ERROR]   TestBalancerWithHANameNodes.testBalancerWithObserverWithFailedNode:230->testBalancerWithObserver:273->Object.wait:-2 Â» TestTimedOut
[ERROR]   TestLargeBlockReport.testBlockReportSucceedsWithLargerLengthLimit:100 Â» Remote
[INFO] 
[ERROR] Tests run: 6867, Failures: 0, Errors: 3, Skipped: 24

hadoop-yetus · 2025-08-01T01:18:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	16m 49s		Maven dependency ordering for branch
+1 💚	mvninstall	38m 51s		branch-3.3 passed
+1 💚	compile	19m 28s		branch-3.3 passed
+1 💚	checkstyle	3m 9s		branch-3.3 passed
+1 💚	mvnsite	3m 10s		branch-3.3 passed
+1 💚	javadoc	2m 27s		branch-3.3 passed
+1 💚	spotbugs	5m 48s		branch-3.3 passed
+1 💚	shadedclient	42m 12s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 31s		the patch passed
+1 💚	compile	21m 20s		the patch passed
+1 💚	cc	21m 20s		the patch passed
+1 💚	javac	21m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 57s	/results-checkstyle-root.txt	root: The patch generated 14 new + 171 unchanged - 0 fixed = 185 total (was 171)
+1 💚	mvnsite	3m 16s		the patch passed
+1 💚	javadoc	2m 32s		the patch passed
+1 💚	spotbugs	6m 51s		the patch passed
+1 💚	shadedclient	42m 28s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 49s		hadoop-common in the patch passed.
-1 ❌	unit	236m 47s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 17s		The patch does not generate ASF License warnings.
		472m 2s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestLargeBlockReport
	hadoop.hdfs.protocol.TestBlockListAsLongs
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/3/artifact/out/Dockerfile
GITHUB PR	#7844
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat
uname	Linux 838bc7f8f6de 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `1915ee0`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/3/testReport/
Max. process+thread count	2268 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/3/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

mccormickt12 · 2025-08-01T17:49:36Z

tests failed again

[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   TestBlockListAsLongs.testFuzz:139->checkReport:156 Â» NoSuchMethod java.nio.Byt...
[ERROR]   TestBalancerWithHANameNodes.testBalancerWithObserverWithFailedNode:230->testBalancerWithObserver:273->Object.wait:-2 Â» TestTimedOut
[ERROR]   TestLargeBlockReport.testBlockReportSucceedsWithLargerLengthLimit:100 Â» Remote
[INFO] 
[ERROR] Tests run: 6867, Failures: 0, Errors: 3, Skipped: 24
[INFO] 
[ERROR] There are test failures.

hadoop-yetus · 2025-08-02T01:41:38Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	16m 19s		Maven dependency ordering for branch
+1 💚	mvninstall	38m 27s		branch-3.3 passed
+1 💚	compile	19m 4s		branch-3.3 passed
+1 💚	checkstyle	2m 59s		branch-3.3 passed
+1 💚	mvnsite	3m 10s		branch-3.3 passed
+1 💚	javadoc	2m 34s		branch-3.3 passed
+1 💚	spotbugs	5m 54s		branch-3.3 passed
+1 💚	shadedclient	41m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s		the patch passed
+1 💚	compile	20m 15s		the patch passed
+1 💚	cc	20m 15s		the patch passed
+1 💚	javac	20m 15s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 11s	/results-checkstyle-root.txt	root: The patch generated 14 new + 171 unchanged - 0 fixed = 185 total (was 171)
+1 💚	mvnsite	3m 42s		the patch passed
+1 💚	javadoc	2m 48s		the patch passed
+1 💚	spotbugs	6m 45s		the patch passed
+1 💚	shadedclient	41m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 56s		hadoop-common in the patch passed.
-1 ❌	unit	231m 11s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 12s		The patch does not generate ASF License warnings.
		461m 45s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestLargeBlockReport
	hadoop.hdfs.protocol.TestBlockListAsLongs
	hadoop.hdfs.server.namenode.TestNameNodeMXBean
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/4/artifact/out/Dockerfile
GITHUB PR	#7844
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat
uname	Linux d6e8e0e0322d 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `d76997a`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/4/testReport/
Max. process+thread count	3138 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7844/4/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

HADOOP-19612 Add RPC header for access token (apache#7803)

86e1950

Add a new auth header to the rpc header proto for access token support. This should support different access tokens within the same connection. Contributed-by: Tom McCormick <[email protected]>

fix cherry pick

e5f42e5

Trigger CI rerun

1915ee0

Trigger CI rerun

d76997a

mccormickt12 mentioned this pull request Aug 4, 2025

Attempt to get unit tests passing. Ignore failing tests and increase … #7854

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-19612 Rpc auth header #7844

HADOOP-19612 Rpc auth header #7844

mccormickt12 commented Jul 30, 2025 •

edited

Loading

Uh oh!

hadoop-yetus commented Jul 30, 2025

Uh oh!

hadoop-yetus commented Jul 31, 2025

Uh oh!

mccormickt12 commented Jul 31, 2025

Uh oh!

hadoop-yetus commented Aug 1, 2025

Uh oh!

mccormickt12 commented Aug 1, 2025

Uh oh!

hadoop-yetus commented Aug 2, 2025

Uh oh!

Uh oh!

HADOOP-19612 Rpc auth header #7844

Are you sure you want to change the base?

HADOOP-19612 Rpc auth header #7844

Conversation

mccormickt12 commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Jul 30, 2025

Uh oh!

hadoop-yetus commented Jul 31, 2025

Uh oh!

mccormickt12 commented Jul 31, 2025

Uh oh!

hadoop-yetus commented Aug 1, 2025

Uh oh!

mccormickt12 commented Aug 1, 2025

Uh oh!

hadoop-yetus commented Aug 2, 2025

Uh oh!

Uh oh!

mccormickt12 commented Jul 30, 2025 •

edited

Loading