Support re-registration of different ContextManagers/Binders during testing #8793

mcculls · 2025-05-10T19:19:42Z

Motivation

In production we lock the registered ContextManager and ContextBinder in a static final variable the first time they are used, via the "initialization-on-demand" pattern. This is to maintain performance because almost every context operation will either use the manager or binder.

However when testing we would like to re-register the ContextManager or ContextBinder, like we can forcibly re-register the tracer instance. This PR adds support for this during testing by using an intermediate provider that always delegates to the latest registered ContextManager or ContextBinder.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-05-10T19:54:10Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/support-context-provider-reregistration-during-tests
git_commit_date	1747050043	1747052025
git_commit_sha	`8bfbf8e`	`6c98b46`
release_version	1.50.0-SNAPSHOT~8bfbf8e6ed	1.50.0-SNAPSHOT~6c98b46774

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747054556	1747054556
ci_job_id	933272610	933272610
ci_pipeline_id	64773795	64773795
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-d33xtjb-project-304-concurrent-1-52in2j8d 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-d33xtjb-project-304-concurrent-1-52in2j8d 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.018 s) : 0, 1018338
Total [baseline] (10.448 s) : 0, 10447591
Agent [candidate] (1.036 s) : 0, 1036412
Total [candidate] (10.474 s) : 0, 10473835
section appsec
Agent [baseline] (1.164 s) : 0, 1164150
Total [baseline] (10.669 s) : 0, 10668622
Agent [candidate] (1.164 s) : 0, 1164329
Total [candidate] (10.695 s) : 0, 10695336
section iast
Agent [baseline] (1.148 s) : 0, 1147875
Total [baseline] (10.857 s) : 0, 10856935
Agent [candidate] (1.153 s) : 0, 1152909
Total [candidate] (10.846 s) : 0, 10845934
section profiling
Agent [baseline] (1.28 s) : 0, 1280164
Total [baseline] (10.95 s) : 0, 10950193
Agent [candidate] (1.288 s) : 0, 1287565
Total [candidate] (10.865 s) : 0, 10864809

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.018 s	-
Agent	appsec	1.164 s	145.812 ms (14.3%)
Agent	iast	1.148 s	129.537 ms (12.7%)
Agent	profiling	1.28 s	261.826 ms (25.7%)
Total	tracing	10.448 s	-
Total	appsec	10.669 s	221.032 ms (2.1%)
Total	iast	10.857 s	409.345 ms (3.9%)
Total	profiling	10.95 s	502.602 ms (4.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	appsec	1.164 s	127.918 ms (12.3%)
Agent	iast	1.153 s	116.498 ms (11.2%)
Agent	profiling	1.288 s	251.154 ms (24.2%)
Total	tracing	10.474 s	-
Total	appsec	10.695 s	221.501 ms (2.1%)
Total	iast	10.846 s	372.099 ms (3.6%)
Total	profiling	10.865 s	390.974 ms (3.7%)

gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.649 ms) : 0, 681649
BytebuddyAgent [candidate] (692.75 ms) : 0, 692750
GlobalTracer [baseline] (239.325 ms) : 0, 239325
GlobalTracer [candidate] (243.214 ms) : 0, 243214
AppSec [baseline] (54.416 ms) : 0, 54416
AppSec [candidate] (55.202 ms) : 0, 55202
Debugger [baseline] (9.714 ms) : 0, 9714
Debugger [candidate] (9.239 ms) : 0, 9239
Remote Config [baseline] (688.078 µs) : 0, 688
Remote Config [candidate] (722.926 µs) : 0, 723
Telemetry [baseline] (9.03 ms) : 0, 9030
Telemetry [candidate] (11.447 ms) : 0, 11447
section appsec
BytebuddyAgent [baseline] (702.24 ms) : 0, 702240
BytebuddyAgent [candidate] (701.957 ms) : 0, 701957
GlobalTracer [baseline] (237.29 ms) : 0, 237290
GlobalTracer [candidate] (237.367 ms) : 0, 237367
AppSec [baseline] (176.321 ms) : 0, 176321
AppSec [candidate] (176.407 ms) : 0, 176407
Debugger [baseline] (5.944 ms) : 0, 5944
Debugger [candidate] (5.975 ms) : 0, 5975
Remote Config [baseline] (629.074 µs) : 0, 629
Remote Config [candidate] (623.906 µs) : 0, 624
Telemetry [baseline] (7.369 ms) : 0, 7369
Telemetry [candidate] (7.444 ms) : 0, 7444
IAST [baseline] (21.601 ms) : 0, 21601
IAST [candidate] (21.892 ms) : 0, 21892
section iast
BytebuddyAgent [baseline] (800.957 ms) : 0, 800957
BytebuddyAgent [candidate] (805.428 ms) : 0, 805428
GlobalTracer [baseline] (230.47 ms) : 0, 230470
GlobalTracer [candidate] (231.077 ms) : 0, 231077
AppSec [baseline] (49.365 ms) : 0, 49365
AppSec [candidate] (48.603 ms) : 0, 48603
Debugger [baseline] (5.942 ms) : 0, 5942
Debugger [candidate] (5.935 ms) : 0, 5935
Remote Config [baseline] (595.931 µs) : 0, 596
Remote Config [candidate] (592.484 µs) : 0, 592
Telemetry [baseline] (7.942 ms) : 0, 7942
Telemetry [candidate] (7.866 ms) : 0, 7866
IAST [baseline] (29.15 ms) : 0, 29150
IAST [candidate] (29.88 ms) : 0, 29880
section profiling
ProfilingAgent [baseline] (110.931 ms) : 0, 110931
ProfilingAgent [candidate] (111.016 ms) : 0, 111016
BytebuddyAgent [baseline] (670.899 ms) : 0, 670899
BytebuddyAgent [candidate] (676.535 ms) : 0, 676535
GlobalTracer [baseline] (377.859 ms) : 0, 377859
GlobalTracer [candidate] (379.758 ms) : 0, 379758
AppSec [baseline] (54.91 ms) : 0, 54910
AppSec [candidate] (54.338 ms) : 0, 54338
Debugger [baseline] (6.2 ms) : 0, 6200
Debugger [candidate] (6.193 ms) : 0, 6193
Remote Config [baseline] (655.872 µs) : 0, 656
Remote Config [candidate] (659.084 µs) : 0, 659
Telemetry [baseline] (8.232 ms) : 0, 8232
Telemetry [candidate] (8.239 ms) : 0, 8239
Profiling [baseline] (110.957 ms) : 0, 110957
Profiling [candidate] (111.043 ms) : 0, 111043

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1028757
Total [baseline] (8.68 s) : 0, 8679882
Agent [candidate] (1.019 s) : 0, 1018781
Total [candidate] (8.637 s) : 0, 8636604
section iast
Agent [baseline] (1.148 s) : 0, 1147842
Total [baseline] (9.247 s) : 0, 9247460
Agent [candidate] (1.149 s) : 0, 1149494
Total [candidate] (9.217 s) : 0, 9216547
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.147 s) : 0, 1146929
Total [baseline] (9.182 s) : 0, 9181654
Agent [candidate] (1.144 s) : 0, 1143530
Total [candidate] (9.236 s) : 0, 9236216
section iast_TELEMETRY_OFF
Agent [baseline] (1.148 s) : 0, 1148320
Total [baseline] (9.21 s) : 0, 9210333
Agent [candidate] (1.145 s) : 0, 1145026
Total [candidate] (9.256 s) : 0, 9255672

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.029 s	-
Agent	iast	1.148 s	119.085 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.147 s	118.172 ms (11.5%)
Agent	iast_TELEMETRY_OFF	1.148 s	119.563 ms (11.6%)
Total	tracing	8.68 s	-
Total	iast	9.247 s	567.578 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.182 s	501.772 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.21 s	530.451 ms (6.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.019 s	-
Agent	iast	1.149 s	130.713 ms (12.8%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.144 s	124.749 ms (12.2%)
Agent	iast_TELEMETRY_OFF	1.145 s	126.245 ms (12.4%)
Total	tracing	8.637 s	-
Total	iast	9.217 s	579.943 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.236 s	599.611 ms (6.9%)
Total	iast_TELEMETRY_OFF	9.256 s	619.068 ms (7.2%)

gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.649 ms) : 0, 687649
BytebuddyAgent [candidate] (681.236 ms) : 0, 681236
GlobalTracer [baseline] (241.819 ms) : 0, 241819
GlobalTracer [candidate] (239.801 ms) : 0, 239801
AppSec [baseline] (56.278 ms) : 0, 56278
AppSec [candidate] (54.532 ms) : 0, 54532
Debugger [baseline] (8.693 ms) : 0, 8693
Debugger [candidate] (7.603 ms) : 0, 7603
Remote Config [baseline] (766.347 µs) : 0, 766
Remote Config [candidate] (709.55 µs) : 0, 710
Telemetry [baseline] (9.883 ms) : 0, 9883
Telemetry [candidate] (11.42 ms) : 0, 11420
section iast
BytebuddyAgent [baseline] (801.507 ms) : 0, 801507
BytebuddyAgent [candidate] (802.23 ms) : 0, 802230
GlobalTracer [baseline] (230.032 ms) : 0, 230032
GlobalTracer [candidate] (230.583 ms) : 0, 230583
IAST [baseline] (28.277 ms) : 0, 28277
IAST [candidate] (27.617 ms) : 0, 27617
AppSec [baseline] (48.716 ms) : 0, 48716
AppSec [candidate] (50.301 ms) : 0, 50301
Debugger [baseline] (5.88 ms) : 0, 5880
Debugger [candidate] (5.937 ms) : 0, 5937
Remote Config [baseline] (598.828 µs) : 0, 599
Remote Config [candidate] (597.374 µs) : 0, 597
Telemetry [baseline] (7.908 ms) : 0, 7908
Telemetry [candidate] (7.918 ms) : 0, 7918
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.347 ms) : 0, 800347
BytebuddyAgent [candidate] (797.885 ms) : 0, 797885
GlobalTracer [baseline] (230.258 ms) : 0, 230258
GlobalTracer [candidate] (229.699 ms) : 0, 229699
IAST [baseline] (30.031 ms) : 0, 30031
IAST [candidate] (29.85 ms) : 0, 29850
AppSec [baseline] (48.477 ms) : 0, 48477
AppSec [candidate] (48.429 ms) : 0, 48429
Debugger [baseline] (5.931 ms) : 0, 5931
Debugger [candidate] (5.814 ms) : 0, 5814
Remote Config [baseline] (593.114 µs) : 0, 593
Remote Config [candidate] (572.656 µs) : 0, 573
Telemetry [baseline] (7.865 ms) : 0, 7865
Telemetry [candidate] (7.893 ms) : 0, 7893
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (801.805 ms) : 0, 801805
BytebuddyAgent [candidate] (798.159 ms) : 0, 798159
GlobalTracer [baseline] (231.017 ms) : 0, 231017
GlobalTracer [candidate] (230.575 ms) : 0, 230575
IAST [baseline] (22.278 ms) : 0, 22278
IAST [candidate] (23.057 ms) : 0, 23057
AppSec [baseline] (55.469 ms) : 0, 55469
AppSec [candidate] (55.382 ms) : 0, 55382
Debugger [baseline] (5.922 ms) : 0, 5922
Debugger [candidate] (5.979 ms) : 0, 5979
Remote Config [baseline] (601.734 µs) : 0, 602
Remote Config [candidate] (602.314 µs) : 0, 602
Telemetry [baseline] (7.696 ms) : 0, 7696
Telemetry [candidate] (7.79 ms) : 0, 7790

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-05-12T12:25:51	2025-05-12T12:33:36
git_branch	master	mcculls/support-context-provider-reregistration-during-tests
git_commit_date	1747050043	1747052025
git_commit_sha	`8bfbf8e`	`6c98b46`
release_version	1.50.0-SNAPSHOT~8bfbf8e6ed	1.50.0-SNAPSHOT~6c98b46774
start_time	2025-05-12T12:25:37	2025-05-12T12:33:21

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747053615	1747053615
ci_job_id	933272611	933272611
ci_pipeline_id	64773795	64773795
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-nudahejj-project-304-concurrent-0-ed8zarb1 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-nudahejj-project-304-concurrent-0-ed8zarb1 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.18 µs) : 366, 405
.   : milestone, 385,
iast (515.661 µs) : 494, 537
.   : milestone, 516,
iast_FULL (738.205 µs) : 715, 762
.   : milestone, 738,
iast_GLOBAL (563.035 µs) : 541, 585
.   : milestone, 563,
iast_HARDCODED_SECRET_DISABLED (529.958 µs) : 507, 553
.   : milestone, 530,
iast_INACTIVE (466.508 µs) : 444, 489
.   : milestone, 467,
iast_TELEMETRY_OFF (513.496 µs) : 490, 537
.   : milestone, 513,
tracing (463.52 µs) : 441, 486
.   : milestone, 464,
section candidate
no_agent (381.29 µs) : 361, 401
.   : milestone, 381,
iast (523.343 µs) : 500, 547
.   : milestone, 523,
iast_FULL (730.418 µs) : 708, 752
.   : milestone, 730,
iast_GLOBAL (563.999 µs) : 542, 586
.   : milestone, 564,
iast_HARDCODED_SECRET_DISABLED (513.07 µs) : 490, 536
.   : milestone, 513,
iast_INACTIVE (471.677 µs) : 449, 495
.   : milestone, 472,
iast_TELEMETRY_OFF (513.596 µs) : 490, 537
.   : milestone, 514,
tracing (465.322 µs) : 443, 488
.   : milestone, 465,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.18 µs [365.592 µs, 404.769 µs]	-
iast	515.661 µs [493.829 µs, 537.493 µs]	130.481 µs (33.9%)
iast_FULL	738.205 µs [714.745 µs, 761.665 µs]	353.025 µs (91.7%)
iast_GLOBAL	563.035 µs [540.941 µs, 585.13 µs]	177.855 µs (46.2%)
iast_HARDCODED_SECRET_DISABLED	529.958 µs [507.172 µs, 552.745 µs]	144.778 µs (37.6%)
iast_INACTIVE	466.508 µs [444.035 µs, 488.981 µs]	81.328 µs (21.1%)
iast_TELEMETRY_OFF	513.496 µs [490.226 µs, 536.766 µs]	128.316 µs (33.3%)
tracing	463.52 µs [441.043 µs, 485.998 µs]	78.34 µs (20.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.29 µs [361.333 µs, 401.247 µs]	-
iast	523.343 µs [500.182 µs, 546.505 µs]	142.054 µs (37.3%)
iast_FULL	730.418 µs [708.344 µs, 752.492 µs]	349.128 µs (91.6%)
iast_GLOBAL	563.999 µs [541.75 µs, 586.247 µs]	182.709 µs (47.9%)
iast_HARDCODED_SECRET_DISABLED	513.07 µs [490.055 µs, 536.085 µs]	131.78 µs (34.6%)
iast_INACTIVE	471.677 µs [448.602 µs, 494.751 µs]	90.387 µs (23.7%)
iast_TELEMETRY_OFF	513.596 µs [489.972 µs, 537.219 µs]	132.306 µs (34.7%)
tracing	465.322 µs [442.597 µs, 488.047 µs]	84.032 µs (22.0%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1340, 1381
.   : milestone, 1360,
appsec (1.747 ms) : 1724, 1770
.   : milestone, 1747,
appsec_no_iast (1.743 ms) : 1720, 1766
.   : milestone, 1743,
code_origins (1.683 ms) : 1657, 1709
.   : milestone, 1683,
iast (1.51 ms) : 1486, 1535
.   : milestone, 1510,
profiling (1.507 ms) : 1483, 1530
.   : milestone, 1507,
tracing (1.503 ms) : 1478, 1528
.   : milestone, 1503,
section candidate
no_agent (1.387 ms) : 1368, 1407
.   : milestone, 1387,
appsec (1.731 ms) : 1707, 1755
.   : milestone, 1731,
appsec_no_iast (1.726 ms) : 1703, 1749
.   : milestone, 1726,
code_origins (1.678 ms) : 1651, 1706
.   : milestone, 1678,
iast (1.519 ms) : 1493, 1544
.   : milestone, 1519,
profiling (1.559 ms) : 1534, 1584
.   : milestone, 1559,
tracing (1.495 ms) : 1470, 1520
.   : milestone, 1495,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.36 ms [1.34 ms, 1.381 ms]	-
appsec	1.747 ms [1.724 ms, 1.77 ms]	386.825 µs (28.4%)
appsec_no_iast	1.743 ms [1.72 ms, 1.766 ms]	383.206 µs (28.2%)
code_origins	1.683 ms [1.657 ms, 1.709 ms]	322.862 µs (23.7%)
iast	1.51 ms [1.486 ms, 1.535 ms]	150.309 µs (11.1%)
profiling	1.507 ms [1.483 ms, 1.53 ms]	146.587 µs (10.8%)
tracing	1.503 ms [1.478 ms, 1.528 ms]	142.817 µs (10.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.387 ms [1.368 ms, 1.407 ms]	-
appsec	1.731 ms [1.707 ms, 1.755 ms]	344.31 µs (24.8%)
appsec_no_iast	1.726 ms [1.703 ms, 1.749 ms]	338.579 µs (24.4%)
code_origins	1.678 ms [1.651 ms, 1.706 ms]	291.108 µs (21.0%)
iast	1.519 ms [1.493 ms, 1.544 ms]	131.692 µs (9.5%)
profiling	1.559 ms [1.534 ms, 1.584 ms]	171.835 µs (12.4%)
tracing	1.495 ms [1.47 ms, 1.52 ms]	108.037 µs (7.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/support-context-provider-reregistration-during-tests
git_commit_date	1747050043	1747052025
git_commit_sha	`8bfbf8e`	`6c98b46`
release_version	1.50.0-SNAPSHOT~8bfbf8e6ed	1.50.0-SNAPSHOT~6c98b46774

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1747054176	1747054176
ci_job_id	933272612	933272612
ci_pipeline_id	64773795	64773795
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-d33xtjb-project-304-concurrent-2-u2pbxxco 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-d33xtjb-project-304-concurrent-2-u2pbxxco 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.381 s) : 15381000, 15381000
.   : milestone, 15381000,
appsec (14.944 s) : 14944000, 14944000
.   : milestone, 14944000,
iast (18.414 s) : 18414000, 18414000
.   : milestone, 18414000,
iast_GLOBAL (18.098 s) : 18098000, 18098000
.   : milestone, 18098000,
profiling (15.403 s) : 15403000, 15403000
.   : milestone, 15403000,
tracing (14.939 s) : 14939000, 14939000
.   : milestone, 14939000,
section candidate
no_agent (15.007 s) : 15007000, 15007000
.   : milestone, 15007000,
appsec (14.993 s) : 14993000, 14993000
.   : milestone, 14993000,
iast (18.85 s) : 18850000, 18850000
.   : milestone, 18850000,
iast_GLOBAL (18.192 s) : 18192000, 18192000
.   : milestone, 18192000,
profiling (15.047 s) : 15047000, 15047000
.   : milestone, 15047000,
tracing (15.315 s) : 15315000, 15315000
.   : milestone, 15315000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.381 s [15.381 s, 15.381 s]	-
appsec	14.944 s [14.944 s, 14.944 s]	-437.0 ms (-2.8%)
iast	18.414 s [18.414 s, 18.414 s]	3.033 s (19.7%)
iast_GLOBAL	18.098 s [18.098 s, 18.098 s]	2.717 s (17.7%)
profiling	15.403 s [15.403 s, 15.403 s]	22.0 ms (0.1%)
tracing	14.939 s [14.939 s, 14.939 s]	-442.0 ms (-2.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.007 s [15.007 s, 15.007 s]	-
appsec	14.993 s [14.993 s, 14.993 s]	-14.0 ms (-0.1%)
iast	18.85 s [18.85 s, 18.85 s]	3.843 s (25.6%)
iast_GLOBAL	18.192 s [18.192 s, 18.192 s]	3.185 s (21.2%)
profiling	15.047 s [15.047 s, 15.047 s]	40.0 ms (0.3%)
tracing	15.315 s [15.315 s, 15.315 s]	308.0 ms (2.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~6c98b46774, baseline=1.50.0-SNAPSHOT~8bfbf8e6ed
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.398 ms) : 2349, 2448
.   : milestone, 2398,
iast (2.173 ms) : 2111, 2235
.   : milestone, 2173,
iast_GLOBAL (2.222 ms) : 2160, 2284
.   : milestone, 2222,
profiling (2.029 ms) : 1980, 2079
.   : milestone, 2029,
tracing (2.015 ms) : 1967, 2063
.   : milestone, 2015,
section candidate
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.397 ms) : 2348, 2447
.   : milestone, 2397,
iast (2.178 ms) : 2117, 2240
.   : milestone, 2178,
iast_GLOBAL (2.231 ms) : 2168, 2293
.   : milestone, 2231,
profiling (2.044 ms) : 1994, 2095
.   : milestone, 2044,
tracing (2.0 ms) : 1952, 2048
.   : milestone, 2000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.47 ms [1.459 ms, 1.482 ms]	-
appsec	2.398 ms [2.349 ms, 2.448 ms]	928.212 µs (63.1%)
iast	2.173 ms [2.111 ms, 2.235 ms]	702.939 µs (47.8%)
iast_GLOBAL	2.222 ms [2.16 ms, 2.284 ms]	751.682 µs (51.1%)
profiling	2.029 ms [1.98 ms, 2.079 ms]	559.032 µs (38.0%)
tracing	2.015 ms [1.967 ms, 2.063 ms]	545.018 µs (37.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.397 ms [2.348 ms, 2.447 ms]	921.935 µs (62.5%)
iast	2.178 ms [2.117 ms, 2.24 ms]	702.866 µs (47.6%)
iast_GLOBAL	2.231 ms [2.168 ms, 2.293 ms]	755.288 µs (51.2%)
profiling	2.044 ms [1.994 ms, 2.095 ms]	568.661 µs (38.5%)
tracing	2.0 ms [1.952 ms, 2.048 ms]	524.161 µs (35.5%)

PerfectSlayer

Something feels weird with those changes.
I get the intention of the PR, but the Javadoc of the context API, or the contract it exposes seems wrong.

Like the register method from ContextManager and ContextBinder states that the custom manager will always be used and replace the current ones:

dd-trace-java/components/context/src/main/java/datadog/context/ContextManager.java

Lines 28 to 33 in fa13c1c

    
             /** 
        
              * Requests use of a custom {@link ContextManager}. 
        
              * 
        
              * @param manager the manager to use (will replace any other manager in use). 
        
              */ 
        
             static void register(ContextManager manager) {

That contradicts what the allowTesting is supposed to do (in addition of having testing related method in our contract isn't especially great but that's another topic)

components/context/src/test/java/datadog/context/ContextProvidersForkedTest.java

mcculls · 2025-05-12T09:07:54Z

Something feels weird with those changes.

Not really, I just need to update the javadoc of the register methods to reflect that the manager / binder will be locked after their first use and cannot then be changed, unless allowTesting is called first. That's the only change needed to make the documentation consistent.

mcculls · 2025-05-12T09:17:51Z

in addition of having testing related method in our contract isn't especially great but that's another topic

The other approach was to expose the Test classes, which IMHO is worse as you're exposing (test) implementation details.

We could move these methods to their own class, or give them an innocuous name like unlock..., but TBH you'll either end up making it harder for people to find them when testing (because it's not clear what to call and why) or you'll end up in the same situation as Config where you have to resort to reflection or instrumentation.

Fundamentally it makes sense to lock the manager and binder after use in this way, because you should not change them after they are used (could introduce lifecycle issues) and it is much more performant. We need to provide a way around this in testing - other parts of the codebase use force... methods and some use reflection.

I want to make clear this is meant for testing only, and I want to avoid people to have to go looking for it - since it relates to the use of register then that's the place it makes sense to put it next too.

Also a reminder that everyday users of the Context API will never even import ContextManager or ContextBinder, they will just use the Context class.

…ation-during-tests

datadog-datadog-prod-us1 · 2025-05-12T12:40:19Z

Datadog Summary

✅ Code Quality ✅ Code Security ✅ Dependencies

Was this helpful? Give us feedback!

mcculls added type: enhancement Enhancements and improvements comp: testing Testing tag: no release notes Changes to exclude from release notes labels May 10, 2025

mcculls force-pushed the mcculls/support-context-provider-reregistration-during-tests branch 3 times, most recently from d063967 to e0900cb Compare May 11, 2025 16:24

mcculls marked this pull request as ready for review May 11, 2025 18:22

mcculls requested review from a team as code owners May 11, 2025 18:22

mcculls requested review from dougqh, PerfectSlayer and mhlidd May 11, 2025 18:22

mcculls force-pushed the mcculls/support-context-provider-reregistration-during-tests branch from e0900cb to 2f375be Compare May 11, 2025 20:08

Support re-registration of different Context providers during testing

fa13c1c

mcculls force-pushed the mcculls/support-context-provider-reregistration-during-tests branch from 2f375be to fa13c1c Compare May 11, 2025 20:15

PerfectSlayer reviewed May 12, 2025

View reviewed changes

components/context/src/test/java/datadog/context/ContextProvidersForkedTest.java Show resolved Hide resolved

mcculls added 2 commits May 12, 2025 13:11

review feedback

a9c3408

Merge branch 'master' into mcculls/support-context-provider-reregistr…

6c98b46

…ation-during-tests

PerfectSlayer approved these changes May 12, 2025

View reviewed changes

mcculls merged commit 90ca83f into master May 12, 2025
590 of 593 checks passed

mcculls deleted the mcculls/support-context-provider-reregistration-during-tests branch May 12, 2025 14:00

github-actions bot added this to the 1.50.0 milestone May 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support re-registration of different ContextManagers/Binders during testing #8793

Support re-registration of different ContextManagers/Binders during testing #8793

Uh oh!

mcculls commented May 10, 2025

Uh oh!

pr-commenter bot commented May 10, 2025 •

edited

Loading

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

mcculls commented May 12, 2025

Uh oh!

mcculls commented May 12, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented May 12, 2025

Uh oh!

Uh oh!

Uh oh!

	/**
	* Requests use of a custom {@link ContextManager}.
	*
	* @param manager the manager to use (will replace any other manager in use).
	*/
	static void register(ContextManager manager) {

Support re-registration of different ContextManagers/Binders during testing #8793

Support re-registration of different ContextManagers/Binders during testing #8793

Uh oh!

Conversation

mcculls commented May 10, 2025

Motivation

Contributor Checklist

Uh oh!

pr-commenter bot commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcculls commented May 12, 2025

Uh oh!

mcculls commented May 12, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented May 12, 2025

Datadog Summary

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented May 10, 2025 •

edited

Loading