-
Notifications
You must be signed in to change notification settings - Fork 312
Optimize IAST Vulnerability Detection #8885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize IAST Vulnerability Detection #8885
Conversation
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 43 metrics, 9 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (995.132 ms) : 0, 995132
Total [baseline] (10.57 s) : 0, 10569593
Agent [candidate] (995.173 ms) : 0, 995173
Total [candidate] (10.573 s) : 0, 10572916
section appsec
Agent [baseline] (1.173 s) : 0, 1173264
Total [baseline] (10.713 s) : 0, 10712828
Agent [candidate] (1.173 s) : 0, 1173131
Total [candidate] (10.733 s) : 0, 10732898
section iast
Agent [baseline] (1.13 s) : 0, 1129802
Total [baseline] (10.795 s) : 0, 10794530
Agent [candidate] (1.134 s) : 0, 1133843
Total [candidate] (10.908 s) : 0, 10908050
section profiling
Agent [baseline] (1.244 s) : 0, 1244379
Total [baseline] (10.955 s) : 0, 10955220
Agent [candidate] (1.248 s) : 0, 1247791
Total [candidate] (10.956 s) : 0, 10955585
gantt
title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.638 ms) : 0, 687638
BytebuddyAgent [candidate] (687.436 ms) : 0, 687436
GlobalTracer [baseline] (241.668 ms) : 0, 241668
GlobalTracer [candidate] (241.849 ms) : 0, 241849
AppSec [baseline] (30.213 ms) : 0, 30213
AppSec [candidate] (30.279 ms) : 0, 30279
Debugger [baseline] (6.025 ms) : 0, 6025
Debugger [candidate] (6.046 ms) : 0, 6046
Remote Config [baseline] (679.358 µs) : 0, 679
Remote Config [candidate] (685.568 µs) : 0, 686
Telemetry [baseline] (8.199 ms) : 0, 8199
Telemetry [candidate] (8.135 ms) : 0, 8135
section appsec
BytebuddyAgent [baseline] (709.926 ms) : 0, 709926
BytebuddyAgent [candidate] (709.232 ms) : 0, 709232
GlobalTracer [baseline] (235.335 ms) : 0, 235335
GlobalTracer [candidate] (234.881 ms) : 0, 234881
IAST [baseline] (22.018 ms) : 0, 22018
IAST [candidate] (23.16 ms) : 0, 23160
AppSec [baseline] (170.783 ms) : 0, 170783
AppSec [candidate] (170.634 ms) : 0, 170634
Debugger [baseline] (5.781 ms) : 0, 5781
Debugger [candidate] (5.774 ms) : 0, 5774
Remote Config [baseline] (600.004 µs) : 0, 600
Remote Config [candidate] (597.803 µs) : 0, 598
Telemetry [baseline] (8.06 ms) : 0, 8060
Telemetry [candidate] (8.105 ms) : 0, 8105
section iast
BytebuddyAgent [baseline] (806.542 ms) : 0, 806542
BytebuddyAgent [candidate] (808.312 ms) : 0, 808312
GlobalTracer [baseline] (232.536 ms) : 0, 232536
GlobalTracer [candidate] (233.028 ms) : 0, 233028
IAST [baseline] (28.743 ms) : 0, 28743
IAST [candidate] (29.366 ms) : 0, 29366
AppSec [baseline] (26.778 ms) : 0, 26778
AppSec [candidate] (27.873 ms) : 0, 27873
Debugger [baseline] (5.825 ms) : 0, 5825
Debugger [candidate] (5.892 ms) : 0, 5892
Remote Config [baseline] (599.117 µs) : 0, 599
Remote Config [candidate] (591.786 µs) : 0, 592
Telemetry [baseline] (7.954 ms) : 0, 7954
Telemetry [candidate] (8.074 ms) : 0, 8074
section profiling
BytebuddyAgent [baseline] (678.808 ms) : 0, 678808
BytebuddyAgent [candidate] (680.679 ms) : 0, 680679
GlobalTracer [baseline] (360.812 ms) : 0, 360812
GlobalTracer [candidate] (361.387 ms) : 0, 361387
AppSec [baseline] (32.196 ms) : 0, 32196
AppSec [candidate] (33.013 ms) : 0, 33013
Debugger [baseline] (10.648 ms) : 0, 10648
Debugger [candidate] (9.076 ms) : 0, 9076
Remote Config [baseline] (1.353 ms) : 0, 1353
Remote Config [candidate] (664.974 µs) : 0, 665
Telemetry [baseline] (8.025 ms) : 0, 8025
Telemetry [candidate] (8.789 ms) : 0, 8789
ProfilingAgent [baseline] (103.377 ms) : 0, 103377
ProfilingAgent [candidate] (103.902 ms) : 0, 103902
Profiling [baseline] (103.401 ms) : 0, 103401
Profiling [candidate] (103.927 ms) : 0, 103927
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (994.682 ms) : 0, 994682
Total [baseline] (8.566 s) : 0, 8566186
Agent [candidate] (996.759 ms) : 0, 996759
Total [candidate] (8.54 s) : 0, 8539558
section iast
Agent [baseline] (1.14 s) : 0, 1139702
Total [baseline] (9.277 s) : 0, 9277466
Agent [candidate] (1.144 s) : 0, 1143997
Total [candidate] (9.323 s) : 0, 9323280
gantt
title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.471 ms) : 0, 687471
BytebuddyAgent [candidate] (688.605 ms) : 0, 688605
GlobalTracer [baseline] (241.352 ms) : 0, 241352
GlobalTracer [candidate] (242.168 ms) : 0, 242168
AppSec [baseline] (30.242 ms) : 0, 30242
AppSec [candidate] (30.33 ms) : 0, 30330
Debugger [baseline] (6.03 ms) : 0, 6030
Debugger [candidate] (6.017 ms) : 0, 6017
Remote Config [baseline] (672.268 µs) : 0, 672
Remote Config [candidate] (682.224 µs) : 0, 682
Telemetry [baseline] (8.19 ms) : 0, 8190
Telemetry [candidate] (8.206 ms) : 0, 8206
section iast
BytebuddyAgent [baseline] (814.282 ms) : 0, 814282
BytebuddyAgent [candidate] (815.895 ms) : 0, 815895
GlobalTracer [baseline] (234.27 ms) : 0, 234270
GlobalTracer [candidate] (234.752 ms) : 0, 234752
IAST [baseline] (28.199 ms) : 0, 28199
IAST [candidate] (29.017 ms) : 0, 29017
AppSec [baseline] (27.722 ms) : 0, 27722
AppSec [candidate] (28.847 ms) : 0, 28847
Debugger [baseline] (5.83 ms) : 0, 5830
Debugger [candidate] (5.906 ms) : 0, 5906
Remote Config [baseline] (576.617 µs) : 0, 577
Remote Config [candidate] (584.193 µs) : 0, 584
Telemetry [baseline] (7.983 ms) : 0, 7983
Telemetry [candidate] (8.204 ms) : 0, 8204
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section baseline
no_agent (4.239 ms) : 4185, 4292
. : milestone, 4239,
iast (9.264 ms) : 9109, 9419
. : milestone, 9264,
iast_FULL (13.684 ms) : 13411, 13958
. : milestone, 13684,
iast_GLOBAL (10.214 ms) : 10032, 10396
. : milestone, 10214,
profiling (8.706 ms) : 8561, 8852
. : milestone, 8706,
tracing (8.028 ms) : 7909, 8146
. : milestone, 8028,
section candidate
no_agent (4.431 ms) : 4382, 4481
. : milestone, 4431,
iast (9.266 ms) : 9116, 9416
. : milestone, 9266,
iast_FULL (14.807 ms) : 14513, 15100
. : milestone, 14807,
iast_GLOBAL (10.38 ms) : 10198, 10562
. : milestone, 10380,
profiling (8.937 ms) : 8799, 9075
. : milestone, 8937,
tracing (7.546 ms) : 7440, 7652
. : milestone, 7546,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section baseline
no_agent (37.416 ms) : 37108, 37723
. : milestone, 37416,
appsec (48.292 ms) : 47849, 48734
. : milestone, 48292,
code_origins (44.777 ms) : 44394, 45159
. : milestone, 44777,
iast (43.814 ms) : 43438, 44190
. : milestone, 43814,
profiling (47.847 ms) : 47398, 48295
. : milestone, 47847,
tracing (44.81 ms) : 44436, 45183
. : milestone, 44810,
section candidate
no_agent (38.417 ms) : 38107, 38726
. : milestone, 38417,
appsec (49.229 ms) : 48786, 49671
. : milestone, 49229,
code_origins (44.818 ms) : 44440, 45196
. : milestone, 44818,
iast (44.904 ms) : 44515, 45294
. : milestone, 44904,
profiling (48.154 ms) : 47691, 48618
. : milestone, 48154,
tracing (43.225 ms) : 42856, 43594
. : milestone, 43225,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section baseline
no_agent (14.927 s) : 14927000, 14927000
. : milestone, 14927000,
appsec (14.819 s) : 14819000, 14819000
. : milestone, 14819000,
iast (18.513 s) : 18513000, 18513000
. : milestone, 18513000,
iast_GLOBAL (18.133 s) : 18133000, 18133000
. : milestone, 18133000,
profiling (15.322 s) : 15322000, 15322000
. : milestone, 15322000,
tracing (14.908 s) : 14908000, 14908000
. : milestone, 14908000,
section candidate
no_agent (15.635 s) : 15635000, 15635000
. : milestone, 15635000,
appsec (15.011 s) : 15011000, 15011000
. : milestone, 15011000,
iast (18.583 s) : 18583000, 18583000
. : milestone, 18583000,
iast_GLOBAL (17.844 s) : 17844000, 17844000
. : milestone, 17844000,
profiling (15.248 s) : 15248000, 15248000
. : milestone, 15248000,
tracing (14.941 s) : 14941000, 14941000
. : milestone, 14941000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~9cde392dc87, baseline=1.51.0-SNAPSHOT~1666056a38
dateFormat X
axisFormat %s
section baseline
no_agent (1.481 ms) : 1469, 1492
. : milestone, 1481,
appsec (2.419 ms) : 2370, 2469
. : milestone, 2419,
iast (2.21 ms) : 2147, 2273
. : milestone, 2210,
iast_GLOBAL (2.258 ms) : 2195, 2321
. : milestone, 2258,
profiling (2.065 ms) : 2014, 2117
. : milestone, 2065,
tracing (2.035 ms) : 1986, 2084
. : milestone, 2035,
section candidate
no_agent (1.482 ms) : 1471, 1494
. : milestone, 1482,
appsec (2.421 ms) : 2371, 2471
. : milestone, 2421,
iast (2.205 ms) : 2142, 2268
. : milestone, 2205,
iast_GLOBAL (2.255 ms) : 2192, 2318
. : milestone, 2255,
profiling (2.064 ms) : 2013, 2116
. : milestone, 2064,
tracing (2.033 ms) : 1984, 2082
. : milestone, 2033,
|
600341f
to
416dbae
Compare
…ad/OverheadController.java Co-authored-by: datadog-datadog-prod-us1[bot] <88084959+datadog-datadog-prod-us1[bot]@users.noreply.github.com>
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/sink/HttpResponseHeaderModuleImpl.java
Outdated
Show resolved
Hide resolved
...s/springboot/src/test/groovy/datadog/smoketest/IastOverheadControlSpringBootSmokeTest.groovy
Show resolved
Hide resolved
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/IastGlobalContext.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadContext.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadContext.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/sink/HttpResponseHeaderModuleImpl.java
Outdated
Show resolved
Hide resolved
...s/springboot/src/test/groovy/datadog/smoketest/IastOverheadControlSpringBootSmokeTest.groovy
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadContext.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadContext.java
Outdated
Show resolved
Hide resolved
import org.springframework.web.bind.annotation.RestController; | ||
|
||
@RestController | ||
public class IastSamplingController { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we introduce a test with different code paths that can be controlled though query/path parameters?, the order in which we trigger vulns is always deterministic and it would be nice to also test with some variability.
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/sink/HttpResponseHeaderModuleImpl.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
dd-java-agent/agent-iast/src/main/java/com/datadog/iast/overhead/OverheadController.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like an extra reviewer on this.
On the startup time regression: I think we can accept the 1ms startup regression for IAST in pet clinic. It does not affect scenarios other than DD_IAST_ENABLED=true
, and the improved detection rate here is worth it.
] | ||
} | ||
|
||
void 'test'() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you improve the name of the test?
hasVulnerability { vul -> vul.type == 'WEAK_HASH' && vul.location.method == 'differentVulns' && vul.evidence.value == 'RIPEMD128'} | ||
|
||
//TODO the current algorithm is not able to detect all the vulnerabilities in the same endpoint if those vulnerabilities are not present in all requests. We need to improve it. | ||
//hasVulnerability { vul -> vul.type == 'WEAK_HASH' && vul.location.method == 'differentVulns' && vul.evidence.value == 'MD2'} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was this discussed in the working group?, I did wonder if the same behavior happens in other tracers or its just our implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, there is a discussion in the working group slack channel and also was mentioned in the last meeting. It should happen in al the tracer as is an algorithm issue not an implementation one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but we have to discuss in the working group the issue about the algorithm degrading when the codepaths can change.
| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.errorprone:error_prone_annotations](https://errorprone.info) ([source](https://github.com/google/error-prone)) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.39.0` -> `2.40.0` | | [org.apache.commons:commons-lang3](https://commons.apache.org/proper/commons-lang/) ([source](https://gitbox.apache.org/repos/asf/commons-lang.git)) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `3.17.0` -> `3.18.0` | | [org.jetbrains.kotlinx.binary-compatibility-validator](https://github.com/Kotlin/binary-compatibility-validator) | plugin | misk/gradle/libs.versions.toml | gradle | patch | `0.18.0` -> `0.18.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.50.1` -> `1.51.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | --- ### Release Notes <details> <summary>google/error-prone (com.google.errorprone:error_prone_annotations)</summary> ### [`v2.40.0`](https://github.com/google/error-prone/releases/tag/v2.40.0): Error Prone 2.40.0 Changes: - Bug fixes and improvements - Releases (including snapshots) have migrated from [OSSRH to the Central Publisher Portal](https://central.sonatype.org/pages/ossrh-eol/#process-to-migrate) Full changelog: google/error-prone@v2.39.0...v2.40.0 </details> <details> <summary>Kotlin/binary-compatibility-validator (org.jetbrains.kotlinx.binary-compatibility-validator)</summary> ### [`v0.18.1`](https://github.com/Kotlin/binary-compatibility-validator/releases/tag/0.18.1) [Compare Source](Kotlin/binary-compatibility-validator@0.18.0...0.18.1) #### What's Changed - Fixed a bug preventing use of cross-compilation support during KLIB dump validation \[[#​304](https://github.com/Kotlin/binary-compatibility-validator/issues/304)]\[[#​306](https://github.com/Kotlin/binary-compatibility-validator/issues/306)] </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.51.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.51.0): 1.51.0 ### Components #### Application Security Management (IAST) - 🐛 Fix verify error when ctor params are used after a call site ([#​9083](DataDog/dd-trace-java#9083) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Limit the maximum size of the location path in IAST vulnerabilities ([#​9028](DataDog/dd-trace-java#9028) - [@​jandro996](https://github.com/jandro996)) - 🐛 Fix IAST gRPC handler with null superclass ([#​8984](DataDog/dd-trace-java#8984) - [@​smola](https://github.com/smola)) - ✨ Optimize IAST Vulnerability Detection ([#​8885](DataDog/dd-trace-java#8885) - [@​jandro996](https://github.com/jandro996)) #### Application Security Management (WAF) - ✨ Upgrade libddwaf-java to 15.0.0 ([#​9022](DataDog/dd-trace-java#9022) - [@​sezen-datadog](https://github.com/sezen-datadog)) - ✨ Extract RestEasy json body response schemas ([#​9015](DataDog/dd-trace-java#9015) - [@​jandro996](https://github.com/jandro996)) - ✨ Extract Jersey json body response schemas ([#​9014](DataDog/dd-trace-java#9014) - [@​jandro996](https://github.com/jandro996)) - ✨ Extract Ratpack json body response schemas ([#​9013](DataDog/dd-trace-java#9013) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Enable API Security by default and make it lazy loading ([#​9009](DataDog/dd-trace-java#9009) - [@​smola](https://github.com/smola)) - ✨ Extract Vert.x json body response schemas ([#​9001](DataDog/dd-trace-java#9001) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Extract Play json body response schemas ([#​8995](DataDog/dd-trace-java#8995) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Fix Jackson nodes introspection for request/response schema extraction ([#​8980](DataDog/dd-trace-java#8980) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Extract Spring json body response schemas ([#​8938](DataDog/dd-trace-java#8938) - [@​sezen-datadog](https://github.com/sezen-datadog)) - ✨ Default obfuscation regexp update ([#​8937](DataDog/dd-trace-java#8937) - [@​sezen-datadog](https://github.com/sezen-datadog)) #### Build & Tooling - ✨ Cancel GitLab running pipeline on new PR push ([#​9023](DataDog/dd-trace-java#9023) - [@​PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Migrate publishing to Maven Central Portal ([#​8807](DataDog/dd-trace-java#8807) - [@​sarahchen6](https://github.com/sarahchen6)) #### Continuous Integration Visibility - 🐛 Fix Test Optimization to work with JDK 24 ([#​9114](DataDog/dd-trace-java#9114) - [@​nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add repo root as safe directory on git client creation ([#​9033](DataDog/dd-trace-java#9033) - [@​daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add PR number tag and improve PR information building ([#​8990](DataDog/dd-trace-java#8990) - [@​daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Update impacted tests logic ([#​8923](DataDog/dd-trace-java#8923) - [@​daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - 🧹 Clean up DSM context injection ([#​8776](DataDog/dd-trace-java#8776) - [@​PerfectSlayer](https://github.com/PerfectSlayer)) #### Database Monitoring - 🐛 Set trace\_injected in try block ([#​9025](DataDog/dd-trace-java#9025) - [@​natashadada](https://github.com/natashadada)) #### Dynamic Instrumentation - 🐛 Add source file tracking enable option ([#​9115](DataDog/dd-trace-java#9115) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Add java.util.Date support ([#​9111](DataDog/dd-trace-java#9111) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Update file probe format ([#​9047](DataDog/dd-trace-java#9047) - [@​jpbempel](https://github.com/jpbempel)) - ✨ add safe local var hoisting ([#​9034](DataDog/dd-trace-java#9034) - [@​jpbempel](https://github.com/jpbempel)) - 🧹 Add new config for debugger upload interval ([#​8959](DataDog/dd-trace-java#8959) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Enable Code Origin with Dynamic instrumentation ([#​8940](DataDog/dd-trace-java#8940) - [@​jpbempel](https://github.com/jpbempel)) #### ML Observability (LLMObs) - 💡 LLM Observability SDK ([#​8781](DataDog/dd-trace-java#8781) - [@​gary-huang](https://github.com/gary-huang), [@​nayeem-kamal](https://github.com/nayeem-kamal)) #### Metrics - 🐛 Ensure client stat reporter is started when the agent is not available at bootstrap ([#​9082](DataDog/dd-trace-java#9082) - [@​amarziali](https://github.com/amarziali)) - ✨ Create metric: appsec.waf.config\_errors ([#​8394](DataDog/dd-trace-java#8394) - [@​sezen-datadog](https://github.com/sezen-datadog)) #### Platform components - ✨ Introduce environment component ([#​9071](DataDog/dd-trace-java#9071) - [@​PerfectSlayer](https://github.com/PerfectSlayer)) #### Profiling - 🐛 Remove annoying warning for smap event parsing ([#​9119](DataDog/dd-trace-java#9119) - [@​jbachorik](https://github.com/jbachorik)) - 🐛 Fix ByteCountingInputStream when reading past EOF ([#​8988](DataDog/dd-trace-java#8988) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Realtime User Monitoring - ✨ Add RUM SDK injection for servlet based web servers ([#​9110](DataDog/dd-trace-java#9110) - [@​PerfectSlayer](https://github.com/PerfectSlayer) [@​amarziali](https://github.com/amarziali)) #### Telemetry - ✨ Update the config origin metric to match what it's mapping ([#​9045](DataDog/dd-trace-java#9045) - [@​sezen-datadog](https://github.com/sezen-datadog)) #### Testing - ✨ Add testing for latest stable version (JDK 24) ([#​8875](DataDog/dd-trace-java#8875) - [@​sarahchen6](https://github.com/sarahchen6)) #### Trace context propagation - 🐛 Fix bug with dropping baggage when `TracePropagationBehaviorExtract=IGNORE` ([#​9037](DataDog/dd-trace-java#9037) - [@​mhlidd](https://github.com/mhlidd)) - 🐛 Fix ArrayIndexOutOfBoundsException in PercentEscaper ([#​9032](DataDog/dd-trace-java#9032) - [@​mhlidd](https://github.com/mhlidd)) #### Tracer core - 🐛 Fix `Error` handling for trace interceptors ([#​9097](DataDog/dd-trace-java#9097) - [@​AlexeyKuznetsov-DD](https://github.com/AlexeyKuznetsov-DD)) - 💡 Add wildcard feature for `DD_TRACE_HEADER_TAGS` and enabling for Http Response headers ([#​9067](DataDog/dd-trace-java#9067) - [@​mhlidd](https://github.com/mhlidd)) #### Tracer public API - 💡 Add LLM Observability SDK ([#​8781](DataDog/dd-trace-java#8781) - [@​gary-huang](https://github.com/gary-huang)) ### Instrumentations #### Akka instrumentation - 🐛 Fix NPE in akka-http and pekko-http integrations ([#​9019](DataDog/dd-trace-java#9019) - [@​mcculls](https://github.com/mcculls)) #### Eclipse Vert.x instrumentation - ✨ Extract Vert.x json body response schemas ([#​9001](DataDog/dd-trace-java#9001) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Write http.route tag as soon as possible in vert.x ([#​8952](DataDog/dd-trace-java#8952) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### JAX-WS instrumentation - 💡⚠️ Enable jax-ws integration by default ([#​9030](DataDog/dd-trace-java#9030) - [@​bm1549](https://github.com/bm1549)) - ✨ Extract Jersey json body response schemas ([#​9014](DataDog/dd-trace-java#9014) - [@​jandro996](https://github.com/jandro996)) #### Mule instrumentation - 🐛 Propagate grizzly http span in filters if nothing is active ([#​9016](DataDog/dd-trace-java#9016) - [@​amarziali](https://github.com/amarziali)) #### Play Framework instrumentation - ✨ Extract Play json body response schemas ([#​8995](DataDog/dd-trace-java#8995) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Ratpack instrumentation - ✨ Extract Ratpack json body response schemas ([#​9013](DataDog/dd-trace-java#9013) - [@​manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Spring instrumentation - ✨ Extract Spring json body response schemas ([#​8938](DataDog/dd-trace-java#8938) - [@​sezen-datadog](https://github.com/sezen-datadog)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 649b690d4c9d7dcb572c457f0802b42b8e3e682e
What Does This Do Implements the new algorithm for detecting IAST vulnerabilities, where vulnerabilities that were already explored in previous runs for a given endpoint are skipped, ensuring that all remaining ones are eventually explored. This addresses the current limitation where only the first matching vulnerabilities are consistently reported, causing others to remain hidden. Changes to OverheadContext The OverheadContext class has been extended to support three separate tracking maps: globalMap Used to track vulnerability detection counts per endpoint across all requests. Keys are strings combining the request method and route (GET /login, POST /submit, etc.). Values are maps from vulnerabilityType → int (count of occurrences). Capped at 4,096 entries using a clear‐on‐overflow strategy, to ensure bounded memory usage. Oldest entries are cleared once the limit is reached. copyMap Created per request to copy the global counts at the start of the request, ensuring a consistent baseline to compare against throughout the lifecycle of the request. requestMap Tracks vulnerability type counts within the request. An additional field, isGlobal, has been added to indicate whether the context is global or request-scoped. If isGlobal is true, the maps are not used, and quota checks proceed using the global strategy only. A new method, resetMaps(), has been added to update globalMap when the request ends and vulnerability data has been reported. Two scenarios are supported: Case 1: Budget not fully used → The entry for the endpoint in globalMap is cleared, since the request stayed within budget. Case 2: Budget fully used → The counts from requestMap are compared to those in copyMap. For each vulnerability type, if the value in requestMap is greater, it is used to update the corresponding entry in globalMap. Changes to OverheadController The method consumeQuota() has been extended to receive a vulnerabilityType and modified to support the new logic: If an OverheadContext is present and not global, and there is remaining quota and a valid span, the controller now invokes a new method maybeSkipVulnerability() to determine whether quota should actually be consumed or not, based on endpoint-specific history. It's better to check the Algorithm execution example flow diagram to understand how this should work Changes to IastRequestContext In releaseRequestContext(), the request now calls resetMaps() on the associated OverheadContext, ensuring globalMap is updated at the end of each request. Motivation [RFC-1029] Optimizing IAST Vulnerability Detection implementation Additional Notes java tracer needs to implement also [RFC-1029-A1] Solution for dynamic http routes (cherry picked from commit 99ecab7)
What Does This Do
Implements the new algorithm for detecting IAST vulnerabilities, where vulnerabilities that were already explored in previous runs for a given endpoint are skipped, ensuring that all remaining ones are eventually explored.
This addresses the current limitation where only the first matching vulnerabilities are consistently reported, causing others to remain hidden.
Changes to OverheadContext
The OverheadContext class has been extended to support three separate tracking maps:
An additional field, isGlobal, has been added to indicate whether the context is global or request-scoped. If isGlobal is true, the maps are not used, and quota checks proceed using the global strategy only.
A new method, resetMaps(), has been added to update globalMap when the request ends and vulnerability data has been reported. Two scenarios are supported:
Changes to OverheadController
The method consumeQuota() has been extended to receive a vulnerabilityType and modified to support the new logic:
If an OverheadContext is present and not global, and there is remaining quota and a valid span, the controller now invokes a new method maybeSkipVulnerability() to determine whether quota should actually be consumed or not, based on endpoint-specific history.
It's better to check the Algorithm execution example flow diagram to understand how this should work
Changes to IastRequestContext
In releaseRequestContext(), the request now calls resetMaps() on the associated OverheadContext, ensuring globalMap is updated at the end of each request.
Motivation
[RFC-1029] Optimizing IAST Vulnerability Detection implementation
Additional Notes
java tracer needs to implement also [RFC-1029-A1] Solution for dynamic http routes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: APPSEC-57267