feat(fastify): add captureBody support #2681

xxzefgh · 2022-05-07T21:30:54Z

Closes: #1906
Closes: #3217

cla-checker-service · 2022-05-07T21:30:57Z

💚 CLA has been signed

xxzefgh · 2022-05-07T21:34:06Z

have not tested this yet, will reopen later

xxzefgh · 2022-05-08T13:43:09Z

Tested with [email protected], http.request.body.original field is populated with request body

apmmachine · 2022-05-08T13:47:31Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-08-08T19:32:33.500+0000
Duration: 39 min 26 sec

Test stats 🧪

Test	Results
Failed	0
Passed	258182
Skipped	0
Total	258182

Steps errors

Expand to view the steps failures

Show only the first 10 steps failures

`Run Tests`

Took 1 min 8 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "12"

`Run Tests`

Took 0 min 27 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "12"

`Archive the artifacts`

Took 0 min 1 sec . View more details here
Description: [2022-08-08T20:03:25.853Z] Archiving artifacts [2022-08-08T20:03:26.596Z] ‘test_output/*.tap’ d

`Run Tests`

Took 1 min 55 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "10"

`Run Tests`

Took 0 min 25 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "10"

`Archive the artifacts`

Took 0 min 1 sec . View more details here
Description: [2022-08-08T20:03:09.062Z] Archiving artifacts [2022-08-08T20:03:09.788Z] ‘test_output/*.tap’ d

`Run Tests`

Took 5 min 18 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "8"

`Run Tests`

Took 5 min 43 sec . View more details here
Description: .ci/scripts/test.sh -b "release" -t "fastify" "8"

`Archive the artifacts`

Took 0 min 1 sec . View more details here
Description: [2022-08-08T20:11:55.276Z] Archiving artifacts [2022-08-08T20:11:55.933Z] ‘test_output/*.tap’ d

`Error signal`

Took 0 min 0 sec . View more details here
Description: hudson.AbortException: script returned exit code 1

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
run module tests for <modules> : Run TAV tests for one or more modules, where <modules> can be either a comma separated list of modules (e.g. memcached,redis) or the string literal ALL to test all modules
run benchmark tests : Run the benchmark test only.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

trentm

@xxzefgh This looks great! Thanks for starting this.

Could you add a test for this to "test/instrumentation/modules/fastify/fastify.test.js"? captureBody support was recently added for hapi instrumentation. Perhaps you could mostly copy how the equivalent test was added for that PR: https://github.com/elastic/apm-agent-nodejs/pull/2618/files#diff-ce2dc92ded3295986f31e8f4178788ae10a651ec2badfda8b8d92fc1b2d2cd7bR3-R118 If you need help understanding how the test code works there, please ask. I'm happy to help.
There is also a commented out test in "test/sanitize-field-names/fastify.test.js" that could be added now that you've implemented this. You could apply this diff to enable it:

--- a/test/sanitize-field-names/fastify.test.js
+++ b/test/sanitize-field-names/fastify.test.js
@@ -3,6 +3,7 @@ const { createAgentConfig } = require('./_shared')
 const agent = require('../..').start(createAgentConfig())
 const {
   resetAgent,
+  assertFormsWithFixture,
   assertRequestHeadersWithFixture,
   assertResponseHeadersWithFixture
 } = require('./_shared')
@@ -27,9 +28,7 @@ function runTest (
     const transaction = data.transactions.pop()
     assertRequestHeadersWithFixture(transaction, expected, t)
     assertResponseHeadersWithFixture(transaction, expected, t)
-    // TODO: uncomment once we fix
-    // https://github.com/elastic/apm-agent-nodejs/issues/1906
-    // assertFormsWithFixture(transaction, expected, t)
+    assertFormsWithFixture(transaction, expected, t)
   })

   // register request handler

You could add Fixes: #1906 to your PR description to mark that this issue fixes the already open issue we had to add captureBody support for fastify.

trentm · 2022-05-09T17:08:33Z

run module tests for fastify

xxzefgh · 2022-05-10T20:43:45Z

@trentm thanks for the input, it will take me probably 1 or 2 weeks to make this a proper PR.

trentm · 2022-05-10T21:10:21Z

@xxzefgh Okay. If you need help, let me know. You should be able to run that one test file directly to test your changes:

node test/instrumentation/modules/fastify/fastify.test.js

xxzefgh · 2022-05-31T21:22:55Z

Hi @trentm, I've added tests as you requested. One issue is that tests are run against v2 (required in devDependencies), which was already supported. I did locally install v3 and it passed, but I'm not sure how tests can be run against multiple versions.

trentm · 2022-05-31T21:28:33Z

@xxzefgh Thanks!

One issue is that tests are run against v2 (required in devDependencies), which was already supported. I did locally install v3 and it passed, but I'm not sure how tests can be run against multiple versions.

We just recently updated devDeps to fastify@3 in #2738

However, we also use test-all-versions (TAV for short) to test all/most supported versions of modules that we instrument.

apm-agent-nodejs/.tav.yml

Lines 475 to 500 in 7e9b9e3

	# https://www.fastify.io/docs/latest/LTS/
	# - #1086 suggests [email protected] was a broken release, skip it.
	fastify-v1:
	name: fastify
	versions: '1.x'
	node: '>=6 <12'
	commands:
	- node test/instrumentation/modules/fastify/fastify.test.js
	- node test/instrumentation/modules/fastify/async-await.test.js
	- node test/instrumentation/modules/fastify/set-framework.test.js
	fastify-v2:
	name: fastify
	versions: '>=2.0.0 <2.4.0 \|\| >2.4.0 <3'
	node: '>=6 <15'
	commands:
	- node test/instrumentation/modules/fastify/fastify.test.js
	- node test/instrumentation/modules/fastify/async-await.test.js
	- node test/instrumentation/modules/fastify/set-framework.test.js
	fastify:
	name: fastify
	versions: '>=3.0.0'
	node: '>=10'
	commands:
	- node test/instrumentation/modules/fastify/fastify.test.js
	- node test/instrumentation/modules/fastify/async-await.test.js
	- node test/instrumentation/modules/fastify/set-framework.test.js

is the current block that defines the fastify versions we test against. I'll kick off those "TAV" tests for this PR after the regular tests run.

trentm · 2022-05-31T21:30:37Z

Actually we could stand to add test/sanitize-field-names/fastify.test.js to the fastify .tav.yml test blocks.

Would you be willing to add this patch to your PR?

diff --git a/.tav.yml b/.tav.yml
index d24137a2..dde25443 100644
--- a/.tav.yml
+++ b/.tav.yml
@@ -484,6 +484,7 @@ fastify-v1:
     - node test/instrumentation/modules/fastify/fastify.test.js
     - node test/instrumentation/modules/fastify/async-await.test.js
     - node test/instrumentation/modules/fastify/set-framework.test.js
+    - node test/sanitize-field-names/fastify.test.js
 fastify-v2:
   name: fastify
   versions: '>=2.0.0 <2.4.0 || >2.4.0 <3'
@@ -492,6 +493,7 @@ fastify-v2:
     - node test/instrumentation/modules/fastify/fastify.test.js
     - node test/instrumentation/modules/fastify/async-await.test.js
     - node test/instrumentation/modules/fastify/set-framework.test.js
+    - node test/sanitize-field-names/fastify.test.js
 fastify:
   name: fastify
   versions: '>=3.0.0'
@@ -500,6 +502,7 @@ fastify:
     - node test/instrumentation/modules/fastify/fastify.test.js
     - node test/instrumentation/modules/fastify/async-await.test.js
     - node test/instrumentation/modules/fastify/set-framework.test.js
+    - node test/sanitize-field-names/fastify.test.js

 finalhandler:
   versions: '*'

xxzefgh · 2022-05-31T21:39:37Z

Ok, I think I was wrong on v2 comment, as I forgot to comment out my patches for v2 before testing 😐

Rebased and added changes to .tav.yml

trentm · 2022-05-31T21:55:28Z

I'm not yet sure what is going on with the node test/instrumentation/modules/fastify/fastify.test.js test failure with node 8.6 (in the GitHub Actions-based test above). I cannot reproduce that failure using node 8.6 on my mac. I'll take a look later when I have a chance.

trentm · 2022-05-31T22:03:23Z

Ah, yes, I can reproduce locally: this happens when using fastify@3 (now in our package-lock.json file) and node 8.6. Fastify@3 only supports node v10 and later. So, we need to add a guard on top of the relevant fastify test files to just skip the test if the node version and fastify version aren't supported.

@xxzefgh Can you please add this patch to your PR:

diff --git a/test/instrumentation/modules/fastify/fastify.test.js b/test/instrumentation/modules/fastify/fastify.test.js
index f9f294ab..ae343585 100644
--- a/test/instrumentation/modules/fastify/fastify.test.js
+++ b/test/instrumentation/modules/fastify/fastify.test.js
@@ -8,6 +8,17 @@ const agent = require('../../../..').start({
   captureBody: 'all'
 })

+const fastifyVer = require('../../../../node_modules/fastify/package.json').version
+const semver = require('semver')
+if (
+  (semver.satisfies(fastifyVer, '1.x') && !semver.satisfies(process.version, '>=6 <12')) ||
+  (semver.satisfies(fastifyVer, '2.x') && !semver.satisfies(process.version, '>=6 <15')) ||
+  (semver.satisfies(fastifyVer, '3.x') && !semver.satisfies(process.version, '>=10'))
+) {
+  console.log(`# SKIP fastify@${fastifyVer} does not support node ${process.version}`)
+  process.exit()
+}
+
 const http = require('http')

 const Fastify = require('fastify')

This reproduces the same version ranges as in the .tav.yml section for fastify.

trentm · 2022-05-31T22:06:14Z

It is possible we'll need to add the same guard to the other test files that test fastify, but let's see. We already have a form of that guard in one of those test files:

apm-agent-nodejs/test/sanitize-field-names/fastify.test.js

Lines 5 to 11 in 7e9b9e3

    
           // fastify >=3 only supports node >=10. 
        
           const fastifyVer = require('../../node_modules/fastify/package.json').version 
        
           const semver = require('semver') 
        
           if (semver.gte(fastifyVer, '3.0.0') && semver.lt(process.version, '10.0.0')) { 
        
             console.log(`# SKIP fastify@${fastifyVer} does not support node ${process.version}`) 
        
             process.exit() 
        
           }

trentm · 2022-06-01T20:44:07Z

run module tests for fastify

trentm · 2022-06-01T20:48:50Z

@elasticmachine, run elasticsearch-ci/docs

trentm · 2022-06-02T20:56:51Z

The "test all versions" tests failed on the captureBody test with fastify@1 and node@10. For some reason that I didn't dig into, the preHandler life-cycle hook doesn't run with fastify@1 (though the hook is listed at https://github.com/fastify/fastify/blob/1.x/docs/Lifecycle.md). However, fastify@1 is long since EOL, so we don't need to support new features for it. Here is a suggested patch for skipping that test for this combination (and a couple other doc tweaks):

diff --git a/docs/supported-technologies.asciidoc b/docs/supported-technologies.asciidoc
index b961ccfc..6b0b684c 100644
--- a/docs/supported-technologies.asciidoc
+++ b/docs/supported-technologies.asciidoc
@@ -46,7 +46,7 @@ These are the frameworks that we officially support:
 router information for full support. We currently support the most popular Koa router called
 https://github.com/alexmingoia/koa-router[koa-router].
 |<<restify,Restify>> |>=5.2.0
-|<<fastify,Fastify>> |>=1.0.0; see also https://www.fastify.io/docs/latest/LTS/[Fastify's own LTS documentation]
+|<<fastify,Fastify>> |>=1.0.0; see also https://www.fastify.io/docs/latest/Reference/LTS/[Fastify's own LTS documentation]
 |<<lambda,AWS Lambda>> |N/A
 |=======================================================================
 
diff --git a/lib/instrumentation/modules/fastify.js b/lib/instrumentation/modules/fastify.js
index 7646822e..cd757ca7 100644
--- a/lib/instrumentation/modules/fastify.js
+++ b/lib/instrumentation/modules/fastify.js
@@ -61,6 +61,7 @@ module.exports = function (fastify, agent, { version, enabled }) {
     })
 
     // Save the parsed req body to be picked up by getContextFromRequest().
+    // (This doesn't work for fastify@1, but that is now EOL.)
     _fastify.addHook('preHandler', (req, reply, next) => {
       req.raw.body = req.body
       next()
diff --git a/test/instrumentation/modules/fastify/fastify.test.js b/test/instrumentation/modules/fastify/fastify.test.js
index ae343585..df05c5bb 100644
--- a/test/instrumentation/modules/fastify/fastify.test.js
+++ b/test/instrumentation/modules/fastify/fastify.test.js
@@ -65,55 +65,58 @@ test('transaction name', function (t) {
   })
 })
 
-test('captureBody', function (t) {
-  t.plan(9)
-
-  const postData = JSON.stringify({ foo: 'bar' })
-
-  resetAgent(data => {
-    assert(t, data, { name: 'POST /postSomeData', method: 'POST' })
-    t.equal(data.transactions[0].context.request.body, postData,
-      'body was captured to trans.context.request.body')
-    fastify.close()
-  })
-
-  var fastify = Fastify()
+// Fastify `captureBody` support is only supported for [email protected] and later.
+if (semver.gte(fastifyVer, '2.0.0')) {
+  test('captureBody', function (t) {
+    t.plan(9)
+
+    const postData = JSON.stringify({ foo: 'bar' })
+
+    resetAgent(data => {
+      assert(t, data, { name: 'POST /postSomeData', method: 'POST' })
+      t.equal(data.transactions[0].context.request.body, postData,
+        'body was captured to trans.context.request.body')
+      fastify.close()
+    })
 
-  fastify.post('/postSomeData', (request, reply) => {
-    reply.send('your data has been posted')
-  })
+    var fastify = Fastify()
 
-  fastify.listen(0, function (err) {
-    t.error(err)
+    fastify.post('/postSomeData', (request, reply) => {
+      reply.send('your data has been posted')
+    })
 
-    // build the URL manually as older versions of fastify doesn't supply it as
-    // an argument to the callback
-    const port = fastify.server.address().port
-    const cReq = http.request(
-      'http://localhost:' + port + '/postSomeData',
-      {
-        method: 'POST',
-        hostname: 'localhost',
-        port,
-        headers: {
-          'Content-Type': 'application/json',
-          'Content-Length': Buffer.byteLength(postData)
+    fastify.listen(0, function (err) {
+      t.error(err)
+
+      // build the URL manually as older versions of fastify doesn't supply it as
+      // an argument to the callback
+      const port = fastify.server.address().port
+      const cReq = http.request(
+        'http://localhost:' + port + '/postSomeData',
+        {
+          method: 'POST',
+          hostname: 'localhost',
+          port,
+          headers: {
+            'Content-Type': 'application/json',
+            'Content-Length': Buffer.byteLength(postData)
+          }
+        },
+        function (res) {
+          t.strictEqual(res.statusCode, 200)
+          res.on('data', function (chunk) {
+            t.strictEqual(chunk.toString(), 'your data has been posted')
+          })
+          res.on('end', function () {
+            agent.flush()
+          })
         }
-      },
-      function (res) {
-        t.strictEqual(res.statusCode, 200)
-        res.on('data', function (chunk) {
-          t.strictEqual(chunk.toString(), 'your data has been posted')
-        })
-        res.on('end', function () {
-          agent.flush()
-        })
-      }
-    )
-    cReq.write(postData)
-    cReq.end()
+      )
+      cReq.write(postData)
+      cReq.end()
+    })
   })
-})
+}
 
 function resetAgent (cb) {
   agent._instrumentation.testReset()

@xxzefgh If you are good adding that to your PR, I can re-run the tests. Thanks for persisting through the slog of testing.

Fixes: #1906

trentm · 2022-08-08T19:31:24Z

@elasticmachine, run elasticsearch-ci/docs

trentm · 2022-08-08T19:31:57Z

run module tests for fastify

…aks; avoid APM server error warning by using apmServerVersion

trentm · 2023-05-02T19:45:31Z

This has languished. I'm adding some small commits to get this over the line.

trentm · 2023-05-02T20:15:12Z

@elasticmachine, run elasticsearch-ci/docs

trentm · 2023-05-02T22:37:20Z

@elasticmachine, run elasticsearch-ci/docs

…e will effectively do this

trentm · 2023-05-02T23:21:34Z

@elasticmachine, run elasticsearch-ci/docs

…re/support-specific-modules * 'main' of github.com:elastic/apm-agent-nodejs: (54 commits) chore: fix dev-utils/ci-tav-slow-jobs.sh (elastic#3319) test: reduce TAV test matrix for slowest jobs (elastic#3321) chore: sync package-lock so 'npm ci' can work (elastic#3318) docs: document `useElasticTraceparentHeader` config var (elastic#3316) chore, test: test driver improvements (elastic#3293) test: drop node 14 from RC tests now that it is EOL (elastic#3315) test: fix running fastify.test.js with node v8 (elastic#3317) feat: add @apollo/server@4 support (elastic#3203) chore: update nvm (elastic#3309) tests: stop testing 'express-graphql' instrumentation (elastic#3304) chore: fix bitrot.js dev util for recent changes (elastic#3308) test: restore testing of Azure Functions on node >=18.x (elastic#3307) fix: support Lambda instrumentation for `contextManager: 'patch'`; refactor Lambda tests (elastic#3305) test: fix fastify TAV test failures (elastic#3314) test: fix @aws-sdk/client-s3 TAV test failures (elastic#3312) feat: add instrumentation for aws-sdk S3 client (elastic#3287) feat(fastify): add captureBody support (elastic#2681) feat: mysql2@3 support (elastic#3301) chore(deps): bump @opentelemetry/exporter-prometheus from 0.37.0 to 0.38.0 in /test/opentelemetry-metrics/fixtures (elastic#3295) chore(deps-dev): bump fastify from 4.16.3 to 4.17.0 (elastic#3296) ...

Closes: #1906 Closes: #3217 Co-authored-by: Trent Mick <[email protected]>

Closes: elastic#1906 Closes: elastic#3217 Co-authored-by: Trent Mick <[email protected]>

github-actions bot added agent-nodejs Make available for APM Agents project planning. community triage labels May 7, 2022

xxzefgh closed this May 7, 2022

xxzefgh reopened this May 8, 2022

trentm reviewed May 9, 2022

View reviewed changes

trentm removed the triage label May 9, 2022

trentm approved these changes May 31, 2022

View reviewed changes

feat(fastify): add captureBody support

12f8bca

Fixes: #1906

trentm added the 8.8-candidate label Feb 2, 2023

trentm removed the 8.8-candidate label Apr 24, 2023

trentm added the 8.9-candidate label Apr 24, 2023

trentm added this to the 8.9 milestone Apr 24, 2023

trentm added 4 commits May 2, 2023 12:20

avoid FSTDEP004 deprecation warning on .listen(); some test style twe…

19f5cb1

…aks; avoid APM server error warning by using apmServerVersion

fix captureBody for fastify@1 even though it is EOL

5543e1b

whitespace nit

bc91d4a

Merge branch 'main' into fastify-capture-body

9b83cd8

trentm mentioned this pull request May 2, 2023

Fastify wont capture body - fix proposed #3217

Closed

3 tasks

trentm linked an issue May 2, 2023 that may be closed by this pull request

Fastify wont capture body - fix proposed #3217

Closed

3 tasks

trentm self-assigned this May 2, 2023

add a changelog entry

7e4a428

testing nit: explicitly t.end() tests, even though the t.plan(N) usag…

46bab02

…e will effectively do this

trentm merged commit ca1ede1 into elastic:main May 3, 2023

trentm added a commit that referenced this pull request May 16, 2023

feat(fastify): add captureBody support (#2681)

0d9da33

Closes: #1906 Closes: #3217 Co-authored-by: Trent Mick <[email protected]>

PeterEinberger pushed a commit to fpm-git/apm-agent-nodejs that referenced this pull request Aug 20, 2024

feat(fastify): add captureBody support (elastic#2681)

07a9327

Closes: elastic#1906 Closes: elastic#3217 Co-authored-by: Trent Mick <[email protected]>

feat(fastify): add captureBody support #2681

feat(fastify): add captureBody support #2681

Uh oh!

Conversation

xxzefgh commented May 7, 2022 • edited by trentm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cla-checker-service bot commented May 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xxzefgh commented May 7, 2022

Uh oh!

xxzefgh commented May 8, 2022

Uh oh!

apmmachine commented May 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Build stats

Test stats 🧪

Steps errors

Run Tests

Run Tests

Archive the artifacts

Run Tests

Run Tests

Archive the artifacts

Run Tests

Run Tests

Archive the artifacts

Error signal

🤖 GitHub comments

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

trentm commented May 9, 2022

Uh oh!

xxzefgh commented May 10, 2022

Uh oh!

trentm commented May 10, 2022

Uh oh!

xxzefgh commented May 31, 2022

Uh oh!

trentm commented May 31, 2022

Uh oh!

trentm commented May 31, 2022

Uh oh!

xxzefgh commented May 31, 2022

Uh oh!

trentm commented May 31, 2022

Uh oh!

trentm commented May 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trentm commented May 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trentm commented Jun 1, 2022

Uh oh!

trentm commented Jun 1, 2022

Uh oh!

trentm commented Jun 2, 2022

Uh oh!

trentm commented Aug 8, 2022

Uh oh!

trentm commented Aug 8, 2022

Uh oh!

trentm commented May 2, 2023

Uh oh!

trentm commented May 2, 2023

Uh oh!

trentm commented May 2, 2023

Uh oh!

trentm commented May 2, 2023

Uh oh!

Uh oh!

xxzefgh commented May 7, 2022 •

edited by trentm

Loading

cla-checker-service bot commented May 7, 2022 •

edited

Loading

apmmachine commented May 8, 2022 •

edited

Loading

`Run Tests`

`Run Tests`

`Archive the artifacts`

`Run Tests`

`Run Tests`

`Archive the artifacts`

`Run Tests`

`Run Tests`

`Archive the artifacts`

`Error signal`

trentm commented May 31, 2022 •

edited

Loading

trentm commented May 31, 2022 •

edited

Loading