feat: OpenTelemetry Bridge

.eslintrc.json

@@ -8,6 +8,8 @@
   "ignorePatterns": [
     "/.nyc_output",
     "/build",
+    "node_modules",
+    "/lib/opentelemetry-bridge/opentelemetry-core-mini",
     "/test/babel/out.js",
     "/test/sourcemaps/fixtures/lib",
     "/test/stacktraces/fixtures/dist",

.gitignore

@@ -11,6 +11,6 @@
 /.nyc_output
 /test_output
 /build
-/node_modules
+node_modules
 /test/benchmarks/.tmp
 /tmp

CHANGELOG.asciidoc

@@ -28,6 +28,24 @@ Notes:
 [[release-notes-3.x]]
 === Node.js Agent version 3.x
 
+==== Unreleased
+
+[float]
+===== Breaking changes
+
+[float]
+===== Features
+
+- Add an experimental <<opentelemetry-bridge>>.  Briefly, the OpenTelemetry
+  Bridge allows one to use the vendor-neutral
+  https://opentelemetry.io/docs/instrumentation/js/api/[OpenTelemetry Tracing
+  API] (https://www.npmjs.com/package/@opentelemetry/api[`@opentelemetry/api`])
+  to manually instrument your code, and have the Elastic Node.js APM agent
+  handle those API calls. ({pull}2641[#2641])
+
+[float]
+===== Bug fixes
+
 
 [[release-notes-3.33.0]]
 ==== 3.33.0 2022/05/05

NOTICE.md

@@ -110,6 +110,22 @@ THE SOFTWARE.
 Parts of "lib/instrumentation/run-context" have been adapted from or influenced
 by TypeScript code in `@opentelemetry/context-async-hooks`.
 
+- **path:** [lib/opentelemetry-bridge/otelutils.js](lib/opentelemetry-bridge/otelutils.js)
+- **author:** OpenTelemetry Authors
+- **project url:** https://github.com/open-telemetry/opentelemetry-js
+- **original file:** https://github.com/open-telemetry/opentelemetry-js/tree/main/packages/opentelemetry-core/src/common/time.ts
+- **license:** Apache License 2.0, https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-core/LICENSE
+
+"lib/opentelemetry-bridge/opentelemetry-core-mini/" includes files adapted from
+code in `@opentelemetry/core`.
+
+- **path:** [lib/opentelemetry-bridge/opentelemetry-core-mini/](lib/opentelemetry-bridge/opentelemetry-core-mini/)
+- **author:** OpenTelemetry Authors
+- **project url:** https://github.com/open-telemetry/opentelemetry-js
+- **original file:** https://github.com/open-telemetry/opentelemetry-js/tree/main/packages/opentelemetry-core/src/
+- **license:** Apache License 2.0, https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-core/LICENSE
+
+
 ## load-source-map
 
 - **path:** [lib/load-source-map.js](lib/load-source-map.js)

README.md

@@ -39,12 +39,12 @@ npm install elastic-apm-node --save
 - [Get started with a custom Node.js stack](https://www.elastic.co/guide/en/apm/agent/nodejs/current/custom-stack.html)
 - [Advanced Setup and Configuration](https://www.elastic.co/guide/en/apm/agent/nodejs/current/advanced-setup.html)
 - [API Reference](https://www.elastic.co/guide/en/apm/agent/nodejs/current/api.html)
+- [OpenTelemetry Bridge](https://www.elastic.co/guide/en/apm/agent/nodejs/current/opentelemetry-bridge.html)
 - [Custom Transactions](https://www.elastic.co/guide/en/apm/agent/nodejs/current/custom-transactions.html)
 - [Custom Spans](https://www.elastic.co/guide/en/apm/agent/nodejs/current/custom-spans.html)
 - [Metrics](https://www.elastic.co/guide/en/apm/agent/nodejs/current/metrics.html)
 - [Performance Tuning](https://www.elastic.co/guide/en/apm/agent/nodejs/current/performance-tuning.html)
 - [Source Map Support](https://www.elastic.co/guide/en/apm/agent/nodejs/current/source-maps.html)
-- [OpenTracing Support](https://www.elastic.co/guide/en/apm/agent/nodejs/current/opentracing.html)
 - [Supported Technologies](https://www.elastic.co/guide/en/apm/agent/nodejs/current/supported-technologies.html)
 - [Upgrading](https://www.elastic.co/guide/en/apm/agent/nodejs/current/upgrading.html)
 - [Troubleshooting](https://www.elastic.co/guide/en/apm/agent/nodejs/current/troubleshooting.html)

docs/api-opentelemetry.asciidoc

@@ -0,0 +1,147 @@
+ifdef::env-github[]
+NOTE: For the best reading experience,
+please view this documentation at https://www.elastic.co/guide/en/apm/agent/nodejs/current/opentelemetry-bridge.html[elastic.co]
+endif::[]
+
+[[opentelemetry-bridge]]
+== OpenTelemetry bridge
+
+NOTE: Added as experimental in v3.34.0.
+To enable it, set <<opentelemetry-bridge-enabled, `opentelemetryBridgeEnabled`>> to `true`.
+
+The Elastic APM OpenTelemetry bridge allows one to use the vendor-neutral
+https://opentelemetry.io/docs/instrumentation/js/api/[OpenTelemetry Tracing API]
+(https://www.npmjs.com/package/@opentelemetry/api[`@opentelemetry/api`]) to
+manually instrument your code, and have the Elastic Node.js APM agent handle
+those API calls. This allows one to use the Elastic APM agent for tracing,
+without any vendor lock-in from adding manual tracing using the APM agent's own
+<<api,public API>>.
+
+
+[float]
+[[otel-getting-started]]
+=== Getting started
+
+The goal of the OpenTelemetry bridge is to allow using the OpenTelemetry API
+with the APM agent. ① First, you will need to add those dependencies to your
+project. The minimum required OpenTelemetry API version is 1.0.0. For example:
+
+[source,bash]
+----
+npm install --save elastic-apm-node @opentelemetry/api
+----
+
+② Second, you will need to configure and start the APM agent. This can be done
+completely with environment variables (so that there is no need to touch
+your application code):
+
+[source,bash]
+----
+export ELASTIC_APM_SERVER_URL='<url of your APM server>'
+export ELASTIC_APM_SECRET_TOKEN='<secret token for your APM server>'  # or ELASTIC_APM_API_KEY=...
+export ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true
+export NODE_OPTIONS='-r elastic-apm-node/start.js'  # Tell node to preload and start the APM agent
+node my-app.js
+----
+
+Or, alternatively, you can configure and start the APM agent at the top of your
+application code as follows. (Note: For automatic instrumentations to function
+properly, this must be executed before other `require` statements and
+application code.)
+
+[source,js]
+----
+require('elastic-apm-node').start({
+    serverUrl: '<url of your APM server>',
+    secretToken: '<secret token for your APM server>', // or, apiKey: '<your API key>'
+    opentelemetryBridgeEnabled: true
+});
+
+// Application code ...
+----
+
+NOTE: These examples show the minimal configuration. See <<configuration,the full APM agent configuration reference>> for other configuration options.
+
+③ Finally, you can use the OpenTelemetry API for any manual tracing in your code.
+For example, the following script uses
+https://open-telemetry.github.io/opentelemetry-js-api/interfaces/tracer.html#startactivespan[Tracer#startActiveSpan()]
+to trace an outgoing HTTPS request:
+
+[source,js]
+----
+const https = require('https')
+const otel = require('@opentelemetry/api')
+const tracer = otel.trace.getTracer('trace-https-request')
+
+tracer.startActiveSpan('makeRequest', span => {
+  https.get('https://httpstat.us/200', (response) => {
+    console.log('STATUS:', response.statusCode)
+    const body = []
+    response.on('data', (chunk) => body.push(chunk))
+    response.on('end', () => {
+      console.log('BODY:', body.toString())
+      span.end()
+    })
+  })
+})
+----
+
+The APM agent source code repository includes
+https://github.com/elastic/apm-agent-nodejs/tree/main/examples/opentelemetry-bridge[some examples using the OpenTelemetry bridge].
+
+
+[float]
+[[otel-architecture]]
+=== Bridge architecture
+
+The OpenTelemetry bridge works similarly to the
+https://github.com/open-telemetry/opentelemetry-js[OpenTelemetry JS SDK]. It
+registers Tracer and ContextManager providers with the OpenTelemetry API.
+Subsequent `@opentelemetry/api` calls in user code will call into those
+providers. The APM agent translates from OpenTelemetry to Elastic APM semantics
+and sends tracing data to your APM server for full support in
+https://www.elastic.co/apm[Elastic Observability's APM app].
+
+Here are a couple examples of semantic translations: The first entry span of a
+service (e.g. an incoming HTTP request) will be converted to an
+{apm-guide-ref}/data-model-transactions.html[Elasic APM `Transaction`],
+subsequent spans are mapped to
+{apm-guide-ref}/data-model-spans.html[Elastic APM `Span`]. OpenTelemetry Span
+attributes are translated into the appropriate fields in Elastic APM's data
+model.
+
+The only difference, from the user's point of view, is in the setup of tracing.
+Instead of setting up the OpenTelemetry JS SDK, one sets up the APM agent
+as <<otel-getting-started,described above>>.
+
+
+[float]
+[[otel-caveats]]
+=== Caveats
+Not all features of the OpenTelemetry API are supported.
+
+[float]
+[[otel-metrics]]
+===== Metrics
+This bridge only supports the tracing API.
+The Metrics API is currently not supported.
+
+[float]
+[[otel-span-links]]
+===== Span Links
+Adding links when
+https://open-telemetry.github.io/opentelemetry-js-api/interfaces/tracer.html[starting a span]
+are not currently supported. Any given links will be silently dropped.
+
+[float]
+[[otel-span-events]]
+===== Span Events
+Span events (https://open-telemetry.github.io/opentelemetry-js-api/interfaces/span.html#addevent[`Span#addEvent()`])
+is not currently supported. Events will be silently dropped.
+
+[float]
+[[otel-baggage]]
+===== Baggage
+https://open-telemetry.github.io/opentelemetry-js-api/classes/propagationapi.html[Propagating baggage]
+within or outside the process is not supported. Baggage items are silently
+dropped.

docs/configuration.asciidoc

@@ -1235,3 +1235,27 @@ require('elastic-apm-node').start({
 })
 ----
 
+
+[[opentelemetry-bridge-enabled]]
+==== `opentelemetryBridgeEnabled`
+
+[small]#Added in: v3.34.0 as experimental#
+
+* *Type:* Boolean
+* *Default:* `false`
+* *Env:* `ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED`
+
+Setting this option to true will enable the <<opentelemetry-bridge,OpenTelemetry Bridge>>. Briefly, the OpenTelemetry Bridge allows one to use the vendor-neutral
+https://opentelemetry.io/docs/instrumentation/js/api/[OpenTelemetry Tracing API]
+(https://www.npmjs.com/package/@opentelemetry/api[`@opentelemetry/api`]) to
+manually instrument your code, and have the Elastic Node.js APM agent handle
+those API calls.
+
+Example usage:
+
+[source,js]
+----
+require('elastic-apm-node').start({
+  opentelemetryBridgeEnabled: true
+})
+----

docs/index.asciidoc

@@ -31,6 +31,8 @@ include::./api.asciidoc[]
 
 include::./metrics.asciidoc[]
 
+include::./api-opentelemetry.asciidoc[]
+
 include::./opentracing.asciidoc[]
 
 include::./log-correlation.asciidoc[]

docs/opentracing.asciidoc

@@ -5,7 +5,10 @@ NOTE: For the best reading experience,
 please view this documentation at https://www.elastic.co/guide/en/apm/agent/nodejs/current/opentracing.html[elastic.co]
 endif::[]
 
-== OpenTracing API
+== OpenTracing bridge
+
+NOTE: https://opentracing.io/[OpenTracing] is discontinued in favor of OpenTelemetry. This Elastic APM OpenTracing bridge is **deprecated**. Consider using the <<opentelemetry-bridge>> instead.
+
 
 The Elastic APM OpenTracing bridge allows creating Elastic APM transactions and spans,
 using the https://opentracing-javascript.surge.sh/[OpenTracing API].

examples/opentelemetry-bridge/.gitignore

@@ -0,0 +1 @@
+/node_modules

examples/opentelemetry-bridge/.npmrc

@@ -0,0 +1 @@
+package-lock=false

examples/opentelemetry-bridge/README.md

@@ -0,0 +1,41 @@
+This directory includes example Node.js scripts showing usage of the
+OpenTelemetry JS API. These can be instrumented with the Elastic Node.js APM
+agent using its OpenTelemetry Bridge.
+
+Setup dependencies via:
+
+    npm install
+
+To run a script using the **Elastic Node.js APM Agent** use:
+
+    export ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true
+    node -r elastic-apm-node/start.js THE-SCRIPT.js
+
+For example:
+
+    export ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true
+    node -r elastic-apm-node/start.js trace-https-request.js
+
+While these examples are written to use the `node -r elastic-apm-node/start.js ...`
+mechanism to start the APM agent. That isn't required. One can still load and
+start the APM agent at the top of a script like this:
+
+```js
+require('elastic-apm-node').start({
+    opentelemetryBridgeEnabled: true
+    // serviceName: ...
+    // serverUrl: ...
+    // secretToken: ...
+})
+```
+
+## Compare to using the OpenTelemetry JS SDK
+
+For comparison, these scripts can be instrumented with the OpenTelemetry JS SDK
+with something like the following:
+
+    node -r ./otel-sdk.js trace-https-request.js
+
+The "otel-sdk.js" is a simplified tracing setup of the OpenTelemetry SDK. For
+the sake of simpler demonstration it writes tracing spans to the console rather
+than sending to some collection service (like Jaeger or Elastic APM).

examples/opentelemetry-bridge/otel-sdk.js

@@ -0,0 +1,35 @@
+'use strict'
+
+// Start a simply-configured OpenTelemetry SDK for Node.js tracing for demo
+// purposes.
+//
+// Based on https://github.com/open-telemetry/opentelemetry-js/blob/main/examples/http/tracer.js
+//
+// Usage:
+//    node -r ./otel-sdk.js MY-SCRIPT.js
+
+// Uncomment this to get OpenTelemetry internal diagnostic messages.
+// const otel = require('@opentelemetry/api')
+// otel.diag.setLogger({
+//   verbose () { console.log('diag VERBOSE:', ...arguments) },
+//   debug () { console.log('diag DEBUG:', ...arguments) },
+//   info () { console.log('diag INFO:', ...arguments) },
+//   warn () { console.log('diag WARN:', ...arguments) },
+//   error () { console.log('diag ERROR:', ...arguments) }
+// }, opentelemetry.DiagLogLevel.ALL)
+
+const { registerInstrumentations } = require('@opentelemetry/instrumentation')
+const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node')
+const { SimpleSpanProcessor, ConsoleSpanExporter } = require('@opentelemetry/sdk-trace-base')
+const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http')
+
+module.exports = (() => {
+  const provider = new NodeTracerProvider()
+  provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()))
+  provider.register()
+  registerInstrumentations({
+    instrumentations: [
+      new HttpInstrumentation()
+    ]
+  })
+})()

examples/opentelemetry-bridge/package.json

@@ -0,0 +1,18 @@
+{
+  "name": "elastic-apm-node-opentelemetry-bridge-examples",
+  "version": "1.0.0",
+  "private": true,
+  "scripts": {
+    "trace-hello-world": "ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true node -r elastic-apm-node/start.js trace-hello-world.js",
+    "trace-https-request": "ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true node -r elastic-apm-node/start.js trace-https-request.js"
+  },
+  "dependencies": {
+    "@opentelemetry/api": "^1.1.0",
+    "@opentelemetry/core": "^1.2.0",
+    "@opentelemetry/instrumentation": "^0.28.0",
+    "@opentelemetry/instrumentation-http": "^0.28.0",
+    "@opentelemetry/sdk-trace-base": "^1.2.0",
+    "@opentelemetry/sdk-trace-node": "^1.2.0",
+    "elastic-apm-node": "file:../.."
+  }
+}