Skip to content

Conversation

@clundquist-stripe
Copy link

@clundquist-stripe clundquist-stripe commented Dec 19, 2025

Summary

When receiving newline-delimited JSON (NDJSON) from services like CloudFlare Logpush via the /services/collector/raw endpoint, each line should be treated as a separate event. Without this option, the entire request body is treated as a single event, which is incorrect for NDJSON payloads.

This change introduces a raw_line_splitting configuration option to the splunk_hec source. When enabled, incoming requests to the raw endpoint are split on newlines, creating a separate event for each line. Empty lines are filtered out, and trailing newlines are handled correctly.

The option defaults to false to maintain backward compatibility with existing configurations.

Includes table-driven tests covering CloudFlare-style NDJSON payloads with various edge cases (multiple events, trailing newlines, empty lines).

See also

Issue Title Relevance
#22969 Sink: Splunk HEC "raw" does not frame events Related - The inverse problem (sink → source). enhancement complements this.
#17236 splunk_hec source does not read metadata from query parame

Vector configuration

sources:
  cloudflare_logs:
    type: splunk_hec
    address: "0.0.0.0:8088"
    raw_line_splitting: true  # Split NDJSON on newlines

sinks:
  splunk:
    type: splunk_hec_logs
    inputs:
      - cloudflare_logs
    endpoint: "https://splunk.example.com:8088"

How did you test this PR?

Unit tests.
I have not tested this integration wise getting real data from CloudFlare yet.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

…SON support

When receiving newline-delimited JSON (NDJSON) from services like CloudFlare
Logpush via the `/services/collector/raw` endpoint, each line should be treated
as a separate event. Without this option, the entire request body is treated as
a single event, which is incorrect for NDJSON payloads.

This change introduces a `raw_line_splitting` configuration option to the
`splunk_hec` source. When enabled, incoming requests to the raw endpoint are
split on newlines, creating a separate event for each line. Empty lines are
filtered out, and trailing newlines are handled correctly.

The option defaults to `false` to maintain backward compatibility with existing
configurations.

Includes table-driven tests covering CloudFlare-style NDJSON payloads with
various edge cases (multiple events, trailing newlines, empty lines).
@clundquist-stripe clundquist-stripe requested a review from a team as a code owner December 19, 2025 21:31
@github-actions github-actions bot added the domain: sources Anything related to the Vector's sources label Dec 19, 2025
@github-actions
Copy link


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@clundquist-stripe
Copy link
Author

Consulting legal on CLA

@clundquist-stripe clundquist-stripe force-pushed the clundquist/support-cloudflare-jsonl-splunk-hec branch from 5d3410e to b087540 Compare December 19, 2025 21:49
@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: ci Anything related to Vector's CI environment domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant