Skip to content

chore(docs): Fix Multiline Parser Doc#3015

Open
crazyuploader wants to merge 4 commits intomainfrom
chore/fix-multiline-parser-doc
Open

chore(docs): Fix Multiline Parser Doc#3015
crazyuploader wants to merge 4 commits intomainfrom
chore/fix-multiline-parser-doc

Conversation

@crazyuploader
Copy link
Copy Markdown
Contributor

Summary

Fixes the doc and example

Closes: #4267

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 3, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
signoz-web Ready Ready Preview, Comment Apr 3, 2026 0:57am

Request Review

@crazyuploader
Copy link
Copy Markdown
Contributor Author

@claude /review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Claude finished @crazyuploader's task in 2m 10s —— View job


Docs Review — parse-multiline-logs.mdx

  • Gather context and read changed files
  • Run JTBD-first rubric and technical verification
  • Post inline findings and summary

Findings

P2

  • Invalid sample date across all log blocks — The sample logs use 2024-20-06 (month 20) which is not a valid calendar date. Likely intended 2024-06-20. Appears in 6 places (lines 23, 28, 49, 54, 132, 137). This was present in the original doc, but since this PR rewrites the page it's a good opportunity to fix. (inline comment)

P3

  • Typo in Troubleshootinglogstransformprocessor should be logstransform processor (two words) on line 190. (inline comment)
  • Minor punctuation — Missing colon/period at end of "Here is an example of multiline logs" (line 20). (inline comment)

Personas and Fit

Target personas: platform engineers and SREs running OTel Collector who need to recombine multiline logs (stack traces, multi-line errors). The doc fits well — it assumes familiarity with collector config and YAML but explains the OTel-specific concepts clearly.

JTBD Coverage

Check Status
Primary job obvious Pass — "recombine multiline logs" is clear
Happy path easy to follow Pass — receiver-level approach is the recommended default path
Time-to-first-success short Pass — 3 steps with clear config snippets
Advanced options out of main flow Pass — logstransform/recombine is correctly positioned as alternative with a warning
Validate section present Pass — ## Validate tells users what to look for in SigNoz
Troubleshooting present Pass — covers common failure modes with symptoms and fixes
Prerequisites Minor gap — no explicit ## Prerequisites section, but the doc context (filelog receiver already in use) makes this acceptable

Technical Accuracy

All OTel technical claims verified against upstream sources:

  • multiline config block with line_start_pattern/line_end_pattern/omit_patternaccurate (Source: filelog receiver README)
  • recombine operator fields (is_first_entry, combine_field, source_identifier) — accurate (Source: recombine operator docs)
  • logstransform processor not in otelcol-contribaccurate (Source: logstransform README)
  • force_flush_period default 500msaccurate

Checklist Coverage (docs-authoring.md)

  • Frontmatter: date, id, title, description, doc_type all present. Tags omitted correctly (applies to both Cloud and Self-Host).
  • doc_type: howto — correct for this task-oriented guide.
  • External links use <a> with target="_blank" rel="noopener noreferrer nofollow" — correct.
  • Code blocks use appropriate language hints (yaml, text).
  • Admonitions used appropriately for info and warning.
  • Images reference existing paths under /img/logs/multiline/.

Open Questions

  • The sample date 2024-20-06 was present in the original doc. This is a good rewrite opportunity to fix it.

Onboarding Label

Onboarding label: skipped — This PR edits an existing doc (file status: modified), not a new ingestion guide.


Overall this is a solid rewrite. The doc is much clearer, better structured, and adds valuable content (k8s-infra Helm guidance, logstransform availability warning, troubleshooting section). The only actionable item is fixing the invalid sample dates.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@crazyuploader crazyuploader marked this pull request as ready for review April 3, 2026 12:44
Copilot AI review requested due to automatic review settings April 3, 2026 12:44
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

Claude finished @crazyuploader's task in 2m 18s —— View job


Docs Review — parse-multiline-logs.mdx

  • Gather context and read changed files
  • Run docs review skill (JTBD rubric, technical verification, inline findings)
  • Post summary

Findings

P2

  • Invalid sample date 2024-20-06 in 5 remaining log blocks — The previous review flagged this; line 23 was fixed but lines 28, 49, 54, 132, 137 still use month 20. (inline comments)
  • Likely incorrect claim: logstransform not in otelcol-contrib — The logstransformprocessor exists in the contrib repo and is included in the default otelcol-contrib build (with "development" stability). The claim on lines 15–16 and 121–124 that it's "not included in the official otelcol-contrib distribution" appears inaccurate. The k8s-infra exclusion may be correct (SigNoz controls that image), but should be stated separately. (inline comment)

P3

  • Vercel build is failing — The deployment shows an error. Worth investigating before merge.

Personas and Fit

Target personas: platform engineers and SREs running OTel Collector who need to recombine multiline logs (stack traces, multi-line errors). The doc assumes familiarity with collector config and YAML, which is appropriate for the audience.

JTBD Coverage

Check Status
Primary job obvious Pass — "recombine multiline logs" is clear
Happy path easy to follow Pass — receiver-level approach is the recommended default
Time-to-first-success short Pass — 3 steps with clear config snippets
Advanced options out of main flow Pass — logstransform/recombine correctly positioned as alternative with warning
Validate section present Pass — ## Validate tells users what to look for
Troubleshooting present Pass — covers common failure modes with symptoms and fixes
Prerequisites Minor gap — no explicit section, but acceptable given the context (filelog receiver already in use)
Next steps Gap — no ## Next Steps to close the loop (e.g., link to log parsing, log pipelines, or alerting docs)

Checklist Coverage (docs-authoring.md)

  • Frontmatter: date, id, title, description, doc_type all present ✓
  • doc_type: howto — correct for this task-oriented guide ✓
  • External links use <a> with target="_blank" rel="noopener noreferrer nofollow"
  • Code blocks use appropriate language hints (yaml, text) ✓
  • Admonitions used appropriately (info, warning) ✓
  • Images reference existing paths under /img/logs/multiline/
  • Images are PNG (not WebP) — pre-existing, not introduced by this PR

Open Questions

  • Is the logstransform processor truly excluded from the SigNoz k8s-infra collector image? If so, the warning should be scoped to k8s-infra specifically, not otelcol-contrib in general.
  • The Vercel build failure should be investigated before merge.

Onboarding Label

Onboarding label: skipped — This PR edits an existing doc (status: modified), not a new ingestion guide.


Overall this is a well-structured rewrite that adds valuable content (k8s-infra Helm guidance, logstransform availability warning, troubleshooting section). Two items need attention before merge: fixing the remaining invalid dates (5 instances) and verifying the logstransform/otelcol-contrib claim.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the “Parsing Multiline Logs” guide to clarify recommended multiline handling in the OpenTelemetry Collector and to improve the configuration examples (receiver-level multiline, Helm preset usage, and an alternative recombine approach).

Changes:

  • Rewrites the doc structure and adds clearer step-by-step guidance for receiver-level multiline configuration (plus k8s-infra Helm preset).
  • Adds a logstransform/recombine alternative section with warnings and troubleshooting/validation guidance.
  • Updates frontmatter metadata (date, description, doc_type) and refreshes examples/screenshots context.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sample log entry uses an invalid date format (2024-20-06 has month 20). This can confuse readers and undermines the ^\d{4}-\d{2}-\d{2} start-pattern example; please correct it to a valid, consistent date (e.g., 2024-06-20).

Copilot uses AI. Check for mistakes.

```python
```text
2024-20-06 18:58:05,898 ERROR:Exception on main handler
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example line has an invalid date (2024-20-06). Please update it (and the other repeated samples) to a valid, consistent date so the multiline pattern example is realistic.

Copilot uses AI. Check for mistakes.
File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example line has an invalid date (2024-20-06). Please update it to a valid date (and keep it consistent with the rest of the doc’s sample logs).

Copilot uses AI. Check for mistakes.

```python
```text
2024-20-06 18:58:05,898 ERROR:Exception on main handler
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sample log line uses an invalid date (2024-20-06). Please correct it to a valid date (e.g., 2024-06-20) to avoid confusing readers.

Copilot uses AI. Check for mistakes.
File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sample log line uses an invalid date (2024-20-06). Please correct it to a valid date and keep it consistent across all examples in this doc.

Copilot uses AI. Check for mistakes.
Comment on lines +121 to +124
The `logstransform` processor has development stability and is **not included in the official
`otelcol-contrib` distribution** or the SigNoz k8s-infra Helm chart. Using it on a standard
k8s-infra installation results in a collector startup error (`unknown type: logstransform`). If
you are on k8s-infra, use the `presets.logsCollection.multiline` approach above instead.
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning states logstransform is not included in the official otelcol-contrib distribution, but other SigNoz docs in this repo configure processors: logstransform: (e.g. data/docs/userguide/fluentbit_to_signoz.mdx, fluentd_to_signoz.mdx, logs_fields.mdx). Please clarify the scope (k8s-infra chart vs other SigNoz collector deployments) and align this statement with the rest of the documentation.

Copilot uses AI. Check for mistakes.
File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Invalid sample date still present: 2024-20-06 (month 20)

The previous review flagged this across all log blocks. Line 23 was fixed to 2024-06-20, but 5 remaining instances were missed. Month 20 is not a valid calendar date and undermines the example's credibility.

Remaining instances: lines 28, 49, 54, 132, 137.

Suggested change
2024-20-06 18:58:05,898 DEBUG:Query Started
2024-06-20 18:58:05,898 DEBUG:Query Started


```python
```text
2024-20-06 18:58:05,898 ERROR:Exception on main handler
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Same invalid date 2024-20-06

Suggested change
2024-20-06 18:58:05,898 ERROR:Exception on main handler
2024-06-20 18:58:05,898 ERROR:Exception on main handler

File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Same invalid date 2024-20-06

Suggested change
2024-20-06 18:58:05,898 DEBUG:Query Started
2024-06-20 18:58:05,898 DEBUG:Query Started


```python
```text
2024-20-06 18:58:05,898 ERROR:Exception on main handler
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Same invalid date 2024-20-06

Suggested change
2024-20-06 18:58:05,898 ERROR:Exception on main handler
2024-06-20 18:58:05,898 ERROR:Exception on main handler

File "python-logger.py", line 9, in make_log
return area[10]
IndexError: string index out of range
2024-20-06 18:58:05,898 DEBUG:Query Started
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Same invalid date 2024-20-06

Suggested change
2024-20-06 18:58:05,898 DEBUG:Query Started
2024-06-20 18:58:05,898 DEBUG:Query Started

Comment on lines +15 to +16
1. **At the receiver** — use the `multiline` configuration on the filelog receiver (recommended for all setups).
2. **At the processor** — use the `recombine` operator in a `logstransform` processor (requires a custom OTel Collector build; not available in `otelcol-contrib` or the k8s-infra chart).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Likely incorrect: logstransform processor IS included in otelcol-contrib

The logstransformprocessor exists in the opentelemetry-collector-contrib repo and is included in the default otelcol-contrib distribution. The claim that it's "not included in the official otelcol-contrib distribution" appears inaccurate — it has "development" stability but is still bundled.

The k8s-infra Helm chart claim may be correct if SigNoz uses a custom collector image that excludes it, but that should be stated separately from the otelcol-contrib claim.

Suggested rewording for lines 15–16 and lines 121–124: clarify that logstransform is available in otelcol-contrib but not in the SigNoz k8s-infra chart's collector image.

Source: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/logstransformprocessor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants