Bug Report

Describe the bug

Tail input is doing the following frequently to detect if the source file has been truncated:

static int adjust_counters(struct flb_tail_config *ctx, struct flb_tail_file *file)
{
  ret = fstat(file->fd, &st);

   [...]

    /* Check if the file was truncated */
    if (file->offset > st.st_size) {

When the underlying filesystem is using Metadata Cache (the file's metadata, i.e. what results of fstat() or fileinfo in windows) might have a small delay in reflecting what the actual state of the file really is. This results in offset being larger than st.st_size (because the stream does reflect the real size of the file, but not it's metadata).

https://techcommunity.microsoft.com/blog/azurestorageblog/accelerate-metadata-heavy-workloads-with-metadata-caching-preview-for-azure-prem/4261442

This leads to the Tail plugin thinking the file was rotated when it was not, resulting in continuous reingestion of the same log files over and over again.

To Reproduce

• Setup a Tail that reads from NFS with metadata cache enabled
• Generate suficient write load on the NFS file
• See the tail issue a debug message "file truncated" when the file was not truncated, resulting in exponential log ingest.

Expected behavior

No false rotations are detected.

Screenshots

Your Environment

• Version used: 4.0.1
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version:
• Filters and plugins:

Additional context