Failed to export events after running for a while

[ediezh]

The only log I see from the container log is the below warning after the pod has been running for a while

W0504 07:57:16.735926       1 reflector.go:302] pkg/kube/watcher.go:95: watch of *v1.Event ended with: The resourceVersion for the provided watch is too old.

Restarting the pod would fix the problem.

But the event exporter should be able to automatically refresh the watched resources list.

[timansky]

having the same problem, is there any healthcheck or example of livinessprobe?

[aauren]

Not sure that this will be the problem for everyone, but the linked issue above appears to be the solution for me when I encountered this problem.

Basically, you should not use the file exporter for stdout:

...
receivers:
  - name: "dump"
     file:
       path: "/dev/stdout"

This is because kubernetes-event-exporter uses a library called lumberjack which will automatically attempt to rotate the file path after ~12 hours. Obviously, /dev/stdout cannot rotated.

People that want to put events on stdout should instead use the stdout receiver:

receivers:
  - name: "dump"
     stdout: {}

[youyongsong]

@mustafaakin-atl upgrade client-go version may solve this issue. kubernetes/kubernetes#95981 this pr add watch health check function.