Support collecting (some) metrics only from secondary replicas #1114
Description
Is your feature request related to a problem? Please describe.
The performance overhead of collecting certain metrics (i.e. collstat-details, but also in general a lot of them) is not trivial. Furthermore there is no difference in the metrics across replicas (assuming no replication lag), making collecting the metrics from every replica redundant.
At the same time, other metrics clearly should be collected from every replica, to ensure a full picture of the cluster.
Describe the solution you'd like
Have a way to activate the collectors conditionally based on the replica status (PRIMARY / SECONDARY).
Describe alternatives you've considered
Running multiple exporter configurations with only a "minimal" config running as a sidecar on each replica and a more "full" config running over the k8s service with the preferSecondary flag in the connection URI. AFAIK running the exporter without direct-connect isn't a recommended approach so I'm not a great fan of this.
Additional context
I haven't considered what happens if sharding is active (we don't use it - yet).