Add a flag to not check external labels using bucket rewrite

[yeya24]

Signed-off-by: yeya24 yb532204897@gmail.com

Fixes #3706

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

Adapt the second approach mentioned in #3706 (comment).

Add a new flag to not check Thanos external labels when uploading blocks to obj store.

Verification

Tested with a local prometheus tsdb block.

Before rewrite:

./promtool tsdb analyze data 01EYV86TDESNRQNT2NN0WK6QC9   --limit=100 |grep build_info
1 prometheus_build_info

Rewrite block with the command below:

thanos tools bucket rewrite --rewrite.to-delete-config-file test.yaml --id 01EYV86TDESNRQNT2NN0WK6QC9  --objstore.config-file /home/yeya24/b.yaml --no-dry-run --prom-blocks
level=info ts=2021-02-26T04:28:21.275951558Z caller=factory.go:46 msg="loading bucket configuration"
level=info ts=2021-02-26T04:28:21.276586951Z caller=tools_bucket.go:836 msg="downloading block" source=01EYV86TDESNRQNT2NN0WK6QC9
level=info ts=2021-02-26T04:28:21.277412002Z caller=tools_bucket.go:872 msg="changelog will be available" file=/tmp/thanos-rewrite/01EZE9GFWXJ1ACN42V72YJ7S7C/change.log
level=info ts=2021-02-26T04:28:21.280528395Z caller=tools_bucket.go:887 msg="starting rewrite for block" source=01EYV86TDESNRQNT2NN0WK6QC9 new=01EZE9GFWXJ1ACN42V72YJ7S7C toDelete="- matchers: \"{__name__=\\\"prometheus_build_info\\\"}\"\n"
level=info ts=2021-02-26T04:28:21.284137549Z caller=compactor.go:41 msg="processed 9.89% of 374 series"
level=info ts=2021-02-26T04:28:21.284549508Z caller=compactor.go:41 msg="processed 19.79% of 374 series"
level=info ts=2021-02-26T04:28:21.284942365Z caller=compactor.go:41 msg="processed 29.68% of 374 series"
level=info ts=2021-02-26T04:28:21.285297219Z caller=compactor.go:41 msg="processed 39.57% of 374 series"
level=info ts=2021-02-26T04:28:21.285640116Z caller=compactor.go:41 msg="processed 49.47% of 374 series"
level=info ts=2021-02-26T04:28:21.285977354Z caller=compactor.go:41 msg="processed 59.36% of 374 series"
level=info ts=2021-02-26T04:28:21.286304648Z caller=compactor.go:41 msg="processed 69.25% of 374 series"
level=info ts=2021-02-26T04:28:21.286629152Z caller=compactor.go:41 msg="processed 79.14% of 374 series"
level=info ts=2021-02-26T04:28:21.286944944Z caller=compactor.go:41 msg="processed 89.04% of 374 series"
level=info ts=2021-02-26T04:28:21.287248931Z caller=compactor.go:41 msg="processed 98.93% of 374 series"
level=info ts=2021-02-26T04:28:21.287294998Z caller=tools_bucket.go:897 msg="wrote new block after modifications; flushing" source=01EYV86TDESNRQNT2NN0WK6QC9 new=01EZE9GFWXJ1ACN42V72YJ7S7C
level=info ts=2021-02-26T04:28:21.298739241Z caller=tools_bucket.go:906 msg="uploading new block" source=01EYV86TDESNRQNT2NN0WK6QC9 new=01EZE9GFWXJ1ACN42V72YJ7S7C
level=info ts=2021-02-26T04:28:21.299116007Z caller=tools_bucket.go:916 msg=uploaded source=01EYV86TDESNRQNT2NN0WK6QC9 new=01EZE9GFWXJ1ACN42V72YJ7S7C
level=info ts=2021-02-26T04:28:21.299130425Z caller=tools_bucket.go:918 msg="rewrite done" IDs=01EYV86TDESNRQNT2NN0WK6QC9
level=info ts=2021-02-26T04:28:21.299161779Z caller=main.go:159 msg=exiting

config:

- matchers: "{__name__=\"prometheus_build_info\"}"

Check the new block, there is no prometheus_build_info metric.

./promtool tsdb analyze data 01EZE9GFWXJ1ACN42V72YJ7S7C --limit=100 | grep build

[yeya24]

It would be good to have this in the next release.

[bwplotka]

LGTM, however, there are some ideas to maybe make it less surpring (:

[yeya24]

This is ready for another round of review!

[yeya24]

Ready for another review.

[kakkoyun]

LGTM 🚀