Replace env_logger with tracing

[ffuugoo]

This PR replaces env_logger with tracing/tracing-subscriber/tracing-log as our log backend (and also removes half of the custom features added in #2187).

(Currently based on top of #2187. Would be rebased onto dev once #2187 is merged.)

TODO:

• Fix DEVELOPMENT.md documentation after removing custom tracing-related features from qdrant crate
• Make tracing required dependency of lib/* crates and remove custom tracing feature everywhere?
  • There's practically no downside to leaving tracing optional in lib/* crates, so I crossed it out.

Further Improvements:

• Add API to change log-level/filters when Qdrant is already running (Add /log_level HTTP endpoint to get/set log-level at runtime #2383)
  • should be pretty trivial to do with tracing_subscriber::reload
• Add configuration parameter to select "span events" to be printed
  • (and an API to change this in runtime)
  • span events are rather verbose to enable them by default, but they are extremely useful for debugging
  • adding configuration parameter should be trivial
  • adding an API - moderately annoying
• Add file appender (and all required configuration/infrastructure), so that logs can be written to a file (Implement logging into a file on disk #2392)
  • RollingFileAppender from tracing-appender crate does not yet support practical rollover strategies
    • max_log_files_feature tokio-rs/tracing#2323 is merged, but not yet released
    • feat(appender): size-based rotation tokio-rs/tracing#2497 is not yet merged
  • there's also tracing-rolling-file, which looks fine, but I haven't tested it yet
• Implement slog drain that would convert slog records into tracing events, preserving slog's "structured" KV pairs
  • slog-stdlog (that we are currently using) is fine
    • but it does not preserve KV pairs
    • I'm not sure if raft crate actually emits any, but it would be kinda silly to use slog if it does not, so I assume it does
    • and so I'd want to capture/log these KV pairs
  • tracing-slog is mostly useless
    • it is extremely simplistic and does not preserve KV pairs
    • which is, like, the whole point of slog and any specific "slog-to-tracing" bridge
    • because slog-stdlog is perfectly fine if you don't want/need KV pairs
• Add configuration parameter to select output format?
  • (and an API to change this in runtime)
  • I can already tell, that this will be super annoying to implement, so not sure if it's worth it

All Submissions:

• Contributions should target the dev branch. Did you create your branch from dev?
• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

1. Does your submission pass tests?
2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
3. Have you checked your code using cargo clippy --all --all-features command?

Changes to Core Features:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you successfully ran tests with your changes locally?

[timvisee]

Nice work!

In development mode it is a lot more noisy now. Can we disable logging on some modules by default to reduce it a bit? Or do you think keeping these is more practical?

Before:

After:

[ffuugoo]

In development mode it is a lot more noisy now. Can we disable logging on some modules by default to reduce it a bit? Or do you think keeping these is more practical?

Some crates are using tracing instead of log, so we are suddenly seeing these logs now. I agree that it makes sense to disable some of the more verbose logs. I personally use debug,hyper=info,h2=info,tower=info,rustls=info.

[timvisee]

I personally use debug,hyper=info,h2=info,tower=info,rustls=info.

Yeah, that seems more reasonable.

Maybe we can set that as default here. What do you think?