gix upgrade and optimizations (#1081)

* upgrade to latest gix v0.46 and make use of free traversal state.

The latest improvements make more data available during commit iteration,
which makes some improvements possible.

Doing so potentially saves some time, even though it's not visible in practice.

* change: bot commits now provide their commit time to first/most-recent commit information.

In practice, this change is unlikely to be noticeable, especially since `--no-bots` is not enabled by default.
However, if it is, bot commits will now contribute to the most recent commit times and the first commit times,
which will help if the only commit is a bot commit.
It also helps with allowing an upcoming multi-threaded implementation that leverages the commit-graph which in
turn will greatly speed up information retrieval in large repositories.

* refactor retrieval of commit graph information to make output clear

This is in preparation for creating a multi-threaded version of it.

* refactor retrieval of commit graph information to make output clear

This is in preparation for creating a multi-threaded version of it.

* refactoring

- Extrat commit graph traversal logic out of CommitMetrics's constructor
- renaming and simplified logic

* minor

* cargo fmt

* remove is_shallow property from GitMetrics

* Single thread for churn and author summary on main thread

* process author sig in seperate thread

* revert

* refactor

* unify logic to update number of commits by author.

More notes related to the `use_commit_graph()` call:

I was thinking about this API for a while, and it's the easiest way to configure this even though it seems redundant. Let me explain, maybe you can find a better solution.

In an even more recent version, `use_commit_graph()` doesn't default to `true`, but defaults to the value of `core.commitGraph`, which in turn defaults to true. We, however, always want to use the commitgraph, under the condition that we can use enough threads *and* actually have a commit-graph.

Now, one would think that `with_commit_graph` is enough to get what we want, but since `use_commit_graph` generally defaults to `true` under the hood it would make another attempt to open the commit-graph, even though there might not be one. This leads to the commit-graph detection to be run twice in the case there is none. Thus we explicitly set our verdict in `use_commit_graph` to avoid that kind of work.

Does it matter, you might ask? Probably not on today's SSDs, a few `stat` calls don't matter, so if you want, you can remove the `use_commit_graph` call. By default, I want an API that allows to avoid any waste, which is why it's implemented that way.

----

More notes related to the overcommit of threads

There isn't really a such a thing like exhausting threads, but indeed this implementation will overcommit a little bit for N threads:

* N threads for author computation (author)
* 1 thread for commit-iteration and author aggregation (aggregator)
* 1 threads for churn computation (churn)

The aggregator thread has to wait for (author) threads and won't be busy doing its work - thanks to the commit-graph these walks are very fast. It won't actually consume much CPU.

The churn thread is the one that runs just as hot if not hotter than the author threads, and I intend to let these contend a little as typically a little bit of over-commit eeks out a little more performance in practice. This is due to IO still playing a role here and once a thread is blocked, it's good to have a choice of more threads to run. Further, the `churn` thread is operating on a best-effort basis so to me its fine to drown it out a little, if the OS decides to do that, but in turn it gets a little additional time while the author threads are joined, so it should even out.

Fearless concurrency is fun :).

* explicitly call use_commit_graph and with_commit_graph

---------

Co-authored-by: o2sh <[email protected]>

Author: Byron

.github/workflows/ci.yml

@@ -52,6 +52,9 @@ jobs:
       - name: Run onefetch
         run: cargo run
 
+      - name: Run onefetch (with commitgraph)
+        run: git commit-graph write --no-progress --reachable && cargo run
+
   formatting:
     runs-on: ubuntu-latest
     steps:

Cargo.lock

@@ -23,10 +23,11 @@ byte-unit = "4.0.19"
 bytecount = "0.6.3"
 clap = { version = "4.3.1", features = ["derive"] }
 clap_complete = "4.3.1"
-gix-features-for-configuration-only = { package = "gix-features", version = "0.29.0", features = [
+crossbeam-channel = "0.5.8"
+gix-features-for-configuration-only = { package = "gix-features", version = "0.30.0", features = [
     "zlib-ng",
 ] }
-gix = { version = "0.45.1", default-features = false, features = [
+gix = { version = "0.46.0", default-features = false, features = [
     "max-performance-safe",
 ] }
 git2 = { version = "0.17.2", default-features = false }
@@ -49,6 +50,7 @@ time-humanize = { version = "0.1.3", features = ["time"] }
 tokei = "12.1.2"
 typetag = "0.2"
 yaml-rust = "0.4.5"
+parking_lot = "0.12"
 
 [dev-dependencies]
 criterion = "0.4.0"

Cargo.toml

@@ -1,6 +1,7 @@
+use super::git::metrics::GitMetrics;
 use crate::{
     cli::NumberSeparator,
-    info::{format_number, utils::git::CommitMetrics, utils::info_field::InfoField},
+    info::{format_number, utils::info_field::InfoField},
 };
 use serde::Serialize;
 use std::fmt::Write;
@@ -65,8 +66,8 @@ pub struct AuthorsInfo {
 }
 
 impl AuthorsInfo {
-    pub fn new(commit_metrics: &CommitMetrics) -> Self {
-        let authors = commit_metrics.authors_to_display.clone();
+    pub fn new(git_metrics: &GitMetrics) -> Self {
+        let authors = git_metrics.authors_to_display.clone();
         Self { authors }
     }
 }

src/info/author.rs

@@ -1,4 +1,4 @@
-use super::utils::{git::CommitMetrics, info_field::InfoField};
+use super::{git::metrics::GitMetrics, utils::info_field::InfoField};
 use crate::{cli::NumberSeparator, info::format_number};
 use serde::Serialize;
 use std::fmt::Write;
@@ -43,11 +43,11 @@ pub struct ChurnInfo {
     pub churn_pool_size: usize,
 }
 impl ChurnInfo {
-    pub fn new(commit_metrics: &CommitMetrics) -> Self {
-        let file_churns = commit_metrics.file_churns_to_display.clone();
+    pub fn new(git_metrics: &GitMetrics) -> Self {
+        let file_churns = git_metrics.file_churns_to_display.clone();
         Self {
             file_churns,
-            churn_pool_size: commit_metrics.churn_pool_size,
+            churn_pool_size: git_metrics.churn_pool_size,
         }
     }
 }

src/info/churn.rs

@@ -1,9 +1,9 @@
-use serde::Serialize;
-
+use super::git::metrics::GitMetrics;
 use crate::{
     cli::NumberSeparator,
-    info::{format_number, utils::git::CommitMetrics, utils::info_field::InfoField},
+    info::{format_number, utils::info_field::InfoField},
 };
+use serde::Serialize;
 
 #[derive(Serialize)]
 #[serde(rename_all = "camelCase")]
@@ -15,10 +15,14 @@ pub struct CommitsInfo {
 }
 
 impl CommitsInfo {
-    pub fn new(commit_metrics: &CommitMetrics, number_separator: NumberSeparator) -> Self {
+    pub fn new(
+        git_metrics: &GitMetrics,
+        is_shallow: bool,
+        number_separator: NumberSeparator,
+    ) -> Self {
         Self {
-            number_of_commits: commit_metrics.total_number_of_commits,
-            is_shallow: commit_metrics.is_shallow,
+            number_of_commits: git_metrics.total_number_of_commits,
+            is_shallow,
             number_separator,
         }
     }