a different internal crate organization to help with build times

[BurntSushi]

While #372 tries to attack the problem of build times with the obvious "make it possible to compile less stuff," this issue promotes a slightly more radical approach. Namely, it is well known that splitting a crate up into multiple crates will help with compile times, presumably because it helps make more effective use of multiple CPUs.

I did the same thing with regex. Although, I didn't do it because of build times. I did it because I wanted to expose lower level primitives outside of a standard regex API (memchr and aho-corasick being examples of that) and I wanted to expose some of the regex engine internals for more flexible use cases (that's regex-syntax and regex-automata). I'd guess that this ended up helping build times, although there's no real comparison point there.

My problem with doing this is: 1) it expands the dependency tree and 2) it introduces more semver boundaries. I am loathe to have more semver boundaries. They are a huge fucking pain in the ass. They make releases and refactoring more annoying. Moreover, you have to be super careful about introducing public dependencies, or else the semver API evolution of one crate is tied to another.

With that said... Jiff doesn't actually have a ton of internal code. It has some. But most of what I'd put into separate crates are already in Jiff's public API.

Anyway, here's what I'm thinking:

• jiff-primitive would have Timestamp, SignedDuration, tz::Offset, civil::{DateTime, Date, Time, ISOWeekDate, Weekday} and maybe some others that I'm thinking about.
• jiff-tz would have the TZif and POSIX time zone parsers. It would have a public dependency on jiff-primitive. Probably TimeZone has to live here too.
• jiff-fmt has what's in jiff::fmt today, and would have public dependencies on jiff-primitive and jiff-tz.
• jiff glues everything together. It still has some non-trivial code in it, like Zoned and Span. Along with TimeZoneDatabase. But otherwise is a facade over the aforementioned crates. Users of Jiff (like users of regex) wouldn't need to worry about the crates above. Jiff's API would stay the same cohesive self. Annoyingly, Jiff would not have public dependencies on any of the above crates.

From my perspective, there are significant challenges with the above blueprint.

• All 4 crates, as they exist inside of Jiff today, would want to use ranged integers. So I would need to make a new jiff-ranged or something that exposes the ranged integer types. I really do not like this idea because the ranged integers are a giant fucking mess IMO. So it's more likely that in order for the above to work, I'd want to rip ranged integers out of Jiff. (This raises the question of what I'd do instead, because we still need to check boundary conditions everywhere.)
• The API surface of these crates is huge, and the thought of writing docs for them and then copying most of those docs verbatim into Jiff proper, and then writing all of the facade wrappers is just... well... it's terrifying. So in order to make this work, I'd probably have to at least settle for the jiff-{primitive,tz,fmt} crates to have light or almost no docs. Basically they'd be "internal only, not intended for public use."
• Changes at the bottom of the crate dependency graph that are needed higher up require an annoying about Cargo interaction: 1) bump version, 2) release, 3) bump version in dependent, 4) release, 5) repeat. I have to do this in ripgrep and it's awful. It's part of the reason why I don't do ripgrep releases often.

This could be done in a piecemeal fashion to get a sense of whether it's worth doing. I think the first thing here would be to start ripping ranged integers out, probably starting with the civil types. Then the other primitives. Once those are free of ranged integers, we could in theory move them out into a separate crate.

[BurntSushi]

Another tact we could take would be a change in API design philosophy. That is, instead of providing a single Jiff crate (even if it's internally split into multiple crates), we could provide a family of crates where it is intended for users to piece them together in the way that they need. This would likely avoid the drudgery of building facades. I generally do not like this style of API design and would rather have everything all in one spot. When things are spread out, I think it hurts discoverability and makes it harder to guide users to the Right Thing. And I think this is especially important in the context of a datetime library.

[BurntSushi]

Ref #369

[hanna-kruppe]

My experience staring at my own projects’ cargo build --timings graph is surely not representative but here’s some observations that may be relevant and which weren’t explicitly mentioned above:

• Splitting into multiple crates can enable more parallelism, but “codegen” (everything after MIR->LLVM IR translation) is already parallelized, so this mostly matters for the rest of the rustc work. I can’t check right now but I believe Jiff compilation today already spends a large fraction of time in codegen even without optimizations. In particular I recall cargo build -j1 of a “hello world with Jiff” program being much slower than without -j1.
• Using more cores for compiling Jiff is only useful if there’s nothing else those cores could be doing, or if dedicating those cores to Jiff instead of other crates shortens the critical path. Both are likely true in projects that depend on Jiff and not much else. But since Jiff has little to no dependencies of its own in most cases, this probably won’t generalize - as soon as you have enough other dependencies that can be compiled independently of Jiff, you can keep your other cores busy with those.
• There’s some easy ways to accidentally negate some or all of the compile time benefits of splitting up crates. Mostly-linear dependency chains between the split up crates, more copies of generic/#[inline]/#[inline(always)] code, one crate or codegen unit taking much longer than the rest and holding up the build almost as much as the monolith crate did before, etc.

So it’s far from clear to me that all the work and overhead you described will actually pay off. There are success stories to learn from (e.g., clap’s split between clap-derive and clap-builder) but also many cases where a single crate out of a multi-crate project ends up dominating compile times for many downstream builds (e.g., cranelift-codegen in the case of Wasmtime, last time I looked into it).

[BurntSushi]

Thanks for that added context! Yeah, the crates I suggested above are likely a linear dependency chain too. :-(

Maybe the path forward here is figuring out how to make codegen faster, perhaps by feeding less codegen to LLVM. Other than the obvious stuff (not making huge functions generic and avoiding inline(always) annotations), I'm not too sure what else to do. I've already tried using fewer generics in places, and in some cases that did have an effect. There's also a perf-inline feature to disable the inline(always) annotations, but that doesn't seem to have much of an effect.

[hanna-kruppe]

Ironically, more liberal use of #[inline] can be beneficial for build times for crates with huge surface area because it may skip a lot of codegen that's never needed by any given consumer. But in the case of Jiff it doesn't currently seem to work out that way. Disabling the perf-inline results in slightly more LLVM IR (221513 vs 201303 lines) but still builds slightly faster on my machine (ca. -0.2s with and without -j1).

Maybe the path forward here is figuring out how to make codegen faster, perhaps by feeding less codegen to LLVM. Other than the obvious stuff (not making huge functions generic and avoiding inline(always) annotations), I'm not too sure what else to do. I've already tried using fewer generics in places, and in some cases that did have an effect.

I'm sure you've heard of it but if you haven't pointed it at Jiff yet, I found cargo-llvm-lines helpful sometimes. From glancing over its output on Jiff, there's some tiny functions related to error handling that contribute significantly from having 100s of instantiations, and a couple that are not very tiny and get instantiated more than once. Since I don't know Jiff's internals at all, I can't say which of these can be reduced with reasonable effort and trade-offs.

Another tool that can help sometimes is rustc's -Zprint-mono-items flag for figuring out what types/constants a given function gets monomorphized for.

[BurntSushi]

Thanks @hanna-kruppe! I was aware of cargo-llvm-lines but had forgotten about it. I hadn't run it on Jiff before.

I opened up #379 with some small experiments.

My inclination here is that there aren't any low hanging fruit (or rather, not some low hanging fruit that is going to help dramatically), and this might just be a matter of going through and ensuring we aren't generating more copies of functions than we should.

One thing that I might try next is looking closer at the printing routines. Several of them are showing up near the top for LLVM. I wonder if me using wtr: W instead of wtr: &mut W is causing unnecessary duplication of some functions.

Haven't had a chance to use -Zprint-mono-items yet, but yeah that seems like it could be quite useful too!

[BurntSushi]

An example of the writer issue is that Jiff, on its own, generates multiple copies of some rather large functions. For example, the function responsible for printing a Span in ISO 8601 format (or even the function that prints in the friendly format). They are generic over a jiff::fmt::Write impl. And Jiff uses two different impls of this trait. One for std::fmt::Formatter and one for String:

https://github.com/BurntSushi/jiff/blob/ca0697735b201085fac11ea46ad616ddba33442b/src/fmt/temporal/mod.rs

jiff/src/span.rs

Lines 3261 to 3263 in ca06977

	temporal::DEFAULT_SPAN_PRINTER
	.print_span(self, StdFmtWrite(f))
	.map_err(|_| core::fmt::Error)

I'm not quite sure how to get this down to one copy. I could make the internal impl accept a &mut dyn jiff::fmt::Write I think, but that's a plausible runtime perf regression. (The functions are so big in the first place because of aggressive inlining because of runtime perf.)

So I'm not sure what's to be done here other than picking over the implementations to try and find cold(er) paths and find a way to put them behind function calls.

[hanna-kruppe]

One general trick to have a &mut dyn Write-style sink without excessive performance cost from dynamic dispatch is to have buffering outside the trait object (BufWriter<&mut dyn Write> in std::io terms). But that's a pain to implement in no_std, and may still regress performance because of the extra data copies and the extra work for tracking when the buffer needs to be flushed.

Perhaps there's a small upper bound on how many bytes the printing of a single Span, Zoned, etc. can produce in a particular format? If so, you could change the printing logic to first write into a sufficiently large byte array on its stack frame, and then push_str the entire output into the sink at once. Only the latter part needs to be monomorphized for each sink type to avoid dynamic dispatch. While this approach still copies the string data once, it has some upsides:

• If the sink is String or something similar, making a single write can reduce data copies by reserving the right capacity up-front (this mostly matters if the string is almost empty before writing to it, including in the ToString case that starts with an empty string).
• For sinks where each write is relatively expensive, such as fmt::Formatter (dynamic dispatch on every write!) or an unbuffered File, it gives you the benefits of buffering without most of the bookkeeping cost.
• For sinks that are already buffered (e.g., io::BufWriter or stdout), it can result in tighter code because every write into the stack buffer is either just a bounds check (-> obviously cold code path) or has no check at all (if you're confident enough to use unsafe for it), as opposed to the repeated bookkeeping for flushing the buffer.
• Owning the buffer and only using it for one piece of data before discarding it means you can populate it in whatever order is most efficient or convenient. For example, binary integer to decimal conversion typically writes digits in reverse order starting from the end of the buffer. Unsure if that's applicable here, but can outweigh the extra memcpy when it does apply.

[BurntSushi]

Yeah Jiff already does that buffer trick for formatting decimal numbers.

For sinks that are already buffered (e.g., io::BufWriter or stdout), it can result in tighter code because every write into the stack buffer is either just a bounds check (-> obviously cold code path) or has no check at all (if you're confident enough to use unsafe for it), as opposed to the repeated bookkeeping for flushing the buffer.

This is one I haven't thought of before! This seems like a potentially huge win.

The friendly duration format max length is probably quite long. Zoned datetimes are too. Probably in excess of 60 bytes off the top of my head. Other cases are smaller, like formatting times. But even formatting a tighter ISO 8601 duration can be quite long because the maximum value of a SignedDuration can be quite large. So the number of digits can really pile up. It feels like it would be excessive to put this all on the stack unfortunately.

Thanks for the other ideas. I'll continue to noodle on this.