Skip to content

fix(font): Apply glyph constraints before thickening and centering before quantizing - #8580

Merged
mitchellh merged 3 commits into
ghostty-org:mainfrom
danielwe:constraints_before_thickening
Oct 3, 2025
Merged

fix(font): Apply glyph constraints before thickening and centering before quantizing#8580
mitchellh merged 3 commits into
ghostty-org:mainfrom
danielwe:constraints_before_thickening

Conversation

@danielwe

@danielwe danielwe commented Sep 10, 2025

Copy link
Copy Markdown
Contributor

In CoreText, when thickening (font smoothing) is enabled or Ghostty is synthesizing a bold face, the glyph bounding box is padded to make sure the thicker glyph can fit. Currently, this happens before applying constraints (scaling and alignment), which makes the size and position of constrained glyphs dependent on font size, font thickening strength, and display DPI.

With this PR, constraints are applied before any other adjustments, and padding is applied directly to the rasterization canvas without modifying any metrics.

For consistency, I also moved constraint application above emboldening in the FreeType code, although under that API, the two operations are orthogonal as far as I can tell.

Secondly, this PR moves glyph centering above bitmap quantization, as centering is generally fractional and will therefore undo the quantizing if done after.

Supersedes #8552.

@danielwe
danielwe requested a review from a team as a code owner September 10, 2025 04:32
@danielwe danielwe changed the title font(fix): Apply glyph constraints before thickening and center before quantizing font(fix): Apply glyph constraints before thickening and centering before quantizing Sep 10, 2025
@danielwe
danielwe force-pushed the constraints_before_thickening branch from 8b19c0f to 6b493c4 Compare September 10, 2025 04:36
@00-kat 00-kat added the font Issue within the font stack (typically src/font) label Sep 11, 2025
@danielwe
danielwe force-pushed the constraints_before_thickening branch from 6b493c4 to efc685c Compare September 11, 2025 03:49
@danielwe danielwe changed the title font(fix): Apply glyph constraints before thickening and centering before quantizing fix(font): Apply glyph constraints before thickening and centering before quantizing Sep 24, 2025
mitchellh added a commit that referenced this pull request Sep 29, 2025
#8563)

> This PR will probably need a rebase on the final outcome of #8550 and
~#8552~ #8580, but I'm putting it out here so folks can begin taking a
look if they want.

This is a rewrite of the code that applies scaling and alignment
constraints. The main intention is to further improve how Nerd Font
icons are matched to the primary font. This PR aligns the calculations
more closely with how the Nerd Font `font-patcher` script works, except
in two cases where we can easily do something unambiguously better (one
because of what's arguably a bug in the script, and one because we do
multi-cell alignment with knowledge of the pixel-rounded cell grid).

A goal of the rewrite is to make the scaling and alignment calculations
as clear and easy to follow as possible.

I'll lead with some screenshots. First the status quo, then this PR.
<img width="505" height="357" alt="Screenshot 2025-09-07 at 17 23 51"
src="https://github.com/user-attachments/assets/8e3ff9fd-3b66-4d54-be38-d54cf3b6cc5b"
/><img width="505" height="357" alt="Screenshot 2025-09-07 at 17 20 39"
src="https://github.com/user-attachments/assets/84fbe076-2e3f-4879-b9b2-91ce86b9ef5f"
/>

Relevant specs: macOS; 1920x1080; Ghostty config:
```ini
font-family = "CommitMono"
font-size = "15"
adjust-cell-height = "+20%"
```

**Points to note**

* Icons are generally larger, making better use of the available space.
* Icons are aligned nearly a pixel lower, better matching the text. This
is because alignment is now calculated from face metrics/bearings, not
the pixel-rounded cell. (See more below.)
* Relative sizes are better matched. Note especially that tall and
narrow icons, like the git branch symbol and icons depicting sheets of
paper, look conspicuously small in the status quo. With this PR, they're
better matched to other icons.
* Look at the letter Z icon I use as prompt character for zsh. It's
_tiny_ in the status quo, but properly sized with this PR. This
demonstrates the most important and clear-cut improvement we make over
`font-patcher`. (See more below.)
* Icons wider than a single cell are now left-aligned rather than
centered across two cells. I think this is preferable and makes better
use of space in most relevant contexts.
- Consider a Neovim bufferline showing the buffer title as a filetype
icon followed by the file name. Padding on the left would be a waste of
space, but having that extra space on the right can improve legibility.
- In listings, such as in the screenshots, columns look tidier when
their left edges are straight rather than ragged.
- This is how `font-patcher` does alignment, and thus what Nerd Font
users and UI designers expect.

**Implementation details**

I won't get too deep in the weeds here; see the code and comments. In
brief:

* `size_horizontal` and `size_vertical` are combined to a single `size`,
which can be `.none, .stretch, .fit, .cover` or `.fit_cover1`. The
latter implements the `pa` rule from `font-patcher`, except it works
better for icons that are small before scaling, like the letter Z prompt
in the screenshots. In short, it preserves aspect ratio while clamping
the size such that the icon `.cover`s at least one cell and `.fit`s
within the available space. See code comments and
ryanoasis/nerd-fonts/pull/1926 for details.
* An alignment mode `.center1` is added, implementing the centering rule
from `font-patcher` that I explained/defended above. In short, we center
the icon _in the first cell_, even it's allowed to span multiple cells.
For icons wider than a single cell, the lower bound that prevents them
from protruding to the left kicks in and turns this into left-alignment.
We keep the regular `.center` rule around for use with emojis, et
cetera.
* Scaling and alignment calculations only use the unrounded face metrics
and bearings. This ensures that pixel rounding of the cell and baseline,
and `adjust-cell-{width,height}`, don't affect scaling or relative
alignment; the icons are always scaled and aligned to the _face_. (The
one place we need to use cell metrics in the calculations is when we use
`cell_width` to obtain the inter-cell padding needed to correctly center
or right-align a glyph across two cells.)
- We can do this with impunity because we're blessed with sprite glyphs
in place of the "icons" that are actually box drawing and block graphics
characters 🙌

**Guide**

The meat of the changes is 100 % in `src/font/face.zig` and
`src/font/nerd_font_codegen.py`. Changes to other files only amount to
a) adding/changing some struct fields to get numbers to where they need
to be (see `src/font/Metrics.zig`), and b) collateral updates to make
otherwise unchanged code and tests work with/take advantage of the
modified structs.

Most files should have a clear and friendly diff. The exception is the
bottom half of `src/font/face.zig`, where the diff is meaningless and
the new code should just be reviewed on its own merits. This is the part
where the `constrain` function is rewritten and refactored. Scarred by
countless hours perusing `font-patcher`, I tried hard to make the math
and logic easy to follow here. I hope I have succeeded 🤞
@mitchellh

Copy link
Copy Markdown
Contributor

Merged some of your other PRs first which caused a conflict, can you please rebase this one.

@danielwe
danielwe force-pushed the constraints_before_thickening branch 2 times, most recently from a6f822e to d11da07 Compare October 1, 2025 17:08
@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

Rebased and ready. Made a tiny change to avoid an unnecessary merge conflict between this #8847.

@qwerasd205 qwerasd205 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you move the code that bails out for glyphs that are too small after all the (relatively) expensive constraint calculations?

Also, as a side note, I'm pretty sure if we're gonna move the FreeType outline emboldening code at all it should be moved before we measure the glyph (additional safety checks may need to be added to make sure that there are outlines to embolden in the first place).

Comment thread src/font/face/coretext.zig Outdated
@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

Why did you move the code that bails out for glyphs that are too small after all the (relatively) expensive constraint calculations?

The idea was (since I was reordering this flow anyway) to make this call based on the actual size of the glyph after it's possibly been scaled and emboldened. In any case, if the constraint is nontrivial it seems unlikely that there's going to be cause for bailing, i.e., we'd probably never actually go through the expensive calculations only to bail right after (except if we've ended up with a font size so small that we bail on every glyph, in which case the user probably has bigger things to worry about than this inefficiency).

But happy to reorder and do this on the unscaled and unpadded rect if that's what you prefer!

@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

I'm pretty sure if we're gonna move the FreeType outline emboldening code at all it should be moved before we measure the glyph

No, the point of this PR is precisely that constraints must be applied to the bbox of the unboldened/unthickened glyph to work correctly.

With Coretext, this is important for everyday usage---you don't want the size of NF icons to depend on the amount of font thickening.

With FreeType, it's perhaps a bit more academic, but it's totally possible for a user to specify a font that does not have a bold variant, but is a Nerd Fonts patched font. In that case, we will be generating synthetically emboldened NF icons (which is kind of dumb, but it is what it is), and an emboldened glyph should not be downscaled to fit in the same bbox as the unboldened glyph (imagine that for letters, it would look like the bold is a smaller font size than non-bold).

@qwerasd205

Copy link
Copy Markdown
Member

Font thickening and emboldening are completely different things.

@qwerasd205

qwerasd205 commented Oct 1, 2025

Copy link
Copy Markdown
Member

I see your argument about bold symbols having the same basic proportions as non-bold ones with synthetic bold, but that's not the effect we'd get if a font did have a bold with symbols that have constraints applied, and it feels odd to me to be inconsistent about that.

Realistically, the symbols I'm more concerned about aren't the ones that are always scaled, but the ones that are simply constrained to cells-- I don't want synthetic bold versions of those symbols to overhang the edge(s) of their cell(s).

@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

the ones that are simply constrained to cells

Which symbols are this? Normally, if a font specifies outlines that go outside the cells, they are just drawn outside the cells, right? There's probably some case I'm forgetting.

@qwerasd205

Copy link
Copy Markdown
Member

// If there's no Nerd Font constraint for this codepoint
// then, if it's a symbol, we constrain it to fit inside
// its cell(s), we don't modify the alignment at all.
.constraint = getConstraint(cp) orelse
if (cellpkg.isSymbol(cp)) .{
.size = .fit,
} else .none,

@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

Thanks, I see.

As an aside, I now realize this won't actually ensure that the symbol is contained within a single cell, only that its outlines aren't wider than one cell. It could still be shifted. For example, Iosevka has a number of (non-NF) symbols that are centered within a 2x advance width (I discovered this yesterday when looking into #8929, though the codepoint discussed there, 25A0, is currently not considered a symbol by isSymbol). Never mind, this is both incorrect and irrelevant.

Also, downscaling without alignment currently just scales down the bearings without any vertical correction, i.e., it pulls the symbol toward the lower left corner. It would probably be better to use the baseline or midline as anchor for vertical scaling, or maybe better, default to center-center alignment when the glyph is rescaled and no alignment is specified. This would also take care of the issue in the previous paragraph. I can submit a separate PR for that, or add to this. #8990

@danielwe

danielwe commented Oct 1, 2025

Copy link
Copy Markdown
Contributor Author

In summary, let me know which changes you want me to push:

  • Bail based on pre-constraint rect instead of post-constraint padded canvas?
  • Measure and constrain post-synthetic bold instead of pre?
  • Re scaling without specified alignment: no change; vertically anchor rescaling at baseline/midline instead of bottom; or default to centered alignment, both vertically and horizontally? Do this in a separate PR or add to this? fix(font): Fix positioning of scaled glyphs that don’t specify alignment #8990

@danielwe
danielwe force-pushed the constraints_before_thickening branch from d11da07 to 2b6f3e8 Compare October 3, 2025 01:23
@danielwe

danielwe commented Oct 3, 2025

Copy link
Copy Markdown
Contributor Author

Alright, I pushed a new version that makes a simpler and more minimal set of changes along the lines you indicated.

  • In FreeType, we measure the possibly emboldened glyph, which is consistent with the rect padding done in CoreText. Thus, both backends apply constraints to the emboldened bbox.
  • We keep the main change: only padding the canvas, not the rect, for font smoothing. Since this padding is always integer (no fractional contribution from emboldening), we keep it separate from frac_{x,y}.

@qwerasd205

Copy link
Copy Markdown
Member

This looks good to me now, I think, but I don't have the energy to do a proper review tonight. Hopefully in the morning I can.

@danielwe

danielwe commented Oct 3, 2025

Copy link
Copy Markdown
Contributor Author

Sounds great!

I'll note that #8990 and #8847 address higher-severity bugs/regressions, so maybe they should take priority if the time window for 1.2.1 is getting tight. That said, this one is probably the quickest and easiest to review.

In Freetype, measure rect after emboldening, so constraints apply to the
true glyph size like in CoreText.

In CoreText, don't let font smoothing affect the rect (only the canvas).
@mitchellh
mitchellh force-pushed the constraints_before_thickening branch from 697b9c7 to f245574 Compare October 3, 2025 19:54
@mitchellh
mitchellh merged commit fd64b83 into ghostty-org:main Oct 3, 2025
10 checks passed
@github-actions github-actions Bot added this to the 1.2.1 milestone Oct 3, 2025
@danielwe
danielwe deleted the constraints_before_thickening branch October 4, 2025 05:35
mitchellh added a commit that referenced this pull request Oct 6, 2025
…fore quantizing (#8580)

In CoreText, when thickening (font smoothing) is enabled or Ghostty is
synthesizing a bold face, the glyph bounding box is padded to make sure
the thicker glyph can fit. Currently, this happens before applying
constraints (scaling and alignment), which makes the size and position
of constrained glyphs dependent on font size, font thickening strength,
and display DPI.

With this PR, constraints are applied before any other adjustments, and
padding is applied directly to the rasterization canvas without
modifying any metrics.

For consistency, I also moved constraint application above emboldening
in the FreeType code, although under that API, the two operations are
orthogonal as far as I can tell.

Secondly, this PR moves glyph centering above bitmap quantization, as
centering is generally fractional and will therefore undo the quantizing
if done after.

Supersedes #8552.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

font Issue within the font stack (typically src/font)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants