Skip to content

Releases: niieani/gpt-tokenizer

3.4.0

07 Nov 20:15
7f880f4

Choose a tag to compare

3.4.0 (2025-11-07)

Features

  • add function calling token counting (#83) (7f880f4)

3.3.0

07 Nov 00:11
59422fd

Choose a tag to compare

3.3.0 (2025-11-07)

Features

  • correct split regexp for o200 and add harmony format support (59422fd), closes #82 #78

3.2.0

09 Oct 23:12
dcc8783

Choose a tag to compare

3.2.0 (2025-10-09)

Features

  • support encode options in isWithinTokenLimit (#80) (dcc8783)

3.1.0

09 Oct 22:47

Choose a tag to compare

3.1.0 (2025-10-09)

Bug Fixes

  • codegen: guard BPE generation against empty output (c46019a)
  • place generated headers after lint directives (70bed74)

Features

  • add new models (gpt-5*) and update pricing data (135a851)

3.0.1

13 Jun 05:04
f77db29

Choose a tag to compare

3.0.1 (2025-06-13)

Bug Fixes

  • add o3-pro and update pricing (52a3b3c)

3.0.0

07 Jun 22:51
1d1d76d

Choose a tag to compare

3.0.0 (2025-06-07)

Features

BREAKING CHANGES

  • changes the default encoding to o200k_base as that is what most modern models use now

2.9.0

05 Mar 02:40
e2506c2

Choose a tag to compare

2.9.0 (2025-03-05)

Features

  • add new models and update pricing (e2506c2)
  • implement 'estimateCost' (4124587)

2.8.1

09 Dec 10:11
0f70122

Choose a tag to compare

2.8.1 (2024-12-09)

Bug Fixes

2.8.0

09 Dec 09:43
15d13b1

Choose a tag to compare

2.8.0 (2024-12-09)

Features

Performance Improvements

  • optimize token counting (c3e533c)

2.7.0

28 Nov 01:50
2d4146a

Choose a tag to compare

2.7.0 (2024-11-28)

Features

  • implement 'countTokens' function (2d4146a), closes #67