Skip to content

Fix byte-level BPE tokenizers reusing SentencePiece classes (Llama 3 …#46686

Closed
kaixuanliu wants to merge 1 commit into
huggingface:mainfrom
kaixuanliu:tokenizer-fix
Closed

Fix byte-level BPE tokenizers reusing SentencePiece classes (Llama 3 …#46686
kaixuanliu wants to merge 1 commit into
huggingface:mainfrom
kaixuanliu:tokenizer-fix

Fix byte-level BPE tokenizers reusing SentencePiece classes (Llama 3 …

1364ebd
Select commit
Loading
Failed to load commit list.
CircleCI Checks / setup_and_quality succeeded Jun 16, 2026 in 6m 31s

Workflow: setup_and_quality

View CircleCI Workflow

View more details on CircleCI Checks