Skip to content

Use cached added-token dicts in per-token decode loops#46535

Open
ishan-1010 wants to merge 1 commit into
huggingface:mainfrom
ishan-1010:fix/per-token-added-encoder-46396
Open

Use cached added-token dicts in per-token decode loops#46535
ishan-1010 wants to merge 1 commit into
huggingface:mainfrom
ishan-1010:fix/per-token-added-encoder-46396