Package note
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Gpt-tokenizer is a pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models. Licensed under MIT, it is popular in the JavaScript ecosystem with 694.8K weekly downloads.
Gpt-tokenizer has an good overall health score of 65/100. Its strongest dimension is security (100/100), while maintenance (40/100) is the area with most room for improvement. This package is in good shape overall.
The best package for every category — one PDF.
Join 500+ devs. No spam.