#

openai-tokenizer

Here are 3 public repositories matching this topic...

daulet / tokenizers

Go, Wasm bindings for HF Tokenizers and Tiktoken

go tokenizer wasm huggingface-tokenizers openai-tokenizer

Updated Mar 30, 2026
Rust

ai-token-calculator

soulteary / ai-token-calculator

Self-hosted browser tool for counting and inspecting text tokens, with a Chinese UI adapted from the OpenAI Tokenizer.

docker golang tokenizer self-hosted openai token-counter openai-tokenizer token-calc

Updated May 13, 2024
Go

flexchar / tiktoken-counter

Tiktoken-counter as standalone API

docker tiktoken token-counter openai-tokenizer

Updated Nov 28, 2025
Python

Add this topic to your repo

To associate your repository with the openai-tokenizer topic, visit your repo's landing page and select "manage topics."