🔤
Tokenizer Playground
Visualize how LLM tokenizers break text into tokens. See BPE merges step-by-step and compare token counts across models.
Input Text
Tokens40
Byte Pair Encoding with merge rulesHello,·world!·This·is·a·simple·test·of·the·tokenizer.
Estimated Token Counts by Model
GPT-4o~4.0 c/t
14
GPT-4~3.7 c/t
15
Claude 3.5~3.5 c/t
16
Llama 3~3.8 c/t
14
Statistics
Characters
53
Words
10
Lines
1
Tokens
40
Chars / Token
1.32
API Pricing Reference
| Model | Est. Tokens | Input $/1M | Output $/1M |
|---|---|---|---|
| GPT-4o | 14 | $2.50 | $10.00 |
| GPT-4 | 15 | $30.00 | $60.00 |
| Claude 3.5 | 16 | $3.00 | $15.00 |
| Llama 3 | 14 | Free | Free |
Uses a simplified BPE implementation for demonstration. Token counts for specific models are heuristic estimates. All processing runs entirely in your browser — nothing is sent to a server.