/orc tools
Products

Orc Tools · Project

Orc Tokenizer

See exactly how GPT tokenizers split your prompt, without sending it anywhere

Liveorc-tokenizer

Open Orc Tokenizer

Orc Tokenizer showing a prompt split into highlighted token bands with IDs and byte counts

How it works

Orc Tokenizer is a browser-based token inspector: paste a prompt, choose a GPT tokenizer preset, and inspect the resulting token boundaries, token IDs, and UTF-8 bytes. Encoding runs entirely on-device using the encoder data shipped by @dqbd/tiktoken, so prompt text is never submitted to an API or stored by the app. It deliberately excludes Claude and Gemini presets rather than approximate them, since their production tokenizers aren't available for reliable local visualization.

Specifics

  • Three tokenizer presets: o200k_base (GPT-5/4o), cl100k_base (GPT-4/3.5), r50k_base (GPT-2)
  • Shows token boundaries, token IDs, and hexadecimal UTF-8 bytes per token
  • Token, character, and UTF-8 byte counts, plus characters-per-token
  • Incremental rendering for long prompts without hiding the full token count
  • No API keys, network calls, or environment variables required to run