In AI, a token is a small chunk of text, such as a word, part of a word or a punctuation mark, that a language model reads and writes. Models don’t see letters or whole sentences; they split everything into tokens first. This matters because tokens decide how much text a model can handle at once, how fast it replies and, on paid plans, how much each request costs.
Key Takeaways
- A token can be a whole word, part of a word, a space or a symbol. Long or rare words are usually split into several pieces.
- OpenAI’s rule of thumb for English is that one token is about four characters, or roughly three-quarters of a word.
- Google’s Gemini documentation says 100 tokens equal about 60–80 English words.
- Everything you send counts, including instructions, files and images. The model’s reply counts as well.
- Different models split text in different ways, so the same prompt can use a different number of tokens on each one.
What Is a Token in AI?
Think of tokens as the building blocks a model uses instead of letters. Before a model can work with your prompt, a tool called a tokeniser cuts the text into pieces. Each piece gets a number, and the model works with those numbers.
A short, common word such as “the” is usually one token. A longer word such as “unbelievable” may break into two or three. Spaces and punctuation can count too. So the token count of a sentence is almost always higher than its word count.
The word also has other meanings, which can cause confusion. In crypto, a token is a digital asset. In web design, a design token is a stored style value, such as a colour. This page covers the AI meaning only.
How Tokens Work
Every request to a language model follows the same path. First, your text is split into tokens. Next, the model reads those tokens and predicts the next token, one at a time. Finally, the model turns the output tokens back into words you can read.
How Text Becomes Tokens
OpenAI gives some handy rules of thumb for English. One token is about four characters, or about three-quarters of a word. That means 100 tokens come to roughly 75 words, and one paragraph is about 100 tokens. For example, OpenAI notes that the quote “You miss 100% of the shots you don’t take” is 11 tokens.
Google’s figures are similar. It says a Gemini token is about four characters, and 100 tokens equal about 60–80 English words. Google also explains that long words get broken into several tokens.
Why Other Languages Can Cost More
English tends to be efficient, but other languages can need more tokens for the same meaning. OpenAI gives the Spanish phrase “Cómo estás” as an example: it is 10 characters, yet it uses 5 tokens. As a result, the same message can cost more to process in some languages.
Images, Audio and Files
Text isn’t the only thing that turns into tokens. According to Google, an image up to 384 pixels in both dimensions counts as 258 tokens, and audio uses 32 tokens per second. Anthropic’s documentation adds that system prompts, tool definitions, images and PDFs all count towards the total.
Input, Output and Thinking
Providers usually track two sides of every request. Input tokens are everything you send, while output tokens are everything the model writes back. Some models also “think” before they answer. Anthropic explains that it bills these thinking tokens as output tokens, and they count towards the model’s limits. In other words, a model that reasons at length can use far more of each token budget than its short final answer suggests.
All of these tokens share one space, called the context window. Once a long chat or a large file fills it, the model can’t take in any more. That’s why token counts matter even when cost isn’t a concern.
Token Examples
- A product description: You ask a model to write a 150-word description. Using OpenAI’s rule that 100 tokens is about 75 words, the reply alone is roughly 200 tokens. Your prompt adds more.
- A blog brief: OpenAI estimates that 1,500 words is about 2,048 tokens. So if you paste a long article and ask for a summary, the input costs far more than the short output.
- A translated ad: You run the same ad copy through a model in English and Spanish. The Spanish version may use more of each token budget, so it can be slower and dearer.
Token Rules of Thumb by Provider
Each provider publishes its own estimates. Here is how they compare:
| Provider | Rough size of one token | Other notes |
|---|---|---|
| OpenAI | About 4 characters, or ¾ of a word in English | 100 tokens ≈ 75 words; offers a free Tokenizer tool |
| Google (Gemini) | About 4 characters | 100 tokens ≈ 60–80 English words; images and audio also use tokens |
| Anthropic (Claude) | Varies by model | Free token counting tool; counts are an estimate and differ between tokenisers |
In short, the rules are close, but they are only estimates. Always check with the tool for the model you actually use.
How to Count and Manage Tokens
- Estimate first. Divide your word count by 0.75 for a rough English token count.
- Use the provider’s tool. OpenAI has a Tokenizer page. Google and Anthropic offer token-counting calls in their developer tools.
- Count the hidden parts. Include system instructions, examples, files and images, not just your question.
- Set an output limit. Most tools let you cap the length of the reply, which also caps its cost.
- Check usage after each run. Google and Anthropic both report input and output usage in every response, so you can track real numbers.
- Trim and reuse. Remove text the model doesn’t need, and reuse a tight, tested prompt rather than pasting long notes each time.
Best Practices
- Be concise, not vague. Short prompts save tokens, but you still need enough detail for a good answer.
- Recount when you switch models. Anthropic advises counting with the model you plan to use, because tokenisers differ.
- Watch both sides. Google notes that the cost of a call depends partly on the number of input and output tokens.
- Plan for long documents. Long inputs fill a model’s context window fast, so split very large files or summarise them first.
Common Mistakes to Avoid
Beginners often assume one word equals one token, which leads to low estimates. Many also forget that pasted files, images and earlier chat messages all add up. Another common slip is to ignore the length of the reply. Finally, some teams compare costs between models without checking how each one splits text. Each mistake can make a project slower or more expensive than planned.
Where You’ll Use Tokens at Work
Token use comes up wherever teams build or run AI tools. For example, remote job listings mention:
- “Hands-on experience with LLM APIs from the main providers, cost and token optimisation” (AI Tooling Engineer).
- Monitoring “output quality, hallucinations, behavioral or model drift, failures, latency, token consumption, and cost” (Senior AI Platform Engineer).
- “Familiarity with modern LLM application concepts, including prompt engineering, context windows, token usage” (Senior Forward Deployed AI Engineer).
These are technical roles, but the same idea matters in marketing. When you use AI to draft emails, ads or blog posts at scale, knowing how each token adds up helps you keep tools fast and budgets under control.
Learn It in Any DMSI Track
AI fundamentals, including strategy and planning, are part of every track. The Full Stack (all four tracks) option covers all ten courses, so you use AI across social media, Google Ads, email and SEO. You also work on a U.S. Project Internship from day one, with unlimited 1-on-1 mentoring.
Next, read how to write your own ChatGPT prompts. You can also learn how a large language model uses tokens to predict text.
Frequently Asked Questions
What is a token in ChatGPT?
It is a chunk of text that the model reads or writes. In English, OpenAI says one token is about four characters, or roughly three-quarters of a word.
How many words is 1,000 tokens?
Using OpenAI’s rule of 100 tokens to about 75 words, 1,000 tokens is roughly 750 English words. Google’s estimate is a little lower, at 60–80 words per 100 tokens.
Do images use tokens?
Yes. For example, Google says a small image of up to 384 by 384 pixels counts as 258 tokens in Gemini.
Why do tokens matter for cost?
Paid AI APIs usually bill by usage. Google, for instance, says the cost of a call depends partly on the number of input and output tokens. So longer prompts and longer replies both tend to cost more.
Is a token the same in every model?
No. Each model family uses its own tokeniser, so the same text can produce a different count. Always check with the tool for the model you use.
Sources: OpenAI Help Center: What are tokens and how to count them; Google AI for Developers: Understand and count tokens; Claude Docs: Token counting; Claude Docs: Context windows; remote job listings on Himalayas.