AI Tokens Are Like Credit: How Tokenizers Work, Token Input and Output, and Why Subscriptions Also Have Limits

A token is the smallest unit that an AI model (LLM) uses to read and write text, as well as the unit that the AI ​​provider uses to charge you. The easiest way to understand it: tokens are like credit. Every time you ask the AI, there are tokens used to read your question, and there are tokens used to write the answer.

The difference is that with telephone credits which are calculated per minute or per SMS, tokens are calculated per word fragment. This article discusses what tokens are, how sentences are converted into numbers, where chunking, vectors, and embedding stand, what exactly you're paying for, and why even monthly AI subscriptions still have usage limits.

Quick summary: what do you need to know?

What are tokens?

AI models do not read letters like humans. Before processing, the text is broken down into chunks that are already known to the model. These snippets can be as short as one character or as long as one word, depending on the language and context. Spaces, punctuation, and word fragments all count.

We tried the sentence "I want to know the earth is flat or the earth is round?" with tokenizer o200k_base (the latest OpenAI models are used). As a result, 42 characters are split into 11 tokens:

There are three interesting things from this example. First, the word "earth" appears twice with the same ID, because the tokenizer always maps the same chunk to the same number. Second, the words "flat" and "round" are broken into two pieces, because the words do not exist in their entirety in the tokenizer dictionary. Third, spaces usually come before words, so "want" and "want" can have different IDs.

Tokens are not the same as characters or words

For English text, OpenAI provides a rough benchmark: 1 token is roughly 4 characters, or about ¾ of a word. Other languages ​​may be different. From our test with the same tokenizer, the sentence "How do I check shipping costs from Surabaya to Jakarta?" (62 characters) to 15 tokens, while the English version (58 characters) is only 13 tokens. Words like “How”, “cost”, and “ship” are broken down into chunks.

Each model also has its own tokenizer. The same text can generate different numbers of tokens in different models. This means that a cheaper price per 1 million tokens does not necessarily mean a cheaper total cost.

How is your question processed until it becomes an answer?

1. Tokenizer. Your sentence is broken down into tokens and converted into a string of numeric IDs.
2. Reading model. Each ID is converted into a vector (a long string of numbers representing meaning), then processed along with all the context: system instructions, chat history, and your questions. All of these are counted as input tokens.
3. Writing model. The model does not immediately output one paragraph. It calculates the probabilities for all possible tokens that might appear next, chooses one, adds it to the context, then repeats until the answer is complete. Each token generated counts as an output token.
4. Detokenizer. The row of model IDs is converted back into text that you can read. This process is often called decode or detokenization.

So where are chunking, vectors and embedding?

This term is often mixed with the process above, even though the position is different. Regular chat does not go through data searches: the model answers from training knowledge stored in the model weights, rather than searching the database every time it is asked.

Chunking, embedding, and similarity search are only used when you want AI to answer based on your own documents, for example store FAQs, product catalogs, or office SOPs. This technique is called RAG (Retrieval-Augmented Generation).

1. Chunking. Documents are cut into small chunks, for example per paragraph or per few hundred tokens.
2. Embedding. Each slice is converted into a vector by an embedding model, for example Gemini Embedding or Qwen3 Embedding, and then stored in a vector database. This step is done once at the beginning, not every time you ask.
3. Search. When there is a question, the question is also converted into a vector with the same embedding model, then the most similar pieces are looked for. Usually the top 3 to 5 cuts (top-k) are taken.
4. Compiled LLM. The piece is attached to the prompt along with the question, then the LLM composes the answer.

The consequence for costs: embedding models have their own rates (usually much cheaper than LLM), and each piece of document pasted into the prompt adds to the LLM token input.

What are we actually paying for?

This is the part that is often misunderstood. The tokenizer and detokenizer process is light, it can even be run on a regular laptop in a matter of milliseconds. What's expensive is the model work in the middle: reading each token input and generating each token output on the GPU server. Tokens are used as a unit of measurement because the amount is proportional to the workload, similar to credit which is a unit of measurement for network usage.

Because of this, API providers charge input and output separately, usually per 1 million tokens. Basic formula:

biaya = (input token × tarif input) + (output token × tarif output)

Some rules that apply to almost all providers:

• Output is more expensive than input, generally 2 to 5 times. Input is read all at once in parallel, while output must be created one token at a time.
• Think tokens are also charged. In the reasoning model, internal thinking steps that do not appear as answers are still counted as output tokens. Even short answers can use lots of tokens.
• Cached input is cheaper. Repeatedly used portions of the same prompt, such as long system instructions, can be billed at a lower rate via prompt caching.
• Cheaper batches. Many providers give a discount of around 50% for requests that do not require an immediate response.

Example of calculating tokens and fees

The following Python script uses library tiktoken to calculate tokens, then estimate costs. The rates below are just examples; always check the official price page for the model you are using.

import tiktoken  # pip install tiktoken

enc = tiktoken.get_encoding("o200k_base")
teks = "Aku ingin tahu bumi datar atau bumi bulat?"
ids = enc.encode(teks)
print(len(teks), "karakter ->", len(ids), "token")
print([enc.decode([i]) for i in ids])

# Estimasi biaya (tarif contoh, dalam USD per 1 juta token)
TARIF_INPUT, TARIF_OUTPUT = 2.00, 10.00
input_tok, output_tok = 1_200, 400
biaya = input_tok / 1e6 * TARIF_INPUT + output_tok / 1e6 * TARIF_OUTPUT
print(f"Estimasi: ${biaya:.4f} per permintaan, ${biaya * 10_000:.2f} untuk 10.000 permintaan")

The result:

42 karakter -> 11 token
['Aku', ' ingin', ' tahu', ' bumi', ' dat', 'ar', ' atau', ' bumi', ' bul', 'at', '?']
Estimasi: $0.0064 per permintaan, $64.00 untuk 10.000 permintaan

Note that the 400 output tokens in this example account for $0.004, which is greater than the 1,200 input tokens which is only $0.0024. That's why limiting the length of answers is often more economical than trimming questions.

Why do long chats become more wasteful?

The model does not "remember" conversations between messages. To keep answers connected, the chat application resends the entire conversation history every time you send a new message. All that history is calculated again as input tokens.

In the simulation above, six short questions in one chat used 11,100 input tokens, three times more than six separate new chats. The longer the conversation and the more files attached, the faster the "credit" is drained.

What about AI with a monthly subscription?

Monthly subscriptions are like "unlimited" internet packages that have reasonable usage limits. You pay a fixed price per month, but behind the scenes the provider still counts the tokens you use, because their server costs are also calculated from that.

The form of the limit is different for each service. Some set the number of messages per hour, some have daily or weekly quotas, and some give different quotas for smarter models. The pattern is the same: the limit runs out faster if you use large models, send long conversations, attach a lot of files, or use deep thinking mode. One long conversation with lots of attachments can use up the same quota as dozens of short questions.

The difference with API: in a subscription, you will not be charged more than the package price, but access is temporarily limited when the quota runs out. In API, access is not limited as long as the balance is sufficient, but the billing increases according to usage.

Tips for saving tokens

1. Start a new chat for a new topic. Irrelevant history only adds token input.
2. Ask for concise answers. Token output is the most expensive part. Write "answer maximum 5 points" or set the output limit in the API.
3. Send only the parts needed. Instead of pasting a 50-page document, paste the relevant sections or use RAG.
4. Use the model according to the task. Classification, short summaries, and data extraction are often sufficient with much cheaper small models.
5. Take advantage of prompt caching for long, repetitive system instructions.
6. Measure before choosing a model. Calculate tokens for your real task examples, because each model's tokenizer is different.

Frequently asked questions

Is it the tokenizer process that gets paid?

No. Tokenizer is just a text to number cutting and translating tool, the process is very light. What you pay for is the model's work to process input tokens and produce output tokens. Tokens become the unit of account, just as credit becomes the unit of network usage.

Does AI always look for similar data before answering?

No. Chat usually answers directly from the knowledge that has been stored in the model during training. Searching for the 3 to 5 most similar data only occurs in the RAG system, namely when the application deliberately connects AI to certain documents or databases.

Why is Indonesian more expensive than English?

Most tokenizer models are trained with predominantly English data, so English words are more often stored intact. Indonesian words, especially those with affixes, are more often broken down into several tokens. The difference isn't always huge, but it's noticeable at high volumes.

One million tokens is roughly how much?

By the English benchmark (1 token ≈ ¾ word), one million tokens is equivalent to about 750,000 words, or several thick books. For Indonesian, the number of words is slightly less.

Do images and files also count as tokens?

Yes. Images, PDFs, and other files are also converted into tokens (or calculated token equivalents) before model processing. The calculation method varies by provider, so large attachments can quickly use up your quota.

If you want to connect Claude or another AI agent to Indonesian data, also read the guide to connecting APIPedia to Claude via MCP. Every tool call there also adds tokens, so the saving tips above still apply.

Source

• OpenAI Help Center: What are tokens and how to count them?
• GitHub: openai/tiktoken, BPE tokenizer for OpenAI models
• BenchLM: LLM Token Pricing, how input and output prices work
• CloudZero: LLM API Pricing, rate comparison and batch discounts and caching
• arXiv: Tokenizations for Austronesian Language Models, a study of language tokenization in Indonesia