Skip to main content
POST
/
v1
/
tokenize
Tokenize
curl --request POST \
  --url https://api.deepinfra.com/v1/tokenize \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "prompt": "<string>",
  "messages": [
    {}
  ],
  "return_token_strs": true
}
'
{
  "count": 123,
  "max_model_len": 123,
  "tokens": [
    123
  ],
  "token_strs": [
    "<string>"
  ]
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Headers

​
xi-api-key
string | null

Body

application/json
​
model
string
required

model name

Example:

"my-org/my-dedicated-llm"

​
prompt
string | null

text to tokenize (completion form)

​
messages
Messages · object[] | null

chat messages to tokenize (chat form)

​
return_token_strs
boolean | null

also return the per-token strings (vLLM)

Response

Successful Response

​
count
integer
required

number of tokens

​
max_model_len
integer
required

the model's maximum context length

​
tokens
integer[]
required

token ids

​
token_strs
string[] | null

per-token strings, if requested (vLLM)