Skip to main content
POST
Openai Chat Completions

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Headers

​
x-deepinfra-source
string | null
​
x-deepinfra-service-tier
string | null

Per-request service tier (priority or flex) for clients that cannot set the service_tier body field. The body field wins when both are present; unrecognized values ride the default tier.

​
xi-api-key
string | null
​
x-api-key
string | null

Body

application/json
​
model
string
required

model name

Example:

"meta-llama/Llama-2-70b-chat-hf"

​
messages
(ChatCompletionToolMessage · object | ChatCompletionAssistantMessage · object | ChatCompletionUserMessage · object | ChatCompletionSystemMessage · object)[]
required

conversation messages: (user,assistant,tool)*,user including one system message anywhere

​
service_tier
enum<string> | null

The service tier used for processing the request. 'priority' processes the request with higher priority (premium rate); 'flex' processes it at lower priority for a discount, served only when spare capacity exists and may be retried/timed out under load. Both apply only to models that support the respective tier. For compatibility, 'auto' is treated as 'priority' and 'standard_only' as 'default'.

Available options:
default,
priority,
flex
​
fail_fast
boolean
default:false

If true, the request is rejected immediately with HTTP 429 when the model has no spare capacity, instead of waiting in the queue. Opt-in; the default (false) keeps standard queueing behavior.

​
models
string[] | null

Ordered list of up to 4 fallback models. The request is attempted on each model in order: when a model rejects it for lack of capacity (HTTP 429 model-busy / flex no-capacity), the next model is tried server-side. The first model that accepts serves the request; the response's model field and billing reflect that model, at that model's pricing. Models before the last are attempted without queueing (as if fail_fast were set); the last model honors the request's own fail_fast value. When models is set, the model field is ignored. Entries must be plain model names (no deploy_id:, custom_hostport, or :revision specifiers); duplicate entries are ignored, keeping the first occurrence.

Required array length: 1 - 4 elements
​
stream
boolean
default:false

whether to stream the output via SSE or return the full response

​
temperature
number
default:1

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic

Required range: 0 <= x <= 2
​
top_p
number
default:1

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

Required range: x <= 1
​
min_p
number
default:0

Float that represents the minimum probability for a token to be considered, relative to the probability of the most likely token. Must be in [0, 1]. Set to 0 to disable this.

Required range: 0 <= x <= 1
​
top_k
integer
default:0

Sample from the best k (number of) tokens. 0 means off

Required range: x >= 0
​
max_tokens
integer | null

The maximum number of tokens to generate in the chat completion.

The total length of input tokens and generated tokens is limited by the model's context length. If explicitly set to None it will be the model's max context length minus input length or 65536, whichever is smaller.

Required range: 0 < x <= 10000000
​
stop

up to 16 sequences where the API will stop generating further tokens

​
stop_token_ids
integer[] | null

Up to 16 token IDs where the API will stop generating further tokens. Merged with the model's built-in stop tokens. Intended for private deployments.

​
n
integer
default:1

number of sequences to return

Required range: 1 <= x <= 4
​
presence_penalty
number
default:0

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

Required range: -2 <= x <= 2
​
frequency_penalty
number
default:0

Positive values penalize new tokens based on how many times they appear in the text so far, increasing the model's likelihood to talk about new topics.

Required range: -2 <= x <= 2
​
tools
FunctionTool · object[] | null

A list of tools the model may call. Currently, only functions are supported as a tool.

​
tool_choice

Controls which (if any) function is called by the model. none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function. required means the model must call a function. defined tool means the model must call that specific tool. none is the default when no functions are present. auto is the default if functions are present.

Available options:
none,
auto,
required
​
response_format
TextResponseFormat · object

The format of the response. Currently, only json is supported.

​
repetition_penalty
number
default:1

Alternative penalty for repetition, but multiplicative instead of additive (> 1 penalize, < 1 encourage)

Required range: 0.01 <= x <= 5
​
user
string | null

A unique identifier representing your end-user, which can help monitor and detect abuse. Avoid sending us any identifying information. We recommend hashing user identifiers.

​
seed
integer | null

Seed for random number generator. If not provided, a random seed is used. Determinism is not guaranteed.

Required range: -9223372036854776000 <= x < 18446744073709552000
​
logprobs
boolean | null

Whether to return log probabilities of the output tokens or not.If true, returns the log probabilities of each output token returned in the content of message.

​
stream_options
StreamOptions · object | null

streaming options

​
reasoning_effort
enum<string> | null

Constrains effort on reasoning for reasoning models. Currently supported values are none, minimal, low, medium, high, xhigh, and max. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Setting to none disables reasoning entirely if the model supports.

Available options:
none,
minimal,
low,
medium,
high,
xhigh,
max
​
reasoning
ChatReasoningSettings · object | null

Reasoning configuration.

​
prompt_cache_key
string | null

A key to identify prompt cache for reuse across requests. If provided, the prompt will be cached and can be reused in subsequent requests with the same key.

​
prompt_cache_options
PromptCacheOptions · object | null

Prompt cache options for this request's prefix, e.g. {"ttl": "1h"}.

​
cache_salt
string | null

Isolates this request's prefix cache: only requests with the same salt reuse each other's cached prefix. Honoured by models served on vLLM; ignored elsewhere.

Minimum string length: 1
​
chat_template_kwargs
Chat Template Kwargs · object | null

Chat template kwargs.

​
media_io_kwargs
MediaIoKwargs · object | null

Per-request media decode options keyed by modality, e.g. {"video": {"num_frames": 16}}. Honoured by video-input models served on vLLM or Dynamo; ignored elsewhere.

​
continue_final_message
boolean | null

If set, the final assistant message is used as a prefix for the model to continue generating from, rather than starting a new turn. Only applicable when the last message in the conversation is an assistant message.

​
ignore_eos
boolean | null

Keep generating until max_tokens instead of stopping at the end-of-sequence token. Only honoured on models tagged with the allow_ignore_eos feature flag; ignored otherwise. Intended for benchmarking, where a fixed output length is needed.

Response

Successful Response