deepinfra/ and CrewAI sends the requests through its native OpenAI client, reading your key from DEEPINFRA_API_KEY, with no base URL to configure and no LiteLLM dependency. Tool calling, streaming, JSON-schema structured outputs and token usage reporting all work over this path, with any chat model from our catalog.
Installation
crewai 1.15.27 and needs no extras. Get your API key from the Dashboard and export it as DEEPINFRA_API_KEY. It is the same key you use for inference, embeddings and Sandboxes.
Configuration
Pass the model id with thedeepinfra/ prefix. DeepInfra ids already have the form org/model, so a full reference has three segments; everything after the prefix reaches the API unchanged, so use the exact id from the catalog.
llm object to every Agent that should run on DeepInfra; different agents in one crew can use different models. Because the whole configuration fits in the model string, it also works wherever CrewAI takes a model name instead of an LLM object, including agents.yaml:
DEEPINFRA_BASE_URL; passing base_url and api_key to LLM directly works as well.
Build a crew
The example below gives an agent a tool and runs a single task. The agent calls the tool through OpenAI-style function calling, receives the result, and writes the answer.usage object on each response, so result.token_usage matches what you are billed for. cached_prompt_tokens shows DeepInfra’s prompt caching kicking in on the second request of the run.
Knowledge and memory with DeepInfra embeddings
CrewAI’s knowledge sources and memory store text as embeddings. Itsopenai embedder accepts an api_base, so point it at DeepInfra and pick an embedding model. The same embedder dictionary works on Crew(embedder=...), on Agent(embedder=...) and in memory configuration.
Keep the same embedding model for the life of a knowledge base or memory store; vectors from different models are not comparable.
Run agent code in a Sandbox
Agents that write code need somewhere safe to run it. DeepInfra Sandboxes are isolated microVMs created with one API call, and thedeepinfra Python SDK wraps them. The tool below runs Python in a fresh sandbox per call and returns the output to the agent.
DeepInfraSandboxPython() to an agent’s tools list like any other tool. A nano sandbox is running within about half a second of the create call, and the context manager terminates it when the call returns, so you pay only for the seconds the code runs. Files the agent needs to keep between calls belong in /workspace; to carry state across calls, create one Sandbox in the tool’s constructor and reuse it instead of the context manager. Note the account limit of 25 active sandboxes when many agents run in parallel.
Choosing a model
The agent loop depends on reliable tool calling. The models below are verified with CrewAI’s function-calling path on DeepInfra; prefix any of them withdeepinfra/ when you pass it to LLM. Context lengths are from the model catalog, which is public and also carries current pricing.
To compare other models, list the catalog with context windows and pricing via
/v1/openai/models?filter=with_meta&sort_by=crewai. Each entry’s metadata block has context_length, max_tokens, and per-million-token pricing; the id is what you pass as model, after the deepinfra/ prefix.Tips
- Context window. CrewAI looks model ids up in a table of OpenAI models to size its context management, and assumes 8,192 tokens for ids it does not know, DeepInfra models included. With the default
respect_context_window=True, an agent on a DeepInfra model starts summarising its conversation at roughly 7,000 tokens even when the model supports far more. Setrespect_context_window=Falseon agents that need the full window; CrewAI then sends the full history and surfaces the provider error if a prompt ever exceeds the model’s real limit. - Model id format. The part after
deepinfra/must beorg/model. A reference without the organisation, such asdeepinfra/DeepSeek-V4-Flash-0731, raises aValueErrorwhen theLLMis constructed rather than failing on the first request. - Older CrewAI versions. Before
crewai1.15.27 thedeepinfra/prefix routed through LiteLLM and neededpip install "crewai[litellm]". If you cannot upgrade, create the LLM withcustom_openai=True,base_url="https://api.deepinfra.com/v1/openai"andapi_key=os.environ["DEEPINFRA_API_KEY"]instead; that form uses CrewAI’s native OpenAI client and still works on current versions. On current versions,is_litellm=Truekeeps the LiteLLM path for those who need it. - Structured outputs. Pass a Pydantic model as
response_formaton theLLMor asoutput_pydanticon aTask. CrewAI sends it as a strict JSON schema, which DeepInfra honours. - Streaming. Set
stream=Trueon theLLMto receive tokens as they are generated. Usage is still reported on the final chunk.
Learn more
CrewAI LLM docs
Every
LLM parameter, the DeepInfra provider section, and the YAML configuration format.Chat Completions
DeepInfra’s OpenAI-compatible endpoint, including Tool Calling and Structured Outputs.
Embeddings
Embedding models for knowledge bases and memory.
Sandboxes
Isolated microVMs for running agent-written code.
Tested with
crewai 1.15.27 and deepinfra 0.3.0.