Skip to main content
CrewAI is a Python framework for building crews of role-playing agents that collaborate on tasks. DeepInfra is one of its built-in providers: prefix a model id with deepinfra/ and CrewAI sends the requests through its native OpenAI client, reading your key from DEEPINFRA_API_KEY, with no base URL to configure and no LiteLLM dependency. Tool calling, streaming, JSON-schema structured outputs and token usage reporting all work over this path, with any chat model from our catalog.

Installation

The DeepInfra provider shipped in crewai 1.15.27 and needs no extras. Get your API key from the Dashboard and export it as DEEPINFRA_API_KEY. It is the same key you use for inference, embeddings and Sandboxes.

Configuration

Pass the model id with the deepinfra/ prefix. DeepInfra ids already have the form org/model, so a full reference has three segments; everything after the prefix reaches the API unchanged, so use the exact id from the catalog.
Pass the llm object to every Agent that should run on DeepInfra; different agents in one crew can use different models. Because the whole configuration fits in the model string, it also works wherever CrewAI takes a model name instead of an LLM object, including agents.yaml:
To route requests through a proxy or gateway, set DEEPINFRA_BASE_URL; passing base_url and api_key to LLM directly works as well.

Build a crew

The example below gives an agent a tool and runs a single task. The agent calls the tool through OpenAI-style function calling, receives the result, and writes the answer.
Example output:
Token counts come from DeepInfra’s usage object on each response, so result.token_usage matches what you are billed for. cached_prompt_tokens shows DeepInfra’s prompt caching kicking in on the second request of the run.

Knowledge and memory with DeepInfra embeddings

CrewAI’s knowledge sources and memory store text as embeddings. Its openai embedder accepts an api_base, so point it at DeepInfra and pick an embedding model. The same embedder dictionary works on Crew(embedder=...), on Agent(embedder=...) and in memory configuration.
Example output:
Keep the same embedding model for the life of a knowledge base or memory store; vectors from different models are not comparable.

Run agent code in a Sandbox

Agents that write code need somewhere safe to run it. DeepInfra Sandboxes are isolated microVMs created with one API call, and the deepinfra Python SDK wraps them. The tool below runs Python in a fresh sandbox per call and returns the output to the agent.
Add DeepInfraSandboxPython() to an agent’s tools list like any other tool. A nano sandbox is running within about half a second of the create call, and the context manager terminates it when the call returns, so you pay only for the seconds the code runs. Files the agent needs to keep between calls belong in /workspace; to carry state across calls, create one Sandbox in the tool’s constructor and reuse it instead of the context manager. Note the account limit of 25 active sandboxes when many agents run in parallel.

Choosing a model

The agent loop depends on reliable tool calling. The models below are verified with CrewAI’s function-calling path on DeepInfra; prefix any of them with deepinfra/ when you pass it to LLM. Context lengths are from the model catalog, which is public and also carries current pricing.
To compare other models, list the catalog with context windows and pricing via /v1/openai/models?filter=with_meta&sort_by=crewai. Each entry’s metadata block has context_length, max_tokens, and per-million-token pricing; the id is what you pass as model, after the deepinfra/ prefix.

Tips

  • Context window. CrewAI looks model ids up in a table of OpenAI models to size its context management, and assumes 8,192 tokens for ids it does not know, DeepInfra models included. With the default respect_context_window=True, an agent on a DeepInfra model starts summarising its conversation at roughly 7,000 tokens even when the model supports far more. Set respect_context_window=False on agents that need the full window; CrewAI then sends the full history and surfaces the provider error if a prompt ever exceeds the model’s real limit.
  • Model id format. The part after deepinfra/ must be org/model. A reference without the organisation, such as deepinfra/DeepSeek-V4-Flash-0731, raises a ValueError when the LLM is constructed rather than failing on the first request.
  • Older CrewAI versions. Before crewai 1.15.27 the deepinfra/ prefix routed through LiteLLM and needed pip install "crewai[litellm]". If you cannot upgrade, create the LLM with custom_openai=True, base_url="https://api.deepinfra.com/v1/openai" and api_key=os.environ["DEEPINFRA_API_KEY"] instead; that form uses CrewAI’s native OpenAI client and still works on current versions. On current versions, is_litellm=True keeps the LiteLLM path for those who need it.
  • Structured outputs. Pass a Pydantic model as response_format on the LLM or as output_pydantic on a Task. CrewAI sends it as a strict JSON schema, which DeepInfra honours.
  • Streaming. Set stream=True on the LLM to receive tokens as they are generated. Usage is still reported on the final chunk.

Learn more

CrewAI LLM docs

Every LLM parameter, the DeepInfra provider section, and the YAML configuration format.

Chat Completions

DeepInfra’s OpenAI-compatible endpoint, including Tool Calling and Structured Outputs.

Embeddings

Embedding models for knowledge bases and memory.

Sandboxes

Isolated microVMs for running agent-written code.
Tested with crewai 1.15.27 and deepinfra 0.3.0.