Skip to main content
CrewAI is a Python framework for building crews of role-playing agents that collaborate on tasks. Its LLM class ships a native OpenAI client that accepts a custom base URL, so it talks to DeepInfra directly with no LiteLLM dependency. Tool calling, streaming, JSON-schema structured outputs and token usage reporting all work over this path, with any chat model from our catalog.

Installation

Get your API key from the Dashboard and export it as DEEPINFRA_API_KEY. It is the same key you use for inference, embeddings and Sandboxes.

Configuration

Create the LLM with custom_openai=True and DeepInfra’s base URL. The model id is passed through unchanged, so use the exact org/model id from the catalog.
custom_openai=True selects CrewAI’s native OpenAI client and skips the check against OpenAI’s own model list, which is what lets a DeepInfra model id through. base_url is required when you set it. Pass the llm object to every Agent that should run on DeepInfra; different agents in one crew can use different models.
Agents defined in agents.yaml take llm as a plain string, which cannot carry a base URL. In a @CrewBase class, build the LLM in a method decorated with @llm and reference the method name from YAML; CrewAI resolves the name to the object when it loads the agent.

Build a crew

The example below gives an agent a tool and runs a single task. The agent calls the tool through OpenAI-style function calling, receives the result, and writes the answer.
Example output:
Token counts come from DeepInfra’s usage object on each response, so result.token_usage matches what you are billed for.

Knowledge and memory with DeepInfra embeddings

CrewAI’s knowledge sources and memory store text as embeddings. Its openai embedder accepts an api_base, so point it at DeepInfra and pick an embedding model. The same embedder dictionary works on Crew(embedder=...), on Agent(embedder=...) and in memory configuration.
Example output:
Keep the same embedding model for the life of a knowledge base or memory store; vectors from different models are not comparable.

Run agent code in a Sandbox

Agents that write code need somewhere safe to run it. DeepInfra Sandboxes are isolated microVMs created with one API call, and the deepinfra Python SDK wraps them. The tool below runs Python in a fresh sandbox per call and returns the output to the agent.
Add DeepInfraSandboxPython() to an agent’s tools list like any other tool. A nano sandbox is running within about half a second of the create call, and the context manager terminates it when the call returns, so you pay only for the seconds the code runs. Files the agent needs to keep between calls belong in /workspace; to carry state across calls, create one Sandbox in the tool’s constructor and reuse it instead of the context manager. Note the account limit of five active sandboxes when many agents run in parallel.

Choosing a model

The agent loop depends on reliable tool calling. The models below are verified with CrewAI’s function-calling path on DeepInfra; context lengths are from the model catalog, which is public and also carries current pricing.
To compare other models, list the catalog with context windows and pricing via /v1/openai/models?filter=with_meta&sort_by=crewai. Each entry’s metadata block has context_length, max_tokens, and per-million-token pricing; the id is what you pass as model.

Tips

  • Context window. CrewAI looks model ids up in a table of OpenAI models to size its context management, and assumes 8,192 tokens for ids it does not know. With the default respect_context_window=True, an agent on a DeepInfra model starts summarising its conversation at roughly 7,000 tokens even when the model supports far more. Set respect_context_window=False on agents that need the full window; CrewAI then sends the full history and surfaces the provider error if a prompt ever exceeds the model’s real limit.
  • The deepinfra/ prefix. LLM(model="deepinfra/Qwen/Qwen3.5-27B") routes through LiteLLM, which reads DEEPINFRA_API_KEY from the environment. It needs pip install "crewai[litellm]"; without LiteLLM installed, constructing the LLM raises an ImportError. The custom_openai=True form above uses CrewAI’s native client and has no extra dependency.
  • Structured outputs. Pass a Pydantic model as response_format on the LLM or as output_pydantic on a Task. CrewAI sends it as a strict JSON schema, which DeepInfra honours.
  • Streaming. Set stream=True on the LLM to receive tokens as they are generated. Usage is still reported on the final chunk.

Learn more

CrewAI LLM docs

Every LLM parameter, provider routing, and the YAML configuration format.

Chat Completions

DeepInfra’s OpenAI-compatible endpoint, including Tool Calling and Structured Outputs.

Embeddings

Embedding models for knowledge bases and memory.

Sandboxes

Isolated microVMs for running agent-written code.
Tested with crewai 1.15.21 and deepinfra 0.3.0.