LLM class ships a native OpenAI client that accepts a custom base URL, so it talks to DeepInfra directly with no LiteLLM dependency. Tool calling, streaming, JSON-schema structured outputs and token usage reporting all work over this path, with any chat model from our catalog.
Installation
DEEPINFRA_API_KEY. It is the same key you use for inference, embeddings and Sandboxes.
Configuration
Create the LLM withcustom_openai=True and DeepInfra’s base URL. The model id is passed through unchanged, so use the exact org/model id from the catalog.
custom_openai=True selects CrewAI’s native OpenAI client and skips the check against OpenAI’s own model list, which is what lets a DeepInfra model id through. base_url is required when you set it. Pass the llm object to every Agent that should run on DeepInfra; different agents in one crew can use different models.
Agents defined in
agents.yaml take llm as a plain string, which cannot carry a base URL. In a @CrewBase class, build the LLM in a method decorated with @llm and reference the method name from YAML; CrewAI resolves the name to the object when it loads the agent.Build a crew
The example below gives an agent a tool and runs a single task. The agent calls the tool through OpenAI-style function calling, receives the result, and writes the answer.usage object on each response, so result.token_usage matches what you are billed for.
Knowledge and memory with DeepInfra embeddings
CrewAI’s knowledge sources and memory store text as embeddings. Itsopenai embedder accepts an api_base, so point it at DeepInfra and pick an embedding model. The same embedder dictionary works on Crew(embedder=...), on Agent(embedder=...) and in memory configuration.
Keep the same embedding model for the life of a knowledge base or memory store; vectors from different models are not comparable.
Run agent code in a Sandbox
Agents that write code need somewhere safe to run it. DeepInfra Sandboxes are isolated microVMs created with one API call, and thedeepinfra Python SDK wraps them. The tool below runs Python in a fresh sandbox per call and returns the output to the agent.
DeepInfraSandboxPython() to an agent’s tools list like any other tool. A nano sandbox is running within about half a second of the create call, and the context manager terminates it when the call returns, so you pay only for the seconds the code runs. Files the agent needs to keep between calls belong in /workspace; to carry state across calls, create one Sandbox in the tool’s constructor and reuse it instead of the context manager. Note the account limit of five active sandboxes when many agents run in parallel.
Choosing a model
The agent loop depends on reliable tool calling. The models below are verified with CrewAI’s function-calling path on DeepInfra; context lengths are from the model catalog, which is public and also carries current pricing.To compare other models, list the catalog with context windows and pricing via
/v1/openai/models?filter=with_meta&sort_by=crewai. Each entry’s metadata block has context_length, max_tokens, and per-million-token pricing; the id is what you pass as model.Tips
- Context window. CrewAI looks model ids up in a table of OpenAI models to size its context management, and assumes 8,192 tokens for ids it does not know. With the default
respect_context_window=True, an agent on a DeepInfra model starts summarising its conversation at roughly 7,000 tokens even when the model supports far more. Setrespect_context_window=Falseon agents that need the full window; CrewAI then sends the full history and surfaces the provider error if a prompt ever exceeds the model’s real limit. - The
deepinfra/prefix.LLM(model="deepinfra/Qwen/Qwen3.5-27B")routes through LiteLLM, which readsDEEPINFRA_API_KEYfrom the environment. It needspip install "crewai[litellm]"; without LiteLLM installed, constructing theLLMraises anImportError. Thecustom_openai=Trueform above uses CrewAI’s native client and has no extra dependency. - Structured outputs. Pass a Pydantic model as
response_formaton theLLMor asoutput_pydanticon aTask. CrewAI sends it as a strict JSON schema, which DeepInfra honours. - Streaming. Set
stream=Trueon theLLMto receive tokens as they are generated. Usage is still reported on the final chunk.
Learn more
CrewAI LLM docs
Every
LLM parameter, provider routing, and the YAML configuration format.Chat Completions
DeepInfra’s OpenAI-compatible endpoint, including Tool Calling and Structured Outputs.
Embeddings
Embedding models for knowledge bases and memory.
Sandboxes
Isolated microVMs for running agent-written code.
Tested with
crewai 1.15.21 and deepinfra 0.3.0.