> ## Documentation Index > Fetch the complete documentation index at: https://docs.deepinfra.com/llms.txt > Use this file to discover all available pages before exploring further. # Deploy Create Llm ## OpenAPI ````yaml https://api.deepinfra.com/openapi.json post /deploy/llm openapi: 3.1.0 info: title: DeepInfra API description: >- The DeepInfra API provides serverless AI inference, custom model deployments, and GPU rentals. version: 1.0.0 servers: - url: https://api.deepinfra.com security: [] tags: - name: Chat Completions description: OpenAI and Anthropic-compatible chat completion endpoints for LLMs. - name: Text Completions description: OpenAI-compatible text completion endpoints. - name: Embeddings description: Generate text embeddings for search and RAG. - name: Image Generation description: Generate, edit, and create variations of images. - name: Audio description: OpenAI-compatible speech synthesis, transcription, and translation. - name: Text to Speech description: ElevenLabs-compatible TTS endpoints and voice management. - name: Inference description: Native DeepInfra inference API for models and deployments. - name: Dedicated Models description: Deploy and manage private model instances with autoscaling. - name: GPU Rentals description: Rent dedicated GPU containers. - name: Models description: Browse, search, and manage AI models. - name: Files & Batches description: File uploads and batch processing. - name: LoRA Adapters description: Create, manage, and query LoRA adapter models. - name: Agents description: Manage agent-framework instances (OpenClaw and friends). - name: Sandboxes description: Create and manage isolated sandbox environments. - name: Account description: User profile, team management, and rate limits. - name: Authentication description: API tokens, SSH keys, scoped JWTs, and login flows. - name: Billing description: Payment methods, usage tracking, and billing. - name: Logs & Metrics description: Query inference logs, deployment logs, and usage metrics. - name: Utilities description: Feedback submission and CLI version. paths: /deploy/llm: post: tags: - Dedicated Models summary: Deploy Create Llm operationId: deploy_create_llm_deploy_llm_post parameters: - name: xi-api-key in: header required: false schema: anyOf: - type: string - type: 'null' title: Xi-Api-Key - name: x-api-key in: header required: false schema: anyOf: - type: string - type: 'null' title: X-Api-Key requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/DeployLLMIn' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/DeploymentOut' '402': content: application/json: schema: $ref: '#/components/schemas/DeepError' description: Payment Required '404': content: application/json: schema: $ref: '#/components/schemas/DeepError' description: Not Found '409': content: application/json: schema: $ref: '#/components/schemas/DeepError' description: Conflict '422': content: application/json: schema: $ref: '#/components/schemas/DeepError' description: Unprocessable Entity security: - HTTPBearer: [] components: schemas: DeployLLMIn: properties: model_name: type: string title: Model Name description: model name for deepinfra (username/mode-name format) gpu: $ref: '#/components/schemas/DeployGPUs' description: The type of GPU the deployment is running on. num_gpus: type: integer maximum: 8 minimum: 1 title: Num Gpus description: Number of GPUs used by one instance default: 1 max_batch_size: type: integer maximum: 256 minimum: 1 title: Max Batch Size description: Maximum number of concurrent requests default: 96 hf: anyOf: - $ref: '#/components/schemas/HFWeights' - type: 'null' base_model: anyOf: - type: string - type: 'null' title: Base Model description: Base public model container_image: anyOf: - type: string - type: 'null' title: Container Image description: Docker image for the deployment (e.g. vllm/vllm-openai:v0.8.4) settings: anyOf: - $ref: '#/components/schemas/ScaleSettings' - type: 'null' extra_args: anyOf: - items: type: string type: array - type: 'null' title: Extra Args description: Extra command line arguments for custom deployments standard_args: anyOf: - $ref: '#/components/schemas/StandardArgs' - type: 'null' description: >- Engine tuning knobs. Values are validated on submission; unsupported or out-of-range values are rejected. preset_id: anyOf: - type: string - type: 'null' title: Preset Id description: >- Apply a stored preset or vLLM recipe by id. A non-empty standard_args or extra_args in this request replaces the preset's whole matching field. type: object required: - model_name - gpu title: DeployLLMIn DeploymentOut: properties: type: $ref: '#/components/schemas/DeployType' default: legacy deploy_id: type: string title: Deploy Id description: Deploy Id examples: - fkj843kjh8 model_name: type: string title: Model Name description: Model Id from huggingface examples: - google/vit-base-patch16-224 version: type: string title: Version description: Model version examples: - d8b79b422843bd59d628bf25b01aded94a9ec1a9b917e69fe460df9ff39ec42b task: type: string title: Task description: Task examples: - image-classification status: type: string title: Status description: Status examples: - deployed fail_reason: type: string title: Fail Reason description: Failure reason examples: - Initialization failed created_at: type: string title: Created At description: Created at examples: - '2021-08-27T17:19:21+00:00' updated_at: type: string title: Updated At description: Updated at examples: - '2021-08-27T17:19:21+00:00' instances: anyOf: - $ref: '#/components/schemas/DeployInstances' - type: 'null' description: Details about number of instances running right now config: anyOf: - $ref: '#/components/schemas/DeployLLMConfig' - type: 'null' description: Immutable deploy configuration settings: anyOf: - $ref: '#/components/schemas/ScaleSettings' - type: 'null' description: Scale Settings standard_args: properties: max_context_size: anyOf: - type: integer maximum: 10000000 minimum: 1 - type: 'null' title: Max Context Size description: Maximum total sequence length (prompt + generation). max_concurrent_requests: anyOf: - type: integer maximum: 1024 minimum: 1 - type: 'null' title: Max Concurrent Requests description: Max number of requests served concurrently. gpu_memory_fraction: anyOf: - type: number maximum: 0.97 minimum: 0.5 - type: 'null' title: Gpu Memory Fraction description: >- Fraction of GPU memory the engine may use for weights + KV cache. max_prefill_tokens: anyOf: - type: integer maximum: 131072 minimum: 512 - type: 'null' title: Max Prefill Tokens description: >- Max tokens processed per prefill/engine step (chunked prefill size). kv_cache_dtype: anyOf: - type: string enum: - auto - fp8 - type: 'null' title: Kv Cache Dtype description: >- KV cache precision. fp8 ~doubles KV capacity at <1% accuracy loss. enable_prefix_caching: anyOf: - type: boolean - type: 'null' title: Enable Prefix Caching description: Reuse KV cache for shared prompt prefixes. quantization: anyOf: - type: string enum: - fp8 - awq - gptq - awq_marlin - gptq_marlin - compressed-tensors - bitsandbytes - type: 'null' title: Quantization description: On-the-fly weight quantization method. additionalProperties: false type: object title: StandardArgs description: Current engine tuning knobs extra_args: anyOf: - items: type: string type: array - type: 'null' title: Extra Args description: >- Current extra engine-specific command-line args (custom-weight deploys only) update_error: anyOf: - type: string - type: 'null' title: Update Error description: >- If the last config update was auto-reverted after an engine crash-loop, the error that caused it (status stays 'running') rollout: anyOf: - $ref: '#/components/schemas/DeployRollout' - type: 'null' description: Per-instance rollout progress of the current config type: object required: - deploy_id - model_name - version - task - status - fail_reason - created_at - updated_at title: DeploymentOut DeepError: properties: error: type: string title: Error description: Error examples: - Model not found type: object required: - error title: DeepError DeployGPUs: type: string enum: - L4-24GB - L40S-48GB - A100-80GB - H100-80GB - H200-141GB - B200-180GB - B300-270GB - RTXPRO6000-96GB - other title: DeployGPUs HFWeights: properties: repo: type: string title: Repo description: huggingface repository i.e username/reponame revision: anyOf: - type: string - type: 'null' title: Revision description: commit sha or branch name token: anyOf: - type: string - type: 'null' title: Token description: huggingface access token with read access to the repo type: object required: - repo title: HFWeights ScaleSettings: properties: min_instances: type: integer title: Min Instances description: Minimum number of model instances to run default: 1 max_instances: type: integer title: Max Instances description: Maximum number of model instances to run default: 1 type: object title: ScaleSettings StandardArgs: properties: max_context_size: anyOf: - type: integer maximum: 10000000 minimum: 1 - type: 'null' title: Max Context Size description: Maximum total sequence length (prompt + generation). max_concurrent_requests: anyOf: - type: integer maximum: 1024 minimum: 1 - type: 'null' title: Max Concurrent Requests description: Max number of requests served concurrently. gpu_memory_fraction: anyOf: - type: number maximum: 0.97 minimum: 0.5 - type: 'null' title: Gpu Memory Fraction description: Fraction of GPU memory the engine may use for weights + KV cache. max_prefill_tokens: anyOf: - type: integer maximum: 131072 minimum: 512 - type: 'null' title: Max Prefill Tokens description: Max tokens processed per prefill/engine step (chunked prefill size). kv_cache_dtype: anyOf: - type: string enum: - auto - fp8 - type: 'null' title: Kv Cache Dtype description: KV cache precision. fp8 ~doubles KV capacity at <1% accuracy loss. enable_prefix_caching: anyOf: - type: boolean - type: 'null' title: Enable Prefix Caching description: Reuse KV cache for shared prompt prefixes. quantization: anyOf: - type: string enum: - fp8 - awq - gptq - awq_marlin - gptq_marlin - compressed-tensors - bitsandbytes - type: 'null' title: Quantization description: On-the-fly weight quantization method. additionalProperties: false type: object title: StandardArgs description: |- The standard-argument allowlist. Construct from a raw ``dict[str, str]`` via :meth:`from_raw`; render to engine flags via :func:`standard_parts`. DeployType: type: string enum: - legacy - llm - lora - tts title: DeployType DeployInstances: properties: running: type: integer title: Running pending: type: integer title: Pending type: object required: - running - pending title: DeployInstances DeployLLMConfig: properties: gpu: $ref: '#/components/schemas/DeployGPUs' description: The type of GPU the deployment is running on num_gpus: type: integer title: Num Gpus description: Number of GPUs used by one instance max_batch_size: type: integer title: Max Batch Size description: Maximum number of concurrent requests weights: anyOf: - $ref: '#/components/schemas/HFWeights' - type: 'null' description: Model weights information type: object required: - gpu - num_gpus - max_batch_size title: DeployLLMConfig DeployRollout: properties: up_to_date: type: integer title: Up To Date description: Instances serving the current config. outdated: type: integer title: Outdated description: Instances still serving a previous config. booting: type: integer title: Booting description: Instances starting up. type: object required: - up_to_date - outdated - booting title: DeployRollout securitySchemes: HTTPBearer: type: http scheme: bearer ````