Multimodal Inputs

Content part types, per-modality limits, and format tradeoffs

Content parts are supplied under messages[].content, following the same shape OpenAI’s SDKs already use. Each part is one of:

Part	Accepts	Notes
text	Plain text	Prompt or question text.
image_url	url or base64 data URI	Single image per request for OCR models; VQA models accept multiple
document_url	url, file URL, or base64 data URI	PDF; enters the page-wise document pipeline

The VLM Run Gateway validates content parts against each model’s capabilities before inference. Mismatches return 400 with capability_violation. Check a model’s accepted input types on Models or GET /v1/openai/models.

Image Inputs

Most OCR and vision models accept a single image_url part per request. VQA models such as qwen/qwen3.5-0.8b accept multiple images in the same message, for side-by-side comparison or multi-image context.

{
  "type": "image_url",
  "image_url": { "url": "https://example.com/photo.jpg" }
}

Use image_resolution to resize an image before inference when a model’s default resolution costs more than the task needs. See Gateway extensions for the full parameter reference.

Document Inputs

Document (PDF) requests are decoded and rasterized at the ingress before per-page inference.

Limits

Limit	Value	Notes
File size	100 MB	Enforced before decoding starts.
Pages	500 (document_max_pages)	Requests exceeding the cap are rejected before inference.

Requests over either limit return 400 with error.code = "invalid_document". See Error Codes.

URL vs. base64

Input	Use when
document_url with a hosted URL	Default choice; keeps the request body small and fast to transmit.
Base64 data URI	Only when the document isn’t already hosted somewhere reachable. Inflates the request body by ~33% and increases request latency for large files.

{
  "type": "document_url",
  "document_url": { "url": "https://example.com/report.pdf" }
}

Prefer a URL over base64 whenever possible, especially for documents over a few megabytes.

For document_dpi and other method-specific knobs, see Method Parameters.

Scanned vs. native PDFs

The VLM Run Gateway rasterizes every page and runs OCR regardless of whether the source PDF has a text layer. Scanned (image-only) and native (text-layer) PDFs are handled the same way, both go through the OCR model rather than a text-extraction shortcut. If you already have a reliable text layer and don’t need OCR, extracting it client-side before calling the Gateway will be faster and cheaper.

Related

Flexible Document OCR

End-to-end recipe from model selection to response parsing.

Streaming

Receive pages as they finish instead of waiting for the whole document.

Models

Which input types each model accepts.

Chat Completions

Full request and response schema.