Overview

Cognesy\Polyglot\BatchInference\BatchInference is an opt-in early access facade for provider-native batch jobs. Submission returns an observed job and a durable reference. The provider processes requests later. Your application decides when to check status, read outcomes, or request cancellation. None of these operations waits for remote inference to finish.

<?php
use Cognesy\Messages\Messages;
use Cognesy\Polyglot\BatchInference\BatchInference;
use Cognesy\Polyglot\BatchInference\Collections\BatchItems;
use Cognesy\Polyglot\BatchInference\Data\BatchItem;
use Cognesy\Polyglot\Inference\Data\InferenceRequest;

$batches = BatchInference::using('mistral');
$job = $batches->submit(BatchItems::of(
    BatchItem::of('document-101', new InferenceRequest(
        messages: Messages::fromString('Summarize this document.'),
    )),
));

$savedReference = $job->reference()->toArray(); // persist in application storage
// @doctest id="f49d"

Operation	Return type	I/O behavior
submit($items, $options)	BatchJob	Prepares one fixed input set, uploads if needed, creates one remote job, returns its first snapshot.
retrieve($reference)	BatchJob	Reads one fresh provider status snapshot.
cancel($reference)	BatchCancellation	Sends a request to stop processing where supported; acknowledgement is not a terminal result.
results($reference)	BatchResults	Checks availability, then exposes a lazy, one-pass stream of outcomes.
listJobs($limit, $cursor)	BatchJobPage	Reads one account/workspace page with an opaque continuation cursor.

BatchJob accessors are pure: calling status(), progress(), or resultsAvailability() does not poll. A completed job may contain failed items. An in-progress xAI job may already have partial results. Use capabilities() to inspect cancellation, listing, typed result reading, partial-result support, input modes, and enforced input limits before submission. Some providers also publish a minimum item count.

The native drivers currently cover OpenAI Chat Completions and Responses, Anthropic, Mistral, Gemini, Qwen, xAI/Grok, Groq, and Together. Each driver has separate wire encoding and status mapping. Public documentation alone does not qualify every model or account for batch processing; review provider details before submitting paid work.

Next: submit and resume, status and cancellation, and results.