Response Handling

Polyglot's PendingInference class represents a pending inference execution. It is returned by the Inference class when you call the create() method. The request is not sent to the underlying LLM until you actually access the response data, making the object a lazy handle over a single inference operation.

Retrieving Text Content

The simplest way to get the model's response is the get() method, which returns the assistant Message:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$pending = Inference::using('openai')
    ->withMessages(Messages::fromString('What is the capital of France?'))
    ->create();

// Get the assistant message
$message = $pending->get();
echo $message->content()->toString(); // "The capital of France is Paris."
// @doctest id="90ea"

Retrieving JSON Data

When you request a JSON response format, use asJsonData() to decode the content directly into an associative array, or asJson() to get the raw JSON string:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;
use Cognesy\Polyglot\Inference\Data\ResponseFormat;

$pending = Inference::using('openai')
    ->withMessages(Messages::fromString('Return JSON with a single "status" field.'))
    ->withResponseFormat(ResponseFormat::jsonObject())
    ->create();

// Decode response content as a PHP array
$data = $pending->asJsonData();
echo $data['status'];

// Or get the raw JSON string
$json = $pending->asJson();
// @doctest id="be7e"

Working with InferenceResponse

For full access to every detail of the model's reply, call response() to get the normalized InferenceResponse object:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$pending = Inference::using('openai')
    ->withMessages(Messages::fromString('What is the capital of France?'))
    ->create();

$response = $pending->response();
$message = $response->message();

// Ordered semantic parts and derived projections
foreach ($message->parts() as $part) {
    echo $part->type() . "\n";
}
echo "Content: " . $message->content()->toString() . "\n";
echo "Reasoning: " . $message->reasoningContent() . "\n";

// Finish reason (returns InferenceFinishReason enum)
echo "Finish reason: " . $response->finishReason()->value . "\n";

// Token usage
$usage = $response->usage();
echo "Input tokens: " . $usage->input() . "\n";
echo "Output tokens: " . $usage->output() . "\n";
echo "Total tokens: " . $usage->total() . "\n";
echo "Cache tokens: " . $usage->cache() . "\n";

// Raw HTTP response data
$httpResponse = $response->responseData();
// @doctest id="5c7b"

Available InferenceResponse Methods

Method	Returns	Description
message()	Message	The complete ordered assistant turn
usage()	InferenceUsage	Token counts for the request
finishReason()	InferenceFinishReason	Why the model stopped generating
responseData()	HttpResponse	The underlying raw HTTP response
isPartial()	bool	Whether this is a partial (streaming) response

Text, reasoning, and tool calls are projections of the message's ordered parts: $message->content()->toString(), $message->reasoningContent(), and $message->toolCalls(). The response envelope does not duplicate those values.

Finish Reasons

The finishReason() method returns an InferenceFinishReason enum. Polyglot normalizes the many vendor-specific strings into a consistent set of values:

Value	Meaning
Stop	The model finished naturally
Length	Output was truncated due to token limits
ToolCalls	The model wants to invoke a tool
ContentFilter	Content was blocked by safety filters
Error	An error occurred during generation
Other	An unrecognized finish reason

Token Usage

The InferenceUsage object provides detailed token breakdowns including cache and reasoning tokens:

<?php

$usage = $response->usage();

$usage->inputTokens;       // Input / prompt tokens
$usage->outputTokens;      // Output / completion tokens
$usage->cacheWriteTokens;  // Tokens written to cache when reported by provider
$usage->cacheReadTokens;   // Tokens read from cache when reported by provider
$usage->reasoningTokens;   // Reasoning / thinking tokens

// Convenience accessors
$usage->input();   // Same as inputTokens
$usage->output();  // outputTokens + reasoningTokens
$usage->cache();   // cacheWriteTokens + cacheReadTokens
$usage->total();   // Sum of all token counts
// @doctest id="2b4f"

Handling Tool Calls

When the model decides to invoke a tool, you can extract the tool call data using asToolCallJsonData() on PendingInference, or inspect the ToolCalls collection on the response object:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;
use Cognesy\Polyglot\Inference\Data\ToolDefinitions;
use Cognesy\Polyglot\Inference\Data\ToolChoice;

$tools = ToolDefinitions::fromArray([
    [
        'type' => 'function',
        'function' => [
            'name' => 'get_weather',
            'description' => 'Get the current weather in a location',
            'parameters' => [
                'type' => 'object',
                'properties' => [
                    'location' => [
                        'type' => 'string',
                        'description' => 'The city and state, e.g. San Francisco, CA',
                    ],
                    'unit' => [
                        'type' => 'string',
                        'enum' => ['celsius', 'fahrenheit'],
                    ],
                ],
                'required' => ['location'],
            ],
        ],
    ],
]);

$response = Inference::using('openai')
    ->with(
        messages: Messages::fromString('What is the weather in Paris?'),
        tools: $tools,
        toolChoice: ToolChoice::auto(),
    )
    ->response();

$message = $response->message();
if ($message->hasToolCalls()) {
    $toolCalls = $message->toolCalls();

    foreach ($toolCalls->all() as $call) {
        echo "Tool: " . $call->name() . "\n";
        echo "Args: " . $call->argsAsJson() . "\n";

        // Access individual argument values
        $location = $call->value('location');
        $unit = $call->value('unit', 'celsius');
    }
}
// @doctest id="eb50"

Quick JSON Extraction from Tool Calls

If you just need the arguments as a PHP array without inspecting the full response, use the shorthand on PendingInference:

<?php

// Single tool call: returns the arguments array directly
$args = $pending->asToolCallJsonData();

// Or as a JSON string
$json = $pending->asToolCallJson();
// @doctest id="cc58"

Note: When a single tool call is present, asToolCallJsonData() returns that call's arguments as an array. When multiple tool calls are present, it returns an array of all tool call data.

Streaming Responses

For long-running completions, streaming lets you display output as it arrives. Call stream() to get an InferenceStream and consume deltas:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$stream = Inference::using('openai')
    ->withMessages(Messages::fromString('Write a short story about a robot.'))
    ->stream();

foreach ($stream->deltas() as $delta) {
    echo $delta->messageChunks->textDelta();
}

// After iteration, get the finalized response
$finalResponse = $stream->final();
echo "\n\nTokens used: " . $finalResponse->usage()->total();
// @doctest id="5e79"

The PartialInferenceDelta Object

Each delta yielded during streaming is a PartialInferenceDelta with the following public properties:

Property	Type	Description
messageChunks	AssistantMessageChunks	Ordered assistant block changes for this event
finishReason	string	Set on the final delta
usage	?InferenceUsage	Token usage (typically on the final delta)
usageIsCumulative	bool	Whether usage counts are cumulative

Stream Methods

The InferenceStream class provides several ways to consume and transform the delta stream:

<?php

// Iterate over visible deltas
foreach ($stream->deltas() as $delta) { /* ... */ }

// Transform each delta
foreach ($stream->map(fn($d) => strtoupper($d->messageChunks->textDelta())) as $text) {
    echo $text;
}

// Reduce to a single value
$fullText = $stream->reduce(
    fn(string $carry, $delta) => $carry . $delta->messageChunks->textDelta(),
    ''
);

// Filter deltas
foreach ($stream->filter(fn($d) => $d->messageChunks->textDelta() !== '') as $delta) {
    echo $delta->messageChunks->textDelta();
}

// Collect all deltas into an array
$allDeltas = $stream->all();

// Get the finalized response (drains the stream if needed)
$response = $stream->final();
// @doctest id="150d"

Using the onDelta Callback

Instead of iterating manually, you can register a callback that fires for each visible delta:

<?php

use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$stream = Inference::using('openai')
    ->withMessages(Messages::fromString('Explain queues in simple terms.'))
    ->stream();

$stream->onDelta(function ($delta) {
    echo $delta->messageChunks->textDelta();

    // Flush output for real-time display
    if (ob_get_level() > 0) {
        ob_flush();
        flush();
    }
});

// Drain the stream to trigger all callbacks
$response = $stream->final();
// @doctest id="eed6"

Stream Lifecycle

The stream is one-shot: once deltas() has been fully iterated, calling it again throws a LogicException. If you need to replay the response, work with the finalized InferenceResponse returned by $stream->final().

Calling final() before the stream is exhausted will automatically drain all remaining deltas, ensuring the finalized response is complete.

Checking for Streaming Mode

If you need to branch your code based on whether a request was configured for streaming, use the isStreamed() method on PendingInference:

<?php

$pending = Inference::using('openai')
    ->withMessages(Messages::fromString('Hello!'))
    ->withStreaming()
    ->create();

if ($pending->isStreamed()) {
    foreach ($pending->stream()->deltas() as $delta) {
        echo $delta->messageChunks->textDelta();
    }
} else {
    echo $pending->get()->content()->toString();
}
// @doctest id="797f"