Extending

Polyglot ships with drivers for over 25 LLM providers and several embeddings providers. When you need to integrate a provider that is not bundled -- or override the behavior of an existing one -- the library exposes clean extension points for both inference and embeddings.

Custom Inference Drivers¶

Inference drivers implement the CanProcessInferenceRequest interface, which defines three methods:

interface CanProcessInferenceRequest
{
    public function makeResponseFor(InferenceRequest $request): InferenceResponse;

    /** @return iterable<PartialInferenceDelta> */
    public function makeStreamDeltasFor(InferenceRequest $request): iterable;

    public function capabilities(?string $model = null): DriverCapabilities;
}
// @doctest id="77df"

Method	Purpose
`makeResponseFor()`	Send a synchronous request and return the complete response
`makeStreamDeltasFor()`	Send a streaming request and yield partial deltas
`capabilities()`	Report driver capabilities (tool calls, JSON mode, vision, etc.)

Registering a Driver Class¶

The simplest approach is to provide a class string. Polyglot will instantiate it with the standard constructor signature ($config, $httpClient, $events):

<?php

use App\Polyglot\AcmeInferenceDriver;
use Cognesy\Messages\Messages;
use Cognesy\Polyglot\Inference\Creation\BundledInferenceDrivers;
use Cognesy\Polyglot\Inference\Config\LLMConfig;
use Cognesy\Polyglot\Inference\Inference;

$drivers = BundledInferenceDrivers::registry()
    ->withDriver('acme', AcmeInferenceDriver::class);

$config = new LLMConfig(
    driver: 'acme',
    apiUrl: 'https://api.acme.com/v1',
    apiKey: (string) getenv('ACME_API_KEY'),
    endpoint: '/chat/completions',
    model: 'acme-large',
);

$text = Inference::fromConfig($config, drivers: $drivers)
    ->withMessages(Messages::fromString('Hello from Acme!'))
    ->get();
// @doctest id="e7e1"

Registering a Driver Spec¶

If your provider speaks the OpenAI wire protocol, you do not need a driver class. Register an InferenceDriverSpec naming the parts that differ; everything you leave out defaults to the OpenAI implementation:

<?php

use Cognesy\Polyglot\Inference\Creation\BundledInferenceDrivers;
use Cognesy\Polyglot\Inference\Data\DriverCapabilities;
use Cognesy\Polyglot\Inference\Drivers\InferenceDriverSpec;

$drivers = BundledInferenceDrivers::registry()
    ->withDriver('acme', new InferenceDriverSpec(
        bodyFormat: AcmeBodyFormat::class,
        capabilities: new DriverCapabilities(responseFormatWithTools: false),
    ));
// @doctest id="c765"

The spec's other fields -- requestAdapter, responseAdapter, usageFormat, messageFormat -- take the same treatment. All bundled providers use this same declarative shape. Providers whose wire protocol or endpoint differs name those provider-specific collaborators in the row; they do not need a provider driver class.

To change behaviour rather than composition, subclass SpecifiedInferenceDriver and name it in the spec. The spec still assembles the five collaborators for it:

<?php

use Cognesy\Polyglot\Inference\Drivers\OpenAI\OpenAIBodyFormat;
use Cognesy\Polyglot\Inference\Drivers\SpecifiedInferenceDriver;

final class LoggingDriver extends SpecifiedInferenceDriver
{
    #[\Override]
    public function makeResponseFor($request): \Cognesy\Polyglot\Inference\Data\InferenceResponse {
        // Add logging, metrics, request transformation, etc.
        return parent::makeResponseFor($request);
    }
}

$drivers = BundledInferenceDrivers::registry()
    ->withDriver('custom', new InferenceDriverSpec(
        bodyFormat: OpenAIBodyFormat::class,
        driverClass: LoggingDriver::class,
    ));
// @doctest id="5ea0"

Registering a Driver Factory¶

withDriver() also accepts a class-string or any callable receiving LLMConfig, CanSendHttpRequests and CanHandleEvents and returning a CanProcessInferenceRequest. Use this when construction needs logic a spec cannot express -- reading an environment variable, choosing between implementations, wiring a decorator:

<?php

use Cognesy\Polyglot\Inference\Creation\BundledInferenceDrivers;

$drivers = BundledInferenceDrivers::registry()
    ->withDriver('custom', fn($config, $httpClient, $events) => new AcmeDriver($config, $httpClient, $events))
    ->withDriver('custom-by-name', AcmeDriver::class);
// @doctest id="2900"

Using the Registry with InferenceRuntime¶

You can pass the driver registry directly when building a runtime:

<?php

use Cognesy\Polyglot\Inference\Config\LLMConfig;
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Polyglot\Inference\InferenceRuntime;

$runtime = InferenceRuntime::fromConfig(
    config: $config,
    drivers: $drivers,
);

$inference = Inference::fromRuntime($runtime);
// @doctest id="da14"

Or use the drivers parameter on Inference::fromConfig() or Inference::using():

<?php

use Cognesy\Polyglot\Inference\Inference;

$text = Inference::using('acme', drivers: $drivers)
    ->withMessages(Messages::fromString('Hello!'))
    ->get();
// @doctest id="b122"

Implementing a Full Driver¶

When building a driver from scratch, you will typically need to implement several adapter components:

Request Adapter -- transforms InferenceRequest into the provider's HTTP request format
Body Format -- structures the request body according to the provider's API schema
Message Format -- converts Polyglot's message format to the provider's format
Response Adapter -- parses the provider's HTTP response into InferenceResponse
Usage Format -- extracts token usage information from the response

For the request adapter, extend BaseHttpRequestAdapter rather than implementing CanTranslateInferenceRequest directly. It owns the request-building skeleton and leaves you the only two methods that vary between providers:

<?php

use Cognesy\Polyglot\Inference\Data\InferenceRequest;
use Cognesy\Polyglot\Inference\Drivers\BaseHttpRequestAdapter;

final class AcmeRequestAdapter extends BaseHttpRequestAdapter
{
    #[\Override]
    protected function toUrl(InferenceRequest $request): string {
        return "{$this->config->apiUrl}{$this->config->endpoint}";
    }

    #[\Override]
    protected function toHeaders(InferenceRequest $request): array {
        return [
            'X-Acme-Key' => $this->config->apiKey,
            'Content-Type' => 'application/json; charset=utf-8',
        ];
    }
}
// @doctest id="6965"

All bundled providers follow this modular adapter pattern and are declared as an InferenceDriverSpec. OpenAI-compatible providers use the default adapters where possible; native protocols and providers with custom URLs or headers select bespoke request or response adapters in their spec row. See BundledInferenceDrivers::registry() for the complete table.

Custom Embeddings Drivers¶

Embeddings drivers implement the CanHandleVectorization interface:

interface CanHandleVectorization
{
    public function handle(EmbeddingsRequest $request): EmbeddingsResponse;
}
// @doctest id="63fc"

The driver owns its complete provider boundary: translating the request, sending it, decoding the provider payload, and adapting that payload into an EmbeddingsResponse. Callers never receive the intermediate HTTP response.

Migrating a v2.6 Embeddings Driver¶

This signature changes in v2.7 and cannot be shimmed by PHP. Update custom implementations in the same deployment that upgrades Polyglot:

- public function handle(EmbeddingsRequest $request): HttpResponse;
- public function fromData(array $data): ?EmbeddingsResponse;
+ public function handle(EmbeddingsRequest $request): EmbeddingsResponse;
// @doctest id="1dd3"

Move the HTTP response decoding and response-adapter call into handle(). Drivers extending BaseEmbedDriver inherit the v2.7 implementation unless they override handle() themselves.

Register a custom embeddings driver using the BundledEmbeddingsDrivers registry, the same pattern used for inference drivers:

<?php

use App\Polyglot\AcmeEmbeddingsDriver;
use Cognesy\Polyglot\Embeddings\Config\EmbeddingsConfig;
use Cognesy\Polyglot\Embeddings\Creation\BundledEmbeddingsDrivers;
use Cognesy\Polyglot\Embeddings\Embeddings;
use Cognesy\Polyglot\Embeddings\EmbeddingsRuntime;

$drivers = BundledEmbeddingsDrivers::registry()
    ->withDriver('acme', AcmeEmbeddingsDriver::class);

$config = new EmbeddingsConfig(
    driver: 'acme',
    apiUrl: 'https://api.acme.com/v1',
    apiKey: (string) getenv('ACME_API_KEY'),
    endpoint: '/embeddings',
    model: 'acme-embed-v1',
    dimensions: 768,
    maxInputs: 100,
);

$embeddings = Embeddings::fromRuntime(
    EmbeddingsRuntime::fromConfig($config, drivers: $drivers)
);
// @doctest id="c87f"

Like inference drivers, you can also pass a callable factory instead of a class string:

<?php

use App\Polyglot\AcmeEmbeddingsDriver;
use Cognesy\Polyglot\Embeddings\Creation\BundledEmbeddingsDrivers;

$drivers = BundledEmbeddingsDrivers::registry()
    ->withDriver('acme', function ($config, $httpClient, $events) {
        return new AcmeEmbeddingsDriver($config, $httpClient, $events);
    });
// @doctest id="403d"

Note: The EmbeddingsDriverRegistry is immutable -- each mutation returns a new instance, matching the same pattern as InferenceDriverRegistry.

Removing or Replacing Bundled Drivers¶

The InferenceDriverRegistry is immutable -- each mutation returns a new instance. You can remove a bundled driver or replace it entirely:

<?php

use Cognesy\Polyglot\Inference\Creation\BundledInferenceDrivers;

// Remove a driver
$drivers = BundledInferenceDrivers::registry()
    ->withoutDriver('ollama');

// Replace a driver
$drivers = BundledInferenceDrivers::registry()
    ->withDriver('openai', MyCustomOpenAIDriver::class);
// @doctest id="8ed1"

Bundled Drivers¶

For reference, Polyglot bundles the following inference drivers:

Driver Name	Built from
`a21`	spec: `A21BodyFormat`
`anthropic`	spec: `AnthropicBodyFormat` + `AnthropicRequestAdapter`
`azure`	spec: `OpenAIBodyFormat` + `AzureOpenAIRequestAdapter`
`bedrock-openai`	spec: `OpenAICompatibleBodyFormat` + `BedrockOpenAIRequestAdapter`
`cerebras`	spec: `CerebrasBodyFormat`
`cohere`	spec: `CohereV2BodyFormat` + `CohereV2RequestAdapter`
`deepseek`	spec: `DeepseekBodyFormat`
`fireworks`	spec: `FireworksBodyFormat`
`gemini`	spec: `GeminiBodyFormat` + `GeminiRequestAdapter`
`gemini-oai`	spec: `GeminiOAIBodyFormat` + `GeminiOAIRequestAdapter`
`glm`	spec: `GlmBodyFormat`
`groq`	spec: `GroqBodyFormat`
`huggingface`	spec: `HuggingFaceBodyFormat` + `HuggingFaceRequestAdapter`
`inception`	spec: `InceptionBodyFormat`
`meta`	spec: `MetaBodyFormat`
`minimaxi`	spec: `MinimaxiBodyFormat`
`mistral`	spec: `MistralBodyFormat`
`openai`	spec: `OpenAIBodyFormat`
`openai-responses`	spec: `OpenResponsesBodyFormat` + `OpenAIResponsesRequestAdapter`
`openresponses`	spec: `OpenResponsesBodyFormat` + `OpenResponsesRequestAdapter`
`openrouter`	spec: `OpenRouterBodyFormat`
`perplexity`	spec: `PerplexityBodyFormat`
`qwen`	spec: `QwenBodyFormat`
`sambanova`	spec: `SambaNovaBodyFormat`
`xai`	spec: `OpenAICompatibleBodyFormat` + `XAiMessageFormat`
`moonshot`	spec: `OpenAICompatibleBodyFormat`
`ollama`	spec: `OpenAICompatibleBodyFormat`
`openai-compatible`	spec: `OpenAICompatibleBodyFormat`
`together`	spec: `OpenAICompatibleBodyFormat`

The full list is defined in BundledInferenceDrivers::registry().

Bundled embeddings drivers include: openai, azure, cohere, gemini, jina, mistral, and ollama.

Listening to Events¶

Both InferenceRuntime and EmbeddingsRuntime dispatch events at key lifecycle points. You can listen for specific events or wiretap all of them:

<?php

use Cognesy\Messages\Messages;
use Cognesy\Polyglot\Inference\Config\LLMConfig;
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Polyglot\Inference\InferenceRuntime;
use Cognesy\Polyglot\Inference\Events\InferenceDriverBuilt;

$runtime = InferenceRuntime::fromConfig(LLMConfig::fromPreset('openai'));

// Listen for a specific event
$runtime->onEvent(InferenceDriverBuilt::class, function (InferenceDriverBuilt $event) {
    echo "Driver built: " . $event->payload['driverClass'] . "\n";
});

// Or listen to all events for debugging
$runtime->wiretap(function ($event) {
    error_log(get_class($event));
});

$response = Inference::fromRuntime($runtime)
    ->withMessages(Messages::fromString('Hello!'))
    ->get();
// @doctest id="4f87"