Advanced Usage

This guide covers advanced patterns and features for power users who need fine-grained control over extraction behavior, streaming, validation, and multi-provider workflows.

Streaming

Streaming lets you receive partial results as the LLM generates them, rather than waiting for the entire response. This is essential for long-running extractions where you want to show progress, or for real-time UIs that display data as it becomes available.

use Cognesy\Instructor\Laravel\Facades\StructuredOutput;

$stream = StructuredOutput::with(
    messages: 'Extract detailed company information from this long document...',
    responseModel: CompanyData::class,
)->withStreaming()->stream();

// Handle partial updates
foreach ($stream->partials() as $partial) {
    echo "Company: " . ($partial->name ?? 'Loading...') . "\n";
    echo "Industry: " . ($partial->industry ?? 'Loading...') . "\n";
    echo "---\n";
}

// Get final complete result
$company = $stream->finalValue();

Streaming with partials()

Each partial is a partially populated instance of your response model. Properties that have not been received yet will be null or their default value. This is useful for broadcasting live updates via WebSockets.

$stream = StructuredOutput::with(
    messages: 'Extract data...',
    responseModel: MyModel::class,
)
->withStreaming()
->stream();

foreach ($stream->partials() as $partial) {
    broadcast(new PartialUpdateEvent($partial));
}

$result = $stream->finalValue();

Streaming Sequences

When extracting an array of items, the sequence() method yields the growing collection as each new item is completed.

$stream = StructuredOutput::with(
    messages: 'Extract all products from this catalog...',
    responseModel: [
        'type' => 'array',
        'items' => ProductData::class,
    ],
)
->withStreaming()
->stream();

foreach ($stream->sequence() as $items) {
    echo "Found: {$items->last()->name}\n";
}

---

Validation and Retries

Automatic Validation

Response models are automatically validated after deserialization. When validation fails, the package sends the error messages back to the LLM with a retry prompt, asking it to correct the response. This loop continues up to max_retries times.

use Symfony\Component\Validator\Constraints as Assert;

final class UserData
{
    public function __construct(
        #[Assert\NotBlank]
        #[Assert\Length(min: 2, max: 100)]
        public readonly string $name,

        #[Assert\Email]
        public readonly string $email,

        #[Assert\Range(min: 18, max: 120)]
        public readonly int $age,
    ) {}
}

// Extraction will retry if validation fails
$user = StructuredOutput::with(
    messages: 'Extract user from: john doe, email: invalid, age: 5',
    responseModel: UserData::class,
    maxRetries: 3,
)->get();

Custom Validators

Implement the CanValidateObject contract for domain-specific validation logic that cannot be expressed with declarative attributes. The validate method must return a ValidationResult instance.

use Cognesy\Instructor\Validation\Contracts\CanValidateObject;
use Cognesy\Instructor\Validation\ValidationResult;

class BusinessRulesValidator implements CanValidateObject
{
    public function validate(object $dataObject): ValidationResult
    {
        if ($dataObject instanceof OrderData) {
            if ($dataObject->total < $dataObject->minimumOrderValue) {
                return ValidationResult::fieldError(
                    field: 'total',
                    value: $dataObject->total,
                    message: "Order total must be at least {$dataObject->minimumOrderValue}",
                );
            }
        }

        return ValidationResult::valid();
    }
}

Custom validators are registered on the StructuredOutputRuntime, not on the facade directly:

use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$runtime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withValidator(new BusinessRulesValidator());

$order = StructuredOutput::withRuntime($runtime)->with(
    messages: 'Extract order...',
    responseModel: OrderData::class,
)->get();

Custom Retry Prompt

Customize the message sent to the LLM when validation fails. The {errors} placeholder is replaced with the actual error messages.

$result = StructuredOutput::with(
    messages: 'Extract data...',
    responseModel: MyModel::class,
    maxRetries: 3,
    retryPrompt: 'The extraction failed validation. Errors: {errors}. Please correct and try again.',
)->get();

---

Data Transformation

Apply transformations to extracted data after deserialization. Transformers run after validation, so they can normalize, enrich, or restructure the data before it reaches your application code.

use Cognesy\Instructor\Transformation\Contracts\CanTransformData;

class NormalizePhoneNumbers implements CanTransformData
{
    public function transform(mixed $data): mixed
    {
        if ($data instanceof ContactData) {
            $data->phone = $this->normalize($data->phone);
        }
        return $data;
    }

    private function normalize(string $phone): string
    {
        return preg_replace('/[^0-9+]/', '', $phone);
    }
}

Custom transformers are registered on the StructuredOutputRuntime, not on the facade directly:

use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$runtime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withTransformer(new NormalizePhoneNumbers());

$contact = StructuredOutput::withRuntime($runtime)->with(
    messages: 'Contact: John, phone: (555) 123-4567',
    responseModel: ContactData::class,
)->get();

// $contact->phone === '+15551234567'

---

Output Modes

Different LLMs support different output modes. The output mode controls the mechanism used to extract structured data from the model's response. You can set the default mode in config/instructor.php or override it per-request via the runtime.

use Cognesy\Instructor\Enums\OutputMode;
use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$jsonSchemaRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::JsonSchema);

$toolsRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::Tools);

$jsonRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::Json);

$mdJsonRuntime = StructuredOutputRuntime::fromProvider(LLMProvider::new())
    ->withOutputMode(OutputMode::MdJson);

// JSON Schema mode (recommended for OpenAI)
$result = StructuredOutput::withRuntime($jsonSchemaRuntime)
    ->with(...)
    ->get();

// Tool/Function calling mode
$result = StructuredOutput::withRuntime($toolsRuntime)
    ->with(...)
    ->get();

// Simple JSON mode
$result = StructuredOutput::withRuntime($jsonRuntime)
    ->with(...)
    ->get();

// Markdown JSON (for Gemini)
$result = StructuredOutput::withRuntime($mdJsonRuntime)
    ->with(...)
    ->get();

---

Few-Shot Learning

Providing input/output examples significantly improves extraction quality, especially for ambiguous or domain-specific data. Each example pairs an input string with a fully populated response model instance.

$person = StructuredOutput::with(
    messages: 'Extract: Jane Doe, 25 years old, jane@example.com',
    responseModel: PersonData::class,
    examples: [
        [
            'input' => 'John Smith is 30 years old and works at john@company.com',
            'output' => new PersonData(
                name: 'John Smith',
                age: 30,
                email: 'john@company.com',
            ),
        ],
        [
            'input' => 'Mary Johnson, age 45',
            'output' => new PersonData(
                name: 'Mary Johnson',
                age: 45,
                email: null,
            ),
        ],
    ],
)->get();

---

System Prompts

System prompts set the overall behavior and domain context for the LLM. They are especially valuable when extracting specialized data.

$medical = StructuredOutput::with(
    messages: $patientNotes,
    responseModel: MedicalRecord::class,
    system: <<<'PROMPT'
        You are a medical records extraction specialist.
        Extract structured data from clinical notes.
        Use standard medical terminology.
        If information is unclear, mark as null rather than guessing.
        PROMPT,
)->get();

---

Tool Descriptions

Customize how the response model is described to the LLM in the tool/function calling interface. This is particularly useful when the auto-generated name or description is not descriptive enough for the model to understand the task.

$result = StructuredOutput::with(
    messages: 'Extract invoice details...',
    responseModel: InvoiceData::class,
    toolName: 'extract_invoice',
    toolDescription: 'Extracts structured invoice data including line items, totals, and payment terms.',
)->get();

---

Multiple Providers

Switch between providers based on the task at hand. Different providers offer different trade-offs in speed, accuracy, cost, and privacy.

class AIService
{
    // Fast, cheap extraction for simple tasks
    public function quickExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('groq')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }

    // High-quality extraction for complex tasks
    public function precisionExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('anthropic')
            ->withModel('claude-3-opus-20240229')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }

    // Local extraction for sensitive data
    public function privateExtract(string $text, string $model): mixed
    {
        return StructuredOutput::connection('ollama')
            ->with(messages: $text, responseModel: $model)
            ->get();
    }
}

---

Cached Context (Prompt Caching)

For repeated extractions with the same system prompt, examples, or large context, use withCachedContext() to signal that the context should be cached by providers that support prompt caching (e.g., Anthropic, OpenAI). This can significantly reduce latency and cost for subsequent calls.

$result = StructuredOutput::withCachedContext(
    system: 'You are a legal document analyzer...',
    examples: $examples,
)->with(
    messages: $newDocument,
    responseModel: LegalAnalysis::class,
)->get();

---

Caching Strategies

Response Caching

Cache extraction results for identical inputs to avoid redundant API calls.

use Illuminate\Support\Facades\Cache;

class CachedExtractor
{
    public function extract(string $text, string $responseModel): mixed
    {
        $cacheKey = 'extract:' . md5($text . $responseModel);

        return Cache::remember($cacheKey, 3600, function () use ($text, $responseModel) {
            return StructuredOutput::with(
                messages: $text,
                responseModel: $responseModel,
            )->get();
        });
    }
}

Semantic Caching

Use embeddings to find cached results for semantically similar (but not identical) inputs.

use Cognesy\Instructor\Laravel\Facades\Embeddings;

class SemanticCache
{
    public function extractWithCache(string $text, string $responseModel): mixed
    {
        // Generate embedding for input
        $embedding = Embeddings::withInputs($text)->first();

        // Check for similar cached results
        $cached = $this->findSimilar($embedding);
        if ($cached) {
            return $cached;
        }

        // Extract and cache
        $result = StructuredOutput::with(
            messages: $text,
            responseModel: $responseModel,
        )->get();

        $this->store($embedding, $result);

        return $result;
    }
}

---

Batch Processing

Process multiple items efficiently, either synchronously or via queued jobs for large batches.

use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Bus;

class BatchExtractor
{
    public function extractBatch(Collection $documents): Collection
    {
        return $documents->map(function ($document) {
            return StructuredOutput::with(
                messages: $document->content,
                responseModel: DocumentData::class,
            )->get();
        });
    }

    // Or with queued jobs for large batches
    public function extractBatchAsync(Collection $documents): void
    {
        $jobs = $documents->map(fn ($doc) => new ExtractDocumentJob($doc));

        Bus::batch($jobs)
            ->name('Document Extraction')
            ->dispatch();
    }
}

---

Error Handling

Graceful Degradation

Wrap extraction calls in try-catch blocks to handle API failures without crashing your application.

use Cognesy\Instructor\Laravel\Facades\StructuredOutput;

class ResilientExtractor
{
    public function extract(string $text): ?PersonData
    {
        try {
            return StructuredOutput::with(
                messages: $text,
                responseModel: PersonData::class,
            )->get();
        } catch (\Throwable $e) {
            Log::warning('Extraction failed', [
                'error' => $e->getMessage(),
                'text' => substr($text, 0, 100),
            ]);

            return null;
        }
    }
}

Fallback Providers

Automatically try alternative providers when the primary one fails. This pattern provides resilience against provider outages and rate limits.

class FallbackExtractor
{
    private array $providers = ['openai', 'anthropic', 'groq'];

    public function extract(string $text, string $model): mixed
    {
        foreach ($this->providers as $provider) {
            try {
                return StructuredOutput::connection($provider)
                    ->with(messages: $text, responseModel: $model)
                    ->get();
            } catch (\Throwable $e) {
                Log::warning("Provider {$provider} failed", [
                    'error' => $e->getMessage(),
                ]);
                continue;
            }
        }

        throw new RuntimeException('All providers failed');
    }
}

---

Performance Optimization

Reduce Token Usage

// Be concise in system prompts
$result = StructuredOutput::with(
    messages: $text,
    responseModel: MyModel::class,
    system: 'Extract data. Be concise.', // Short system prompt
)->get();

// Use smaller models for simple extractions
$result = StructuredOutput::withModel('gpt-4o-mini')
    ->with(messages: $text, responseModel: SimpleModel::class)
    ->get();

Parallel Extraction

Use Laravel's concurrency features to run multiple extractions simultaneously.

use Illuminate\Support\Facades\Concurrency;

$results = Concurrency::run([
    fn () => StructuredOutput::with(messages: $text1, responseModel: Model::class)->get(),
    fn () => StructuredOutput::with(messages: $text2, responseModel: Model::class)->get(),
    fn () => StructuredOutput::with(messages: $text3, responseModel: Model::class)->get(),
]);