Module completions

Completions functionality and route handler.

Functions

completions

OpenAI-compatible completions endpoint handler.

create_streamer

Creates a SSE streamer for chat completions with optional callbacks.

handle_error

Handle route / generation errors and logging them.

match_responses

Matches and processes different types of model responses into appropriate completion responses.

parse_request

Parses and validates a completion request.

process_non_streaming_response

Process non-streaming completion responses.

Type Aliases

CompletionOnChunkCallback

A callback function that processes streaming response chunks before they are sent to the client.

CompletionOnDoneCallback

A callback function that is executed when the streaming response completes.

CompletionResponder

Represents different types of completion responses.

CompletionStreamer

A streaming response handler.