Module chat_completion

Chat Completions functionality and route handler.

Functions

chatcompletions

OpenAI-compatible chat completions endpoint handler.

create_streamer

Creates a SSE streamer for chat completions with optional callbacks.

handle_error

Handle route / generation errors and logging them.

match_responses

Matches and processes different types of model responses into appropriate chat completion responses.

parse_request

Parses and validates a chat completion request.

process_non_streaming_response

Process non-streaming chat completion responses.

Type Aliases

ChatCompletionOnChunkCallback

A callback function that processes streaming response chunks before they are sent to the client.

ChatCompletionOnDoneCallback

A callback function that is executed when the streaming response completes.

ChatCompletionResponder

Represents different types of chat completion responses.

ChatCompletionStreamer

A streaming response handler.