stormlog.infer.openai_client

OpenAI-compatible Chat Completions client used by inference profiling.

Classes

ChatCompletionResult(text, started_at_ns, ...)

Parsed response and client-observed timing metadata.

OpenAIChatCompletionsClient(*, endpoint, ...)

Minimal OpenAI-compatible Chat Completions HTTP client.
class stormlog.infer.openai_client.ChatCompletionResult(text, started_at_ns, ended_at_ns, e2e_latency_ms, ttft_ms, first_chunk_latency_ms, chunk_interarrival_ms=<factory>, usage=None, finish_reason=None)[source]

Bases: object

Parsed response and client-observed timing metadata.

Parameters:

  • text (str)

  • started_at_ns (int)

  • ended_at_ns (int)

  • e2e_latency_ms (float)

  • ttft_ms (float | None)

  • first_chunk_latency_ms (float | None)

  • chunk_interarrival_ms (list[float])

  • usage (dict[str, Any] | None)

  • finish_reason (str | None)

text: str
started_at_ns: int
ended_at_ns: int
e2e_latency_ms: float
ttft_ms: float | None
first_chunk_latency_ms: float | None
chunk_interarrival_ms: list[float]
usage: dict[str, Any] | None = None
finish_reason: str | None = None
class stormlog.infer.openai_client.OpenAIChatCompletionsClient(*, endpoint, model, timeout_seconds, api_key=None, max_tokens_field='max_tokens')[source]

Bases: object

Minimal OpenAI-compatible Chat Completions HTTP client.

Parameters:

  • endpoint (str)

  • model (str)

  • timeout_seconds (float)

  • api_key (str | None)

  • max_tokens_field (Literal['max_tokens', 'max_completion_tokens'])

complete(*, prompt, output_tokens, stream, stream_include_usage)[source]
Parameters:

  • prompt (str)

  • output_tokens (int)

  • stream (bool)

  • stream_include_usage (bool)

Return type:

ChatCompletionResult