stormlog.infer.config

Configuration models for inference profiling.

Functions

parse_int_list(value, *, field_name)	Parse a comma-separated positive integer list.
resolve_endpoint(*, endpoint, base_url)	Resolve either a full chat-completions endpoint or a /v1 base URL.

Classes

ProfileConfig(endpoint, model, concurrency, ...)	Resolved configuration for one inference profiling run.
WorkloadCase(case_id, concurrency, ...)	One inference profiling workload shape.

stormlog.infer.config.parse_int_list(value, *, field_name)

Parse a comma-separated positive integer list.

Parameters:

• value (str)

• field_name (str)

Return type:

list[int]

stormlog.infer.config.resolve_endpoint(*, endpoint, base_url)

Resolve either a full chat-completions endpoint or a /v1 base URL.

Parameters:

• endpoint (str | None)

• base_url (str | None)

Return type:

str

class stormlog.infer.config.WorkloadCase(case_id, concurrency, input_tokens, output_tokens)

Bases: object

One inference profiling workload shape.

Parameters:

• case_id (str)

• concurrency (int)

• input_tokens (int)

• output_tokens (int)

case_id: str

concurrency: int

input_tokens: int

output_tokens: int

class stormlog.infer.config.ProfileConfig(endpoint, model, concurrency, input_tokens, output_tokens, output_path, duration_seconds=None, request_count=1, stream=True, stream_include_usage=True, timeout_seconds=60.0, warmup_requests=0, seed=0, api_key=None, max_tokens_field='max_tokens', tokenizer='auto', tokenizer_model=None, tiktoken_encoding=None, strict_token_counts=False, system_sampler='auto', sample_interval_seconds=1.0)

Bases: object

Resolved configuration for one inference profiling run.

Parameters:

• endpoint (str)

• model (str)

• concurrency (tuple[int, ...])

• input_tokens (tuple[int, ...])

• output_tokens (tuple[int, ...])

• output_path (str)

• duration_seconds (float | None)

• request_count (int | None)

• stream (bool)

• stream_include_usage (bool)

• timeout_seconds (float)

• warmup_requests (int)

• seed (int)

• api_key (str | None)

• max_tokens_field (Literal['max_tokens', 'max_completion_tokens'])

• tokenizer (str)

• tokenizer_model (str | None)

• tiktoken_encoding (str | None)

• strict_token_counts (bool)

• system_sampler (str)

• sample_interval_seconds (float)

endpoint: str

model: str

concurrency: tuple[int, ...]

input_tokens: tuple[int, ...]

output_tokens: tuple[int, ...]

output_path: str

duration_seconds: float | None = None

request_count: int | None = 1

stream: bool = True

stream_include_usage: bool = True

timeout_seconds: float = 60.0

warmup_requests: int = 0

seed: int = 0

api_key: str | None = None

max_tokens_field: Literal['max_tokens', 'max_completion_tokens'] = 'max_tokens'

tokenizer: str = 'auto'

tokenizer_model: str | None = None

tiktoken_encoding: str | None = None

strict_token_counts: bool = False

system_sampler: str = 'auto'

sample_interval_seconds: float = 1.0

cases()

Return type:

list[WorkloadCase]