stormlog.infer.cache_state

Requested and verified prefix-cache state for each workload case.

A benchmark can ask for a cold cache and can call a reset endpoint before each case, such as vLLM’s /reset_prefix_cache (available when the server runs in development mode) or SGLang’s /flush_cache. Asking is not proof: until an engine adapter can read the cache’s contents, every case records its cache state as unverified and says why.

A 2xx answer is not proof of a reset either. vLLM answers HTTP 200 with {"success": false} while blocks are still held, so the body is read: a reset is acknowledged only when the server says success: true, refused when it keeps saying anything else, and accepted but unconfirmed when a 2xx body has no success field.

Functions

cache_lines(cache)	A text-report line when a cache state was requested or reset.
cache_state_record(*, session_id, case_id, ...)	The infer.cache_state record written before a case runs.
cache_summary(record)	The cache block of a case report; older artifacts did not record one.
reset_cache(url, *, timeout_seconds[, ...])	POST to a reset endpoint and record what happened; never raises.
run_kind(requested, warmup_requests[, reset])	Whether a case was designed as a cold start or a warmed-up steady state.

Classes

CacheReset(url, at_ns[, status, error, ...])

The outcome of a cache reset, over every attempt it took.

class stormlog.infer.cache_state.CacheReset(url, at_ns, status=None, error=None, success=None, attempts=1, answered_at_ns=None)

Bases: object

The outcome of a cache reset, over every attempt it took.

success is True when the body said "success": true, False when it had a success field with any other value, and None without one. at_ns is when the first attempt was sent; answered_at_ns is when the answer recorded here, the last attempt’s, came back.

Parameters:

• url (str)

• at_ns (int)

• status (int | None)

• error (str | None)

• success (bool | None)

• attempts (int)

• answered_at_ns (int | None)

url: str

at_ns: int

status: int | None = None

error: str | None = None

success: bool | None = None

attempts: int = 1

answered_at_ns: int | None = None

property answer: str | None

acknowledged, refused or accepted_unverified; None if no 2xx.

property succeeded: bool

The server accepted the reset, whether or not it confirmed it.

property acknowledged: bool

The server said the reset happened.

to_record()

Return type:

dict[str, Any]

stormlog.infer.cache_state.reset_cache(url, *, timeout_seconds, api_key=None, retry_seconds=10.0)

POST to a reset endpoint and record what happened; never raises.

A reset the server refuses (success: false) is tried again every half second for up to retry_seconds. No attempt starts after that, though the last one may take up to timeout_seconds to answer. The API key, when there is one, goes along as it does with every request, since the reset route usually sits behind the same server.

Parameters:

• url (str)

• timeout_seconds (float)

• api_key (str | None)

• retry_seconds (float)

Return type:

CacheReset

stormlog.infer.cache_state.run_kind(requested, warmup_requests, reset=None)

Whether a case was designed as a cold start or a warmed-up steady state.

This names the run’s design, not evidence about the cache. A cold start whose reset failed is unspecified: the failure shows the cache was not cleared.

Parameters:

• requested (str)

• warmup_requests (int)

• reset (CacheReset | None)

Return type:

str

stormlog.infer.cache_state.cache_state_record(*, session_id, case_id, requested, reset, warmup_requests)

The infer.cache_state record written before a case runs.

Parameters:

• session_id (str)

• case_id (str)

• requested (str)

• reset (CacheReset | None)

• warmup_requests (int)

Return type:

dict[str, Any]

stormlog.infer.cache_state.cache_summary(record)

The cache block of a case report; older artifacts did not record one.

Parameters:

record (dict[str, Any] | None)

Return type:

dict[str, Any]

stormlog.infer.cache_state.cache_lines(cache)

A text-report line when a cache state was requested or reset.

Parameters:

cache (Any)

Return type:

list[str]