stormlog.infer.vllm_hook.writer

The vLLM hook’s raw log: one epoch directory per process and role.

Records go into a bounded in-memory queue and a daemon thread writes them, so a patched vLLM call never waits on a disk. Each record’s fields are serialized once, as they are queued, and the queue bounds the number of records and the exact size of that JSON until written; a record that does not fit, or is too large on its own, is dropped and counted. The thread checks its deadlines after every record, so heartbeats, flush requests and sealing stay on time under a backlog. Each line is written whole or not at all. Segments are sealed by renaming .part to .jsonl; a failed seal is counted and retried. A status file is rewritten every heartbeat, and keeps being rewritten after the disk cap stops record writing, so loss stays visible. See docs/vllm_execution.md.

Functions

remove_old_epochs(root, retain_hours, *[, now])

Delete epoch directories untouched for longer than retain_hours.

Classes

EpochWriter(root, role, *[, limits, ...])	Write one process role's raw log; every public method is thread-safe.
WriterLimits([max_bytes, segment_bytes, ...])	Bounds on memory, disk and timing for one epoch.

class stormlog.infer.vllm_hook.writer.EpochWriter(root, role, *, limits=None, status_fields=None)

Bases: object

Write one process role’s raw log; every public method is thread-safe.

Parameters:

• root (Path)

• role (str)

• limits (WriterLimits | None)

• status_fields (Callable[[], dict[str, Any]] | None)

emit(kind, fields)

Queue one record; drop and count it when it does not fit.

A record the queue cannot take at any size, or whose long strings alone pass record_bytes, is dropped unserialized, so neither a backlog nor a client’s long request IDs cost the caller an encoding. The rest are serialized now. fields must not reuse the common fields’ names, which the writer thread adds with the sequence number.

Parameters:

• kind (str)

• fields (dict[str, Any])

Return type:

None

count_error()

Return type:

None

close(goodbye=True)

Write the last records and seal; wait at most close_seconds.

Parameters:

goodbye (bool)

Return type:

None

class stormlog.infer.vllm_hook.writer.WriterLimits(max_bytes=268435456, segment_bytes=8388608, seal_seconds=60.0, heartbeat_seconds=1.0, queue_records=20000, queue_bytes=33554432, record_bytes=4194304, batch_records=256, close_seconds=2.0)

Bases: object

Bounds on memory, disk and timing for one epoch.

Parameters:

• max_bytes (int)

• segment_bytes (int)

• seal_seconds (float)

• heartbeat_seconds (float)

• queue_records (int)

• queue_bytes (int)

• record_bytes (int)

• batch_records (int)

• close_seconds (float)

max_bytes: int = 268435456

segment_bytes: int = 8388608

seal_seconds: float = 60.0

heartbeat_seconds: float = 1.0

queue_records: int = 20000

queue_bytes: int = 33554432

record_bytes: int = 4194304

batch_records: int = 256

close_seconds: float = 2.0

stormlog.infer.vllm_hook.writer.remove_old_epochs(root, retain_hours, *, now=None)

Delete epoch directories untouched for longer than retain_hours.

Parameters:

• root (Path)

• retain_hours (float)

• now (float | None)

Return type:

int