stormlog.infer.otlp_wire

Count an OTLP/HTTP protobuf trace export’s messages before it is parsed.

Parsing an ExportTraceServiceRequest costs memory per message far more than per byte: an empty span is 2 bytes on the wire and over a hundred in upb’s arena, a kilobyte as a pure-Python message. Unknown fields, and the elements of a repeated string or number, are kept one by one as well. So the receiver charges a parse by what it builds, counted here first by a linear scan of the wire format that builds nothing. The scan reads its schema from the installed opentelemetry-proto descriptors, so it follows every field the parser will build, those a newer version adds included, and knows which fields the parser keeps as unknown. The same scan counts the spans, so an export with too many is refused before ParseFromString runs.

The scan stops as soon as a count passes its cap; what it returns then is over that cap, which is all the receiver needs to refuse the export. It runs in Python, about 40 ms for a batch of 512 vLLM request spans but seconds for a body of millions of tiny fields, so it also stops, with TimeoutError, once a deadline passes. Anything protobuf could not parse as this message is a ValueError. So are two things the pure-Python backend of protobuf 4 would parse: groups (wire types 3 and 4), which no OTLP message uses, and messages nested more than 100 levels below the request, protobuf’s default limit, where upb and later pure-Python versions stop.

Functions

count_message(data, root, *, max_messages, ...)	count_trace_request() for a message of any type, by its Descriptor.
count_trace_request(data, *, max_messages, ...)	Count what parsing an encoded export would build.
message_schema(root)	The schema of root, a message Descriptor, and of every message type it can hold.
trace_request_descriptor()	The installed ExportTraceServiceRequest descriptor; ImportError without the infer-otlp extra.

Classes

WireCounts(messages, spans, values, ...)

What parsing the export would build.

class stormlog.infer.otlp_wire.WireCounts(messages, spans, values, elements, unknown_bytes)

Bases: object

What parsing the export would build.

values counts every attribute (KeyValue) and every element of an array value, wherever they sit: at least the attribute values the spans keep. elements counts what the parser keeps one by one outside a message’s own fields: each unknown field (a field number the schema does not define, or a wire type its field does not take), and each element of a repeated string, bytes or number field. unknown_bytes is the size of the unknown fields, tags included.

Parameters:

• messages (int)

• spans (int)

• values (int)

• elements (int)

• unknown_bytes (int)

messages: int

spans: int

values: int

elements: int

unknown_bytes: int

stormlog.infer.otlp_wire.count_message(data, root, *, max_messages, max_spans, max_elements, deadline=None)

count_trace_request() for a message of any type, by its Descriptor.

Parameters:

• data (bytes | bytearray)

• root (Any)

• max_messages (int)

• max_spans (int)

• max_elements (int)

• deadline (float | None)

Return type:

WireCounts

stormlog.infer.otlp_wire.count_trace_request(data, *, max_messages, max_spans, max_elements, deadline=None)

Count what parsing an encoded export would build.

Stops once messages, spans or elements passes its cap. Raises TimeoutError once time.monotonic() passes deadline.

Parameters:

• data (bytes | bytearray)

• max_messages (int)

• max_spans (int)

• max_elements (int)

• deadline (float | None)

Return type:

WireCounts

stormlog.infer.otlp_wire.message_schema(root)

The schema of root, a message Descriptor, and of every message type it can hold.

Parameters:

root (Any)

Return type:

_Schema

stormlog.infer.otlp_wire.trace_request_descriptor()

The installed ExportTraceServiceRequest descriptor; ImportError without the infer-otlp extra.

Return type:

Any