stormlog.scrub

Shared scrubbing primitives for what Stormlog records or sends elsewhere.

An exporter builds its output from allowlists of fields; these helpers are the layers underneath, shared so that every surface redacts the same way. Each exporter documents its own allowlists. See docs/scrubbing.md.

Functions

`is_forbidden_key_name`(name)	Whether a key's name says its value may be a credential.
`redact_url`()	A URL as it may be recorded: no credentials and no query string.
`replace_spans`(text, spans, *[, limit])	`text` with each span, overlapping or touching spans merged, redacted.
`scrub_text`(text, *[, max_bytes, secrets])	Free text with credentials removed, then cut to `max_bytes`.
`truncate_utf8`(text, max_bytes)	The longest prefix of `text` whose UTF-8 encoding fits `max_bytes`.
`url_secrets`(url)	The values in a URL that may be credentials, for `KnownSecrets`.

Classes

KnownSecrets([values])

The exact credentials Stormlog was given, redacted wherever they appear.

stormlog.scrub.redact_url(url: str, *, origin_only: bool = False) → str[source]

stormlog.scrub.redact_url(url: None, *, origin_only: bool = False) → None

A URL as it may be recorded: no credentials and no query string.

Either can carry a token, so only the scheme, host, port and path are kept, and a removed query is marked. origin_only drops the path too, for a destination whose path is not known to be safe: a token can sit in a path segment as easily as in a query.

class stormlog.scrub.KnownSecrets(values=())[source]

Bases: object

The exact credentials Stormlog was given, redacted wherever they appear.

Allowlists decide which fields leave; this is the backstop behind them. Each value is matched in every spelling it most often travels in: as given; percent-encoded (inside a URL) in either case of hex digit, with any characters left plain, and with + for a space; JSON-escaped (inside a JSON string), with or without unicode escapes in either case, surrogate pairs, an escaped / and the short escapes; and each character may be spelled differently from the next. Also base64, standard and URL-safe, padded or not, of the value on its own: an encoded token, or a Basic authorization header when the user:password pair is registered.

Parameters:: values (Iterable[str | None])

add(value)[source]

Register one value; empty and short values are skipped.

Parameters:: value (str | None)
Return type:: None

redact(text)[source]

text with every form of every registered value replaced.

Every occurrence is found in the original text first, overlapping ones included, and the spans are merged before anything is replaced, so replacing one value can never uncover part of another.

Parameters:: text (str)
Return type:: str

spans(text)[source]

Where each form of each value occurs in text, as (start, end).

Parameters:: text (str)
Return type:: list[tuple[int, int]]

found_in(text)[source]

Whether any spelling of any registered value occurs in text.

Parameters:: text (str)
Return type:: bool

stormlog.scrub.replace_spans(text, spans, *, limit=None)[source]

text with each span, overlapping or touching spans merged, redacted.

With limit, only text[:limit] is kept: a span that starts before it is redacted whole, and nothing after it is copied.

Parameters:

text (str)
spans (Iterable[tuple[int, int]])
limit (int | None)

Return type:

str

stormlog.scrub.url_secrets(url)[source]

The values in a URL that may be credentials, for KnownSecrets.

The user name, on its own whatever the password (it is often a token), the password, the user:password pair a Basic header would encode, decoded and also as written in the URL for a client that did not decode it, and every query value. A fragment is never sent to a server, so it is not read.

Parameters:: url (str | None)
Return type:: list[str]

stormlog.scrub.truncate_utf8(text, max_bytes)[source]

The longest prefix of text whose UTF-8 encoding fits max_bytes.

A character is never split. A lone surrogate, which UTF-8 cannot encode, becomes ?.

Parameters:

text (str)
max_bytes (int)

Return type:

str

stormlog.scrub.scrub_text(text, *, max_bytes=None, secrets=None)[source]

Free text with credentials removed, then cut to max_bytes.

For text an exporter has consent to send, such as an error message, and only as a layer under its allowlist: patterns catch the common shapes of a credential, not every secret. Every match, of the exact values in secrets and of the patterns, is found in the same text before any is replaced; the cut comes last, so it never leaves part of a secret that a whole match would have removed.

Parameters:

text (str)
max_bytes (int | None)
secrets (KnownSecrets | None)

Return type:

str

stormlog.scrub.is_forbidden_key_name(name)[source]

Whether a key’s name says its value may be a credential.

An exporter refuses to admit such a key from any outside source (an environment variable or a flag), even one an operator names explicitly, because the value cannot be checked.

Parameters:: name (str)
Return type:: bool