Skip to content

Normalization

Canonical forms for indicator values.

Normalization never replaces the original sieve value. Callers keep both. The steps are Unicode NFKC, refanging, trimming, and a type-specific rewrite.

Functions:

  • normalize_value –

    Return the canonical form of one indicator value.

  • fold_domain –

    Lower-case a domain, drop a trailing dot, and encode it with IDNA.

  • apply_input_nfkc –

    Apply NFKC per character and record original spans.

normalize_value 

normalize_value(kind: str, value: str) -> str

Return the canonical form of one indicator value.

Parameters:

  • kind (str) –

    An :class:~qarbaal.types.IOCType member, for example IOCType.URL.

  • value (str) –

    Sieve value or raw text. Defanged text is refanged first.

Returns:

  • str –

    The normalized value. Unparseable addresses are still cleaned

  • str –

    (case, surrounding space) and returned.

fold_domain 

fold_domain(value: str) -> str

Lower-case a domain, drop a trailing dot, and encode it with IDNA.

Parameters:

  • value (str) –

    A host name. It may contain Unicode labels.

Returns:

  • str –

    An ASCII punycode host when IDNA accepts it, otherwise the lower-case name.

apply_input_nfkc 

apply_input_nfkc(text: str) -> tuple[str, list[tuple[int, int]]]

Apply NFKC per character and record original spans.

Compatibility characters such as full-width dots become their ASCII equivalents. Each output character maps back to the source character that produced it, so later offset tracking still points at the input.

Parameters:

  • text (str) –

    Original input.

Returns: