Package org.apache.tika.ml.chardetect

Record Class StructuralEncodingRules.Utf8Stats

java.lang.Object
java.lang.Record
org.apache.tika.ml.chardetect.StructuralEncodingRules.Utf8Stats
Enclosing class:
StructuralEncodingRules
public static record StructuralEncodingRules.Utf8Stats(int errors, int sequences, boolean truncatedTailInvalid) extends Record
Single-pass tally of UTF-8 structure over the sample: malformed-sequence events and complete valid multi-byte sequences. Same walk as StructuralEncodingRules.checkUtf8(byte[]) but without the early exit — scans the entire range, resyncing after each error.

errors counts one event per bad lead, orphaned continuation, overlong, surrogate, or out-of-range codepoint regardless of how many bytes the bad sequence spans — matching Java's new String(bytes, UTF_8) U+FFFD-per-error semantics, the convention callers use for tolerance thresholds. A truncated sequence at probe-end is neither an error nor a sequence.

sequences gauges how much genuine multi-byte UTF-8 evidence the probe carries independent of its error count.

truncatedTailInvalid: the probe ends in a truncated sequence whose partial continuations are provably NOT UTF-8 (e.g. E0 41 at probe-end). Not an error event (no U+FFFD equivalent — the decoder would ask for more input), but it disqualifies the sample from StructuralEncodingRules.Utf8Result.LIKELY_UTF8/StructuralEncodingRules.Utf8Result.AMBIGUOUS.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Utf8Stats(int errors, int sequences, boolean truncatedTailInvalid)
    Creates an instance of a Utf8Stats record class.

  • Method Summary

    Modifier and Type
    Method
    Description
    final boolean
    equals(Object o)
    Indicates whether some other object is "equal to" this one.
    int
    errors()
    Returns the value of the errors record component.
    final int
    hashCode()
    Returns a hash code value for this object.
    int
    sequences()
    Returns the value of the sequences record component.
    StructuralEncodingRules.Utf8Result
    toResult()
    Collapse to the StructuralEncodingRules.checkUtf8(byte[]) tri-state: any invalidity → NOT_UTF8; else ≥1 complete multi-byte sequence → LIKELY_UTF8 (a lone truncated lead is no structural evidence); else AMBIGUOUS (pure ASCII, or truncated-lead-only).
    final String
    toString()
    Returns a string representation of this record class.
    boolean
    truncatedTailInvalid()
    Returns the value of the truncatedTailInvalid record component.

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • Utf8Stats

      public Utf8Stats(int errors, int sequences, boolean truncatedTailInvalid)
      Creates an instance of a Utf8Stats record class.
      Parameters:
      errors - the value for the errors record component
      sequences - the value for the sequences record component
      truncatedTailInvalid - the value for the truncatedTailInvalid record component

  • Method Details

    • toResult

      public StructuralEncodingRules.Utf8Result toResult()
      Collapse to the StructuralEncodingRules.checkUtf8(byte[]) tri-state: any invalidity → NOT_UTF8; else ≥1 complete multi-byte sequence → LIKELY_UTF8 (a lone truncated lead is no structural evidence); else AMBIGUOUS (pure ASCII, or truncated-lead-only).

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • errors

      public int errors()
      Returns the value of the errors record component.
      Returns:
      the value of the errors record component

    • sequences

      public int sequences()
      Returns the value of the sequences record component.
      Returns:
      the value of the sequences record component

    • truncatedTailInvalid

      public boolean truncatedTailInvalid()
      Returns the value of the truncatedTailInvalid record component.
      Returns:
      the value of the truncatedTailInvalid record component