Package org.apache.tika.ml.chardetect
package org.apache.tika.ml.chardetect
-
ClassDescriptionReads an encoding-detection probe sized by content, not raw bytes.Charset relationships used for lenient (lenient) evaluation of charset detectors.Structural false-CJK veto: measures how badly a probe fails to decode under a legacy multi-byte CJK charset, robustly against embedded UTF-8.Cheap byte-wise decode-equivalence check for single-byte charsets.Byte-level HTML tag stripper used as a preprocess for charset detection.Result of a strip operation: new content length and the number of well-formed tags (including comments) successfully parsed.Naive-Bayes pipeline detector: structural checks for wide Unicode + BOMs before falling through to the bigram NB classifier for everything else.Naive-Bayes byte-bigram charset classifier.Per-bigram contribution to the per-class score, used for diagnostic tools that want to understand why a probe scores one class over another.Script / writing-system family used by
NaiveBayesBigramEncodingDetector.CAP_PER_BIGRAM_NATS.Score result returned byNaiveBayesBigramEncodingDetector.scoreClassesAndCount(byte[]).Raw per-class logits from a single MoE specialist.SPI contract for an MoE charset-detection specialist.Fast, rule-based encoding checks that run before the statistical model.Outcome of the UTF-8 structural check.Feature extractor for the UTF-16 specialist of the mixture-of-experts charset detector.UTF-16 specialist detector of the mixture-of-experts charset detection architecture.