Package org.apache.tika.langdetect.charsoup.core

package org.apache.tika.langdetect.charsoup.core

Related Packages

Package	Description
org.apache.tika.langdetect.charsoup

Class	Description
CharSoupFeatureExtractor	Extracts character n-gram features from text using the hashing trick (FNV-1a).
CharSoupModel	INT8-quantized multinomial logistic regression model for language detection.
FeatureExtractor	Common interface for feature extractors used by the bigram language detector.
SaltedNgramFeatureExtractor	Feature extractor using positional salt (BOW/EOW/FULL_WORD) instead of sentinel characters in n-grams.
ScriptAwareFeatureExtractor	Production feature extractor for the CharSoup language detection model.
ScriptCategory	Coarse Unicode script categories for language detection.
ShortTextFeatureExtractor	Production feature extractor for the CharSoup short-text language detection model.
WordTokenizer	General-purpose word tokenizer that shares the same preprocessing pipeline as CharSoupFeatureExtractor: NFC normalization, URL/email stripping, case folding via Character.toLowerCase(int).