Package org.apache.tika.ml.chardetect.tools

package org.apache.tika.ml.chardetect.tools

Related Packages

Package	Description
org.apache.tika.ml.chardetect

Classes

Class	Description
BenchmarkCharsetDetectors	Micro-benchmark comparing charset detector throughput.
BuildCharsetTrainingData	Generates charset-detection training, devtest, and test data from MADLAD-400 and Cantonese Wikipedia sentence files.
DiagnoseDiscrimination	Diagnostic for the win-1252-vs-MacRoman / IBM850 / windows-1250 / windows-1257 / etc. confusion problem.
EvalCharsetDetectors	Compares MojibusterEncodingDetector against ICU4J and juniversalchardet.
RebalanceCharsetTraining	Rebalance the per-class training corpus so all Western European Latin SBCS classes have the same underlying source text — eliminating the "common ASCII bigrams pull toward MacRoman" bias surfaced by DiagnoseDiscrimination.
TrainNaiveBayesBigram	Naive-Bayes byte-bigram charset classifier trainer.