Uses of Class
org.apache.tika.ml.junkdetect.BigramTables

Packages that use BigramTables

Package	Description
org.apache.tika.ml.junkdetect
org.apache.tika.ml.junkdetect.tools

Uses of BigramTables in org.apache.tika.ml.junkdetect

Methods in org.apache.tika.ml.junkdetect that return BigramTables

Modifier and Type	Method	Description
static BigramTables	BigramTables.readFrom(DataInputStream dis)	Inverse of BigramTables.writeTo(DataOutputStream).

Methods in org.apache.tika.ml.junkdetect with parameters of type BigramTables

Modifier and Type	Method	Description
static int	JunkDetector.codepointToIndex(BigramTables tables, int cp)	Binary-search a codepoint in the script's index.
static double	JunkDetector.computeF1MeanLogP(int[] cps, BigramTables tables)	Codepoint-array form of JunkDetector.computeF1MeanLogP(String, BigramTables).
static double	JunkDetector.computeF1MeanLogP(String text, BigramTables tables)	Mean log-prob over the codepoint pairs in text using the given script's bigram tables.

Method parameters in org.apache.tika.ml.junkdetect with type arguments of type BigramTables

Modifier and Type	Method	Description
static Map<String,double[]>	JunkDetector.bucketSumsAndCounts(int[] cps, Map<String, BigramTables> tablesByScript)	Per-script {sumLogP, count} buckets for a codepoint sequence, scored with the given per-script tables.

Uses of BigramTables in org.apache.tika.ml.junkdetect.tools

Methods in org.apache.tika.ml.junkdetect.tools that return BigramTables

Modifier and Type	Method	Description
static BigramTables	TrainJunkModel.buildBigramTablesFromCounts(HashMap<Long,long[]> pairCounts, HashMap<Integer,long[]> unigramCounts, long unigramTotal, int minBigramCount, double loadFactor, int keyIndexBits)	Builds the BigramTables carrier from pre-tallied pair/unigram counts.
static BigramTables	TrainJunkModel.trainBigramTablesForScript(Path trainFile, int minBigramCount, double loadFactor, int keyIndexBits)	Builds the BigramTables carrier for the dominant script bucket of one *.train.gz file.

Method parameters in org.apache.tika.ml.junkdetect.tools with type arguments of type BigramTables

Modifier and Type	Method	Description
static float[]	TrainJunkModel.calibrateBucketScript(Path file, String script, Map<String, BigramTables> tables)	Bucket-model z1 calibration {mu, sigma} for one script, sampling windows from file and taking that script's bucket mean log-prob per window (the inference z1 input).
static void	TrainJunkModel.saveModel(TreeMap<String, BigramTables> f1Tables, TreeMap<String,float[]> f1Calibrations, float[] blockTable, float[] blockCal, float[] controlCal, float[] combinerWeights, List<String> scriptBuckets, float[] scriptTransTable, float[] scriptTransCal, Path output)	Writes a model file in the current binary format.