Package org.apache.tika.eval.core.tokens
Class AnalyzerManager
- java.lang.Object
-
- org.apache.tika.eval.core.tokens.AnalyzerManager
-
public class AnalyzerManager extends Object
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description org.apache.lucene.analysis.AnalyzergetCommonTokensAnalyzer()This analyzer should be used to generate common tokens lists from large corpora.org.apache.lucene.analysis.AnalyzergetGeneralAnalyzer()This analyzer should be used to extract all tokens.static AnalyzerManagernewInstance(int maxTokens)
-
-
-
Method Detail
-
newInstance
public static AnalyzerManager newInstance(int maxTokens)
-
getGeneralAnalyzer
public org.apache.lucene.analysis.Analyzer getGeneralAnalyzer()
This analyzer should be used to extract all tokens.- Returns:
-
getCommonTokensAnalyzer
public org.apache.lucene.analysis.Analyzer getCommonTokensAnalyzer()
This analyzer should be used to generate common tokens lists from large corpora. It is not used by tika-eval in profiling or comparing.- Returns:
-
-