Package org.apache.tika.eval.core.tokens
Class AnalyzerManager
- java.lang.Object
-
- org.apache.tika.eval.core.tokens.AnalyzerManager
-
public class AnalyzerManager extends Object
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description org.apache.lucene.analysis.Analyzer
getCommonTokensAnalyzer()
This analyzer should be used to generate common tokens lists from large corpora.org.apache.lucene.analysis.Analyzer
getGeneralAnalyzer()
This analyzer should be used to extract all tokens.static AnalyzerManager
newInstance(int maxTokens)
-
-
-
Method Detail
-
newInstance
public static AnalyzerManager newInstance(int maxTokens)
-
getGeneralAnalyzer
public org.apache.lucene.analysis.Analyzer getGeneralAnalyzer()
This analyzer should be used to extract all tokens.- Returns:
-
getCommonTokensAnalyzer
public org.apache.lucene.analysis.Analyzer getCommonTokensAnalyzer()
This analyzer should be used to generate common tokens lists from large corpora. It is not used by tika-eval in profiling or comparing.- Returns:
-
-