org.apache.tika.eval.tokens

Class AnalyzerManager

java.lang.Object

org.apache.tika.eval.tokens.AnalyzerManager

public class AnalyzerManager
extends Object

Method Summary

Modifier and Type	Method and Description
org.apache.lucene.analysis.Analyzer	getCommonTokensAnalyzer() This analyzer should be used to generate common tokens lists from large corpora.
org.apache.lucene.analysis.Analyzer	getGeneralAnalyzer() This analyzer should be used to extract all tokens.
static AnalyzerManager	newInstance(int maxTokens)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

newInstance

public static AnalyzerManager newInstance(int maxTokens)
                                   throws IOException

Throws:

IOException

getGeneralAnalyzer

public org.apache.lucene.analysis.Analyzer getGeneralAnalyzer()

This analyzer should be used to extract all tokens.

Returns:

getCommonTokensAnalyzer

public org.apache.lucene.analysis.Analyzer getCommonTokensAnalyzer()

This analyzer should be used to generate common tokens lists from large corpora. It is not used by tika-eval in profiling or comparing.

Returns: