Package org.apache.tika.eval.core.tokens
Class CommonTokenCountManager
- java.lang.Object
-
- org.apache.tika.eval.core.tokens.CommonTokenCountManager
-
public class CommonTokenCountManager extends Object
-
-
Constructor Summary
Constructors Constructor Description CommonTokenCountManager()
CommonTokenCountManager(Path commonTokensDir, String defaultLangCode)
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
close()
CommonTokenResult
countTokenOverlaps(String langCode, Map<String,org.apache.commons.lang3.mutable.MutableInt> tokens)
Deprecated.Set<String>
getLangs()
org.apache.commons.lang3.tuple.Pair<String,LangModel>
getLangTokens(String lang)
Set<String>
getTokens(String lang)
-
-
-
Method Detail
-
countTokenOverlaps
@Deprecated public CommonTokenResult countTokenOverlaps(String langCode, Map<String,org.apache.commons.lang3.mutable.MutableInt> tokens) throws IOException
Deprecated.- Throws:
IOException
-
getLangTokens
public org.apache.commons.lang3.tuple.Pair<String,LangModel> getLangTokens(String lang)
- Parameters:
lang
-- Returns:
- pair of actual language code used and a set of common tokens for that language
-
close
public void close() throws IOException
- Throws:
IOException
-
-