Class CommonTokenOverlapCounter

java.lang.Object

org.apache.tika.eval.app.tools.CommonTokenOverlapCounter

public class CommonTokenOverlapCounter
extends Object

Dev tool that reports pairs of languages whose common-token lists overlap by more than 1%.

The shipped runtime resources are Bloom filters, which cannot be enumerated, so this reads the raw token<TAB>df<TAB>cf source lists directly. Point it at a directory of those files (e.g. tika-eval-core/src/test/resources/common_tokens).

Constructor Summary

Constructors

Constructor	Description
CommonTokenOverlapCounter()

Method Summary

Modifier and Type	Method	Description
static void	main(String[] args)

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CommonTokenOverlapCounter

public CommonTokenOverlapCounter()

Method Details

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception