Configuring Digesters

Tika can hash documents during parsing — useful for deduplication, integrity verification and forensics. Digests land in metadata under tk:digest:<ALGORITHM> for the default HEX encoding, or tk:digest:<ALGORITHM>:<ENCODING> for the others, e.g. tk:digest:SHA-256:BASE32. The key carries the algorithm’s JCA name (SHA-256, SHA3-512); config names it by the enum spelling below (SHA256, SHA3_512).

Two implementations are available, both configured in the parse-context section:

Factory Config key Algorithms

CommonsDigesterFactory

commons-digester-factory

MD2, MD5, SHA1, SHA256, SHA384, SHA512 (Apache Commons Codec)

BouncyCastleDigesterFactory

bouncy-castle-digester-factory

All of the above plus SHA3_256, SHA3_384, SHA3_512 (BouncyCastle provider)

Encodings: HEX (default, lowercase), BASE32 and BASE64, both RFC 4648.

JSON Configuration

tika-config-commons-digests-basic.json
{
  "parse-context": {
    "commons-digester-factory": {
      "digests": [
        { "algorithm": "MD5" },
        { "algorithm": "SHA256" },
        { "algorithm": "SHA512" }
      ]
    }
  }
}

Each entry takes an optional encoding — see tika-config-digests.json:

{ "algorithm": "SHA256", "encoding": "BASE32" }

Using BouncyCastle for SHA3 Algorithms

Swap the factory key; SHA3 algorithms are spelled SHA3_256, SHA3_384, SHA3_512.

tika-config-bc-digests-multiple.json
{
  "parse-context": {
    "bouncy-castle-digester-factory": {
      "digests": [
        { "algorithm": "MD5" },
        { "algorithm": "SHA256" },
        { "algorithm": "SHA3_512" }
      ]
    }
  }
}

Skipping the container digest

To digest only the embedded documents of a container (say, the PDFs inside a zip) and not the container itself, set skipContainerDocumentDigest alongside digests:

tika-config-digests-skip-container.json
{
  "parse-context": {
    "commons-digester-factory": {
      "digests": [ { "algorithm": "MD5" } ],
      "skipContainerDocumentDigest": true
    }
  }
}

Programmatic Configuration

CommonsDigesterFactory factory = new CommonsDigesterFactory();
factory.setDigests(Arrays.asList(
    new DigestDef(DigestDef.Algorithm.SHA256),
    new DigestDef(DigestDef.Algorithm.MD5, DigestDef.Encoding.BASE32)));
factory.setSkipContainerDocumentDigest(true);

ParseContext context = new ParseContext();
context.set(DigesterFactory.class, factory);

AutoDetectParser parser = new AutoDetectParser();
try (TikaInputStream tis = TikaInputStream.get(inputStream)) {
    parser.parse(tis, handler, metadata, context);
}

Command Line

tika-app’s `--digest=<algorithm> flag covers the Commons algorithms only; the digest appears in the metadata output.

java -jar tika-app.jar --digest=SHA256 document.pdf

For SHA3, use a JSON config with bouncy-castle-digester-factory as above. See the CLI reference.

Source