Adding a Metadata Key

Every metadata key Tika can emit is a Property constant (or, for runtime-minted names like scraped HTML <meta> tags, a KeyPrefix) — there are no bare String keys naming a population of writable metadata. (One String constant remains as an exception: TikaCoreProperties.EMBEDDED_RESOURCE_TYPE_KEY is an internal building block that constructs the Property name for EMBEDDED_RESOURCE_TYPE — not an independent key — and is excluded from the registry’s key count for that reason.) That closed/open key space is tracked in a generated, build-gated registry, so adding a key involves one extra step beyond writing the Java.

Add the constant

A curated tk: key can only be minted from inside org.apache.tika.metadata, via the package-private reserved* factories — a public Property factory rejects a tk:/X-TIKA: name at construction time. Add it to TikaCoreProperties (or another class in that package) as usual:

Property MY_NEW_KEY = Property.reservedInternalText(TIKA_META_PREFIX + "my-new-key");

A parser module coining its own key uses a public factory in its own namespace instead:

Property MY_NEW_KEY = Property.internalText("myformat:my-new-key");

Naming conventions (frozen for 4.0):

  • Tika-coined keys use the tk: namespace, kebab-case, no underscores.

  • External-standard names are used verbatim, including the standard’s own prefix (dc:, xmp:, cp:, extended-properties:).

  • HTTP headers stay bare — no http: namespace (Content-Type, Content-Encoding, Location).

Document-derived names: declare a KeyPrefix

When the key names come from the document or an external tool (custom document properties, format-specific attribute names, NER labels), they can’t be constants. Declare a KeyPrefix once, as a static final field — never per-parse, never from document text — and write through Metadata#add(KeyPrefix, String, String):

static final KeyPrefix MYFORMAT = KeyPrefix.file("myformat:", "myformat's own header fields");
...
metadata.add(MYFORMAT, nameFromDocument, value);              // String value
metadata.add(MYFORMAT, nameFromDocument, date.toInstant());   // source-typed date

The route is append-only (repeated names accumulate, losslessly transcribing the source) and never throws on hostile input: blank, over-length, or flooding names are skipped with a WARN. Use KeyPrefix.file(…​) for names read out of the document, KeyPrefix.tool(…​) for names coined by an external tool or service.

Regenerate the registry

The registry — three JSON files under tika-metadata-schema/src/main/resources/org/apache/tika/metadata/, listing every declared key, every open-namespace prefix, and a field-provenance table — is generated from the live Property/KeyPrefix declarations, never hand-edited. A committed copy is the reviewable audit trail (a rename or dropped key shows up as a diff), and CI fails if it’s stale.

Run this after adding, renaming, or removing a Property or KeyPrefix:

tika-metadata-schema/regen.sh

It installs the modules the change touched, regenerates all three registry files, sanity-checks the diff, and runs the gate tests. Commit the Java change and the regenerated JSON together.

Flags are on regen.sh --help. The registry design is in tika-metadata-schema/README.md; the traps this script routes around (classpath scanning quirks, exec:java vs. a forked classpath) are in .skills/metadata-schema/SKILL.md.

After a rename

The compiler won’t catch a stale string literal like metadata.get("Message-From"). Grep the repo for the old key and replace it with the constant:

grep -rn '"Message-' --include=*.java . | grep -v /target/