Metadata Changes in Tika 4.x
This document details the metadata key changes in Apache Tika 4.x.
Overview
Tika 4.x puts every "user generated" metadata key under a prefix. The change buys three things:
-
Security: user-controlled content can no longer overwrite a value Tika computed.
-
Clarity: the prefix says which parser or source a key came from.
-
Consistency: one naming approach across all parsers.
The headline prefix moves:
| Category | Change |
|---|---|
HTML custom metadata |
Prefixed with |
MAPI metadata |
Curated keys move from |
Resource name |
|
Unrecognized image metadata |
Prefixed with |
QuattroPro metadata |
Moved from |
Tika-native namespace consolidation (tk:)
Tika 4.0.0 consolidates every Tika-asserted metadata key — keys Tika computes about the
parse, not values copied out of the document — under a single tk: prefix, normalized to
kebab-case. This replaces the pre-4.0.0 X-TIKA: prefix and the scattered tika:, tika_pg:,
rendering:, signature:, imagereader: prefixes and bare hasSignature /
embeddedResourceType keys. This is a clean break: there is no automatic remapping. Code that
references these keys by string must be updated per the table below (code using the
TikaCoreProperties / TikaPagedText / Rendering constants needs no change — the constants
now resolve to the tk: names).
The tk: namespace is also a trust boundary: a file-controlled key can no longer enter it via a
String write. The pre-4.0.0 X-TIKA: prefix stays reserved (rejected on String writes) during
the 4.x line so a crafted file cannot forge it, even though Tika no longer emits it.
File-claimed namespaces (dc:, xmp:, pdf:, office:, html:, standards and passthrough)
are unchanged — they are the file’s assertions, not Tika’s, and keep their verbatim prefixes.
The canonical Content-Type key is likewise unchanged (it is the routing media type). But the
Tika-added variants of it — hint/override/magic-detected — are Tika’s own assertions about
detection, so they move into tk: (e.g. Content-Type-Hint → tk:content-type-hint). The Java
constant did move even though the key string did not: Metadata.CONTENT_TYPE is now
HttpHeaders.CONTENT_TYPE, and a Property rather than a String — see
Java constants moved.
| 3.x key | 4.0.0 key |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Dropped, not renamed: 3.x X-TIKA:content_handler (the handler’s simple class name) has no 4.x
equivalent; tk:content-handler-type covers the use case with a different value shape (the
handler-type enum name), so the old key is not remapped.
New in 4.0.0, with no 3.x predecessor: tk:chunks, tk:content-handler-type,
tk:content-type-magic-detected, tk:decoded-charset, tk:detection-content-length,
tk:encoding-detection-trace, tk:resource-name-extension-inferred,
tk:truncated-content-for-detection, tk:exception:embedded-depth-limit-reached, and
tk:page:numbers (only the singular tk:page:number has a 3.x predecessor,
tika_pg:page_number).
Reserved-namespace families migrate by prefix:
| 3.x prefix | 4.0.0 prefix |
|---|---|
|
|
|
|
|
|
Format-namespace key casing
The same casing rule applies inside format namespaces, decided by who authored the key name: names Tika coined are kebab-cased; names that come from the file or an external standard/schema keep their verbatim spelling. So the namespace prefix (the domain) is independent of casing (the name’s origin).
-
Kebab-cased (Tika-coined names):
pdf:structural flags (pdf:hasMarkedContent→pdf:has-marked-content),access-permission:,zip:,database:,pst:,dwg:,wordperfect:,quattropro:, and the few camelCasemsoffice:stragglers. -
Left verbatim (standard/schema/file names): Dublin Core (
dc:), the XMP schemas (xmp*:), EXIF/tiff:,iptc:,photoshop:,geo:, the CF/CMIP attributes undercf:, MAPI property names, email headers, the ECMA-376 OOXML core/extended property names (AppVersion,lastModifiedBy,TotalTime, …), and the PDF/A/X/VT/UA sub-namespaces (pdfaid:,pdfx:, …) and PDF Info-dict names (pdf:trapped,pdf:docinfo:producer, …).
(The access_permission: prefix is also normalized to access-permission:. The
extended-properties:HiddedSlides entry additionally corrects a long-standing spelling typo.)
| 3.x key | 4.0.0 key |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Also new in 4.0.0, with no 3.x predecessor: the entire zip:* integrity-check family
(zip:central-directory-only-entries, zip:compressed-size, zip:compression-method,
zip:detector-data-descriptor-required, zip:detector-zip-file-opened,
zip:duplicate-entry-names, zip:integrity-check-result, zip:local-header-only-entries,
zip:uncompressed-size, zip:unix-mode, zip:version-made-by), and
vlm:prompt-tokens/vlm:completion-tokens (the tika-vlm module is new in 4.x).
dwg:applicationComment/applicationName/applicationVersion/productInfo are
omitted above: 3.x passed the raw field names from dwgread through unprefixed, and there
is no verified 3.x-to-4.x mapping for these four fields.
|
Other Open-Namespace Prefix Renames
A handful of open-vocabulary KeyPrefix families (document/tool-derived suffixes, so no
flat key-for-key row can enumerate them) also moved prefix in this release:
| 3.x form | 4.0.0 prefix |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
bare ISO 19115 field names; the space-delimited |
|
bare GDAL field names ( |
|
|
|
bare MIF field names |
|
bare IDML field names |
|
bare audio field names ( |
|
bare HDF attribute names |
|
bare NetCDF global-attribute names |
|
bare Climate-Forecast convention keys ( |
|
bare ISA-Tab field names |
|
bare QuickTime |
|
bare |
|
The compatibility filter bridges families whose 3.x form carries a recognizable marker
(NER_*, iworks:*, MDB_PROP:*, …) in both directions. Families whose 3.x keys were
completely bare (HDF, NetCDF, GDAL’s residual, ISA-Tab, QuickTime, NetworkParser, audio
SPI, the bare ISO 19115/GROBID-adjacent one-off keys File-Type-Description, version,
and Password) are restored only on egress (V4_TO_V3): on ingest a bare name carries
nothing that identifies its family, so rewriting would mis-map same-named keys from other
sources, and the filter deliberately passes them through.
This table is a summary by prefix, not exhaustive. metadata-migration-3x-4x.json
(bundled in tika-core, generated by the TIKA-4797 migration join) is the machine-readable
key-for-key list of every enumerable rename and drop, including egress-only rows (marked
"egressOnly":true). The open-vocabulary families in the table above cannot be enumerated
key-for-key — their suffixes come from documents — and exist as prefix rules inside
LegacyKeyMigrationFilter rather than as JSON rows; a script migrating an index should
handle both the JSON and the prefix families above.
Migration Steps
Java constants moved, even where the key string did not
Metadata no longer implements HttpHeaders, Message, TIFF and the other constant-holding
interfaces, so a constant referenced through Metadata no longer compiles:
// 3.x
metadata.get(Metadata.CONTENT_TYPE);
// 4.x -- same key string, different home, and now a Property rather than a String
metadata.get(HttpHeaders.CONTENT_TYPE);The distinction matters because the two move independently. Content-Type the key string is
unchanged, so nothing in the rename tables below mentions it — but Metadata.CONTENT_TYPE the
Java constant is gone. If your compiler says cannot find symbol: variable CONTENT_TYPE, this
is why. The same applies to every constant listed under Deprecations and Removals in
Migrating to Tika 4.x.
Key strings that moved
Code that reaches for a metadata key by its String name has to be updated. The call itself
is unchanged; only the key string moves:
// 3.x
String value = metadata.get("resourceName");
// 4.x
String value = metadata.get("tk:resource-name");The families most often hit this way:
| Source | 3.x key | 4.x key |
|---|---|---|
HTML custom metadata |
|
|
Unrecognized image metadata |
|
|
Office custom properties |
|
|
Resource name |
|
|
Curated MAPI keys |
|
|
|
|
Restoring 3.x Key Names (LegacyKeyMigrationFilter)
Code that can’t migrate to the new key names immediately doesn’t have to fork Tika: the opt-in
LegacyKeyMigrationFilter metadata filter rewrites keys at the emit edge, in either direction,
using the same committed migration table as this document. Enable it via parse-context
(see Serialization and Configuration for how metadata
filters are configured in general):
{
"parse-context": {
"legacy-key-migration-filter": {}
}
}The default direction (V4_TO_V3) rewrites Tika’s 4.x-shaped output back into 3.x key
spellings, so an unmigrated consumer keeps working against 4.x’s output. To rewrite in the
other direction instead — 3.x-shaped input names mapped forward to their 4.x spellings on the
way in — set direction to V3_TO_V4:
{
"parse-context": {
"legacy-key-migration-filter": {"direction": "V3_TO_V4"}
}
}table overrides the bundled migration table with a custom one, same row shape
({"v3": "…", "v4": "…"}, "v4": "DROPPED" for a 3.x key with no 4.x successor).
Write API Changes (reserved-key guard)
The tk: namespace isn’t just renamed in 4.0.0 — writing to it changed shape. In 3.x, a
String write to a reserved key (X-TIKA:*) simply succeeded: a document-controlled
custom property literally named X-TIKA:Parsed-By could overwrite Tika’s own computed
value. In 4.0.0 Metadata#set(String, String) and #add(String, String) throw
IllegalArgumentException on a reserved tk: (or legacy X-TIKA:) name instead, so the
failure is loud rather than a silent security gap.
For parser authors
-
Document-derived key names (HTML
<meta>names, custom document properties, CSV headers, …) go through a declaredKeyPrefixandMetadata#add(KeyPrefix, String, String)— the single write route for names Tika does not control. See Adding a Metadata Key.-
Append-only: a repeated name accumulates values.
-
Never throws on hostile input: blank, over-length, or flooding names are skipped with a WARN.
-
Source-typed dates (OLE
VT_DATE, OOXMLvt:filetime): use theadd(KeyPrefix, String, Instant)overload — stores canonical ISO-8601. If the source’s date fails to parse, keep the raw text via the String overload. -
Known, bounded vocabularies belong in curated
Propertyconstants instead.
-
-
Parser modules enforce the route with a forbiddenapis ban on
Metadata.set/add(String, String)(metadata.forbiddenapis.signaturesFile, opt-in per modulepom.xml). -
Parsing untrusted files at scale? Configure a
StandardMetadataLimiter: it bounds key size, value size, values-per-field, and total metadata size per parse. -
Property’s public factories (`Property.externalText(…),internalText(…), etc.) throwIllegalArgumentExceptionat construction if given a reserved name — a curatedtk:Propertycan only be built from Tika’s own package-private factories. Since these calls usually initializestatic finalconstants, the throw surfaces as anExceptionInInitializerErrorat class-load time (andNoClassDefFoundErroron later touches of the same class), far from the offending declaration — check the initializer error’s cause for the reserved name. -
The throw is a
RuntimeException(IllegalArgumentException), not aTikaException. A harness that narrowly catchesTikaExceptionaround a parse will see this as a new, uncaught failure mode when an unmigrated parser hits a crafted file. -
Some former
Stringconstants are now SIMPLE (single-valued)Propertyconstants instead of BAG. A secondadd()call on one throwsPropertyTypeException(also anIllegalArgumentException) instead of silently accumulating. Useset()if the field is genuinely single-valued, or a BAG-typed key if it can legitimately repeat.
For integrators
-
A manual copy loop (
for (String n : src.names()) dest.set(n, src.get(n))) both collapses multi-valued keys to their last value and throws on everytk:key that every parse output already contains (tk:parsed-by,tk:content, …). Replace it withMetadata#putAll(Metadata), which preserves multi-values and copies reserved keys through their trusted route. -
Metadata#setAll(Properties)is removed; there is no replacement — it wrote a raw map directly intoMetadata, bypassing both the limiter and the reserved-key guard. UseputAll(Metadata)or individualset/addcalls instead. -
/pipesand/asyncuserMetadatabehavior is unchanged: a request’s owntk:-named entries still land in that request’s output. This is deliberate (the deserializer usesMetadata’s trusted reconstruction route, not the guarded `Stringroute) and is bounded to the requester’s own request; it is not affected by the guard changes above.