Apache Tika 4.2.0

Tika 4.2.0 is the second minor release of the 4.x line. The most notable changes over 4.1.0 are summarised below; the CHANGES-4.2.0.txt file carries the complete list. Note the breaking change in unpack output, which is now a Frictionless Data Package by default; see "Breaking Changes and Deprecations" before upgrading.

Upgrading from 3.x? Start with the migration guides. The 3.x line remains supported. See the download page for both.

Breaking Changes and Deprecations

  • Unpack output is a Frictionless Data Package by default (tika-server /unpack, tika-app -z/-Z, pipes UNPACK); unpack-config.outputFormat=REGULAR (--unpack-format=REGULAR) restores the old layout (TIKA-4681). A document without attachments is still a package, so /unpack returns 200, not 204; unpack-config.emptyPackage=SKIP restores the 204 (TIKA-4971).
  • The package's metadata.json is now rmeta.json and gets the metadata filters; a package sent to the container's emitter replaces its <emitKey>.json. REGULAR .metadata.json sidecars are the file's row from the metadata list. FrictionlessResource.name() is the resource name (the filename moved to resourceName()); FrictionlessResource.create and DataPackage.addResource are removed (TIKA-4971).
  • Config is untrusted unless the operator says so. @OperatorOnly properties (paths, executables, class names, URLs, credentials, operator-sized limits) are refused wherever a request builds a component, and an @OperatorOnly class can't be built from a request at all. Java code setting one through a JsonConfig wraps it in JsonConfig.trusted(json); JsonConfig constructors now need tika-serialization. Newly locked: pdf-parser imageGraphicsEngineFactoryClass and maxMainMemoryBytes, raw-tiff maxTotalPreviewBytes, VLM completionsPath. The RuntimeConfig subclasses are removed (TIKA-4965, TIKA-4974).
  • Per-request limits and timeouts may tighten the operator's, never loosen them. pipes.trustedRequests=true marks requests as the operator's own (no clamp, may set @OperatorOnly properties); PipesForkParser sets it, and tika-server and tika-grpc refuse to start with it (TIKA-4978, TIKA-4985).
  • A duplicate key in configuration JSON is an error, not last-wins. A request's http-fetcher config may set only httpRequestHeaders; a file-system emitter's runtime config naming basePath fails as an unknown key. forbidden-apis bans JsonConfig.json() outside ConfigDeserializer and PluginJson, also in projects inheriting tika-parent (point json.forbiddenapis.signaturesFile at an empty file to opt out). ExtensionConfig no longer implements JsonConfig (TIKA-4975).
  • Dates: Metadata.getDate() reads zone-less values as UTC, not the JVM's zone. Archive-entry and mail-attachment file times move from dcterms:* to fs:created/fs:modified. DateNormalizingMetadataFilter honors offsets, normalizes every value and completes partial dates (2019 -> 2019-01-01T12:00:00Z). MailDateParser is replaced by TikaDates; MailUtil moved to tika-parser-mail-module, leaving tika-parser-mail-commons a deprecated shim, to be removed in 4.3.0 (TIKA-4917).
  • OCR engines (tesseract-ocr-parser, tess4j-parser) under "parsers", deprecated in 4.1.0, now fail config load: configure them under "engines" and name them in "text-recognizers". A VLM parser may stay under "parsers" as a document parser (ContentEnricher.parsesDocuments()). openai-embedding-filter, jina-embedding-filter and openai-image-embedding-parser, deprecated in 4.1.0, are removed; use an engine and an inference binding (TIKA-4985).
  • text-recognizers: an inline entry must have exactly one key, setting both _min-width and min-width is an error, and gated entries load as GatedEnricher, not SizeGatedEnricher (TIKA-4972).
  • tika-server transfer limits: maxRequestReadMillis (408) and maxResponseWriteMillis, both 5 min; idleTimeoutMillis 120s (was CXF's fixed 200s); maxInFlightRequestBytes (429). Raise the deadlines for very large or slow transfers; cut-offs are logged at INFO by the TransferLimitLog logger. Cleartext HTTP/2 (h2c), on by default since 4.0.0, now needs allowHttp2 (TIKA-4991).
  • tika-server TLS: a tlsConfig block turns TLS on; "active" is deprecated (true ignored with a warning, false refused, removed in 4.3.0), and TikaServerConfig.getTlsConfig() is null without a block. Protocol and cipher suite policy is the JVM's (jdk.tls.server.protocols, jdk.tls.server.cipherSuites, jdk.tls.disabledAlgorithms): includedProtocols, excludedProtocols, includedCipherSuites and excludedCipherSuites, none of which worked as documented, are refused at startup. clientAuthenticationWanted is removed (true refused, false ignored with a warning), as tika-server never reads client certificates (TIKA-4994).
  • embedded-limits maxDepth defaults to 20 (was unlimited) and replaces output-limits maxPackageEntryDepth, which is to be removed in 4.3.0 with its accessors on OutputLimits and SecureContentHandler (#3319).
  • Pipes plugins load only from the zips in plugin-roots; a plugin on the classpath needs -Dtika.plugins.classpath=true, which PipesForkParser sets when it finds no plugins directory holding tika-pipes-file-system (TIKA-4918, TIKA-4985). A plugin-roots naming a directory without the plugin zips, which the classpath used to mask, now fails with "Unknown type: ... Available: []"; tika-app and tika-server treat the docs' PLUGINS_PATHS placeholder as unset and use their bundled plugins directory (TIKA-4984).
  • image/vnd.microsoft.icon moves from ImageParser to the new ICOParser, which also parses cursors (TIKA-4937); animated cursors (RIFF ACON) are application/x-navi-animation, even when named *.cur (TIKA-4985).
  • ISArchiveParser(String location) is removed; pass a file-backed TikaInputStream (TikaInputStream.get(Path)) (TIKA-4986).
  • tika-bundle-standard no longer embeds dependencies that are OSGi bundles themselves (commons-*, pdfbox, fontbox, bouncycastle, jsoup, asm, xz, xmpcore, dd-plist); deploy them alongside it (TIKA-4934). Its imports are now mandatory, it registers its encoding detector, language detector and metadata filter providers, and it requires Java 17 (TIKA-4955).

Security

  • The default zip-bomb ratio (output-limits.zipBombRatio) is 1000, up from 100. It is measured across the whole embedded tree against the container's bytes, so nested compression is still caught; at 100, ordinary compressed logs and text tripped it. Set zipBombRatio to 100 to restore the old limit (TIKA-4985).
  • The Gemini VLM parser sends its API key in x-goog-api-key, not in the URL that error messages and logs quote (TIKA-4985).
  • XML parsers are configured through Apache Commons Secure XML (TIKA-4935); Tika's own entity-expansion limit is retired in favor of the JDK's (TIKA-4940); and forbidden-apis rejects building XML parsers outside XMLReaderUtils (TIKA-4939). New tk:xml-external-reference and -count list the external DTDs and entities a document referenced, which Tika never resolves (TIKA-4941). XFA is read with SAX; XMLReaderUtils.getXMLInputFactory() is deprecated (TIKA-4938).
  • The tika-app GUI's Formatted Text view no longer lets a document make Swing construct classes or fetch URLs: only plain elements and attributes reach it, and remote images are not loaded (TIKA-4992).
  • tika-server TLS: a keystore configured without "active": true no longer serves plain HTTP. 2-way TLS accepts client certificates issued by a CA in the truststore, which it had rejected since 2.4.0; their revocation is checked through the certificate's OCSP responder or CRL distribution point and fails closed (tlsConfig checkRevocation=false turns it off). Rejected client certificates are logged at INFO with the reason (TIKA-4994).

Server, Pipes and Operations

  • The file-system emitter can gzip its output ("compression": "GZIP"); tika-eval reads .json.gz extracts and measures them uncompressed (TIKA-4989).
  • tika-server activates catalog presets with --preset <name> (repeatable) or TIKA_PRESETS (comma-separated), overriding the config file's entry (TIKA-4988).
  • PipesParser.start() and PipesForkParser.start() bring the forks up before the first parse (TIKA-4953), and tika-server starts its forks before opening its port, so a fork that can't start fails startup, not the first request (TIKA-4983).
  • PipesForkParser: a ParseMode, ContentHandlerFactory or EmbeddedLimits in the per-call ParseContext overrides PipesForkParserConfig, and the caller's context is no longer modified (TIKA-4981).
  • Forked pipes servers get their config and auth token over stdin, never through the temp dir; server-manager and PipesServer signatures change (TIKA-4952).
  • maxFilesProcessedPerProcess defaults to 100,000, up from 10,000 (TIKA-4950).
  • Pipes forks inherit the parent's default locale unless forkedJvmArgs set user.language (TIKA-4920).
  • Pipes UNPACK deletes its temp dir when the fetch or parse fails (TIKA-4985).
  • Parser loading and fork startup are about 1.3s faster: parsers no longer build a default EncodingDetector the loader then replaces (TIKA-4954).
  • Quieter pipes logging: per-parse timing lines move to TRACE, lifecycle lines to DEBUG (TIKA-4949).
  • OSGi: tika-serialization is a bundle, so JsonConfig constructors work (TIKA-4982); tika-core installs without a Service Loader Mediator (TIKA-4945); the DefaultParser and DefaultDetector services no longer recurse into a StackOverflowError (TIKA-4942); OOXML, HTTP and GeoGebra parsing and charset detection work in tika-bundle-standard again (TIKA-4955).

Extraction and Formats

  • docx: OMML equations are wrapped in <span class="omml"> (TIKA-4968); REF, PAGEREF, NOTEREF, HYPERLINK \l and simple fields link to their bookmarks (TIKA-4967); bold and italics from character styles are applied, with opt-in includeCharacterStyleClasses (TIKA-4966).
  • .doc: stored line and paragraph counts are extracted (TIKA-4694).
  • ExecutableParser emits a PE file's icon (its first icon group) as an embedded THUMBNAIL .ico; extractAllIcons=true emits every group, extractIcons=false none (TIKA-4936).
  • BigTIFF is read through metadata-extractor 2.22.0: EXIF and DNG tags from BigTIFF DNGs, plain BigTIFF no longer fails, and little-endian BigTIFF is detected by content (TIKA-4987).
  • Previews from Olympus ORF, Panasonic RW2, Epson ERF, Kodak KDC and Leaf MOS; Epson ERF is detected by content. A metadata-extractor tag descriptor that throws skips the tag instead of failing the parse (TIKA-4980).
  • OneNote: table and list structure in FSSHTTPB files (TIKA-4943); section and page GUID metadata (TIKA-4944).
  • Detection: *.rs is text/x-rust (TIKA-4977); a MIME email nesting multipart/related is message/rfc822, not MHTML (TIKA-4973); XML with an element like <htmlPath> is no longer HTML (TIKA-4962); XUL is detected by content and parsed by the HTML parser, as XMLParser fails on its DTD entities (TIKA-4985); OS/2 bitmap array and TGA detection is tighter (TIKA-4937).
  • Image checks (byte-size, min-image-size, blank-image-check; new tika-image-processing module) keep blank and tiny images from text recognizers; SizeGatedEnricher is deprecated (TIKA-4972).
  • VLM parsers: output gets tk:ocr-warnings (an empty Claude response is a warning, not an exception), vlm:skipped-reason and vlm:image-pixels; they take temperature and retry dropped HTTP connections (TIKA-4972). The OpenAI-compatible parser has opt-in useMaxCompletionTokens for models that reject max_tokens (#3350).
  • The embed task writes tk:chunks once per binding, not once per image or page (TIKA-4970).
  • Opt-in "metadata-list": {"typedValues": true} writes numbers, booleans and tk:chunks as JSON values rather than strings; new Property.ValueType.JSON (TIKA-4969).
  • Parsers no longer carry one document's state into the next, or share it between concurrent parses: IDML page counts, an ENVI header's multi-line field (#3406), an IPTC/ANPA wire's agency format and publisher (TIKA-4995), and ISArchiveParser's ISA-Tab directory and study file name (TIKA-4986).

Other Changes

  • tika-app -J writes the metadata list when the container throws, with the exception in tk:exception:container-exception on the first record, as /rmeta does; it used to print nothing. The exit status still reports the failure (TIKA-4985).
  • A RAR entry junrar fails to extract is recorded as tk:exception:embedded-stream-exception on the container; what was extracted before the failure is kept and the rest of the archive is parsed (junrar 8.1.2, TIKA-4985).
  • JSoupParser emits CDATA text again (TIKA-4963).
  • OutlookExtractor falls back to the Date: header when a .msg has no MAPI date (TIKA-4959).
  • The MP4 parser no longer warns of an EOFException on every file (TIKA-4957).
  • GeoGebraParser no longer needs jackson-databind (TIKA-4956).
  • tika-server's welcome page lists endpoints inherited from superclasses (XMP on /meta) under their real paths (TIKA-4991).

Contributors

The following people have contributed to Tika 4.2.0 -- via code, patches, pull requests, bug reports, review and discussion. Thank you!

  • Adam Rauch
  • Arpit
  • Dominik Schmidt
  • Ed Merks
  • Gary D. Gregory
  • Henry Lindeman
  • Michael Schobel-Thoma
  • mstcomit
  • Naveed
  • Philipp Ottlinger
  • Piotr P. Karwasz
  • Tilman Hausherr
  • Tim Allison
  • Vivek Dinesh Suryawanshi
  • Zili Chen

    See https://s.apache.org/uknad for more details on these contributions.