Migrating Tika Server to 4.x
- Overview
- New
/tikaEndpoint Structure - Breaking Changes
- Removed Endpoints
- Renamed Endpoints
- Handler-Type Changes on
/tika - Error Response Bodies Are Now JSON
/pipesand/asyncWire Formats/metaIs Now Pipes-Backed- Accept Header Routing Removed
- Removed Features
- Command-Line Flags
/pipesand/asyncRequireallowPipes; Per-Request Config RequiresallowPerRequestConfig
- Configuration Changes
- New Features
- Advanced: Shared Server Mode
Overview
Tika Server 4.x introduces pipes-based parsing for the main content-extraction endpoints (/tika, /rmeta, /unpack, /meta), which provides process isolation for those operations. This improves stability and resource management but introduces some breaking changes.
New /tika Endpoint Structure
The /tika endpoint has been simplified with path-based routing:
| Method | Path | Config? | Output |
|---|---|---|---|
PUT |
|
- |
raw Markdown |
PUT |
|
- |
raw text (body only) |
PUT |
|
- |
raw HTML |
PUT |
|
- |
raw XML |
PUT |
|
- |
raw Markdown |
PUT |
|
- |
JSON (default handler: markdown) |
PUT |
|
- |
JSON with specified handler (text, txt, html, xml, body, markdown, md, ignore) |
POST |
|
YES |
raw Markdown (multipart with optional config) |
POST |
|
YES |
raw output in the named format |
POST |
|
YES |
JSON (default handler: markdown) |
POST |
|
YES |
JSON with specified handler |
Using PUT endpoints (simple)
# Get plain text
curl -T document.pdf http://localhost:9998/tika/text
# Get JSON with metadata and text
curl -T document.pdf http://localhost:9998/tika/json
# Get JSON with HTML content
curl -T document.pdf http://localhost:9998/tika/json/html
Using POST endpoints (with configuration)
POST endpoints accept multipart requests with a file part and optional config part:
# Parse with custom PDF parser settings (requires allowPerRequestConfig)
curl -X POST http://localhost:9998/tika/config/json \
-F "file=@document.pdf" \
-F "config={\"pdf-parser\":{\"ocr\":{\"strategy\":\"no_ocr\"}}};type=application/json"
Breaking Changes
Removed Endpoints
/tika/main and /tika/form/main (Boilerpipe)
The Boilerpipe content extraction endpoints have been removed. These endpoints used BoilerpipeContentHandler which is not compatible with pipes-based parsing.
Migration: Use /tika/text for plain text extraction.
/tika/form, /tika/form/*
All /form endpoints have been removed. Use the simplified endpoint structure above.
Migration: Use PUT endpoints for simple requests, POST multipart for requests with configuration.
/tika/config, /tika/form/config
The 3.x forms of these endpoints are gone. The /tika/config* paths in the table above are their replacements, and they work differently: per-request configuration now travels as a config part in a multipart POST, and requires allowPerRequestConfig.
Migration: POST /tika/config for raw output, POST /tika/config/json for JSON, each with a config part in the multipart request.
/translate/*
The translation endpoints have been removed, along with tika-server’s dependency on tika-translate.
These endpoints were never usable as shipped. Every request had to name a fully-qualified translator implementation class in the URL path, and each of the bundled implementations wraps a commercial API that requires credentials the server has no way to supply — there is no configuration surface for them and none was ever wired up. With the default distribution, every request returned an error.
Migration: Call the translation service you use directly, or invoke tika-translate from your own code. Tika’s translation support is unaffected; only the server endpoints are gone.
Renamed Endpoints
/language/stream and /language/string are now both /language
The /stream and /string suffixes never described anything a caller could choose between. Every one of these endpoints has always taken the request body and @Consumes("/"); the two /language paths differed only in whether JAX-RS bound the body to a String or an InputStream, and both then ran the identical detection. The suffixes existed to disambiguate Java method signatures, not to offer an option.
Collapsing them also removes the /detect/stream vs /detectors near-collision. /detectors is unchanged — it lists the server’s configured detectors and does not detect anything.
Migration: drop the suffix. PUT /detect/stream becomes PUT /detect; PUT /language/stream and PUT /language/string both become PUT /language. Request bodies and headers are unchanged, but behavior is not:
-
/detectnow runs in the fork pool (detection opens containers over caller-supplied bytes), so it can return429or503withRetry-After, and413, like the parsing endpoints. A failure reading the body is now a500; 3.x returned200withapplication/octet-streamas if detection had succeeded. -
/languagecaps detection input at the first 100,000 characters and uses the defaultLanguageDetectoron the classpath; 3.x pinned it to Optimaize.
Handler-Type Changes on /tika
/tika/text is body-only again
3.x served this from a BodyContentHandler. 4.0.0 prereleases ran it over the whole XHTML document, so output began with the document title as bare text. This is restored to 3.x behaviour: /tika/text returns body content only.
Migration: if you relied on the prerelease behaviour, PUT /tika/json/text still runs the whole-document handler and returns the content in a JSON envelope.
/tika/json and /tika/config/json default to markdown
Previously they hardcoded plain text. With no handler named in the path they now use the server default, which is markdown — matching /rmeta. Name one explicitly to pin it: /tika/json/text, /tika/json/html, /tika/json/xml, /tika/json/body, /tika/json/md.
Raw /tika and POST /tika/config default to Markdown
3.x — and the 4.0 prereleases — returned XHTML from the bare PUT /tika endpoint (and XHTML from the bare POST /tika/config). Both now return Markdown by default, matching /tika/json and /rmeta. Use /tika/xml for XHTML, /tika/html for HTML, or /tika/text for body-only text (and the /tika/config/{xml,html,text} POST variants for the configured equivalents).
Unrecognized handler names are rejected
/rmeta/txet used to fall back to the default handler and return output that looked correct. Any unrecognized handler name in the path is now a 400 naming the valid types (text, txt, html, xml, body, markdown, md, ignore). This applies to /tika/json/{handler} and the /rmeta/{handler} family.
Error Response Bodies Are Now JSON
In 3.x, error responses from /tika, /rmeta, and /unpack returned a plain-text
body such as "Parse failed: TIMEOUT". In 4.x these endpoints return a JSON body with
at least a status field:
{"status": "TIMEOUT"}
A message field is included when one is available and may contain a server-side stack
trace, e.g. {"status": "TIMEOUT", "message": "Task timed out after 60000ms"}.
The HTTP status codes are also more precise:
-
UNSPECIFIED_CRASH,TIMEOUT, andOOMreturn503— transient process failures, not a500server misconfiguration. -
Pool saturation (
CLIENT_UNAVAILABLE_WITHIN_MS) returns429with aRetry-Afterheader, not503or200— see Backpressure. -
An unknown or reserved fetcher/emitter (
FETCHER_NOT_FOUND,EMITTER_NOT_FOUND) returns400, not500— the request is permanently malformed, so retrying will not help. -
Two distinct limits return
413. A request body overmaxRequestSizeBytesis rejected by a request filter with a plain-text413— both a declaredContent-Lengthover the limit and a chunked body that turns out to be too large. Separately, a parse result too large for the pipes IPC channel (pipes.maxIpcPayloadBytes) returns413with the JSON status body{"status":"PAYLOAD_LIMIT_EXCEEDED"}. -
/pipesand/asyncnow signal the outcome through the HTTP status (the same429/503/400/413mappings, plusRetry-After) instead of always returning200with the failure only in the body.
The raw /tika, /tika/text, /tika/html, /tika/xml, and /tika/md endpoints return
422 with the extracted content as the body when a container-level exception occurs during
a partial parse; the exception itself is not appended to the raw body — use /rmeta for the
structured exception.
Migration: Clients that parse plain-text error bodies must switch to JSON. Clients
that branch only on HTTP status code should account for the 429/400/413 mappings
above and stop treating UNSPECIFIED_CRASH as 500.
/pipes and /async Wire Formats
/pipes response body. /pipes now returns the same JSON body as /tika, /rmeta,
and /unpack — {"status":"<RESULT_STATUS>","message":"…"} — where status is the
PipesResult status enum (EMIT_SUCCESS, EMIT_SUCCESS_PARSE_EXCEPTION,
PARSE_EXCEPTION_NO_EMIT, TIMEOUT, OOM, FETCHER_NOT_FOUND, …) and message carries
the exception when there is one. It previously used a /pipes-only shape
({"status":"ok"|"process_crash"|"application_error", …} with type, parse_exception,
and a stringified emitted). A parse that threw is now reported by its status enum, not as
"ok".
/pipes also rejects with 400: a malformed request body (the reason is in the response),
and any emit strategy other than EMIT_ALL — the /pipes response carries only
status and message, so a passback strategy would silently discard the parsed data. Use
/rmeta if you want the data passed back.
/async request body. POST /async now requires an object envelope,
{"tuples":[ … ]}, instead of a bare JSON array; the envelope leaves room for future
batch-level fields. A bare array (or any body without a tuples array) is rejected with 400,
as is a tuple naming a fetcher or emitter the server does not have — validated at POST time,
before anything is queued. A batch larger than the queue’s total capacity is also a 400
telling the caller to split it: retrying cannot help. 429 with Retry-After is reserved
for transient fullness, where the same batch can succeed later.
FetchEmitTuple wire format. The per-tuple parse-context key is parse-context;
3.x used parseContext. An unknown field anywhere in a tuple is rejected with a 400
naming the field and listing the known ones, instead of being silently ignored.
/meta Is Now Pipes-Backed
/meta previously parsed in-process, in the request-handling JVM, with no crash
isolation and its own ad hoc error handling (500 for most parse failures, 400
for a field that couldn’t be extracted from an incompletely-parsed document). It
now shares the same pipes-backed PipesParser as /tika, /rmeta, and /unpack
(see
Endpoints
and Forked-Process Groups), with the same crash isolation and the same
per-document exception handling as those endpoints (see
Error Responses):
-
/meta,/meta/form,/meta/confignow return200 OKwith the exception embedded intk:exception:container-exception, instead of500. -
/meta/{field}now returns422 Unprocessable Entityfor a genuine parse exception, instead of500or400.
Migration: clients that treated any non-200 from /meta as "parse failed"
should check the new status codes above. Clients that inspected the response body
for error text should check tk:exception:container-exception (full-object
endpoints) or the 422 body (/meta/{field}).
The default representation is now JSON, not CSV. A /meta request without an
Accept header returned CSV in 3.x; it now returns JSON. CSV is still available with
Accept: text/csv.
Two changes to the returned metadata come with this, neither of which produces an error:
-
/metano longer returns alanguagefield. Language detection previously ran inline on this endpoint via a dedicated content handler that buffered text solely to detect the language, which meant holding the document text twice to populate one field. That handler was removed./metadeliberately parses with theignorecontent handler, so there is no text for a language detector to work from.Migration: configure a language-detection metadata filter (
charsoup-metadata-filter,optimaize-metadata-filter, oropen-nlp-metadata-filter) and use/rmetaor/tika/json, which capture content. The detected value arrives astk:detected-language, withtk:detected-language-confidence. Note that these filters readtk:content, so they are no-ops on/metaand on any endpoint configured with theignorehandler. -
/metasuppresses embedded parsing without stamping an exception flag. It skips embedded documents with a document selector, so — unlike the 4.0 prereleases, which set an embedded depth limit of0and recordedtk:exception:embedded-depth-limit-reachedon every container — notk:exception:*flag is set merely for suppressing embedded content.
Accept Header Routing Removed
The /tika endpoint no longer routes on the Accept header. Name the format in the path
instead — /tika/text, /tika/html, /tika/json, and the rest of the table above.
Removed Features
-
Fetcher-based streaming - The
InputStreamFactorypattern for fetching documents via HTTP headers (fetcherName,fetchKey) has been removed. All documents are now processed through the pipes infrastructure: a body at or belowmaxInlineBytes(default 10 MB) travels inside the request to the forked worker and touches no disk; a larger one is spooled to a temp file the worker reads.
Command-Line Flags
-
The no-op
-a/--pluginsConfigflag has been removed; delete it from launch scripts, where it now fails option parsing. -
--helpnow exits0(it previously exited non-zero).
/pipes and /async Require allowPipes; Per-Request Config Requires allowPerRequestConfig
This replaces the enableUnsecureFeatures flag that alpha-1 briefly used, and before
that, enabling these capabilities simply by listing endpoints under server.endpoints.
enableUnsecureFeatures no longer exists: a config that still carries it fails to start
with an "Unrecognized field" error naming the key, rather than silently ignoring it.
The single flag has been split into two, so that granting batch/fetcher access and
granting per-request parser configuration are separate decisions:
| Was | Now |
|---|---|
|
|
|
|
The capabilities are two default-false flags in the server section:
-
allowPipesgates the/pipesand/asyncendpoints, which drive process-isolated batch parsing through your fetchers and emitters. Selecting either withoutallowPipescauses the server to refuse to start with a clear error. -
allowPerRequestConfiggates per-request parser configuration: the/configfamily of endpoints and the multipartconfigpart. When off, such requests are rejected with 403.
/status is no longer gated: it exposes only aggregate counters, so it is enabled simply by listing status under endpoints.
The endpoints allowlist now also gates SPI-provided resources: a discovered resource
whose root path matches a named endpoint binds only when that endpoint is enabled (e.g.
the application/rdf+xml XMP resource serves /meta, so omitting meta removes it too).
An SPI resource with a custom root path still loads unconditionally — installing the jar
is the opt-in.
Migration: if your config selects pipes or async, add "allowPipes": true; if you rely on per-request config, add "allowPerRequestConfig": true:
{
"server": {
"allowPipes": true,
"allowPerRequestConfig": true,
"endpoints": ["tika", "rmeta", "pipes", "async", "status"]
}
}
Configuration Changes
Removed Configuration Options
The following TikaServerConfig options have been removed:
-
returnStackTrace- Removed as misleading. It gated only the error-response body, while exception detail — including messages carrying file paths and document fragments — continued to travel intk:exception:*metadata on successful parses and inside/unpackzips. Setting it tofalsedid not keep stack traces inside the server, which is what the name implied. Exception detail is now always returned; filter Tika’s output with aMetadataFilterbefore forwarding it somewhere less trusted. -
taskTimeoutMillis- Was the total time allowed per task before the forked process was killed; that is nowparse-context.timeout-limits.totalTaskTimeoutMillis(same meaning, new home). The default changed: 3.xtaskTimeoutMillisdefaulted to300000(5 minutes), whiletotalTaskTimeoutMillisdefaults to3600000(1 hour) — an upgrader who never set the 3.x value gets a much larger worst-case per-document bound. SettotalTaskTimeoutMillis: 300000to keep the 3.x behavior. 4.x also adds a second, independent axis with no pre-4.0 equivalent —progressTimeoutMillis, a stall detector that kills the task only after a period of genuine silence, not merely aftertotalTaskTimeoutMillisof wall-clock time — worth setting deliberately rather than leaving at its default. See Timeouts. -
taskPulseMillis- No longer needed -
minimumTimeoutMillis- No longer needed -
logLevel- Renamed torequestLogLevel, with a narrower meaning: it only sets the level at which each request URI is logged (empty, the default, disables request logging), and no longer changes any other log level. A config still carryinglogLevelfails to start. -
digest,digestMarkLimit- Moved to the parse-context digester configuration; configure digests there instead of in theserversection. -
idBase- Renamed toid. -
preventStopMethod,maxFiles,javaPath,maxRestarts,numRestarts,forkedStatusFile,maxForkedStartupMillis,tempFilePrefix,noFork- Vestigial options from the pre-4.0 spawn-child server model, which no longer exists (the-noForkCLI flag is gone too). Delete them from your config; leaving any in place now fails startup, because unrecognized config keys are rejected. -
portis now a single integer. The 3.x multi-instance port ranges and lists (-p 9995-9998,-p 9995,9997) are no longer supported; run one server per port.
Configuration via HTTP Headers Removed
4.x configures parsing through the tika-config file, not through request headers. The remaining per-request configuration headers have been removed and are now silently ignored if sent — the request succeeds, the header has no effect:
| Removed header | Replacement |
|---|---|
|
|
|
|
|
|
|
no replacement — see below |
|
Use an explicit handler path ( |
The X-Tika-OCR* and X-Tika-PDF* header families were removed earlier in the
4.x line. Parser configuration is now supplied as JSON.
{
"parse-context": {
"output-limits": {
"writeLimit": 100000,
"throwOnWriteLimit": false
},
"embedded-limits": {
"maxCount": 10
}
}
}
Migration: move these settings into your tika-config. For per-request values,
POST to a /config endpoint (/tika/config, /tika/config/json, /rmeta/config,
/meta/config) with the JSON above as the multipart config part; this requires
allowPerRequestConfig=true.
|
Two capabilities are genuinely gone, not relocated. Per-request output bounds without Client-supplied metadata ( |
Transport headers are unaffected: Content-Disposition,
Content-Type and Content-Length still describe the payload and still
influence detection.
The way Content-Type influences detection changed. In 3.x, parsing ran
in-process and a request Content-Type acted as a hard override that forced the
type. In 4.x, parsing runs in a forked worker and the header is carried across as
a soft hint: detection keeps it only when it equals or specializes the type
detected from the content, and otherwise ignores it (TIKA-4825). A 3.x client
that forced an unrelated type onto arbitrary bytes (for example text/plain)
will now see that type ignored in favor of content-based detection. Supply the
correct Content-Type (or a filename) to refine within the detected hierarchy.
|
Pipes Configuration (for /pipes and /async)
No pipes or fetcher configuration is required to start the server: the default-on endpoints
(/tika, /rmeta, /meta, /unpack, /detect) run on the built-in \_\_bytes fetcher for
content carried inside the request, an internally-configured spool fetcher for anything larger,
and a default pipes config, and the server starts with no config file at all. You
need a pipes section and your own file-system-fetcher only when you enable allowPipes
and want /pipes//async to fetch documents from a directory you control:
{
"fetchers": {
"file-system-fetcher": {
"file-system-fetcher": {
"basePath": "/path/to/your/input"
}
}
},
"parse-context": {
"timeout-limits": {
"progressTimeoutMillis": 30000
}
},
"pipes": {
"numClients": 2
},
"plugin-roots": "path/to/plugins"
}
The 3.x pipes keys staleFetcherTimeoutSeconds and staleFetcherDelaySeconds are gone.
A pipes config still carrying either fails startup: unknown pipes keys are rejected, not
ignored.
|
Set Setting |
|
|
maxRequestSizeBytes now defaults to 1 GiB
The request-body cap was unlimited by default in the prereleases; it now defaults to 1 GiB.
Bodies above maxInlineBytes are spooled to disk,
so this also bounds how much one request can write, and a body over the limit is rejected with
413.
Migration: if you legitimately post bodies larger than 1 GiB, set "maxRequestSizeBytes"
in the server section above your largest payload (or -1 to restore no limit).
http-fetcher verifies TLS by default
The http-fetcher now verifies TLS certificates by default (verifySsl defaults to true).
A pipeline that fetched from a self-signed or internal host and previously succeeded will
start failing at the TLS handshake.
Migration: trust the host’s CA (preferred), or set verifySsl: false on the fetcher if
you accept the risk.
New Features
Process Isolation
All parsing now occurs in isolated child processes, providing:
-
Protection against parser crashes affecting the server
-
Memory isolation (OOM in parser doesn’t crash server)
-
Configurable timeouts at the pipes level
This isolation carries a throughput and memory cost relative to a single in-JVM
parser, and the size of that cost depends on your document mix and the pool
configuration (per-client versus shared-server, numClients, per-fork heap).
See Performance and Isolation Trade-offs for the
comparison and tuning guidance.
Backpressure: 429 Separates "Busy" From "Broken"
In 3.x a saturated server had no way to say so. /tika, /rmeta, and /unpack parsed in
the request-handling JVM, so load simply piled onto request threads, and a client could not
tell overload apart from a document that broke the parse.
Because parsing now runs in a fixed pool of forks, 4.x reports the two conditions differently:
-
The pool was full →
429, statusCLIENT_UNAVAILABLE_WITHIN_MS. Nothing failed; the same request can succeed later, andRetry-Aftersays how long the server thinks that is. -
A fork died on this document →
503, statusOOM,TIMEOUT, orUNSPECIFIED_CRASH. The replacement fork is up within seconds (Retry-After: 5), but resubmitting the same bytes will likely reproduce the failure. -
The request is malformed →
400. Retrying never helps.
A client, load balancer, or queue consumer can now back off on the first, quarantine or route elsewhere on the second, and drop the third — instead of hammering a busy server or failing blind on a single opaque status.
The cost is a sizing decision 3.x did not ask of you: the pool is exactly numClients
forks wide. 429 under normal load is what "too small for this request volume" looks like;
forks starving each other of CPU is what "too large for this host" looks like. Neither
raises an error naming numClients, so size it deliberately — see
Forked-JVM CPU and Heap Sizing.
Advanced: Shared Server Mode
For memory-constrained environments, an experimental shared server mode is available. Instead of running N separate server processes (one per client), all clients share a single server process.
|
This mode sacrifices reliability for reduced memory usage. One crash, OOM, or timeout affects all in-flight requests. Only use if you fully understand the tradeoffs. |
{
"pipes": {
"numClients": 4,
"useSharedServer": true,
"forkedJvmArgs": ["-Xmx4g"]
}
}
See Shared Server Mode for details.