Setting Limits

Untrusted documents can be pathological: deeply nested, self-expanding, or crafted to run forever. Every limit below is configured in the parse-context section of the JSON config, loaded into the ParseContext, and enforced throughout the parse.

Limits at a glance

parse-context key Class Bounds

embedded-limits

EmbeddedLimits

Depth and count of embedded documents. Unlimited by default.

output-limits

OutputLimits

Extracted characters, XML nesting, package nesting, zip-bomb ratio.

timeout-limits

TimeoutLimits

Total wall-clock budget per task and the stall detector.

unpack-config

UnpackConfig

Total bytes written by ParseMode.UNPACK. Pipes only.

standard-metadata-limiter-factory

StandardMetadataLimiterFactory

Metadata size, per-field size, key size, values per field, field allow/deny lists.

Configuring limits

Every limit group is a parse-context block. This is the configuration exercised by AllLimitsTest (tika-serialization/src/test/resources/configs/all-limits-test.json):

{
  "parsers": ["default-parser"],
  "parse-context": {
    "embedded-limits": {
      "maxDepth": 10,
      "throwOnMaxDepth": false,
      "maxCount": 1000,
      "throwOnMaxCount": false
    },
    "output-limits": {
      "writeLimit": 100000,
      "throwOnWriteLimit": false,
      "maxXmlDepth": 100,
      "maxPackageEntryDepth": 10,
      "zipBombThreshold": 1000000,
      "zipBombRatio": 100
    },
    "timeout-limits": {
      "totalTaskTimeoutMillis": 3600000,
      "progressTimeoutMillis": 60000
    },
    "standard-metadata-limiter-factory": {
      "maxTotalBytes": 1048576,
      "maxFieldSize": 102400,
      "maxKeySize": 1024,
      "maxValuesPerField": 100
    }
  }
}

TikaLoader.loadParseContext() loads them all; each class has a static get(ParseContext) that returns the configured instance or defaults:

ParseContext context = TikaLoader.load(configPath).loadParseContext();

EmbeddedLimits embedded = EmbeddedLimits.get(context);
OutputLimits output = OutputLimits.get(context);
TimeoutLimits timeouts = TimeoutLimits.get(context);

To set them programmatically, construct and put them on the context — the same pattern for every limit class:

context.set(EmbeddedLimits.class, new EmbeddedLimits(10, true, 500, false));
context.set(OutputLimits.class, new OutputLimits(50000, true, 50, 5, 500000, 50));
context.set(TimeoutLimits.class, new TimeoutLimits(7200000, 120000));

Tests: AllLimitsTest, EmbeddedLimitsTest, OutputLimitsTest, and TimeoutLimitsTest under tika-serialization/src/test/java/org/apache/tika/config/.

Embedded document limits

EmbeddedLimits bounds how deep and how many embedded documents are parsed.

Setting Default Description

maxDepth

-1 (unlimited)

Maximum nesting depth. Recursion stops at the limit; siblings at the current level still parse.

throwOnMaxDepth

false

Throw EmbeddedLimitReachedException at maxDepth instead of continuing and setting tk:exception:embedded-depth-limit-reached=true.

maxCount

-1 (unlimited)

Maximum total embedded documents. Processing stops immediately when reached.

throwOnMaxCount

false

Throw EmbeddedLimitReachedException at maxCount instead of continuing and setting tk:exception:embedded-resource-limit-reached=true.

With maxDepth=1, depth-1 siblings all parse and their children do not:

container.zip (depth 0)
├── doc1.docx (depth 1)      PARSED
│   ├── image1.png (depth 2) NOT PARSED
│   └── embed.xlsx (depth 2) NOT PARSED
├── doc2.pdf  (depth 1)      PARSED
└── doc3.txt  (depth 1)      PARSED

Output limits

OutputLimits bounds extracted text and structural expansion.

Setting Default Description

writeLimit

-1 (unlimited)

Maximum characters of text to extract. Extraction stops when reached.

throwOnWriteLimit

false

Throw WriteLimitReachedException at writeLimit instead of stopping and setting tk:exception:write-limit-reached=true.

maxXmlDepth

100

Maximum XML element nesting depth. Guards against XML bombs.

maxPackageEntryDepth

10

Maximum depth of nested package entries (zip within zip).

zipBombThreshold

1,000,000

Extracted characters before the zip-bomb ratio check activates.

zipBombRatio

100

Maximum ratio of extracted characters to input bytes read before flagging a zip bomb.

Timeout limits

TimeoutLimits applies two independent bounds: one on total wall-clock time, one on time since the parser last reported progress.

Setting Default Description

totalTaskTimeoutMillis

3,600,000 (1 hour)

Wall-clock budget for the whole task, embedded documents included. Every per-parser timeout is clipped to what remains of this budget, however it is itself configured.

progressTimeoutMillis

120,000 (2 minutes)

How long the task may go silent before it counts as hung. Enforced — the task actually killed — only when the parse runs in a forked JVM (Pipes, or tika-app --fork); in-process library use has nothing to kill the parsing thread.

throwOnDeadline

false

Throw EmbeddedLimitReachedException on totalTaskTimeoutMillis exhaustion instead of skipping the remaining embedded documents and reporting PARTIAL_TIMEOUT. See throwOnDeadline for the pipes-mode caveat before enabling it.

progressTimeoutMillis is a stall detector, not a ceiling on any single operation. A per-parser timeout (e.g. tesseract-ocr-parser.timeoutMillis) is honored up to min(configured, time remaining in totalTaskTimeoutMillis); progressTimeoutMillis never enters that calculation.

Waits on a bounded external call checkpoint progress periodically, so a legitimate 10-minute readpst invocation does not need progressTimeoutMillis raised to 10 minutes. The stall detector fires only on genuine silence: an in-JVM hang, or a process so wedged that checkpointing cannot report past it.

Budgets compose recursively — a PDF inside a zip inside an email draws from the one totalTaskTimeoutMillis budget for the top-level task. For worked scenarios and how each outcome is reported, see Timeouts.

Embedded byte extraction limits

ParseMode.UNPACK writes embedded bytes out; UnpackConfig.maxUnpackBytes caps the total.

Setting Default Description

maxUnpackBytes

10 GiB

Maximum total bytes extracted from all embedded documents per file. -1 is unlimited (not advisable for untrusted input); 0 means zero bytes, not unlimited.

At the limit, extraction stops for the remaining embedded documents and already-extracted bytes are kept. The parse still reports success: the truncation shows up only as a WARN in the fork’s log, not in the metadata or the result status.

{
  "pipes": { "parseMode": "UNPACK" },
  "parse-context": {
    "unpack-config": { "maxUnpackBytes": 104857600 }
  }
}

See Extracting Embedded Bytes for the rest of UnpackConfig, and UnpackModeTest in tika-pipes/tika-pipes-integration-tests.

Metadata limits

Configuring a MetadataWriteLimiterFactory in the ParseContext makes Metadata.newInstance(parseContext) return a Metadata with limits already applied, so every subsequent write is filtered.

StandardMetadataLimiterFactory factory = new StandardMetadataLimiterFactory();
factory.setMaxTotalBytes(1024 * 1024);
factory.setMaxFieldSize(100 * 1024);
factory.setMaxValuesPerField(100);

ParseContext context = new ParseContext();
context.set(MetadataWriteLimiterFactory.class, factory);
Metadata metadata = Metadata.newInstance(context);
Setting Default Description

maxTotalBytes

10 MB

Total estimated size of all metadata in UTF-16 bytes. Further metadata is dropped and tk:warn:truncated-metadata is set.

maxFieldSize

100 KB

Maximum size of a single field’s value(s) in UTF-16 bytes. Longer values are truncated.

maxKeySize

1024

Maximum metadata key length in UTF-16 bytes. Longer keys are truncated.

maxValuesPerField

10

Maximum values on a multi-valued field. Extra values are dropped.

includeFields

empty (all)

If non-empty, only these fields are stored, plus the always-included fields below.

excludeFields

empty (none)

Never stored, unless always-included.

includeEmpty

false

Whether to store empty or null values.

Always-included fields

StandardMetadataLimiter.ALWAYS_SET_FIELDS and ALWAYS_ADD_FIELDS bypass includeFields/excludeFields and the total-size budget, because parser dispatch and error reporting depend on them. Everything except tk:content is still truncated at max(maxFieldSize, 300).

Always set: Content-Type, Content-Length, Content-Encoding, Content-Disposition, tk:content-type-override, tk:content-type-parser-override, tk:content-type-hint, tk:content, tk:resource-name, tk:exception:container-exception, access-permission:extract-content, access-permission:extract-for-accessibility.

Always added (multi-valued): tk:parsed-by, tk:exception:embedded-exception.

Detecting truncation

Any drop or truncation sets tk:warn:truncated-metadata:

if ("true".equals(metadata.get(TikaCoreProperties.TRUNCATED_METADATA))) {
    log.warn("Metadata was truncated for: " + resourceName);
}

Recommendations

  1. Set limits whenever the content is untrusted.

  2. Use includeFields to capture only the metadata you need.

  3. Check tk:warn:truncated-metadata rather than guessing.

  4. Combine with process isolation — limits protect against memory blowups, process isolation protects against crashes.

  5. Test with adversarial files; MockParser (tika-core test jar) simulates hangs, OOMs, and huge output.

See also