Package org.apache.tika.parser.inference

Class InferenceDispatcher

java.lang.Object

org.apache.tika.parser.inference.InferenceDispatcher

All Implemented Interfaces:

TransientParseState, ParseHook

public final class InferenceDispatcher
extends Object
implements ParseHook, TransientParseState

Matches offered units against the inference bindings and runs the tasks. Built once at config load and run as a ParseHook: every document, top-level or embedded, is offered once by the auto-detect parser. Units are buffered per binding for the whole top-level parse and flushed at its end, so a document is one request per binding, not one per unit; the buffer is bounded by each binding's maxChunks and maxBytes, and its bytes live in files the dispatcher owns until the flush. At the flush every unit is aimed at the metadata that is still read: under the recursive wrapper, the copy it kept for the unit's document (an inline part or a page render lands on its parent); outside it, where the parse hands back one metadata object and the embedded documents' own are discarded, the top-level document. A request narrows the bindings through InferenceSelection. Failures mark the document and never fail the parse. TEXT is not offered during the parse: whoever holds the finished metadata list runs text(java.util.List<org.apache.tika.metadata.Metadata>, org.apache.tika.parser.ParseContext) over it.

Since:

Apache Tika 4.1

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	InferenceDispatcher.Bound	A binding resolved to its engine and tasks.

Constructor Summary

Constructors

Constructor	Description
InferenceDispatcher(List<InferenceDispatcher.Bound> bound)

Method Summary

Modifier and Type	Method	Description
void	discard(ParseContext context)	Clears the buffer and its files without running anything.
void	end(Metadata root, boolean failed, ParseContext context)	Runs the buffered units, or drops them when the parse failed: no engine call for a document nobody gets.
void	flush(Metadata root, ParseContext context)	Runs every binding's tasks over its buffered units; clears the buffer and its files.
List<InferenceDispatcher.Bound>	getBound()
void	offer(MediaType type, Metadata metadata, Metadata parent, Path bytes, ParseContext context)	After the document's parser ran cleanly.
void	offer(InputKind kind, MediaType type, Metadata target, Metadata parent, Path source, ParseContext context)	Buffers the bytes at source for every binding that takes them; dropped past a binding's budget.
void	offerPage(MediaType type, Metadata document, Metadata parent, int page, Path bytes, ParseContext context)	A page of document rendered by its own parser; page is 1-based, bytes and parent as in ParseHook.offer(org.apache.tika.mime.MediaType, org.apache.tika.metadata.Metadata, org.apache.tika.metadata.Metadata, java.nio.file.Path, org.apache.tika.parser.ParseContext).
void	start(Metadata root, ParseContext context)	Resolves the request's selection once, at the top of the parse, so a misnamed binding fails the request rather than one embedded document.
void	text(List<Metadata> metadataList, ParseContext context)	The InputKind.TEXT stage.
boolean	wants(MediaType type, Metadata metadata, ParseContext context)	Whether this hook wants the bytes of a document of this detected type.
boolean	wants(InputKind kind, MediaType type, ParseContext context)	Whether any binding that runs for this request takes this kind and type.
boolean	wantsPages(MediaType renderType, Metadata document, ParseContext context)	Whether this hook wants the pages of the document being parsed rendered as images of this type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

InferenceDispatcher

public InferenceDispatcher(List<InferenceDispatcher.Bound> bound)

Method Details

getBound

public List<InferenceDispatcher.Bound> getBound()

start

public void start(Metadata root,
 ParseContext context)
           throws TikaException

Resolves the request's selection once, at the top of the parse, so a misnamed binding fails the request rather than one embedded document.

Specified by:

start in interface ParseHook

Throws:

TikaException

wants

public boolean wants(MediaType type,
 Metadata metadata,
 ParseContext context)
              throws TikaException

Description copied from interface: ParseHook

Whether this hook wants the bytes of a document of this detected type.

Specified by:

wants in interface ParseHook

Throws:

TikaException

offer

public void offer(MediaType type,
 Metadata metadata,
 Metadata parent,
 Path bytes,
 ParseContext context)
           throws IOException,
TikaException

Description copied from interface: ParseHook

After the document's parser ran cleanly. bytes is the document pinned to a file for this call only; parent is the document it is embedded in, null at top level.

Specified by:

offer in interface ParseHook

Throws:

IOException

TikaException

wantsPages

public boolean wantsPages(MediaType renderType,
 Metadata document,
 ParseContext context)
                   throws TikaException

Description copied from interface: ParseHook

Whether this hook wants the pages of the document being parsed rendered as images of this type. Its parser asks once, before rendering anything for the hooks.

Specified by:

wantsPages in interface ParseHook

Throws:

TikaException

offerPage

public void offerPage(MediaType type,
 Metadata document,
 Metadata parent,
 int page,
 Path bytes,
 ParseContext context)
               throws IOException,
TikaException

Description copied from interface: ParseHook

A page of document rendered by its own parser; page is 1-based, bytes and parent as in ParseHook.offer(org.apache.tika.mime.MediaType, org.apache.tika.metadata.Metadata, org.apache.tika.metadata.Metadata, java.nio.file.Path, org.apache.tika.parser.ParseContext).

Specified by:

offerPage in interface ParseHook

Throws:

IOException

TikaException

end

public void end(Metadata root,
 boolean failed,
 ParseContext context)

Runs the buffered units, or drops them when the parse failed: no engine call for a document nobody gets.

Specified by:

end in interface ParseHook

wants

public boolean wants(InputKind kind,
 MediaType type,
 ParseContext context)
              throws TikaException

Whether any binding that runs for this request takes this kind and type.

Throws:

TikaException

offer

public void offer(InputKind kind,
 MediaType type,
 Metadata target,
 Metadata parent,
 Path source,
 ParseContext context)
           throws IOException,
TikaException

Buffers the bytes at source for every binding that takes them; dropped past a binding's budget. The top-level document's own file is referenced, since the stream that owns it is open until after the flush; anything else (an embedded document's spool, a page render) is copied into a file the dispatcher owns before its parser deletes it.

Throws:

IOException

TikaException

flush

public void flush(Metadata root,
 ParseContext context)

Runs every binding's tasks over its buffered units; clears the buffer and its files.

discard

public void discard(ParseContext context)

Clears the buffer and its files without running anything.

text

public void text(List<Metadata> metadataList,
 ParseContext context)
          throws TikaException

The InputKind.TEXT stage. Runs the TEXT bindings over the finished metadata list of a document tree, so every document's text goes to the engine in one pass and a batch may hold chunks of the container and of its attachments alike. Call it after the parse and before any metadata filters. A document that released other inputs to inference (TikaCoreProperties.INFERENCE_RELEASED without TEXT) is skipped. Failures mark the first document of the list.

Throws:

TikaException