OpenSearch Plugin

The OpenSearch plugin (tika-pipes-opensearch) provides an emitter (writes parsed docs to an OpenSearch index) and a reporter (writes per-document processing status to OpenSearch).

Interface Component name Class

Emitter

opensearch-emitter

OpenSearchEmitter

Reporter

opensearch-pipes-reporter

OpenSearchPipesReporter

Shared HTTP Client Settings

Both the emitter and the reporter take a nested httpClientConfig block. The block is required on both — omitting it fails at startup with a NullPointerException, not a config error.

The emitter and the reporter read the same three fields — userName, password and verifySsl. The remaining fields are accepted by the JSON parser but not applied to the HTTP client by either component, so setting them has no effect:

Field Default Description

userName / password

none

Basic-auth credentials. Omit both for an anonymous client.

verifySsl

true

Verify server certificates and hostnames against the JVM trust store.

authScheme

basic (fixed)

Not read. The client always uses basic auth; setting this to ntlm changes nothing.

connectionTimeoutMillis

120000 (fixed)

Not read. The connect timeout is the HTTP client’s own default of 120000 ms.

socketTimeoutMillis

120000 (fixed)

Not read. The socket read timeout is the HTTP client’s own default of 120000 ms.

proxyHost / proxyPort

none

Not read. Neither component configures an outbound proxy.
The Elasticsearch plugin applies all of these; only the OpenSearch plugin is limited to the three above. If you need a proxy or non-default timeouts against OpenSearch, set them at the JVM level.

Both components verify TLS certificates and hostnames by default, so an https:// openSearchUrl is encrypted and authenticated. If the cluster uses a private CA, add it to the JVM truststore. To disable verification — self-signed certificates are the usual reason — set verifySsl on httpClientConfig for the emitter and the reporter separately:

"httpClientConfig": { "verifySsl": false }

With verifySsl: false the client accepts any certificate from any host and skips the hostname check, so anything on the network path can read and alter what is indexed. It logs a WARN saying so at startup. Prefer trusting the CA.

OpenSearch Emitter (opensearch-emitter)

Writes parsed documents to an OpenSearch index.

{
  "emitters": {
    "ose": {
      "opensearch-emitter": {
        "openSearchUrl": "https://opensearch.example.com:9200/tika-docs",
        "idField": "doc_id",
        "attachmentStrategy": "PARENT_CHILD",
        "updateStrategy": "OVERWRITE",
        "commitWithin": 1000,
        "embeddedFileFieldName": "embedded",
        "httpClientConfig": {
          "userName": "admin",
          "password": "REDACTED",
          "authScheme": "basic",
          "connectionTimeoutMillis": 10000,
          "socketTimeoutMillis": 60000
        }
      }
    }
  }
}

Configuration

Field Default Description

openSearchUrl

required

Full URL of the target OpenSearch index, e.g., https://opensearch.example.com:9200/tika-docs.

idField

required

Validated non-blank, but not otherwise read: the _id is the emit key (attachments get the emit key plus a random suffix).

attachmentStrategy

SEPARATE_DOCUMENTS

How attached/embedded documents are indexed. SEPARATE_DOCUMENTS — each attachment becomes its own top-level document; PARENT_CHILD — attachments are nested under the parent in a parent/child relation. Absent behaves as SEPARATE_DOCUMENTS.

updateStrategy

UPSERT

How existing documents are handled. OVERWRITE — replaces an existing document at _id; UPSERT — merges into an existing document. Absent behaves as UPSERT.

commitWithin

unused

Accepted by the config parser but never read. Control refresh visibility on the OpenSearch side instead.

embeddedFileFieldName

none

Name of the field used to hold embedded-file content (used by PARENT_CHILD).

httpClientConfig

required

See Shared HTTP Client Settings.

OpenSearch Reporter (opensearch-pipes-reporter)

Writes per-document processing status records to an OpenSearch index. Useful for building dashboards over pipeline activity.

{
  "pipes-reporters": {
    "opensearch-pipes-reporter": {
      "openSearchUrl": "https://opensearch.example.com:9200/tika-status",
      "includes": ["PARSE_SUCCESS", "PARSE_EXCEPTION_NO_EMIT", "OOM", "TIMEOUT"],
      "keyPrefix": "tika_",
      "includeRouting": true,
      "httpClientConfig": {
        "userName": "admin",
        "password": "REDACTED",
        "authScheme": "basic",
        "connectionTimeoutMillis": 10000,
        "socketTimeoutMillis": 60000
      }
    }
  }
}

pipes-reporters accepts multiple reporters keyed by component name — see Pipes Reporters for how multiple reporters compose.

Configuration

Field Default Description

openSearchUrl

required

Full URL of the status index, e.g., https://opensearch.example.com:9200/tika-status.

includes

none

Set of RESULT_STATUS names to include (e.g., PARSE_SUCCESS, PARSE_SUCCESS_WITH_EXCEPTION). Names are resolved with RESULT_STATUS.valueOf, so an unrecognised name fails at startup. If unset, all are reported.

excludes

none

Set of RESULT_STATUS names to skip. Mutually exclusive with includes; setting both is a config error.

keyPrefix

none

Prefix prepended to status field names in the emitted documents.

includeRouting

false

If true, include OpenSearch routing info in each status record.

httpClientConfig

required

See Shared HTTP Client Settings.

Complete Pipeline Example

A filesystem iterator/fetcher feeding the OpenSearch emitter and reporter — the usual shape for ingesting a directory into an OpenSearch index.

{
  "content-handler-factory": {
    "basic-content-handler-factory": {
      "type": "TEXT",
      "writeLimit": -1,
      "throwOnWriteLimitReached": true
    }
  },
  "fetchers": {
    "fsf": {
      "file-system-fetcher": {
        "basePath": "/data/input",
        "extractFileSystemMetadata": false
      }
    }
  },
  "emitters": {
    "ose": {
      "opensearch-emitter": {
        "openSearchUrl": "https://opensearch.example.com:9200/tika-docs",
        "idField": "doc_id",
        "attachmentStrategy": "PARENT_CHILD",
        "updateStrategy": "OVERWRITE",
        "commitWithin": 1000,
        "embeddedFileFieldName": "embedded",
        "httpClientConfig": {
          "userName": "admin",
          "password": "REDACTED",
          "authScheme": "basic",
          "connectionTimeoutMillis": 10000,
          "socketTimeoutMillis": 60000
        }
      }
    }
  },
  "pipes-iterator": {
    "file-system-pipes-iterator": {
      "basePath": "/data/input",
      "countTotal": true,
      "fetcherId": "fsf",
      "emitterId": "ose"
    }
  },
  "pipes-reporters": {
    "opensearch-pipes-reporter": {
      "openSearchUrl": "https://opensearch.example.com:9200/tika-status",
      "includes": ["PARSE_SUCCESS", "PARSE_EXCEPTION_NO_EMIT", "OOM", "TIMEOUT"],
      "keyPrefix": "tika_",
      "includeRouting": true,
      "httpClientConfig": {
        "userName": "admin",
        "password": "REDACTED",
        "authScheme": "basic",
        "connectionTimeoutMillis": 10000,
        "socketTimeoutMillis": 60000
      }
    }
  },
  "pipes": {
    "parseMode": "RMETA",
    "onParseException": "EMIT",
    "numClients": 4
  }
}

Notes

  • The OpenSearch plugin’s HTTP client is REST-based; it does not depend on the OpenSearch transport client.

  • For Elasticsearch deployments, use the parallel Elasticsearch plugin instead — the field names differ (esUrl vs. openSearchUrl) and ES adds API-key auth.

  • Don’t check real credentials into source control — the password values in the examples above are placeholders.