CSV Plugin

The CSV plugin (tika-pipes-csv) provides an iterator that reads work items from a CSV file. It is iterator-only — pair it with a fetcher and emitter.

Interface Component name Class

Iterator

csv-pipes-iterator

CSVPipesIterator

CSV Iterator (csv-pipes-iterator)

Reads each row of the CSV as a work item and emits one FetchEmitTuple per row.

{
  "pipes-iterator": {
    "csv-pipes-iterator": {
      "csvPath": "/data/work-items.csv",
      "idColumn": "doc_id",
      "fetchKeyColumn": "source_path",
      "emitKeyColumn": "output_path",
      "fetcherId": "fsf",
      "emitterId": "fse"
    }
  }
}

Configuration

In addition to the required fetcherId / emitterId (see Wiring an Iterator):

Field Default Description

csvPath

required

Path to the CSV file on disk.

fetchKeyColumn

required

Column whose value becomes the fetch key on each emitted tuple. Setting it also makes fetcherId required.

emitKeyColumn

none

Column whose value becomes the emit key on each emitted tuple. Falls back to fetchKeyColumn.

idColumn

none

Column whose value becomes the tuple’s id. Falls back to fetchKeyColumn.

Every column named above is consumed as a key and left out of the metadata; all remaining columns become metadata, keyed by their header. A column whose header collides with a reserved Tika-native key is logged and skipped rather than aborting the row.

Notes

  • The file must be UTF-8 and is parsed with CSVFormat.EXCEL (comma-delimited).

  • The CSV must have a header row — column names in the config refer to header values, not column indexes. A blank header cell aborts the run.

  • For very large CSV files, the iterator streams rows rather than loading them all into memory.

  • For row-shaped work items in JSONL (one JSON object per line), use the JSON iterator instead.