Class BatchProcessBuilder


  • public class BatchProcessBuilder
    extends Object
    Builds a BatchProcessor from a combination of runtime arguments and the config file.
    • Constructor Detail

      • BatchProcessBuilder

        public BatchProcessBuilder()
    • Method Detail

      • build

        public BatchProcess build​(InputStream is,
                                  Map<String,​String> runtimeAttributes)
                           throws IOException
        Builds a BatchProcess from runtime arguments and a input stream of a configuration file. With the exception of the QueueBuilder, the builders choose how to adjudicate between runtime arguments and the elements in the configuration file.

        This does not close the InputStream!

        Parameters:
        is - inputStream
        runtimeAttributes - incoming runtime attributes
        Returns:
        batch process
        Throws:
        IOException
      • build

        public BatchProcess build​(Node docElement,
                                  Map<String,​String> incomingRuntimeAttributes)
        Builds a FileResourceBatchProcessor from runtime arguments and a document node of a configuration file. With the exception of the QueueBuilder, the builders choose how to adjudicate between runtime arguments and the elements in the configuration file.
        Parameters:
        docElement - document element of the xml config file
        incomingRuntimeAttributes - runtime arguments
        Returns:
        FileResourceBatchProcessor
      • getNumConsumers

        public static int getNumConsumers​(Map<String,​String> attrs)
        numConsumers is needed by both the crawler and the consumers. This utility method is to be used to extract the number of consumers from a map of String key value pairs.

        If the value is "default", not a parseable integer or has a value < 1, then AbstractConsumersBuilder's getDefaultNumConsumers()

        Parameters:
        attrs - attributes from which to select the NUM_CONSUMERS_KEY
        Returns:
        number of consumers