Package org.apache.tika.pipes.fetcher.http

Class HttpFetcher

java.lang.Object

org.apache.tika.plugins.AbstractTikaExtension

org.apache.tika.pipes.fetcher.http.HttpFetcher

All Implemented Interfaces:

Fetcher, RangeFetcher, TikaExtension, org.pf4j.ExtensionPoint

public class HttpFetcher
extends AbstractTikaExtension
implements Fetcher, RangeFetcher

Based on Apache httpclient

Field Summary

Fields

Modifier and Type	Field	Description
static Property	HTTP_CONTENT_ENCODING
static Property	HTTP_CONTENT_TYPE
static String	HTTP_FETCH_PREFIX
static Property	HTTP_FETCH_TRUNCATED
static String	HTTP_HEADER_PREFIX
static Property	HTTP_NUM_REDIRECTS	Number of redirects
static Property	HTTP_STATUS_CODE	http status code
static Property	HTTP_TARGET_IP_ADDRESS
static Property	HTTP_TARGET_URL	If there were redirects, this captures the final URL visited

Fields inherited from class org.apache.tika.plugins.AbstractTikaExtension

pluginConfig

Constructor Summary

Constructors

Constructor	Description
HttpFetcher(ExtensionConfig pluginConfig, HttpFetcherConfig httpFetcherConfig)

Method Summary

Modifier and Type	Method	Description
static HttpFetcher	build(ExtensionConfig pluginConfig)
TikaInputStream	fetch(String fetchKey, long startRange, long endRange, Metadata metadata, ParseContext parseContext)
TikaInputStream	fetch(String fetchKey, Metadata metadata, ParseContext parseContext)	Fetches a resource and returns it as a TikaInputStream.
HttpFetcherConfig	getHttpFetcherConfig()
JwtGenerator	getJwtGenerator()
static Map<String,Collection<String>>	parseHeaders(String headersString)
void	setHttpClient(org.apache.http.client.HttpClient httpClient)
void	setHttpClientFactory(HttpClientFactory httpClientFactory)
void	setHttpFetcherConfig(HttpFetcherConfig httpFetcherConfig)
void	setJwtGenerator(JwtGenerator jwtGenerator)

Methods inherited from class org.apache.tika.plugins.AbstractTikaExtension

getExtensionConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.tika.pipes.api.fetcher.RangeFetcher

fetch

Methods inherited from interface org.apache.tika.plugins.TikaExtension

getExtensionConfig

Field Details

HTTP_HEADER_PREFIX

public static String HTTP_HEADER_PREFIX

HTTP_FETCH_PREFIX

public static String HTTP_FETCH_PREFIX

HTTP_STATUS_CODE

public static Property HTTP_STATUS_CODE

http status code

HTTP_NUM_REDIRECTS

public static Property HTTP_NUM_REDIRECTS

Number of redirects

HTTP_TARGET_URL

public static Property HTTP_TARGET_URL

If there were redirects, this captures the final URL visited

HTTP_TARGET_IP_ADDRESS

public static Property HTTP_TARGET_IP_ADDRESS

HTTP_FETCH_TRUNCATED

public static Property HTTP_FETCH_TRUNCATED

HTTP_CONTENT_ENCODING

public static Property HTTP_CONTENT_ENCODING

HTTP_CONTENT_TYPE

public static Property HTTP_CONTENT_TYPE

Constructor Details

HttpFetcher

public HttpFetcher(ExtensionConfig pluginConfig,
 HttpFetcherConfig httpFetcherConfig)

Method Details

build

public static HttpFetcher build(ExtensionConfig pluginConfig)
                         throws TikaConfigException,
IOException

Throws:

TikaConfigException

IOException

fetch

public TikaInputStream fetch(String fetchKey,
 Metadata metadata,
 ParseContext parseContext)
                      throws IOException,
TikaException

Description copied from interface: Fetcher

Fetches a resource and returns it as a TikaInputStream.

Specified by:

fetch in interface Fetcher

Parameters:

fetchKey - the key identifying the resource to fetch (interpretation depends on the implementation, e.g., file path, URL, S3 key)

metadata - metadata object to be updated with resource information

parseContext - the parse context

Returns:

a TikaInputStream for reading the resource content

Throws:

IOException - if an I/O error occurs during fetching

TikaException - if a Tika-specific error occurs during fetching

fetch

public TikaInputStream fetch(String fetchKey,
 long startRange,
 long endRange,
 Metadata metadata,
 ParseContext parseContext)
                      throws IOException,
TikaException

Specified by:

fetch in interface RangeFetcher

Throws:

IOException

TikaException

parseHeaders

public static Map<String,Collection<String>> parseHeaders(String headersString)

setHttpClientFactory

public void setHttpClientFactory(HttpClientFactory httpClientFactory)

setHttpFetcherConfig

public void setHttpFetcherConfig(HttpFetcherConfig httpFetcherConfig)
                          throws TikaConfigException

Throws:

TikaConfigException

setHttpClient

public void setHttpClient(org.apache.http.client.HttpClient httpClient)

getHttpFetcherConfig

public HttpFetcherConfig getHttpFetcherConfig()

setJwtGenerator

public void setJwtGenerator(JwtGenerator jwtGenerator)

getJwtGenerator

public JwtGenerator getJwtGenerator()