org.apache.tika.pipes.fetcher.s3

Class S3Fetcher

java.lang.Object

org.apache.tika.pipes.fetcher.AbstractFetcher

org.apache.tika.pipes.fetcher.s3.S3Fetcher

All Implemented Interfaces:

Initializable, Fetcher, RangeFetcher

public class S3Fetcher
extends AbstractFetcher
implements Initializable, RangeFetcher

Fetches files from s3. Example string: s3://my_bucket/path/to/my_file.pdf This will parse the bucket out of that string and retrieve the path.

Constructor Summary

Constructors

Constructor and Description
S3Fetcher()

Method Summary

Modifier and Type	Method and Description
void	checkInitialization(InitializableProblemHandler problemHandler)
InputStream	fetch(String fetchKey, long startRange, long endRange, Metadata metadata)
InputStream	fetch(String fetchKey, Metadata metadata)
void	initialize(Map<String,Param> params) This initializes the s3 client.
void	setBucket(String bucket)
void	setCredentialsProvider(String credentialsProvider)
void	setExtractUserMetadata(boolean extractUserMetadata) Whether or not to extract user metadata from the S3Object
void	setMaxConnections(int maxConnections)
void	setPrefix(String prefix) prefix to prepend to the fetch key before fetching.
void	setProfile(String profile)
void	setRegion(String region)
void	setSpoolToTemp(boolean spoolToTemp)

Methods inherited from class org.apache.tika.pipes.fetcher.AbstractFetcher

getName, setName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.tika.pipes.fetcher.Fetcher

getName

Constructor Detail

S3Fetcher

public S3Fetcher()

Method Detail

fetch

public InputStream fetch(String fetchKey,
                         Metadata metadata)
                  throws TikaException,
                         IOException

Specified by:

fetch in interface Fetcher

Throws:

TikaException

IOException

fetch

public InputStream fetch(String fetchKey,
                         long startRange,
                         long endRange,
                         Metadata metadata)
                  throws TikaException,
                         IOException

Specified by:

fetch in interface RangeFetcher

Throws:

TikaException

IOException

setSpoolToTemp

@Field
public void setSpoolToTemp(boolean spoolToTemp)

setRegion

@Field
public void setRegion(String region)

setProfile

@Field
public void setProfile(String profile)

setBucket

@Field
public void setBucket(String bucket)

setPrefix

@Field
public void setPrefix(String prefix)

prefix to prepend to the fetch key before fetching. This will automatically add a '/' at the end.

Parameters:

prefix -

setExtractUserMetadata

@Field
public void setExtractUserMetadata(boolean extractUserMetadata)

Whether or not to extract user metadata from the S3Object

Parameters:

extractUserMetadata -

setMaxConnections

@Field
public void setMaxConnections(int maxConnections)

setCredentialsProvider

@Field
public void setCredentialsProvider(String credentialsProvider)

initialize

public void initialize(Map<String,Param> params)
                throws TikaConfigException

This initializes the s3 client. Note, we wrap S3's RuntimeExceptions, e.g. AmazonClientException in a TikaConfigException.

Specified by:

initialize in interface Initializable

Parameters:

params - params to use for initialization

Throws:

TikaConfigException

checkInitialization

public void checkInitialization(InitializableProblemHandler problemHandler)
                         throws TikaConfigException

Specified by:

checkInitialization in interface Initializable

Parameters:

problemHandler - if there is a problem and no custom initializableProblemHandler has been configured via Initializable parameters, this is called to respond.

Throws:

TikaConfigException