org.apache.tika.parser.txt

Class Icu4jEncodingDetector

java.lang.Object

org.apache.tika.parser.txt.Icu4jEncodingDetector

All Implemented Interfaces:

Serializable, EncodingDetector

public class Icu4jEncodingDetector
extends Object
implements EncodingDetector

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Icu4jEncodingDetector()

Method Summary

Modifier and Type	Method and Description
Charset	detect(InputStream input, Metadata metadata) Detects the character encoding of the given text document, or null if the encoding of the document can not be detected.
int	getMarkLimit()
boolean	isStripMarkup()
void	setIgnoreCharsets(List<String> charsetsToIgnore)
void	setMarkLimit(int markLimit) How far into the stream to read for charset detection.
void	setStripMarkup(boolean stripMarkup) Whether or not to attempt to strip html-ish markup from the stream before sending it to the underlying detector.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Icu4jEncodingDetector

public Icu4jEncodingDetector()

Method Detail

detect

public Charset detect(InputStream input,
                      Metadata metadata)
               throws IOException

Description copied from interface: EncodingDetector

Detects the character encoding of the given text document, or null if the encoding of the document can not be detected.

If the document input stream is not available, then the first argument may be null. Otherwise the detector may read bytes from the start of the stream to help in encoding detection. The given stream is guaranteed to support the mark feature and the detector is expected to mark the stream before reading any bytes from it, and to reset the stream before returning. The stream must not be closed by the detector.

The given input metadata is only read, not modified, by the detector.

Specified by:

detect in interface EncodingDetector

Parameters:

input - text document input stream, or null

metadata - input metadata for the document

Returns:

detected character encoding, or null

Throws:

IOException - if the document input stream could not be read

isStripMarkup

public boolean isStripMarkup()

setStripMarkup

@Field
public void setStripMarkup(boolean stripMarkup)

Whether or not to attempt to strip html-ish markup from the stream before sending it to the underlying detector.

The underlying detector may still apply its own stripping if this is set to false.

Parameters:

stripMarkup - whether or not to attempt to strip markup before sending the stream to the underlying detector

getMarkLimit

public int getMarkLimit()

setMarkLimit

@Field
public void setMarkLimit(int markLimit)

How far into the stream to read for charset detection. Default is 12000.

Parameters:

markLimit -

setIgnoreCharsets

@Field
public void setIgnoreCharsets(List<String> charsetsToIgnore)