Class StandardHtmlEncodingDetector

  • All Implemented Interfaces:
    Serializable, EncodingDetector

    public final class StandardHtmlEncodingDetector
    extends Object
    implements EncodingDetector
    An encoding detector that tries to respect the spirit of the HTML spec part 12.2.3 "The input byte stream", or at least the part that is compatible with the implementation of tika.

    https://html.spec.whatwg.org/multipage/parsing.html#the-input-byte-stream

    If a resource was fetched over HTTP, then HTTP headers should be added to tika metadata when using detect(java.io.InputStream, org.apache.tika.metadata.Metadata), especially HttpHeaders.CONTENT_TYPE, as it may contain charset information.

    This encoding detector may return null if no encoding is detected. It is meant to be used inside a CompositeEncodingDetector. For instance:

     
         EncodingDetector detector = new CompositeEncodingDetector(
           Arrays.asList(
             new StandardHtmlEncodingDetector(),
             new Icu4jEncodingDetector()));
     

    See Also:
    Serialized Form
    • Constructor Detail

      • StandardHtmlEncodingDetector

        public StandardHtmlEncodingDetector()
    • Method Detail

      • detect

        public Charset detect​(InputStream input,
                              Metadata metadata)
                       throws IOException
        Description copied from interface: EncodingDetector
        Detects the character encoding of the given text document, or null if the encoding of the document can not be detected.

        If the document input stream is not available, then the first argument may be null. Otherwise the detector may read bytes from the start of the stream to help in encoding detection. The given stream is guaranteed to support the mark feature and the detector is expected to mark the stream before reading any bytes from it, and to reset the stream before returning. The stream must not be closed by the detector.

        The given input metadata is only read, not modified, by the detector.

        Specified by:
        detect in interface EncodingDetector
        Parameters:
        input - text document input stream, or null
        metadata - input metadata for the document
        Returns:
        detected character encoding, or null
        Throws:
        IOException - if the document input stream could not be read
      • getMarkLimit

        public int getMarkLimit()
      • setMarkLimit

        @Field
        public void setMarkLimit​(int markLimit)
        How far into the stream to read for charset detection. Default is 8192.