Class HtmlContentCleaner

java.lang.Object

org.apache.tika.ml.junkdetect.HtmlContentCleaner

public final class HtmlContentCleaner
extends Object

Junk-detection HTML→text cleaning: strip tags, then expand entities to codepoints. Called by both TrainJunkModel and JunkFilterEncodingDetector so training and inference prepare text identically (no drift).

Unlike charset detection's HtmlByteStripper.stripTagsAndEntities(byte[], int, int, byte[], int) (which drops entities as charset-neutral ASCII noise), junk detection expands them: the resulting codepoints — cross-script under a wrong decoding — are what expose mojibake.

Method Summary

Modifier and Type	Method	Description
static String	clean(String s)	Strip HTML tags (entities preserved through the strip), then expand entities to real codepoints.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

clean

public static String clean(String s)

Strip HTML tags (entities preserved through the strip), then expand entities to real codepoints. No-op-ish on plain text (no tags/entities).