Use TSaxon, a variant of Saxon, and TagSoup to help transform HTML.
Stylesheets written in XSLT are the standard method of taking XML documents in one format and transforming them into HTML, XML documents in a different format, XHTML, or plain-text documents.
There are many XSLT processors. Michael Kay’s Saxon Version 6.5.3 (http://saxon.sourceforge.net/#F6.5.3) is a particularly mature and successful implementation for XSLT 1.0 and XPath 1.0. It is packaged as a Java JAR file called saxon.jar. You can download this JAR with the 6.5.3 distribution from the Saxon site on Sourceforge.
Now suppose, for example, that we want to extract just the header
elements (h1
, h2
,
h3
, etc.) from an XHTML document and display them
as progressively indented plain text (i.e., each
h1
element is unindented, each
h2
element is indented by a single space,
h2
by two spaces, etc.).
The XSLT stylesheet outline.xsl does exactly
what we want. It specifies an output method of text, and matches the
h1
through h6
elements in the
XHTML input, taking the content of each one and prepending the
correct number of spaces. The textual content of other elements is
suppressed.
The following command, executed in your working directory, will process outline.xsl and the XHTML document outline.xhtml using Saxon and will display the resulting indented plain text:
java -jar saxon.jar outline.xhtml outline.xsl
It so happens that outline.html contains only
h1
, h2
, and
h3
elements (borrowed from the XML specification)
for the sake of brevity, but if you add other elements to it, you
will see that the content of those elements is not displayed by this
command.
The
great bulk of documents on the Web are not well-formed XHTML, but are
HTML, so unmodified XSLT processors cannot be applied to them. For
example, the HTML document outline.html is
similar to outline.xhtml, but the end tags for
the h1
, h2
, and
h3
elements are missing. If you try to apply Saxon
to outline.html, you will get no output except
an error message saying the document is not well-formed.
In this situation, you can use TSaxon instead of Saxon. TSaxon
(http://www.ccil.org/~cowan/XML/tagsoup/tsaxon)
is a variant of Saxon 6.5.3 with an HTML parser called TagSoup
packaged in it. It behaves exactly like Saxon unless you give it the
-H
(HTML) switch, which causes it to substitute
the TagSoup parser for the standard XML parser packaged with Saxon.
You can download the latest version of TSaxon from the web site, or just use the version that came in the file archive. TSaxon’s version of saxon.jar exists in the file archive under the subdirectory TSaxon. In the TSaxon subdirectory, issuing the following command will process outline.html using the TSaxon and the TagSoup parser and the stylesheet:
java -jar saxon.jar -H outline.html outline.xsl
The reason that TSaxon is also packaged as the file saxon.jar is that it is completely backward-compatible and can be used as a drop-in replacement for Saxon. Saxon is stable, but TSaxon is not yet stable, because TagSoup (http://www.ccil.org/~cowan/XML/tagsoup) is still under active development. Nevertheless, TSaxon is still quite useful with even very messy HTML: less than one percent of a sample of over a thousand HTML files (downloaded from the Web at random) could not be correctly processed with the current version.
—John Cowan
18.218.238.134