Parsing RSS Feeds
Out of the box, the parser only understands XML tags that have the same name as HTML tags. So this example:
import org.htmlparser.Parser;
import org.htmlparser.filters.NodeClassFilter;
import org.htmlparser.tags.TitleTag;
import org.htmlparser.util.NodeIterator;
import org.htmlparser.util.NodeList;
import org.htmlparser.util.ParserException;
/*
* RSS (RDF Site Summary - formerly called Rich Site Summary) is a method of
* describing news or other Web content that is available for "feeding"
* (distribution or syndication) from an online publisher to Web users.
* RSS is an application of the Extensible Markup Language (XML) that adheres
* to the World Wide Web Consortium's Resource Description Framework (RDF).
* Originally developed by Netscape for its browser's Netcenter channels,
* the RSS specification is now available for anyone to use.
*/
public class ResourceDescriptionFrameworkSiteSummary
{
public static void main (String[] args) throws ParserException
{
Parser parser;
NodeList list;
parser = new Parser ("http://sourceforge.net/export/rss2_sftopstats.php?feed=mostactive_weekly");
list = parser.extractAllNodesThatMatch (new NodeClassFilter (TitleTag.class));
for (NodeIterator iterator = list.elements (); iterator.hasMoreNodes (); )
System.out.println (iterator.nextNode ().toPlainTextString ());
}
}
Will only find the TITLE tags, which may be what we want:
Rank 1: Gaim (100% activity)
Rank 2: Azureus - BitTorrent Client (99.9934% activity)
Rank 3: eGroupWare: Enterprise Collaboration (99.9867% activity)
Rank 4: WinMerge (99.9801% activity)
Rank 5: phpMyAdmin (99.9735% activity)
Rank 6: guliverkli (99.9668% activity)
Rank 7: phpGedView (99.9602% activity)
Rank 8: AMSN (99.9536% activity)
Rank 9: dotproject (99.9469% activity)
Rank 10: ScummVM (99.9403% activity)
However, with some custom tags defined, it can handle the heirarchy of the XML:
import org.htmlparser.Node;
import org.htmlparser.Parser;
import org.htmlparser.PrototypicalNodeFactory;
import org.htmlparser.filters.NodeClassFilter;
import org.htmlparser.tags.CompositeTag;
import org.htmlparser.tags.Tag;
import org.htmlparser.tags.TitleTag;
import org.htmlparser.util.NodeIterator;
import org.htmlparser.util.NodeList;
import org.htmlparser.util.ParserException;
/*
* RSS (RDF Site Summary - formerly called Rich Site Summary) is a method of
* describing news or other Web content that is available for "feeding"
* (distribution or syndication) from an online publisher to Web users.
* RSS is an application of the Extensible Markup Language (XML) that adheres
* to the World Wide Web Consortium's Resource Description Framework (RDF).
* Originally developed by Netscape for its browser's Netcenter channels,
* the RSS specification is now available for anyone to use.
*/
class Item extends CompositeTag { public String[] getIds () { return (new String[] { "ITEM" }); } }
class Title extends CompositeTag { public String[] getIds () { return (new String[] { "TITLE" }); } }
class Description extends CompositeTag { public String[] getIds () { return (new String[] { "DESCRIPTION" }); } }
class Link extends CompositeTag { public String[] getIds () { return (new String[] { "LINK" }); } }
class Guid extends CompositeTag { public String[] getIds () { return (new String[] { "GUID" }); } }
class PubDate extends CompositeTag { public String[] getIds () { return (new String[] { "PUBDATE" }); } }
public class ResourceDescriptionFrameworkSiteSummary
{
public static void main (String[] args) throws ParserException
{
Parser parser;
PrototypicalNodeFactory factory;
NodeList list;
Item item;
NodeList kids;
Node node;
Tag tag;
String name;
parser = new Parser ("http://sourceforge.net/export/rss2_projsummary.php?group_id=24399");
factory = new PrototypicalNodeFactory (true); // empty
factory.registerTag (new Item ());
factory.registerTag (new Title ());
factory.registerTag (new Description ());
factory.registerTag (new Link ());
factory.registerTag (new Guid ());
factory.registerTag (new PubDate ());
parser.setNodeFactory (factory);
list = parser.extractAllNodesThatMatch (new NodeClassFilter (Item.class));
for (NodeIterator iterator = list.elements (); iterator.hasMoreNodes (); )
{
item = (Item)iterator.nextNode ();
kids = item.getChildren ();
if (null != kids)
for (int i = 0; i < kids.size (); i++)
{
node = kids.elementAt (i);
if (node instanceof Tag)
{
tag = (Tag)node;
name = tag.getTagName ();
if (name.equals ("TITLE") || name.equals ("DESCRIPTION"))
System.out.println (tag.toPlainTextString ());
}
}
}
}
}
This isn't as pretty as it could be, but you get the idea:
Project name: HTML Parser
Project description: HTML Parser is a library, written in Java, which allows you to parse HTML (HTML 4.0 supported).
It has been used by people on live projects. Developers appreciate how easy it is to use. The architecture is flexible, allowing you to extend it easily.
Developers on project: 16
Project administrators: <a href="http://sourceforge.net/users/derrickoswald/">derrickoswald</a>, <a href="http://sourceforge.net/users/somik/">somik</a>
Activity percentile (last week): 98.3413%
Most recent daily statistics (24 Jan 2004): Ranking: 251, Activity percentile: 98.34%,
Downloadable files: 25615 total downloads to date
Most recent daily statistics (24 Jan 2004): Download count: 19
Mailing lists (public): 4
Public mailing lists: htmlparser-developer, htmlparser-announce, htmlparser-user, htmlparser-cvs
Discussion forums (public): 2, containing 110 messages
Public discussion forums: Open Discussion, Help, htmlparser-user, htmlparser-developer
Tracker: Bugs (1 open/158 total)
Tracker description: Bug Tracking System
Tracker: Support Requests (1 open/20 total)
Tracker description: Tech Support Tracking System
Tracker: Patches (0 open/0 total)
Tracker description: Patch Tracking System
Tracker: Feature Requests (2 open/10 total)
Tracker description: Feature Request Tracking System
CVS (8169 commits/809 adds)
Most recent daily statistics (24 Jan 2004): Commit count: 0; Add count: 0 <br><a href="http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/htmlparser/">[Web-based access to repository]</a>