4 The Gatherer

• 4.1 Overview
• 4.2 Basic setup
• 4.3 RootNode specifications
  • 4.3.1 RootNode filters
  • 4.3.2 Example RootNode configuration
  • 4.3.3 Using extreme values -- ``robots''
  • 4.3.4 Gatherer enumeration vs. candidate selection
• 4.4 Extracting data for indexing: The Essence summarizing subsystem
  • 4.4.1 Default actions of ``stock'' summarizers
  • 4.4.2 Summarizing SGML data
    • Location of support files
    • The SGML to SOIF table
    • Errors and warnings from the SGML Parser
    • Creating a summarizer for a new SGML-tagged data type
    • The SGML-based HTML summarizer
    • Other examples
  • 4.4.3 Summarizer components distribution
    • Using ``Rainbow'' to summarize MIF and RTF documents
    • The translation table
  • 4.4.4 Customizing the type recognition, candidate selection, presentation unnesting, and summarizing steps
    • Customizing the type recognition step
    • Customizing the candidate selection step
    • Customizing the presentation unnesting step
    • Customizing the summarizing step
• 4.5 Post-Summarizing: Rule-based tuning of object summaries
• 4.6 Gatherer administration
  • 4.6.1 Setting variables in the Gatherer configuration file
  • 4.6.2 Local file system gathering for reduced CPU load
  • 4.6.3 Gathering from password-protected servers
  • 4.6.4 Controlling access to the Gatherer's database
  • 4.6.5 Periodic gathering and realtime updates
  • 4.6.6 The local disk cache
  • 4.6.7 Incorporating manually generated information into a Gatherer
• 4.7 Troubleshooting