Developing MagicStats Themes User Guide
=======================================

Chris Lattner, Cherokee Communication Corp.

1. Developing MagicStats Themes
===============================

MagicStats is a powerful web server analysis tool.  This power comes
from the fact that it is possible to completely customize the results
of the engine through the use of Themes.  A Theme is a collection of
pages that present information gathered by MagicStats in a particular
layout.  Themes do not calculate data on their own, instead, they
format the data produced by MagicStats plugins.

The purpose of this manual is to describe the method used to create a
custom theme.  This assumes that the concepts of Variables, Filters,
and Themes are well understood; these basic building blocks are used
to detail the inner workings of themes and the plugins that they use
for data mining.

1.1. Naming the Theme
---------------------

When developing a theme, the first step is to choose a name for the
theme.  For this example, we will develop a theme named Entropy.  To
start a new theme, first create a subdirectory in your Themes
directory to hold the theme:

devotion:~/.MagicStats/Themes> mkdir Entropy
devotion:~/.MagicStats/Themes> cd Entropy/
devotion:~/.MagicStats/Themes/Entropy>

This directory will hold the configuration file for the theme as well
as the Skeleton Files for the theme and any additional resources the
theme needs (such as graphics).

1.2. Creating the Configuration File
------------------------------------

Once the directory for the theme exists, you must create a
configuration file that specifies which Skeleton files (Skel Files) to
use.  This file must be named <ThemeName>.cfg and must reside in the
<ThemeName> directory.

devotion:~/.MagicStats/Themes/Entropy> touch Entropy.cfg
devotion:~/.MagicStats/Themes/Entropy> vi Entropy.cfg
devotion:~/.MagicStats/Themes/Entropy>

The configuration file is the central coordination unit for the theme.
 It specifies how data flows between the individual pages of the theme
and how frequently the pages are updated.  The configuration file
follows the MagicStats standard .cfg file conventions.

1.3. Creating the Skeleton Files
--------------------------------

For each page generated by the theme, there is a corresponding Skel
File. Skel files consist of a modified form of HTML.  As a matter of
fact, Skel files are really just standard HTML with a few extra tags,
and an expanded syntax.  Anyone who can write HTML and understand how
it works can create their own MagicStats theme.

For the example of the Entropy theme, we will create x files.

1.4. Supplying additional theme resources
-----------------------------------------

Themes need graphics and stuff occasionally. Logo example?

2. Developing the Configuration File
====================================

2.1. Variable Export Table
--------------------------

2.2. Variable Inheritance
-------------------------

2.3. Skel File Descriptors
--------------------------

2.4. A Word on Update Frequencies
---------------------------------

3. Developing Skel Files
========================

Every page of output that MagicStats generates has a corresponding
Skel file.  A Skel file is a type of a template for a page that
MagicStats creates.  Skel files provide formatting and layout of the
raw information that the MagicStats engine provides.

A Skel file is a superset of standard HTML; it extends the syntax and
adds a few new tags to the mix.  As a matter for fact, any plain HTML
file that is input to MagicStats will be passed straight through as
output with no changes.  The only reason to make changes or use any of
the HTML enhancements is to provide dynamic content.  There are many
ways to insert data, with the <MSInsert> tag and with the <MSPlugin>
tag.  The <MSInsert> tag is the simplest, so we'll start there.

3.1. Inserting Variables and Data
---------------------------------

One you have prepared your static HTML page for the theme, you should
have places that need some simple dynamic content inserted in it.  An
example of this could be a date stamp for the statistics, the name of
the web site being monitored, or a link to another page, relative to
the date of this one.

The <MSInsert> tag follows syntax similar to standard HTML, with a few
exceptions.  One of these exceptions is that it may be placed inside
of other tags, in places normally not valid in HTML.  One example of
this is to link to a site whose URL is stored in the $SiteURL
variable:

<a href="<MSInsert=($SiteURL)>">Link to my site!</a>

When this is processed by MagicStats, that tag is replaced before it
reaches the browser instead, the HTML code may end up looking like
this:

<a href="http://www.magicstats.com/">Link to my site!</a>

which is now completely valid HTML.

Generally, the <MSInsert> tag can insert either arbitrary Skel
Expressions (See Page #6) or a number of special tags, with optional
formatting attributes.  Inserting an expression works like shown above
just use <MSInsert={expression}> and the result of the expression is
inserted in place of the <MSInsert> tag.

There are a number built in tags that may be used to insert dynamic
data.  These tags are: Version, ElapsedSeconds, MagicStatsURL, Date,
UpdateDate, YesterdayDate, TomorrowDate, NextMonth, and LastMonth. 
Many of the tags take a formatting parameter, which is set with
standard HTML parameter passing conventions.  An example of inserting
the date the statistics were generated:

 This page updated on <MSInsert=UpdateDate Format=TimeDate><br>

Which outputs the following HTML:

This page updated on Sat Apr 03 19:27:00 1999<br>

For descriptions of each of the tags and the parameters that may be
used to customize the output, please see the "<MSInsert> Tag" section
of the Skel File Reference (Page #7).

3.2. Using Plugins
------------------

Sometimes, simple variable and text substitutions are not enough. If
you need to insert more complex and structured data into your reports,
you must use Plugins.  A plugin is a well described extension to the
MagicStats engine.  Plugins are used to do most of the analysis and
formatting of the statistics for MagicStats.  A plugin may be either
built in or provided as an add on, in an external file.  For sake of
clarity, we will only be discussing a single type of plugin, the Page
Plugin.

3.2.1. General Plugin Properties
--------------------------------

As mentioned before, plugins are well-documented extensions to the
MagicStats engine that are used to process data. Every plugin has a
name that identifies it, as well as a set of properties that may be
set in the Skel file itself.  Each instance of a plugin operates
independently of the others, allowing a Theme author to use multiple
plugins of the same type, each with different properties set.

Multiple plugins are typically bound together into a single file and
distributed.  Plugins are compiled modules that are not cross
platform.  This means that a Solaris plugin will not work under
Windows, and visa versa.  Plugins live in the Plugins/ subdirectory of
the MagicStats output directory (typically ~/.MagicStats).

3.2.2. Plugin Syntax
--------------------

Plugins are used with the <MSPlugin> tag.  The full usage works
something like this:

<MSPlugin={PluginName} {Property}={Exp} {Property}={Exp}  >

The <MSPlugin> tag consists of the name of the Plugin to use followed
by a list of the properties that the plugin accepts.  For the Entropy
Theme, we make use of the HourlyGraph plugin to give a listing of the
number of hits to the site by the hour.  This plugin is useful for
tracking the popularity of your site vs. the time of day.  Our usage
looks like this:

<MSPlugin=HourlyGraph
  Filter=($PageFilter)

  Graph.Width=500 Graph.Height=300
  Graph.BGColor=($BGColor)
  Graph.Java.BarColor="#0000FF"
  Graph.Java.BarEndColor="#FFFF00"
  Graph.Java.BarStyle=2
  Graph.Java.BarCenter=128

  Graph.ASCII.MaxBarLength=70
  Graph.Image.MaxBarLength=500>

As you can see, this plugin takes a lot of properties, but almost all
are optional.  The properties passed to the plugin may be separated
into two classes of properties, those that affect the data the plugin
gathers and those that affect how the plugin formats the resulting
data.

3.2.3. Data Gathering Properties
--------------------------------

Plugins usually have a small set of properties that affect which data
they gather.  In the case of the HourlyGraph plugin, above, the only
property in this class is the "Filter" property.  If two plugins of
the same type all have identical Data Gathering Properties, but have
different Output Formatting Properties, they share a common instance
of the plugin specified this is one simple way that MagicStats
maintains high efficiency, while providing excellent flexibility and
extensibility.

3.2.4. Output Formatting Properties
-----------------------------------

Output Formatting Properties are the opposite of Data Gathering
Properties - they control only the silly little cosmetic details of
how the data is presented.  In the case of the HourlyGraph Plugin, the
unspecified Graph.Format property controls whether the data is
presented as a table or a graph.  The default is obviously a graph.

3.2.5. What properties does Plugin X recognize?
-----------------------------------------------

For a complete listing of the properties that each plugin recognizes,
every plugin is provided with a complete data-sheet.  These
data-sheets are available in the same directory that the plugin
resides in.  To determine where the plugin data-sheet is, type the
following command:

MagicStats2 -P | grep -i PluginName

This should print one line that describes the specific plugin,
including the name, version and location.  The location is relative to
your statistics output directory.  If for example, the output location
said "$(MS)/Plugins/PluginPack1/", try putting in Plugins/PluginPack1/
after the root of your statistics directory.  For example, here at
MagicStats.com, my location would be at:
"http://www.magicstats.com/~sabre/stats/Plugins/PluginPack1/".  If a
plugin is listed as "Built In", try looking in the Plugins/BuiltIn/
subdirectory.  If you would simply like to browse around and see what
is available, try browsing with your web browser in the Plugins/
directory.

3.3. Conditional HTML
---------------------

Some themes need to vary the output they produce depending on the
surroundings.  Perhaps the host web server does not support
referencing URLs, perhaps the theme author wants to provide two modes,
classic and fancy, perhaps the theme author is just weird.  Whatever
the case, MagicStats provides a powerful mechanism for using
Conditional HTML in themes.

Using Conditional HTML is done through the use of a few new tags. 
These tags allow you to form "if" statements in the HTML source code. 
A simple example of this would be:

<font size=+1 <MSIF ($Mode) == "Fancy"> face=Verdana </MSIF> >

If the user of the theme defined the ($Mode) variable to be "Fancy",
the following text would be in the Verdana font, instead of the
default font.  Notice that, like the other tags, Conditional HTML tags
may be nested inside of other tags (and may in fact, be nested inside
of each other).  There are actually many tags that can be used for
conditional statements.  <MSIF> you see above, but also supported is
<MSELSE> and <MSELSEIF>.  Here is a more complex example:

<MSIF ($Mode) == "Fancy"> <font face=Verdana>
<MSELSEIF ($Mode) == "Elegant"> <I>
<MSELSE>
   <MSIF ($Style) == "Monospace"> <tt></MSIF>
<MSENDIF>

Note that the HTML output by MagicStats includes no mention of the
<MS> tags.  If $Mode was "Plain" and $Style was "Monospace," the only
output of this series of statements would be "\n <tt>\n" where the
'\n' characters were newline characters.  This ensures total
compatibility with all possible web-browsers.

4. Deploying a MagicStats theme
===============================

There comes a time when your theme is finally polished and perfected
and you would like to distribute it to the world.  This section
describes exactly what you need to do to ensure that your theme is
ready for the world.

4.1. Validate your Theme
------------------------

It takes very little time to run your theme through the MagicStats
Theme Lint tool, and this little time can help you locate many
problems.

devotion:~/.MagicStats/Themes/Entropy> MagicStats2 -lint Entropy
***place output here***
devotion:~/.MagicStats/Themes/Entropy>

As you can see, a few problems slipped through the cracks, and though
these are easily fixed, it is likely that they would cause someone a
problem.  Now we are ready to move onto documentation and packaging.

4.2. Theme Documentation
------------------------

4.3. Theme Distribution and Packaging
-------------------------------------

Themes are typically distributed in slightly special form of .tar
files.  Tar files blah blah blah

5. Configuration File Reference
===============================

5.1. General Rules
------------------

5.2. Variable Export Table
--------------------------

5.3. Expressions
----------------

6. Skel File Reference
======================

Since Skel files are the primary means of laying out a theme, it is
important that all information pertinent to creating them is available
to you.  Here is the most important information laid out in a
(hopefully) organized format.

6.1. Skel Expressions
---------------------

Skel expressions are used throughout the MagicStats Skel file.  An
expression is any text that has special meaning to the MagicStats
parsing engine.  To assist in your development, MagicStats has a
number of advanced features in its language for helping you deal with
complex cases.

6.2. Simple Literal Strings
---------------------------

The simplest expression in MagicStats is a literal string.  This may
either be a bare word, such as the "Graph.Java.BarCenter=128"
statement in the Plugins section above, or it may be a quoted word
such as the "<MSIF ($Mode) == "Fancy">" example from the Conditional
HTML section.  Bare words must be extremely simple words (like the 128
and the Graph.Java.BarCenter), that may consist of only letters,
numbers, and the '.' character.

Strings quoted with either the double or single quote characters may
contain any characters that are not the quote character.  If you need
to produce the quote character in the string, you may escape it with
the '\' character.  Here is an example:

"my name is \"fred\""   == 'my name is "fred"'

As you can see, there are a number of escape characters that are
escaped to provide functionality that would be difficult to type.  A
table of the recognized escape sequences is below.

Character                          Meaning
.................................. .................................. 
\n                                 Newline
\t                                 Tab
\"                                 " Character
\'                                 ' Character
\\                                 \ Character

6.2.1. Variables
----------------

Variable references are expressions.  When you say ($Variable) in your
Skel File, you are referencing a variable expression - that whose
value is equal to the variable's value.  You will almost certainly be
using variable in your expressions, because doing stuff with static
text is no fun at all!

6.2.2. What is Truth?
---------------------

The MagicStats engine considers a logical expression True unless it is
one of the following three cases:

1.   An empty string, i.e. ""

2.   The string "false", without regards to the case of the string.

3.   The string "0"

Although it is possible to use string values and evaluate their
Truth-value, it is much more common to use these to allow users to
specify flags as either "True" or "False".  The logical operators
listed below will always return a value of "1" or "0" as their result.

6.2.3. Binary Operators
-----------------------

A number of binary operators are available for use to manipulate
strings.  Binary Operators take two parameters, one on each side. 
They perform an operation on their two parameters and return the
result.  An example of a binary operator is the '+' operator, used to
concatenate strings.  For the purpose of demonstration, all of these
examples will assume the variable $Name is set to "MagicStats."

The '+' operator (string concatenation) simply appends the right
string to the left string and returns the result.  For example,
"($Name) + " 2.0 - Best of breed analysis tool"" would return the
string "MagicStats 2.0 - Best of breed analysis tool".

The '==' and '<>' operators are used to test for case sensitive string
equality.  The '==' operator (string equals) returns '1' if the
parameters are equal, and '0' if not.  The '<>' operator (string not
equals) returns the opposite.  The '!=' operator is a synonym for the
'<>' operator.  For example, "($Name) == "MagicStats"" would return a
value of '1'.

The last two operators are logical operators.  The '||' operator
(logical OR) returns '1' if either of it's parameters is True (see the
section on Page #7 on "What is Truth?").  The '&&' operator (logical
AND) returns '1' only if both of it's parameters are True. Example:
"<MSIF ($Name) && "1">Name Defined</MSIF>" would add the string "Name
Defined" to the HTML output file if ($Name) was True.  In this case,
it is defined to "MagicStats", which is considered True.

6.2.4. Unary Operators
----------------------

So far, the only unary operator is the '!' operator (logical NOT). 
The '!' operator returns the logical inverse of an expression.  If the
expression is True, it returns '0', if the expression is False, it
returns '1'.  For example: "!($Name)" would return a value of '0',
because "MagicStats" is considered to be True.

6.3. <MSInsert> Tag
-------------------

Version, ElapsedSeconds, MagicStatsURL, Date, UpdateDate,
YesterdayDate, TomorrowDate, NextMonth, LastMonth

FORMAT Attribute parameters:

Format                             strftime equilivant
.................................. .................................. 
NUMERIC                            %Y%m%d
NUMERICMONTHLY                     %Y%m
LONG                               %A, %B %d, %Y
SHORT                              %m/%d/%Y
TIMEDATE                           %c
Arbitrary strftime                 Same as the Format

6.4. <MSPlugin> Tag
-------------------

6.4.1. Usage:
-------------

<MSPlugin={PluginName} {Property}={Exp} {Property}={Exp} >

6.5. Conditional HTML Tags
--------------------------

MagicStats enables the use of "Conditional HTML."  Conditional HTML is
used

6.5.1. Usage:
-------------

<MSIF {Exp}> {html} <MSENDIF>
<MSIF {Exp}> {html} <MSELSE> {html} <MSENDIF>
<MSIF {Exp}> {html} <MSELSEIF {Exp}> {html} <MSELSE> {html} <MSENDIF>

6.5.2. Tag Descriptions:
------------------------

<MSIF> <MSELSE> <MSENDIF> </MSIF> <MSELSEIF> <MSELSIF> <MSELIF>

7. Index
========

<MSIF> <MSInsert> <MSElse> <MSElseIf> <MSElsIf> <MSElIf> <MSEndIf>
<MSPlugin>

8. Glossary
===========

Theme, Plugin, Filter, Variable, .cfg file, Variable Export Table,
Skel Expression, Conditional HTML, Escape Character (\n and such)

