# Chapter 6

| # Advanced Data Model Realization Techniques

> This chapter includes the following sections:

- What are Filters--page 65
- Adding Filters to Data Model Realization--page 69
- Advanced Data Acquisition Mechanisms--page 78
- Other Node Types based on their Operational Behavior--page 79
- refreshQualifiers & Other Qualifiers--page 80
- Data Model Realization Specifications with Tcl procedures as DAQ--page 88
- Data Model Realization Specifications with C libraries and Tcl/TOE Command Extensions as DAQ--page 94
- Another DAQ Service--page 102
- Performance Considerations--page 103

## What are Filters

> The filter file defines data filters implemented using Tcl/TOE procedures. These filters are used to extract the pertinent information from the raw results of data acquisition commands. This enables the agent to use raw extensions or system commands (such as `df` for Solaris software) to acquire data, with the processing/parsing of the output being performed within the agent, not through external utilities such as awk or sed.

> > Note - For more information on Tcl/TOE, refer to the Appendix A.

### Standard Extensions for File Name

    <module><-subspec>-d.flt

> This file is optional for a module, and only exists if the module is using filter functions. Only Tcl/TOE procedures can be defined in this file.

### Examples of Filters

#### CPU Data Filter

> The UNIX command `vmstat 10 2` returns four lines of data in the format:

>     procs     memory            page            disk          faults      cpu
>     r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 s2   in   sy   cs us sy id
>     0 0 0  24120  7040   0  63  6  2  4  0  1  0  4  2  1  215  347  167  5 11 83
>     0 0 0 280020  1172   0   1  0  8 18  0  2  0  1  0  0   39   52   46  1  1 98

> This data is passed as the datalist argument in the `cpuFilter` procedure. The procedure parses the percent idle, system and user fields from the fourth line of data, computes the percent busy, and returns the results. The code is written in Tcl.

>     proc cpuFilter { datalist } {
>         set line [ lindex $datalist 3 ]
>         set user [ string range $line 69 71 ]
>         set system [ string range $line 72 74 ]
>         set idle [ string range $line 75 77 ]
>         set busy [ expr 100 - $idle ]
>         return "$idle $busy $system $user"
>     }

#### User Data Filter

> The UNIX command `who` returns data in the following format:

>     tom        console      Oct  2 09:54
>     tom        pts/1        Oct  4 23:43    (superior)
>     tom        pts/0        Oct  2 09:55
>     tom        pts/2        Oct  2 09:55
>     tom        pts/5        Oct  2 09:55
>     tom        pts/4        Oct  3 09:29
>     tom        pts/3        Oct  2 09:55

> This data is passed to the `userFilter` procedure as the datalist argument. The procedure loops through each line to determine the console user and count the number of unique users and sessions.

>     proc userFilter { datalist } {
>         set console none
>         set ucount 0
>         set scount 0
>         foreach line $datalist {
>             set name [ lindex $line 0 ]
>             set source [ lrange $line 5 end ]
>             if { [ lindex $line 1 ] == "console" } { set console $name }
>             if { [ catch { set users($name) } ] } {
>                 set users($name) ""
>                 incr ucount
>             }
>             if { [ catch { set sessions($name:$source) } ] } {
>                 set session($name:$source) ""
>                 incr scount
>             }
>         }
>         return [ list $console $ucount $scount ]
>     }

#### Load Data Filter

> The UNIX command `uptime` returns data in the following format:

>      11:45pm  up 4 day(s), 16:29,  2 users,  load average: 0.00, 0.00, 0.01

> This data is passed to the loadFilter procedure where the 1-, 5-, and 15-minute load averages are picked out using a regular expression pattern.

>     proc loadFilter { datalist } {
>         regexp {^.*up *(.*), +[0-9]+ +users*,.+: *([^,]+), *([^,]+), *([^,]+)}
>         [ lindex $datalist 0 ] dummy a b c d
>         return [ list $b $c $d $a ]
>     }

#### File System Data Filter

> The UNIX command `df -kF ufs` returns data in the following format:

>     Filesystem            kbytes    used   avail capacity  Mounted on
>     /dev/dsk/c0t0d0s0      22847   11912   8655    58%    /
>     /dev/dsk/c0t0d0s6     246167  185193  36364    84%    /usr
>     /dev/dsk/c0t0d0s3     105943    3183  92170     4%    /var
>     /dev/dsk/c0t0d0s7     793382       9 714043     1%    /export1
>     /dev/dsk/c0t0d0s5     288855   43131 216844    17%    /opt

> This data is passed to the f`ileFilter` procedure where the desired data fields are picked out from each line.

>     proc fileFilter { datalist } {
>         #
>         # pick out appropriate fields for each line
>         #
>         set result ""
>         set continuation ""
>         foreach line [ lrange $datalist 1 end ] {
>             set line $continuation$line
>             if { [ llength $line ] < 6 } {
>                 set continuation "$line "
>                 continue
>             } else {
>                 set continuation ""
>             }
>             regsub {%} $line "" line
>             lextract $line 1 kbytes 3 avail 4 capacity 5 mount
>             lappend result $mount $kbytes $avail $capacity
>         }
>         return $result
>     }

## Adding Filters to Data Model Realization

### Example Data Model File

> The following code example lists the Solaris Example Model file,
> `solaris-example-models-d.x`. It has three independent manged objects; CPU, system, and file system.

> CODE EXAMPLE 6-1 The `solaris-example-models-d.x` File

>     #
>     # Solaris Managed Object and Property Models
>     #
>     type = reference

>     #
>     # Cpu Managed Object
>     #
>     cpu = { [ use MANAGED-OBJECT ]

>         mediumDesc             = CPU Properties
>         consoleHint:mediumDesc = base.modules.solaris-example:cpu

>         idle = { [ use PERCENT MANAGED-PROPERTY ]
>             shortDesc       = Idle
>             mediumDesc      = CPU Idle Time
>             fullDesc        = Percentage of time the CPU is in the

>                                idle state
>             units           = %

>         }

>         busy = { [ use PERCENT MANAGED-PROPERTY ]
>             shortDesc       = Busy
>             mediumDesc      = CPU Busy Time
>             fullDesc        = Percentage of time the CPU is in the

>                                busy state
>             units           = %

>         }

>         system = { [ use PERCENT MANAGED-PROPERTY ]
>             shortDesc       = System
>             mediumDesc      = CPU System Time
>             fullDesc        = Percentage of time the CPU is running

>                                in system mode
>             units           = %

>         }

>         user = { [ use PERCENT MANAGED-PROPERTY ]
>             shortDesc       = User
>             mediumDesc      = CPU User Time
>             fullDesc        = Percentage of time the CPU is running

>                                in user mode
>             units           = %

>         }

>     }

>     #
>     # System Managed Object
>     #
>     system = { [ use MANAGED-OBJECT ]

>         mediumDesc             = System Information
>         consoleHint:mediumDesc = base.modules.solaris-example:system

>         userstats = { [ use MANAGED-PROPERTY-CLASS ]
>             mediumDesc             = User Statistics
>             consoleHint:mediumDesc = \

>              base.modules.solaris-example:system.userstats

>             consoleUser = { [ use STRING MANAGED-PROPERTY ]
>                 shortDesc     = User
>                 mediumDesc    = Console User
>                 fullDesc      = User currently logged in on the console

>                 consoleHint:mediumDesc = \

>              base.modules.solaris-example:system.userstats.consoleUser
>             }

>             numUsers = { [ use INT MANAGED-PROPERTY ]
>                 shortDesc       = #Users
>                 mediumDesc      = Number of Users
>                 fullDesc        = Number of unique users currently

>                                    logged in

>                 consoleHint:mediumDesc = \

>                  base.modules.solaris-example:system.userstats.numUsers
>             }

>             numSessions = { [ use INT MANAGED-PROPERTY ]
>                 shortDesc       = Sessions
>                 mediumDesc      = Number of User Sessions
>                 fullDesc        = Number of currently active user

>                                    sessions

>             }

>             primaryUser = { [ use STRING MANAGED-PROPERTY ]
>                 shortDesc       = User
>                 mediumDesc      = Primary System User
>                 fullDesc        = The login name of the primary user

>             }

>         }

>         load = { [ use MANAGED-PROPERTY-CLASS ]
>             mediumDesc             = Load Average
>             consoleHint:mediumDesc = \

>              base.modules.solaris-example:system.load

>             one = { [ use FLOAT MANAGED-PROPERTY ]
>                 shortDesc       = 1min
>                 mediumDesc      = 1 Min Load Avg
>                 fullDesc        = The one minute load average

>                 consoleHint:mediumDesc = \

>                  base.modules.solaris-example:system.load.one

>             }

>             five = { [ use FLOAT MANAGED-PROPERTY ]
>                 shortDesc       = 5min
>                 mediumDesc      = 5 Min Load Avg
>                 fullDesc        = The five minute load average

>                 consoleHint:mediumDesc = \

>                  base.modules.solaris-example:system.load.five
>             }

>         }
>     }

>     #
>     # Filesystem Table
>     #
>     filesystems = { [ use MANAGED-OBJECT ]
>         mediumDesc           = Filesystems

>         fileTable = { [ use MANAGED-OBJECT-TABLE ]
>             mediumDesc       = Filesystem Property Table

>             fileEntry = { [ use MANAGED-OBJECT-TABLE-ENTRY ]
>                 mediumDesc   = Filesystem
>                 index        = mount

>                 mount = { [ use STRING MANAGED-PROPERTY ]
>                     shortDesc     = Mount Pt
>                     mediumDesc    = Filesys Mount Point
>                     fullDesc      = The mount point for the filesystem

>                 }

>                 size = { [ use INT MANAGED-PROPERTY ]
>                     shortDesc      = Filesys Sz
>                     mediumDesc     = Filesystem Size
>                     fullDesc       = Total filesystem size in KBytes
>                     units          = KB

>                 }

>                 avail = { [ use INT MANAGED-PROPERTY ]
>                     shortDesc      = FilesysAvl
>                     mediumDesc     = Filesystem Space
>                     fullDesc       = Available filesys diskspace in KB
>                     units          = KB

>                 }

>                 pctUsed = { [ use PERCENT MANAGED-PROPERTY ]
>                     shortDesc      = Disk Used
>                     mediumDesc     = Filesystem Capacity
>                     fullDesc       = Percentage of Disk Space Used
>                     units          = %

>                 }
>             }
>         }
>     }

> The `solaris-example.properties` file is shown below. This file also contains the module parameter internationalization key and strings:

> CODE EXAMPLE 6-2 The `solaris-example.properties` File

>     #
>     # Module Parameters
>     #
>     moduleName=Solaris Example
>     moduleType=operatingSystem
>     moduleDesc=This is an example module monitoring cpu, load,

>     and filesystem statistics.

>     #
>     # Node Descriptions
>     #
>     cpu=CPU Properties

>     system=System Information

>     system.userstats=User Statistics
>     system.userstats.consoleUser=Console User
>     system.userstats.numUsers=Number of Users

>     system.load=Load Average
>     system.load.one=1 Min Load Avg
>     system.load.five=5 Min Load Avg

### Example Data Model Realization File Using Tcl Filters

> The following code example lists the Solaris example model realization file using Tcl filters developed in the section, "Examples of Filters".

> CODE EXAMPLE 6-3 Solaris Example Model Realization File

>     [ use MANAGED-MODULE ]
>     [ requires template solaris-example-models-d ]

>     #
>     # Load Module Parameters
>     #
>     [ load solaris-example-m.x ]

>     #
>     # Define services required by this module
>     #
>     _services = { [ use SERVICE ]
>         #
>         # Standard Bourne Shell
>         #
>         sh = {
>             command = "pipe://localhost//bin/sh;transport=shell"
>             max     = 2
>         }
>     }

>     #
>     # Load filters required by this module
>     #
>     _filters = { [ use PROC ]
>         [ source solaris-example-d.flt ]
>     }

>     #
>     # Cpu Information uses the cpuFilter which is already discussed
>     # in the previous section
>     #
>     cpu = { [ use templates.solaris-example-models-d.cpu _filters ]
>         type                = active
>         refreshService      = _services.sh
>         refreshCommand      = vmstat 10 2
>         refreshFilter       = cpuFilter
>         refreshInterval     = 60
>     }

>     #
>     # System User and Load Information uses the userFilter
>     # which is already discussed
>     # in the previous section
>     #
>     system = { [ use templates.solaris-example-models-d.system ]
>         userstats = { [ use _filters ]
>             type                = active
>             refreshService      = _services.sh
>             refreshCommand      = who
>             refreshFilter       = userFilter
>             refreshInterval     = 120

>             primaryUser = {
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshCommand  = solaris-example-primary-user-d.sh
>                 refreshInterval = 86400
>             }
>         }

>         load = {
>             one = {
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshCommand  = echo 10.2
>                 refreshInterval     = 60
>             }
>             five = {
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshCommand  = echo 10.2
>                 refreshInterval     = 60
>             }
>         }

>     }

>     #
>     # Filesystem Information uses the fileFilter
>     # which is already discussed
>     # in the previous section
>     #
>     filesystems = { [use templates.solaris-example-models-
>     d.filesystems
>                       _filters]
>         type                = active
>         refreshService      = _services.sh
>         refreshCommand      = df -kF ufs
>         refreshFilter       = fileFilter
>         refreshInterval     = 120
>     }

> The Solaris Example Primary `user-d.sh` file is shown below:

> CODE EXAMPLE 6-4 The solaris-example-primary-user-d.sh File

>     #!/bin/sh

>     echo "I am a primary user (from Sh)"

> > Note - All the filters used, for example, CPUfilter, and so on, in the example above, are defined in the file `solaris-example-d.flt`.

### Loading the DAQ Services

#### Tcl Filters

> If the DAQ was implemented using Tcl filters, the Filters file must be loaded into a container object. Nodes that want to call a procedure defined in the Filters file must inherit this object.

> The _services.<shell> object can then be used by other objects for Tcl shell DAQ services.

### RefreshQualifier for Filters

#### refreshFilter

> Refresh filters can be specified in active and derived nodes:

>     refreshFilter = <Tcl command or procedure>

> The refreshFilter qualifier specifies a Tcl command or procedure that is used to process the data acquired by the refresh command. The Tcl procedure must take a single argument that is the result returned by the refresh command. The result of the refresh filter is cascaded into the managed properties. Recall that if the refresh command is implemented as a UNIX command or shell script, any data returned on stderr constitutes a data acquisition error. As a result, no data is passed to the refresh filter, regardless if data was returned on stdout too.

#### Solaris Example--Loading the Filter File

> In the following example, the Solaris Example Filter file is loaded into the filters object. This object can be inherited by other objects that want to use the procedures defined in the Filter file. One such object is the CPU managed object.

> The following is a code example of loading the filter file:

> CODE EXAMPLE 6-5 Loading the Filter File

>     _filters = { [ use PROC ]
>         [ source solaris-example-d.flt ]
>     }
>     cpu = { [ use templates.solaris-example-models-d.cpu _filters ]
>     ... }

## Advanced Data Acquisition Mechanisms

> These are other techniques and are discussed in detail later on in this chapter.

#### Tcl/TOE Code

> Standard Tcl/TOE commands, such as `file` to get file statistics, can be used to acquire data. In addition, Tcl procedures can be written to filter raw results returned by UNIX programs. Tcl provides many useful commands for parsing strings (`regexp,` `regsub,` and so forth.).

> In general, Tcl procedures are preferred over using shell scripts when filtering data as they are typically easier to implement and more efficient. When returning data to the agent from Tcl procedures or commands, the data elements must be elements in a Tcl list.

#### C Code Libraries and Tcl/TOE Command Extensions

> C-code libraries and Tcl/TOE command extensions can also be written to perform DAQ. This is accomplished by packaging the DAQ functions as shared object libraries that can be dynamically loaded by the Sun Management Center agent.

> When returning more than one item of data from Tcl commands, the data elements must be elements in a Tcl list.

> The next few sections introduce concepts necessary to understand advanced data model realization techniques.

## Other Node Types based on their Operational Behavior

### Passive Nodes

> Nodes that do not actively collect data but instead have data cascaded into them are known as passive nodes. By default, all nodes are passive, unless otherwise specified using the type qualifier.

> A node can be explicitly specified to be passive using the following specification:

> type = passive

> Passive managed property nodes can specify an update filter to process that data being cascade into it.

>     updateFilter = <Tcl procedure or command>

> If no update filter or other qualifier is required, it is unnecessary to explicitly declare a passive node in the agent file at all. For such nodes, it is sufficient to model them in the model file only. For more details on `updateFilter`, refer to the section, `"updateFilter"`.

### Derived Nodes

> A node is specified to be a derived node using the following specification:

>     type = derived

> Derived nodes establish dependency relationships with the nodes on which they rely through the use of the refresh triggers specification. Nodes can be triggered by the change in value or status of another node, and refresh automatically when such an event occurs. Derived nodes can also refresh at an interval, although this is usually unnecessary if the triggers are specified properly.

> A derived node can use other MIB nodes as the service(s) for its refresh. In other words, its value is often a function of the values or qualifiers of one or more other managed properties. Through the use of derived variables, it is possible to create nodes whose value represents averages, rates of change, specific digital filters (for example: high pass, low pass, or band pass or other useful calculated information).

> All derived nodes must specify the following refresh parameters:

>     refreshService = <service object>
>     refreshCommand = <command to run in the context of refreshService>
>     refreshTrigger = <node name>[:<event>] [<node name>[:<event>] ...]

> The `refreshCommand` could also be Tcl commands and procedures.

> > Note - The refresh interval is optional for a derived node. Derived nodes may be forced to update at periodic intervals, although this is usually unnecessary if the refresh triggers are speficied properly. For more information on refreshTrigger, refer to the next section.

## refreshQualifiers & Other Qualifiers

> The following refresh qualifiers can also be specified in active and derived nodes.

### timeoutInterval

> The timeout interval can be specified for active and derived nodes.

>     timeoutInterval = <timex specification>

> If the refresh command does not complete within the specified timeoutInterval, the command is aborted. In that case, the alarm state of the node is marked indeterminate (unknown value). The default is no timeout. This can be used in conjunction with `refreshMode = sync` (described later) to ensure that the agent does not hang while collecting data.

### refreshTrigger

> Refresh triggers must be specified in derived nodes.

>     refreshTrigger = <node name>[:<event>] [<node name>[:<event>] ...]

> Derived nodes establish dependency relationships with one or more nodes on which they rely through the use of the refresh triggers specification. Nodes can be triggered off the change in value or status of another node, and refreshes automatically when such an event occurs.

> The refreshTrigger specifies the name of the node that the derived node depends on. The possible events are:

> status | event generated upon status change of the node
> refresh | event generated when an active node is to refresh its value
> update | event generated in managed properties when the data values are updated
> set | event generated in the node when a SNMP set is made

> If no event is specified, the occurrence of any of the above events in the specified node will trigger the execution of the refresh operation. If an event is specified, only the occurrence of the specified event on the specified node triggers the execution of the refresh operation.

#### Specifying Node Name

> <node name> can be specified two ways. If the triggering node is a direct child of the current node, or is the direct child of a superior of the current node, the object name can be used directly. For example, in the Solaris Example Module, the CPU managed object has several child managed properties, including: idle, user, system, busy, and average (derived). Any of these refresh triggers are valid for the derived average node:

> `refreshTrigger` | `busy:update`
> `refreshTrigger` | `idle:refresh`
> `refreshTrigger` | `system:status`

> If the triggering node does not meet the above criteria, the full name of the triggering node must be specified. Wildcards are allowed. It is important to take care that the name is uniquely specified. Otherwise, the first node matching the name becomes the trigger.

> The average node could trigger off the 15 minutes load average as follows:

>     refreshTrigger = *solaris*load.fifteen

> Wildcards can be used as placeholders only for full node names.You cannot use wildcards to partially specify a node name. For example, the following is not valid, because the node name solaris is not fully specified:

>     refreshTrigger = *sol*load.fifteen

#### Specifying RefreshTriggers from a Node in Another Module

> The triggering node must not reside in a different module from the current node. Otherwise, if the other module is unloaded, the triggering relationship is lost, and both modules must be reloaded to restore the relationship.

> > Note - This cannot be used for active nodes.

### refreshParams

> Refresh params can be specified in active and derived nodes:

>     refreshParams = <params>

> The refreshParams qualifier is used to specify arguments to be passed to the refresh command. If the refresh mode is set to `multi`, `multilist`, or `multiecm`, and the refresh params specifies a space separated list of arguments, the refresh command is executed once for each argument. The next section describes refreshMode.

### refreshMode

> The refresh mode can be specified in active and derived nodes:

>     refreshMode = async | sync

> The refreshMode qualifier specifies the execution mode of the refresh command.

#### async

> By default, active nodes have a refresh mode of async. This specification implies that the refresh operation is asynchronous. That is, the agent is allowed to process other events during the execution of the refresh command.

#### sync

> If the refresh command must return immediately with the result, setting the refreshMode to sync reduces the overhead associated with an asynchronous command.

### initInterval

> The initialization interval can be specified in active and derived nodes.

>     initInterval = <timex specification>

> The initInterval specifies, the time window within which the node must run the refreshCommand for the first time after the module initializes.

> > Note - initInterval does not specify an exact time at which the node will first issue its refresh command. Rather, this interval specifies a time range. Sometime within that time range, the first refresh command is issued. The exact time is randomized by the agent to spread out load.

> If initInterval is specified as `-1`, the first execution of the refreshCommand is executed as specified by the refreshInterval.

> The initialization interval enables module designers to control the rate at which the module nodes begin their data acquisition operations. This might be done to prioritize the order in which nodes initialize, or to avoid large spikes in data acquisition activity during module start up.

### initHoldoff

> The initialization hold off can be specified in active and derived nodes:

>     initHoldoff = <timex specification>

> The initHoldoff qualifier specifies the time, in Timex specification, to wait before running the refresh command the first time.

> The initialization hold off is typically used to delay the execution of a specific node that depends on the value of another node that must execute first. Effectively, the initHoldoff time is added to the initInterval time, thereby delaying initialization. initHoldoff must never be set to less than 2 seconds.

> Here are some examples of how initHoldoff and initInterval interact:

>     initInterval = 10
>     initHoldoff = 2

> The node will initialize sometime between 2 and 12 seconds.

>     initInterval = 10
>     initHoldoff = 30

> The node will initialize sometime between 30 and 40 seconds.

### Check Qualifiers

> checkCommand, checkService and checkInterval

> The checkCommand, checkService, and checkInterval can be specified for active and derived nodes to perform check operations.

>     checkService = <service specification>
>     checkCommand = <command to execute in service context>
>     checkInterval = <timex specification>

> The check operation provides a mechanism for triggering refresh operations based on some criteria tested by the check operation. Typically, the check operation is a lighter weight operation and is performed at a higher frequency relative to the refresh operation. This mechanism enables managed objects to be monitored in a more timely fashion without the performance penalty of executing the refresh operation at a higher frequency.

> The check operation triggers the execution of the refresh operation only when the value returned by the check command differs from the value from the previous check.

> For example, when monitoring the contents of a file, the refresh operation can involve reading the contents of the file at every refresh interval. To monitor the file more effectively, the last modification date of the file can be checked at a higher frequency to determine if the file has changed. If the check detects that the file has changed, the refresh operation is triggered. Hence, the check mechanism provides an efficient way to monitor the file since checking the last modification date of a file (using stat system call) is much more lightweight than having to open, read, and close the file.

> checkService specifies the service used to run the checkCommand. If this qualifier is not specified, the checkCommand shall use the refreshService service.
> checkInterval specifies the interval, in `timex` specification, at which to run the checkCommand. This checkInterval must be specified if the checkCommand is specified.
> checkInterval and refreshInterval operate completely independently of each other. refreshInterval specifies the interval at which refreshes will definitely occur. checkInterval specifies the interval at which refreshes can occur, depending on the result of the check condition.

> For example:

>     checkInterval = 10
>     refreshInterval = 300

> This means that every 10 seconds checkCommand is executed. If the check passes, the refresh command is invoked. But regardless of those checks, every 5 minutes the refresh command is invoked.

### updateFilter

> Passive managed property nodes can specify an update filter to process the data being cascaded into it.

>     updateFilter = <Tcl command or procedure>

> The update filter specifies a Tcl command or procedure to process the data being cascaded into the passive node. Like the refreshFilter, the Tcl procedure specified by the updateFilter takes a single argument that is the data that is cascaded into the passive node.

> To execute a user-defined Tcl procedure, the procedure must be available in the current node's context. To execute a user-defined Tcl command extension, the appropriate Tcl package must be loaded as described in the earlier section Loading the DAQ Services.

### refreshService

> Besides the `refreshService` command discussed in the previous chapter, the other `refresh` services are:

- SNMP stack (for doing data acquisition from other agents)
- Internal service that enables access to built-in or dynamically-loaded extensions to the agent process
- Another node in the MIB tree (from which you can acquire data)

#### SNMP Service

> `refreshService = .services.snmp`

> Specify the SNMP service when the refresh command is an SNMP get request for acquiring data from another SNMP agent. The SNMP service object is created by all Sun Management Center agents for general SNMP communications.

#### Internal Service

> `refreshService = _internal`

> Specify the internal service when the refresh command is a Tcl/TOE command or procedure to be executed in the current node's context.

> To execute user-defined Tcl procedures, the procedures must be available in the current node's context.

> To execute user-defined Tcl command extensions, the appropriate Tcl package must be loaded as described in the earlier section"Loading the DAQ Services".

#### Superior Service

> `refreshService = _superior`

> Specify the superior service when the refresh command is a Tcl/TOE command or procedure to be executed in the context of the current node's superior in the tree hierarchy.

> For example, in the Solaris Example module, the CPU managed object node contains a managed property node called "`busy`." In that example, the CPU node is the superior of the busy node.

> To execute user-defined Tcl procedures, the procedures must be available in the superior's context.

> To execute user-defined Tcl command extensions, the appropriate Tcl package must be loaded as described in the earlier section Loading the DAQ Services.

#### MIB Node Service

> `refreshService = <node name>`

> Use this type of service when the refresh command is to be executed in the context of another MIB node. The refresh command specified must be available in the context of the specified MIB node.

> This specification is often used for derived nodes that specify the MIB node whose value the derived node depends on.

> For example, in the Solaris Example Module, the CPU managed object has several child managed properties, including: idle, user, system, busy, and average (derived). So, the refresh service for the derived average node is set to one of these other children, such as:

> `refreshService = busy`

> For information on what are valid <node name> specifications, refer to the description of refreshTrigger in the section on "refreshTrigger".

> > Note - The specified node could reside in an entirely different module. However, this type of interdependency between modules is not suggested, since the modules can be loaded or unloaded independently.

## Data Model Realization Specifications with Tcl procedures as DAQ

### Example Data Model File

> The following code example lists the additional specifications needed in the Solaris Example Model file:

> CODE EXAMPLE 6-6 Solaris Example Model File

>     # Additional managed property average needs to be managed and
>     # is to be added to the managed object cpu
>     # in the models file of solaris-example from the previous section o
>     # on the Example Data Model File.
>     cpu = { [ use MANAGED-OBJECT ]
>            .....
>            ....
>         average = { [ use PERCENTHI MANAGED-PROPERTY ]
>             shortDesc       = AvgCPU
>             mediumDesc      = Average CPU Usage
>             fullDesc        = Average percentage of time CPU is busy
>             units           = %

>         }

>     }

> The following is an example of the Data Model Realization file using Tcl procedures as DAQ:

> CODE EXAMPLE 6-7 Solaris Example Model Realization File

>     [ use MANAGED-MODULE ]
>     [ requires template solaris-example-models-d ]

>     #
>     # Load Module Parameters
>     #
>     [ load solaris-example-m.x ]

>     #
>     # Define services required by this module
>     #
>     _services = { [ use SERVICE ]
>         #
>         # Standard Bourne Shell
>         #
>         sh = {
>             command = "pipe://localhost//bin/sh;transport=shell"
>             max     = 2
>         }
>     }

>     #
>     # Load filters required by this module
>     #
>     _filters = { [ use PROC ]
>         [ source solaris-example-d.flt ]
>     }

>     _procedures = {
>             [ use PROC ]
>             [ source solaris-example-system.prc ]
>             [ source solars-example-average-d.prc]

>     }

>     #
>     # Cpu Information
>     #
>     cpu = { [ use templates.solaris-example-models-d.cpu _filters ]
>         type                = active
>         refreshService      = _services.sh
>         refreshCommand      = vmstat 10 2
>         refreshFilter       = cpuFilter
>         refreshInterval     = 60

>         average = { [ use _procedures ]
>             type            = derived
>             refreshService  = _internal
>             refreshTrigger  = busy:update
>             refreshCommand  = getAverage

>         }
>     }

>     #
>     # System User and Load Information
>     #
>     system = { [ use templates.solaris-example-models-d.system }
>         userstats = { [ use _filters ]
>             type                = active
>             refreshService      = _services.sh
>             refreshCommand      = who
>             refreshFilter       = userFilter
>             refreshInterval     = 120

>             numUsers{ [ use _procedures ]
>             refreshService = _internal
>             updateFilter = numUsersFilter

>             primaryUser = {  [ use _procedures]
>                 type            = active
>                 refreshService  = _internal
>                 refreshCommand  = getPrimaryUser
>                 initInterval    = 10
>                 refreshInterval = 86400
>             }
>         }

>         load = {
>             one = {
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshMode = sync
>                 refreshCommand  = echo 10.2
>                 initInterval = 20
>                 refreshInterval     = 60

>             }
>             five = {  [ use _procedures ]
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshCommand  = echo 10.2
>                 initInterval = 60
>                 refreshInterval     = 60
>                 checkService = _internal
>                 checkCommand = testFive
>                 checkInterval = 10
>             }
>         }

>     }

>     #
>     # Filesystem Information
>     #
>     filesystems = { [use templates.solaris-example-models\
>     d.filesystems _filters]
>         type                = active
>         refreshService      = _services.sh
>         refreshCommand      = df -kF ufs
>         refreshFilter       = fileFilter
>         refreshInterval     = 120
>     }

> The following is an example of the corresponding `solaris-example-system.prc` File:

> CODE EXAMPLE 6-8 The `solaris-example-system.prc` File

>     # This is called by the primaryUser node
>     proc getPrimaryUser{}{
>      set primaryUser "Hello"
>      return $primaryUser
>     }

>     # This is the updateFilter for numUsers
>     proc numUsersFilter{ index value } {
>         if { $value > 0 } {
>             return $value
>         } else {
>             return 1
>         }

>     }

>     # This is the checkCommand
>     proc testFive{}{
>        set result 10.2
>        return $result
>     }

> The following is an example of the corresponding solaris-example-average-d.flt File:

> CODE EXAMPLE 6-9 The `solaris-example-average-d.flt` File

>     # The corresponding solaris-example-average-d.flt file
>     # finds the value of the other nodes and finds their average.

>     proc getAverage{} {
>        set busy [ toe_send [ locate busy ] getValue ]
>        set idle [ toe_send [ locate idle ] getValue ]
>        set user [ toe_send [ locate user] getValue ]
>        set system [ toe_send [ locate system] getValue ]
>        set average [ expr  ( $busy + $idle + $user + $system )/4 ]
>     }

### Standard Extension for File Name

    <module><-subspec>-d.prc

> Objects in the MIB tree can perform special data acquisition functions or alarm status actions. These specialized functions can be specified as Tcl procedures and placed in a module-specific procedures file. This provides a simple mechanism to override or extend the functionality of the core MIB object primitives.

> This file is optional for a module, and only exists if the module is using procedures. Only Tcl/TOE procedures can be defined in this file.

### Loading the DAQ Services

> The following section describes the Tcl procedures and node types based on the operational behavior to be used and refresh qualifiers.

#### Tcl Procedures

> If the DAQ was implemented using Tcl procedures, the Procedures file must be loaded into a container object. Nodes that need to call a procedure defined in the Procedure file must then inherit this object.

>     _procedures = { [ use PROC ]
>          [ source <modules><-subspec>-d.prc ]
>     }
>     <node> = { [ use <PRIMITIVE> _procedures ] ... }

### Node Type Based on Operational Behavior

> Node types can be active, derived or passive.

### Refresh Qualifiers

> All the refresh qualifiers discussed earlier are applicable. Also, the refreshService will be `_internal`.

## Data Model Realization Specifications with C libraries and Tcl/TOE Command Extensions as DAQ

### Solaris Example Data Model Realization File

> This is an example of the migration of DAQ functionality to Tcl command extensions. The discussion of this example follows in the next few sections.

> CODE EXAMPLE 6-10 Agent File Modifications

>     [ requires package ssi ]
>     [ requires template solaris-example-models-d ]
>     [ use MANAGED-MODULE ]
>     [ load solaris-example-m.x ]

>     _services = { [ use SERVICE ]
>         sh = {
>             command = "pipe://localhost//bin/sh;transport=shell"
>             max             = 2
>         }
>     }
>     #
>     # default the refresh service for the entire module
>     #
>     refreshService      = _internal

>     cpu = { [ use templates.solaris-example-models-d.cpu ]
>         type                = active
>         refreshMode         = sync
>         refreshCommand      = ssinfo cpu
>         refreshInterval     = 60

>         average = {
>             type            = derived
>             refreshTrigger  = busy:update
>             refreshCommand  = digitalFilter [ valueOf busy.0 ]
>             refreshParams   = 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
>         }
>     }
>     system = { [ use templates.solaris-example-models-d.system ]
>         userstats = { [ use _filters ]
>             type                        = active
>             refreshCommand              = ssinfo user
>             refreshInterval             = 120

>             primaryUser = {
>                 type            = active
>                 refreshService  = _services.sh
>                 refreshCommand  = solaris-example-primary-user-d.sh
>                 initInterval    = 10
>                 refreshInterval = 86400
>             }
>         }
>      load = { [ use _filters ]
>             type                        = active
>             refreshCommand              = ssinfo load
>             refreshInterval             = 120
>         }
>     }

>     filesystems = { [use templates.solaris-example-models-
>     d.filesystem ]
>         type                = active
>         refreshMode         = sync
>         refreshCommand      = ssinfo file
>         refreshInterval     = 120
>     }

>     [ load solaris-example-d.def ]

> Shown below are code fragments from the `ssi` package files.

> CODE EXAMPLE 6-11 Code Fragments From `ssi` Package File

>     packages.h
>     .
>     .
>     .
>     #define PKG_SSI  "ssi"
>     extern int Ssi_Init();
>     .
>     .
>     .
>     pkgssi.c
>     .
>     .
>     .
>     int
>     Ssi_Init(interp)
>         Tcl_Interp *interp;
>     {
>         int code;

>         code = Tcl_PkgProvide(interp, PKG_SSI, "1.0");
>         if (code != Tcl_OK) {
>             return code;
>         }
>         /* --- create "ssinfo" command --- */
>         Tcl_CreateCommand(interp, "ssinfo", cmdSsinfo, (ClientData)
>     "ssinfo",
>                           (Tcl_CmdDeleteProc *) NULL);
>         return(Tcl_OK);
>     }

>     int
>     cmdSsinfo(dummy, interp, argc, argv)
>         ClientData dummy;                   /* Not used. */
>         Tcl_Interp *interp;                 /* Current interpreter. */
>         int argc;                           /* Number of arguments. */
>         char **argv;                        /* Argument strings. */
>     {
>     .
>     .
>     .
>     switch(*argv[1]) {
>             case "l":
>             case "L":
>                 if(!strcasecmp(argv[1], "load")) {
>                     float one, five, fifteen;
>             	     code = ssiGetLoadAverage(&one,&five,&fifteen);
>             	       if (code == 0) {
>      sprintf(buf, "%3.2f %3.2f %3.2f", one, five, fifteen);
>                         Tcl_AppendResult(interp, buf, (char *)NULL);
>                     }
>                     else {
>                         ssi_error = 1;
>                     }
>                        found_option = 1;
>                 }
>                 break;
>     .
>     .
>     .
>         return(status);
>     }

> Shown below are code fragments of the DAQ C library code:

> CODE EXAMPLE 6-12 DAQ C code

>     siSolaris.c
>     A code fragment from the ssi code is shown below.
>     .
>     .
>     .
>     int ssiGetLoadAverage(float *pfOneMin, float *pfFiveMin, float
>     *pfFifteenMin)
>     {
>         long    laAveNRun[ 3 ];
>         int             iSize;

>         if ( !bSSIInit )
>         {
>             return ( ssiAPI_not_init );
>         }

>         if ( !( NlistArray[ X_AVENRUN ].n_value ) )
>         {
>              int iEntry = X_AVENRUN;
>              if ( initKvmEntries ( &iEntry , 1 ) != 0 )
>              {
>                  return ( ssiAPI_kvm_nlist_failed );
>              }
>         }

>         iSize = sizeof ( laAveNRun );
>         if ( kvm_read(pKD, NlistArray[ X_AVENRUN ].n_value,
>                        (char *)(laAveNRun), iSize) != iSize )
>         {
>                 return ( ssiAPI_loadavg_failed );
>         }

>         *pfOneMin  = (float)loaddouble ( laAveNRun[ 0 ] );
>         *pfFiveMin = (float)loaddouble ( laAveNRun[ 1 ] );
>         *pfFifteenMin  = (float)loaddouble ( laAveNRun[ 2 ] );

>         return ( 0 );
>     } /* end ssiGetLoadAverage () */
>     .
>     .
>     .

### Solaris Example - Tcl Command Extension

> The migration of functionality from scripts to C involves the following steps:

- Writing a C library to perform the functions performed by the scripts
- Writing a Tcl extension to integrate the library functions
- Modifying the agent definition file to make use of the new Tcl extensions

> > Note - The initialization procedure always returns an integer with the value TCL_OK. See CODE EXAMPLE 6-11 for more information.

### Writing a C Library

> Although the C-code data acquisition functions can be put directly in the Tcl extension, placing the functions in a generic library enables other C programs to use the same functions, which improves code coverage and increases the reliability of the code. For example, the following is a C subroutine used to retrieve the system load average:

> CODE EXAMPLE 6-13 Code Fragment Used to Retrieve System Load Average

>     int ssiGetLoadAverage(float *pfOneMin, float *pfFiveMin, float *pfFifteenMin)
>     {
>         long    laAveNRun[ 3];
>         int             iSize;

>         if ( !bSSIInit )
>         {
>             return ( ssiAPI_not_init );
>         }

>         if ( !( NlistArray[ X_AVENRUN ].n_value ) )
>         {
>              int iEntry = X_AVENRUN;
>              if ( initKvmEntries ( &iEntry , 1 ) != 0 )
>              {
>                  return ( ssiAPI_kvm_nlist_failed );
>              }
>         }
>     iSize = sizeof ( laAveNRun );
>         if ( kvm_read(pKD, NlistArray[ X_AVENRUN ].n_value,
>                        (char *)(laAveNRun), iSize) != iSize )
>         {
>                 return ( ssiAPI_loadavg_failed );
>         }

>         *pfOneMin  = (float)loaddouble ( laAveNRun[ 0 ] );
>         *pfFiveMin = (float)loaddouble ( laAveNRun[ 1 ] );
>         *pfFifteenMin  = (float)loaddouble ( laAveNRun[ 2 ] );
>         return ( 0 );
>     } /* end ssiGetLoadAverage () */

> This function can then be combined with other functions that determine system information to create a library of functions that access the kernel and determine system specific information.

### Writing a Tcl Extension

> To use C library functions in Tcl, Tcl extension files, referred to as `packages`, must be created. These packages define an initialization procedure that enables Tcl commands to run C code. When creating packages, consider the following issues:

- Package naming
- Writing the initialization procedure
- Returning data to Tcl

> These topics are described in the following sections. Refer to the Tcl documentation for information about Tcl functions (Tcl_*).

#### Package Naming

> To enable a single Tcl application to incorporate many different packages without experiencing naming conflicts, a naming convention must be followed. This convention specifies a short, unique prefix for each package. For example, a package that uses the system- specific C library functions may use a prefix of 'ssi' (system specific interface). This prefix is then used to name the initialization function (described in the next section) and the package shared object file (`pkgssi.so`).

#### Init Function

> Each package must include an initialization function. This function is called when the package is loaded. The name of the initialization procedure must contain the package prefix with the first letter capitalized followed by `_Init`. For example the initialization function for the `ssi` package would be named `Ssi_Init`.

#### Package Registration

> The initialization function is used for package and command registration.

> Package registration ensures that no other versions of the same package is being used currently or will be loaded later. Package registration is done using the `Tcl_PkgProvide` command. For example, the registration of the `ssi` package version 1.0 is:

>     code = Tcl_PkgProvide(interp, "ssi", "1.0");
>     if (code != Tcl_OK) {
>        return code;
>     }

#### Command Registration

> Commands provided by the package are registered in the Tcl interpreter using the Tcl_CreateCommand function. There must be one call to Tcl_CreateCommand for each Tcl command created. For example:

>     Tcl_CreateCommand(interp, "ssinfo", cmdSsinfo, (ClientData)
>     "ssinfo",
>                           (Tcl_CmdDeleteProc *) NULL);

> In this example, a single Tcl command (ssinfo) is created to access all the ssi library functions. The Tcl_CreateCommand function creates a link between the Tcl ssinfo command and the C cmdSsinfo function. From the `cmdSsinfo` function, the appropriate `ssi` function is called based on the ssinfo command arguments. In this example, a design decision was made to create a single Tcl command to access all the `ssi` library functions. The other possibility is to create a Tcl command for each possible `ssi` library function. It is up to the developer to decide which is better.

#### Returning Data into Tcl

> After the C library function has been called to acquire data, the data must be returned to Tcl interpreter. This is done by using the following Tcl commands:

- `Tcl_AppendElement`
- `Tcl_AppendResult`

> For example to return the data from the `ssiGetLoadAverage` function call, the function Tcl_AppendResult is used:

>     code = ssiGetLoadAverage(&one,&five,&fifteen);
>     if (code == 0) {
>         sprintf(buf, "%3.2f %3.2f %3.2f", one, five, fifteen);
>         Tcl_AppendResult(interp, buf, (char *)NULL);
>     }

> > Note - Data must be returned to the Tcl interpreter as a string.

### Loading the DAQ Services

#### Tcl Command Extension Packages

> If DAQ is implement using a Tcl command extension package, the package shared object must be loaded using the following directive:

>     [ requires packages <package name> ]

> This loads the package so that the Tcl commands provided by the package is available in the agent context. For example, to load a package named `pkgssi.so`, the command is:

>     [ requires packages ssi ]

### Node Type Based on Operational Behavior

> These can be active, derived or passive.

### Refresh Qualifiers

> To use the Tcl commands from the package, the refreshCommand must be modified to call the appropriate package function. In addition, the refreshService must be set to _internal to denote that the refreshCommand is to be run in the agent context. Finally, the refreshMode can be set to sync to optimize the function call. Other refreshQualifiers will remain the same.

## Another DAQ Service

### Tcl Shell Service

|  | To enable the agent to execute Tcl command extensions in a separate subshell process:

| 1. | Create binary extensions in the form of a Tcl package.

|  | This is described in the "C Code Libraries and Tcl/TOE Command Extensions".

| 2. | Create a simple Tcl wrapper script to load the packages required by the subshell.

|  | By convention, Tcl wrapper scripts are named with a `.Tcl` extension and use the pkgload command to load the required Tcl packages.

| 3. | Add a Tcl shell service object in the agent file.

|  | The Tcl shell service object is an object maintaining a pipe to one or more Tcl subshell processes where commands can be directed and the results returned asynchronously.

> The module configuration file specification for a Tcl shell service object is:

>      _services = { [ use SERVICE ]
>             <shell> = {
>     command = "pipe://localhost//<Tcl shell name>;transport=shell"
>                 max = <max shells>
>             }
>         }

> where:

> <shell> is the name of the Tcl shell.
> <Tcl shell name> is the name of the Tcl script containing the pkgload commands.
> <max shells> specifies the maximum number of shell subprocess to spawn. This is typically set to 1 or 2.

> The _services.solarisShell object can be used by other objects for Tcl shell DAQ services.

#### Solaris Example--Tcl Shell

> The _services.solarisShell can be used to collect data asynchronously in a separate sub-shell.

## Performance Considerations

> Some data acquisition mechanisms are more efficient than others. In general, C-code libraries and Tcl command extensions are much more efficient than UNIX commands and shell scripts.

> For example, CPU usage statistics can be computed using a shell script that executes the UNIX command (`vmstat`) and parses the result using (`awk`). Similarly, these CPU statistics can also be computed by running `vmsta`t directly and parsing the results using a Tcl procedure. Using a Tcl procedure to parse data is slightly more efficient than using UNIX filter commands like `sed` and `awk`.

> Alternatively, this information can be much more efficiently computed using a Tcl command extension and C system calls that do not include the overhead of creating processes and parsing data in a shell script or in Tcl.

> > Note - Because the Sun Management Center agent is single threaded when running Tcl commands, it is assumed that all Tcl commands return their results with little delay. If it is expected that the Tcl command will take a significant amount of time to return its result, a Tcl shell that loads the appropriate Tcl package can be spawned as a subprocess of the agent. The Tcl command can then be directed to the subprocess and the result can be returned asynchronously to ensure that the agent is not blocked by the execution of the command.

Copyright © 2000 Sun Microsystems, Inc. All Rights Reserved.