Monitoring the contents of log files

Log file monitoring is an option available in the performance monitoring rules. This monitoring agent scans log files on managed Windows devices for specific strings or expressions, and generates alerts when they are found. This is useful if you want to be alerted when a particular condition exists that can be traced through a log file.

You can monitor a text file generated by any application, including .htm and .xml files (however, Unicode files can’t be monitored). After you specify which file to monitor and define rules using regular expressions, the file will be monitored as long as the log file monitoring rule is contained in a ruleset that is in effect on that device.

The first time text in the log file matches a regular expression, an alert is generated. The alert is generated only once for that file even if there are multiple matches. Later, if the file changes so there is no longer a matching condition, then the agent begins scanning for that regular expression again and will generate an alert on the next occurrence of the match.

You can also scan log backup files that are created when a log file becomes too large and older entries in the file are appended to a different file (a “rolling” log file). However, "wrapped” log files, which remove older entries within a single log file to make room for new entries, are not supported.

For this monitoring option, you must specify the location and exact name of the file on the managed device, and you specify the search criteria with a regular expression. When a string in the file matches the expression, an alert action is generated if you have defined a Log file monitoring alert type in the appropriate alerting ruleset.

You can include log file monitoring in any alerting ruleset you have defined. The following procedure describes the five general steps for setting up log file monitoring:

Create a log file monitoring rule in an alert ruleset.
Specify which log file to monitor on the managed devices.
Define the monitoring rules for that file, using regular expressions.
Select a severity level for the rule and name the instance so it will be identified in alerts.
Apply action and time rules and save the rule in the alert ruleset.

To set up a log file monitoring rule

In the core server console, click Tools > Configuration > Agent Settings > Alerting.
Under Alert rulesets, select the ruleset you want to edit, then click Edit on the toolbar.
In the left column of the Alert ruleset window that opens, click Alerts. Under the Monitor folder in the list of alerts, click Log file monitoring.
On the toolbar, click New.
In the Log file monitoring dialog box, type a name and description for the log file monitoring rule.
To change the frequency at which the item is monitored, change the Polling interval settings.
Click Log file configuration to specify which log files are monitored, what you are monitoring for, and how you will be alerted.
Regular expressions are used to define what content in the log file should be monitored. When the monitoring service finds a match for the regular expression in the log file, it follows the alert rules to notify you of the occurrence.
Click Manage. In the Regular expression management dialog box, add a descriptive name and a regular expression, then click Add. Repeat for each regular expression you want to use for monitoring log files. When you have added them all, click OK.
You can add as many regular expressions as you want in this dialog box. Note that you need to create a new rule for each expression that you want to search for, and each rule is applied to only one log file. In other words, each rule includes one regular expression and one log file.
Select a regular expression in the Regular expression drop-down list.
Enter the path and complete filename of the log file you want to monitor in the Log file path box. This must be a specific filename, and only that filename will be monitored (for example, c:\logs\error.txt)
If you want to include backup files for the log file, enter the path and complete filename of the backup file in the Backup log file path box (this step is optional). This also needs to be a complete path and filename for a specific file.
Type an Instance descriptive name. This identifies the log file monitoring rule in the alert notifications you receive.
Select the severity level you want to apply to this alerting rule.
If you want to monitor only new entries in the log file (beginning at the time the monitoring rule is deployed to the device), click Monitor changes to log files. (This option is typically used for log files so the agent doesn’t keep scanning the same existing text.)
If you want to monitor all existing and all new entries in the log file, click Monitor entire log file. (This option is typically used to monitor other less dynamic files, such as configuration files.)
Click OK to add the rule to the list of logfile monitoring rules.
Repeat steps 4-15 to add other logfile monitoring rules.
After you have created the logfile monitoring rules you want, you need to add them to the ruleset. You can add multiple monitoring rules and apply action and time rules to them, depending on how you want to be notified when log file changes trigger alerts.
Click OK to save your changes and exit the Alert rules dialog box.
In the Rules summary list, select the rule you created and click Edit in the toolbar.
Adjust the Time as necessary. Click the State icons to select the states that you want notifications for. Dimmed states won't trigger notifications. Select Health if you want an alert that contributes to the device's health status.
Click Save to exit the dialog boxes when you are done.
Now let's deploy the alerting agent setting you created. In the Agent settings window toolbar, click the Create a task button and then click Change settings.
In the type column, click Alerting, and beside it select your new alerting agent setting.
Click Save, and the Scheduled tasks window will open with your new change settings task selected.
Add targets to the task, then right-click the task and click Start now > All.
Monitor the task progress. When the task finishes your new alert ruleset will be active on the devices you targeted.

Notes

Log file monitoring is supported only for managed Windows devices.
When you edit or delete a rule, affected devices won't get the updates until the next time they run a security scan. If you want the update to happen sooner, schedule an agent setting update for the alert setting you modified.
This feature maps log files into memory to use less memory during a search. Runtime memory is allocated for this as linear regular expression searches occur. Because Windows locks the file when it is mapped into memory, you may encounter issues with some applications.