HP Operations Manager for Windows

Configure the message storm detection mechanism


Message storm detection is disabled by default. By setting appropriate values in the Server Configuration dialog box, you can configure the message storm detection mechanism according to the individual needs of your managed environment. You can configure what to interpret as a message storm by setting configuration values in the following namepaces:

The following table shows the values, which are the same in both namespaces used by the message storm detection mechanism. Note that these values have to be defined separately for each of the two available mechanisms. The assigned configuration values however can differ between two mechanisms.

Value Name Possible Values Description

Enable message storm detection based on time created

Enable message storm detection based on time received

True, False Used to enable or disable message storm detection. Set it to true to enable, set to false to disable.
Time interval to analyze Any value from 1 to 604800 Time interval in seconds over which the message flow is analyzed.
Number of messages for beginning of message storm Any value from 2 to 1000000 Used in combination with the time interval to analyze.

If based on the message property TimeCreated, then it specifies the maximum number of messages which can be created on a managed node within the defined time interval.

If based on the property TimeReceived, then it specifies the maximum number of messages which can be received on the management server from a managed node within the defined time interval.

If this value is exceeded, then this indicates a message storm. In this case the mgmt server automatically creates a high priority message and sends it immediately to the console. This message contains information about the affected node, message count, and time interval. The severity of this message can be configured with the value "Severity of the message storm begin message". Optionally, both an automatic and operator- initiated action can be associated with this message by using the values "Automatic action of the message storm begin message" and "Operator-initiated action of the message storm begin message". The actions could contain the command to start the "opcragt" tool to stop the agent on the affected node on demand.

Suppress messages during message storm True, False Defines whether to suppress messages for a managed node for which a message storm has been detected. If the option suppression is selected, then all messages for the affected managed node will be suppressed until the message storm is over. The end of a message storm is detected automatically.

Use the value false if you do not want messages to be suppressed; use the value true if you want messages to be suppressed. Suppressed messages are not stored in the database and are lost.

Number of messages for end of message storm Any value from 1 to 1000000 Defines the number of messages to fall below until a message storm is indicated as over. If the message storm is over, then the message suppression is stopped automatically for the affected node. Then the management server creates a high priority message and sends it immediately to the console to inform operators about the end of the message storm.

This message contains information about the affected node, message count, and time interval. The severity of this message can be configured with the registry value "Severity of the message storm end message". Optionally, both an automatic and operator- initiated action can be associated with this message by using the registry values "Automatic action of the message storm end message" and "Operator-initiated action of the message storm end message".

Severity of the message storm begin message 2,4, 8, 16, 32 Severity of the high priority message sent to the console to indicate the start of a message storm. Use the following numeric values:
 2	 normal 
 4	 warning
 8	 minor
16	 major
32	 critical
Severity of the message storm end message 2, 4, 8, 16, 32 Severity of the high priority message sent to the console to indicate that a message storm is over. Use the following numeric values:
 2	 normal 
 4	 warning
 8	 minor
16	 major
32	 critical
Automatic action of the message storm begin message String Name of a command file (.cmd or .bat) or name of a script file (.vbs) to be associated as an automatic action to the high priority message that is sent when a message storm is detected. Run time options and variables can be optionally added following the command/script file name.

The automatic action is executed on the management server immediately as a high priority action. The action execution does not involve the agent. It is also executed if the agent is stopped or is not installed on the management server. Possible tasks for the automatic action could be:

  • Stop the agent on the affected managed node by using the "opcragt" tool.
  • Send an email to the operators.
  • Page the operators.
Assign an empty string if no automatic action should be assigned to the message. Use the file %OvInstallDir\bin\OvMsgStormStartAutoTmpl.cmd as the template for a command file that could be assigned as an automatic action to a high priority message.
Automatic action of the message storm end message String Name of a command file (.cmd or .bat) or name of a script file (.vbs) to be associated as an automatic action to the high priority message that is sent when a message storm is over. Run time options and variables can be optionally added following the command/script file name.

The automatic action is executed on the management server immediately as a high priority action. The action execution does not involve the agent. It is also executed if the agent is stopped or is not installed on the management server. A possible task for the automatic action could be to send an email to the operators to indicate the end of a message storm.

Assign an empty string if no automatic action should be assigned to the message.

Operator-initiated action of the message storm begin message String Name of a command file (.cmd or .bat) or name of a script file (.vbs) to be associated as an operator-initiated action for the high priority message that is sent when a message storm is detected. Run time options and variables can be optionally added following the command/script file name.

A possible task in the operator-initiated action could be to restart the agent on the affected managed node by using the "opcragt" tool.

Assign an empty string if no operator-initiated action should be assigned to the message. Use the file %OvInstallDir\bin\OvMsgStormStartOperatorTmpl.cmd as a template for a command file that could be assigned as an operator-initiated action for a high priority message.

Operator-initiated action of the message storm end message String Name of a command file (.cmd or .bat) or name of a script file (.vbs) to be associated as an operator-initiated action for the high priority message that is sent when a message storm is over. Run time options and variables can be optionally added following the command/script file name.

Possible tasks depend on the managed environment and user requirements. Assign an empty string if no operator initiated action should be assigned to the message.

Message key prefix for message storm messages Any string User-defined prefix of the message key to add to the high priority messages generated when a message storm is detected and when a message storm is over. One of the following keys is constructed when a message storm is detected:

<MsgKeyPrefix>:TimeCreated:start:<NodeID>

<MsgKeyPrefix>:TimeReceived:start:<NodeID>

One of the following keys is constructed when a message storm is over:

<MsgKeyPrefix>:TimeCreated:end:<NodeID>

<MsgKeyPrefix>:TimeReceived:end:<NodeID>

Note NOTE:
Defining this value ensures that a message created at the end of a message storm automatically acknowledges the message created when the message storm was detected.

Assign an empty string if no message key should be created and to suppress automatic acknowledgement.

Service name for message storm messages Any string Service name that should be assigned to the high priority messages. The following format is used:

<ServiceName>@@<NodeID>

Note NOTE:
If message storm detection is enabled, then it is necessary to insert all values correctly. Missing incorrect values will disable message storm detection.

Change the configuration dynamically

After the initial configuration of the message storm detection mechanism, the OvEpMessageActionServer service must be restarted to read the configuration values.

Subsequent changes to the configuration values, however, do not require a restart of the OvEpMessageActionServer service. Values can be changed at any time during production and the new values take effect immediately, so it is possible to dynamically enable and disable the message storm detection mechanism.

Note NOTE:
Any dynamic change to the configuration values while the message storm detection mechanism is enabled will reset the message flow analysis by starting a new measuring time interval. Metrics collected up to this point are ignored.

Related Topics: