Configure the message storm detection mechanism
Message storm detection is disabled by default. By setting appropriate values in the Server Configuration dialog box, you can configure the message storm detection mechanism according to the individual needs of your managed environment. You can configure what to interpret as a message storm by setting configuration values in the following namepaces:
- Message Storm Detection Based on Time Created
- Message Storm Detection Based on Time Received
The following table shows the values, which are the same in both namespaces used by the message storm detection mechanism. Note that these values have to be defined separately for each of the two available mechanisms. The assigned configuration values however can differ between two mechanisms.
| Value Name | Possible Values | Description |
|
Enable message storm detection based on time created Enable message storm detection based on time received |
True, False | Used to enable or disable message storm detection. Set it to true to enable, set to false to disable. |
| Time interval to analyze | Any value from 1 to 604800 | Time interval in seconds over which the message flow is analyzed. |
| Number of messages for beginning of message storm | Any value from 2 to 1000000 | Used in combination with the time interval to
analyze.
If based on the message property TimeCreated, then it specifies the maximum number of messages which can be created on a managed node within the defined time interval. If based on the property TimeReceived, then it specifies the maximum number of messages which can be received on the management server from a managed node within the defined time interval. If this value is exceeded, then this indicates a message storm. In this case the mgmt server automatically creates a high priority message and sends it immediately to the console. This message contains information about the affected node, message count, and time interval. The severity of this message can be configured with the value "Severity of the message storm begin message". Optionally, both an automatic and operator- initiated action can be associated with this message by using the values "Automatic action of the message storm begin message" and "Operator-initiated action of the message storm begin message". The actions could contain the command to start the "opcragt" tool to stop the agent on the affected node on demand. |
| Suppress messages during message storm | True, False | Defines whether to suppress messages for a managed
node for which a message storm has been detected. If the option
suppression is selected, then all messages for the affected managed
node will be suppressed until the message storm is over. The end of
a message storm is detected automatically.
Use the value false if you do not want messages to be suppressed; use the value true if you want messages to be suppressed. Suppressed messages are not stored in the database and are lost. |
| Number of messages for end of message storm | Any value from 1 to 1000000 | Defines the number of messages to fall below until
a message storm is indicated as over. If the message storm is over,
then the message suppression is stopped automatically for the
affected node. Then the management server creates a high priority
message and sends it immediately to the console to inform operators
about the end of the message storm.
This message contains information about the affected node, message count, and time interval. The severity of this message can be configured with the registry value "Severity of the message storm end message". Optionally, both an automatic and operator- initiated action can be associated with this message by using the registry values "Automatic action of the message storm end message" and "Operator-initiated action of the message storm end message". |
| Severity of the message storm begin message | 2,4, 8, 16, 32 | Severity of the high priority message sent to the
console to indicate the start of a message storm. Use the following
numeric values:
2 normal 4 warning 8 minor 16 major 32 critical |
| Severity of the message storm end message | 2, 4, 8, 16, 32 | Severity of the high priority message sent to the
console to indicate that a message storm is over. Use the following
numeric values:
2 normal 4 warning 8 minor 16 major 32 critical |
| Automatic action of the message storm begin message | String | Name of a command file (.cmd or .bat) or name of a
script file (.vbs) to be associated as an automatic action to the
high priority message that is sent when a message storm is
detected. Run time options and variables can be optionally added
following the command/script file name.
The automatic action is executed on the management server immediately as a high priority action. The action execution does not involve the agent. It is also executed if the agent is stopped or is not installed on the management server. Possible tasks for the automatic action could be:
|
| Automatic action of the message storm end message | String | Name of a command file (.cmd or .bat) or name of a
script file (.vbs) to be associated as an automatic action to the
high priority message that is sent when a message storm is over.
Run time options and variables can be optionally added following
the command/script file name.
The automatic action is executed on the management server immediately as a high priority action. The action execution does not involve the agent. It is also executed if the agent is stopped or is not installed on the management server. A possible task for the automatic action could be to send an email to the operators to indicate the end of a message storm. Assign an empty string if no automatic action should be assigned to the message. |
| Operator-initiated action of the message storm begin message | String | Name of a command file (.cmd or .bat) or name of a
script file (.vbs) to be associated as an operator-initiated action
for the high priority message that is sent when a message storm is
detected. Run time options and variables can be optionally added
following the command/script file name.
A possible task in the operator-initiated action could be to restart the agent on the affected managed node by using the "opcragt" tool. Assign an empty string if no operator-initiated action should be assigned to the message. Use the file %OvInstallDir\bin\OvMsgStormStartOperatorTmpl.cmd as a template for a command file that could be assigned as an operator-initiated action for a high priority message. |
| Operator-initiated action of the message storm end message | String | Name of a command file (.cmd or .bat) or name of a
script file (.vbs) to be associated as an operator-initiated action
for the high priority message that is sent when a message storm is
over. Run time options and variables can be optionally added
following the command/script file name.
Possible tasks depend on the managed environment and user requirements. Assign an empty string if no operator initiated action should be assigned to the message. |
| Message key prefix for message storm messages | Any string | User-defined prefix of the message key to add to
the high priority messages generated when a message storm is
detected and when a message storm is over. One of the following
keys is constructed when a message storm is detected:
One of the following keys is constructed when a message storm is over:
NOTE:Defining this value ensures that a message created at the end of a message storm automatically acknowledges the message created when the message storm was detected. Assign an empty string if no message key should be created and to suppress automatic acknowledgement. |
| Service name for message storm messages | Any string | Service name that should be assigned to the high
priority messages. The following format is used:
<ServiceName>@@<NodeID> |
NOTE:If message storm detection is enabled, then it is necessary to insert all values correctly. Missing incorrect values will disable message storm detection.
Change the configuration dynamically
After the initial configuration of the message storm detection mechanism, the OvEpMessageActionServer service must be restarted to read the configuration values.Subsequent changes to the configuration values, however, do not require a restart of the OvEpMessageActionServer service. Values can be changed at any time during production and the new values take effect immediately, so it is possible to dynamically enable and disable the message storm detection mechanism.
NOTE:Any dynamic change to the configuration values while the message storm detection mechanism is enabled will reset the message flow analysis by starting a new measuring time interval. Metrics collected up to this point are ignored.
Related Topics: