This article applies to:
• Alerting
• Product edition: All
• Feature Category: Alerts
Problem description
When attempting to create new alert definitions "Cannot Save alert. (400) Exceeded limit settings: 1200 alerts allowed per customer" messages are received.
Cannot save alert. (400)
Exceeded limit setting: 1200 alerts allowed per customer
Note: The log excerpts/Messages are only examples. Date, time, and environmental variables may vary depending on your environment.
Resolution
The inability to create a new alert definition can be limited due to alert limits instituted for all customers. These limits allow for the tenant to be optimally managed. While limits may be increased on a case by case basis, it is highly recommended that we review and perform alert clean-up for deleted, snoozed, permanently snoozed, or orphaned Alerts that are counting against the limit.
The alert limit is calculated with consideration to all alerts in all states to give us on overall value for the total alerts present. It is highly encouraged for customers that perform alert creation via automation to periodically perform alert maintenance to the alerts to ensure only viable alerts are maintained in the system at all times.
To further help customers delve into their alerts and provide a summary view of their alerts in their currently given state, we provide an API that will allow administrators to view the alerts that are present. Below is an example of the output from the API, that shows the states and number of alerts in those states.
https://<<cluster>>.wavefront.com/api-docs/ui/#!/Alert/getAlertsSummary
{ "status": { "result": "OK", "message": "", "code": 200 },
"response":
{ "no_data_smoke": 0,
"checking": 1098,
"active_severe": 3,
"no_data_warn": 162,
"active_warn": 11,
"active": 15,
"no_data": 243,
"active_smoke": 1,
"snoozed": 33,
"affected_by_maintenance": 0,
"trash": 43,
"no_data_severe": 38,
"no_data_info": 23,
"active_info": 0,
"invalid": 0
}
}
From the example output, it is visible there are several alerts that are in a "Snoozed" state and possibly 'Snoozed Forever' and in "Trash". Please review Wavefront documentation on Alert Lifecycles.
Continued alert maintenance, will ensure that the limits are rarely reached and that we are keeping only valid meaningful alerts in the system at all times.
Comments