prometheus, grafana, alertmanager: number of alerts

 prometheus alerts counts

from : https://jaanhio.me/blog/visualizing-alerts-metrics-grafana/ + https://community.grafana.com/t/how-to-get-the-time-range-selected-on-the-dashboard-into-a-variable/2868/3

(sum by (alertname) (changes(ALERTS_FOR_STATE[$__range]) AND ignoring(alertstate) max_over_time(ALERTS{alertstate="firing"}[$__range])) + (count by (alertname) (changes(ALERTS_FOR_STATE[$__range]) AND ignoring(alertstate) max_over_time(ALERTS{alertstate="firing"}[$__range])) * 1))

Then use a grafana panel as "Gauge" with the following options :

* Value options: show calculate, Last *

* Orientation = horizontal, and 

Number of alerts by alert name of the last 2 months

PromQL = sum by(alertname) (changes(ALERTS_FOR_STATE[65d]))

Number of alerts by instance over the last 2 months

PromQL = sum by(instance_name) (changes(ALERTS_FOR_STATE[65d]))

Opsgenie webinar / ressources

opsgenie is a tool allowing filtering and routing of monitoring-triggered alerts (nagios, AWS SNS, datadog, ...) to specific channels (SMS, phone-call, Slack, Jira, ...).

Main features on top of this :

• time-table (who's on-call) 
• alerts / incident resolution centralization
• third party integrations with 100+ tools

Opsgenie Learning Center :  https://docs.opsgenie.com/

[video] Opsgenie : "What do we do?"  https://www.youtube.com/watch?v=yphtZ9z2TtA&feature=youtu.be

[video] Opsgenie: "First Look" https://www.youtube.com/watch?v=pyM2dROKn6g

Opsgenie Pricing : https://www.atlassian.com/software/opsgenie/pricing

Implement nagios to opsgenie Heartbeats :

• basic demo : https://www.youtube.com/watch?v=wsN2E_ZHlkE&feature=youtu.be
• https://docs.opsgenie.com/docs/monitoring-nagios
• https://docs.opsgenie.com/docs/heartbeat-monitoring
• https://docs.opsgenie.com/docs/heartbeat-api