There are some principles to follow when designing an alert system, even a simple one that will be part of a dashboard:

• Balance alert sensitivity to minimize false positives: People will learn quickly to ignore alerts if they rarely identify an actual problem. This has to be balanced against missing too many real issues though. A cost benefit calculation can be used to help in this decision.
  [cost of investigating a problem] * [number of alerts] < [cost of actual problem] * [probability of true positive]
  The left side of that equation should be noticeably less than the right side.
• Be wary of alert fatigue: A long list of continuous alerts is daunting, and human brains tend to become conditioned over time to recognize it as a normal situation. Alerts will start to become background noise and will not get a response. Anyone with an Android smartphone knows how this feels - too many alerts are exhausting. There does not need to be an alert for every problem. Keep it to a manageable amount for the big ones.
• Make alerts like a to-do list: A user should not have to search for where the problem is; it should be listed out for them if there are any. If there are no issues discovered, a blank list can be comforting - like leaving work an hour early!
• Incorporate a tracking system for alert responses: Nothing is more frustrating than spending a couple of hours investigating a problem someone else has already discovered and corrected.