Azure Data Factory Alerts provide an automated response that can be beneficial to monitor and audit Azure Data Factory activity. These alerts are very proactive and more efficient than manual monitoring operations. Alerts can be fired on both success and failure of a pipeline based on the rule configuration.
Alert Rule
Azure Data Factory Alerts use an alert rule which states the criteria upon which the alerts should trigger. We can enable or disable the alert rules.
To create a new alert rule, we must configure three properties;
- Alert Rule Name and Severity
- Target Criteria
- Configure Email/SMS/Push/Voice notification
Steps to figure the alert rule
1. Navigate to monitoring tab in Azure Data Factory. Select Alerts & Metrics panel and select New Alert Rule.
2. Set Alert Rule Name and add severity to the alert.
3. In Target Criteria, select Azure Data Factory metrics on which the alerts must be triggered. There will be a monthly rate as per the configured criteria.
Here I have selected ‘Failed pipeline runs metrics’ which will trigger only when a pipeline activity fails.
This will navigate to configure alert logic tab.
- Set Show History to compare the metric values within that time period
- Select the dimension values so it could filter on the right time series. Here the dimensions are pipeline name and failure type. These dimensions vary for each data factory metrics.
- Alert Logic – To compare the metric value with threshold calculated based on time aggregati
Based on the above alert logic condition, we can receive alerts on the very first pipeline failure.
- Set the period and frequency based on which the above time aggregation in alert logic condition works
After adding this criterion, we will be redirected to alert rule page. Only two criteria can be added.
4. In configure Email/SMS/Push/Voice notification, an action group must be set
- Action Group: An action group defines a set of notification preferences and actions included by Azure alerts
- We can either create a new action group or use an existing action group
- To create a new action group,
- Enter a name in the Action group name box and enter a name in the short name box which is used in each alerts sent using this group.
- Then select Add Notification and enter a name for Action Name
There are four options to receive the alerts and Carrier charges may apply for some of these services.
- SMS
- Azure App push notifications
- Voice
5. After configuring all the mentioned properties, click on to create alert rule and declare the enable option. The alerts will be triggered only if it is enabled.
If the selected pipeline fails, an alert will be triggered and the fired alert will be in active state. We can check this alert in the monitor service from the Azure portal.
During alert active state, if the same pipeline run completed successfully, then alert will be deactivated. For the same reason, we receive an automated deactivated severity alert mail. If we want to disable these deactivated alerts, we can control this through suppress notification under action group.
These alerts provide around a clock service, to manage and monitor the data factory. This will allow us to quick wit the problems that affect any dependency process and increases the data integration process. But these alerts come up with regular monthly rates.
To know more about Azure pricing information click here. Azure data factory alerts is one of the ways to receive notification in case of a pipeline run failure.
To learn more about other alternatives for triggering alerts in case of pipeline run failure refer here.
