Configure Probes

OpsCenter Probes allow you to configure once-per-minute queries against recent and currently-running query history (using Snowflake’s QUERY_HISTORY( ) table function), and take action based on defined conditions.

There are 2 elements to an OpsCenter Probe:

• What to Check For: This is the Probe Condition, which is a valid WHERE clause that can be applied to a query of the INFORMATION_SCHEMA.QUERY_HISTORY() table function.
• What to Do: If the Probe Condition evaluates to TRUE, the Probe can send a notification, cancel the query, or both.

OpsCenter Probes can send notifications to the user who ran the query which matched the Condition (“Author” notification), a fixed list of recipients (“Other” notification), or both. Each kind of notification (“Author” or “Other”) have their own method of notification.

The supported methods of notification are:

• Email
• Slack message

One Probe can send a notification to the author via one of these mechanisms. For example, one Probe cannot send both an email and a Slack message to the author of a query. However, the author and other recipients may have differing methods of notification. One probe may email the author of a query and send a Slack message to a fixed list of other recipients.

Sending email and Slack notificatons requires enabling additional features. Please see Enabling Notifications and Slack Notifications respectively in the Advanced Features section of this Guide.

Next, let’s set up a Probe to monitor queries based on certain parameters. We’ll start by building a new Probe to check for queries which run longer than 5 minutes, and cancel them if they are found.

Click on the “Probes” link in the left-hand navigation menu:

To create a new Probe, we’ll click “New”:

We simply choose a name for our new Probe, and then specify the conditions under which we want to be alerted. Note that the Condition is any valid WHERE clause that can be applied to the INFORMATION_SCHEMA.QUERY_HISTORY() table function.

In this case, we want to monitor by runtime, so let’s use the EXECUTION_TIME column to start. Note this column contains execution times in milliseconds, so in order to find queries with durations longer than 5 minutes, we need to use 300,000 (5 * 60 * 1000) milliseconds. We do this by entering EXECUTION_TIME > 300000 in the “Condition” field.

We also need to add one more check in our condition, to ensure that our Probe does not cancel the OpsCenter app itself (which runs as a query in Snowflake). We’ll do this for now by excluding any Streamlit queries, using this clause AND NOT (query_text ilike 'execute streamlit%').

Note that when provided as a Probe Condition, the WHERE clause does not include the keyword WHERE.

Finally, we need to select what we want to happen when queries are found that meet our condition. In this case, we will select “cancel”.

Cick “Create” to create the Probe. After a few seconds, you’ll be taken back to the list of Probes, and you’ll see your new Probe listed:

From here, you can create further Probes, or modify/delete existing Probes.