Prerequisites
The connector should already be set up, with a Connector Profile and a Linked Account. See Getting Started on the Databricks Workspace connector page.How Databricks webhooks work
StackOne creates and manages the webhook automatically when events are enabled on this connection. It creates a notification destination named StackOne in your workspace and deletes it when events are disabled or the account is disconnected. A destination is a channel, not a subscription. Databricks only sends a notification to it from jobs (or tasks) whose notifications include the StackOne destination, so you attach it to each job you want events from.
Make the service principal a workspace admin
Creating a notification destination requires workspace admin rights. Add the service principal whose Client ID this connection uses to the workspace admins group, signed in to your Databricks workspace as a workspace admin.
Open the groups settings
Navigate to Settings > Identity and access, then click Manage next to Groups.
Open the admins group
In the Groups list, click admins.
Add a member
On the Members tab, click Add members.
Select the service principal
Search for the service principal by name or Application ID, select it, and click Add.
- The Application ID is the Client ID entered on this connection.
Confirm the StackOne destination
After events are enabled on this connection, the StackOne destination appears in the workspace notification destinations.
Open the notification destinations
Navigate to Settings > Notifications, then click Manage next to Notification destinations.
Check the StackOne destination
Confirm that a destination named StackOne is listed. Don’t edit or delete it, as StackOne manages it.
Send job notifications to StackOne
Repeat these steps for every job you want events from. Notifications can also be added to individual tasks in the job’s task settings; task-level events identify the task run.
Open the job notifications
In Jobs & Pipelines, open the job. In the Job notifications panel, click Edit notifications.
Add a notification
In the Job notifications dialog, click Add notification.
Select the StackOne destination
Open Select a destination and, under System destinations, select StackOne.
Choose the events and save
Tick the events to send to StackOne, then click Save.
- Failure is ticked by default.
- Duration warning and Streaming backlog can only be ticked once the job has a matching metric threshold (see the next section).
- Maintenance start and Maintenance complete apply to continuous jobs.
Set metric thresholds for warning events
Duration warning and Streaming backlog notifications fire only when the job has a matching metric threshold.
Add a metric threshold
On the job page, under Job health configuration, click Add metric thresholds. Choose Run duration for duration warnings, or a Streaming backlog (bytes), Streaming backlog (duration), Streaming backlog (files), or Streaming backlog (records) metric for streaming jobs, then set the threshold.
Available webhook events
The following Databricks events can be enabled. Only events selected in StackOne are delivered, and only for jobs or tasks whose notifications include the StackOne destination.
Job run events
Events fired as a job or task run progresses.
- Job Start (
jobs.on_start) — Fired when a job run or task run starts. - Job Success (
jobs.on_success) — Fired when a run completes successfully, including runs that succeeded with failures. - Job Failure (
jobs.on_failure) — Fired when a run fails. - Job Duration Warning (
jobs.on_duration_warning_threshold_exceeded) — Fired when a run exceeds the job’s run duration threshold.
Other job events
A catch-all for job notifications without a dedicated event.
- Other Job Notification (
jobs.*) — Fired for any other job notification, such as streaming backlog or continuous-job maintenance. The event type is passed through exactly as Databricks sends it.
Delivery format
Details of how Databricks delivers job notifications to StackOne.
- Databricks sends no event ID or timestamp, so StackOne uses the run ID as the event ID.
- Task-level notifications are sent once per task attempt. A retried task, including a serverless automatic retry, produces new events with a new task run ID.
- Job-level notifications are not sent for retried tasks.