An adapter turns a notification decision into a side effect: an SMTP session, an HTTP POST, a process, a Telegram message. jaque has seven, one per contact type, and none of them is trusted to remember anything. Every decision is written to the event log before an attempt is made and its outcome is written after, so whoever delivers can be killed, restarted or replaced by another process and resume from the log with no duplicate that already landed and no notification still owed. Delivery is at least once, by design, and idempotent by notification ID.

Status: the path is tested end to end against a fake receiver, including a flaky one, but no deployment has paged a human yet; see the status page. JAQUE_TELEGRAM_TOKEN is the environment form of -telegram-token by the general rule that every flag has a JAQUE_ prefixed variable, documented in CLI flags.

1. The seven adapters

Adapter Contact type Mechanism
email email net/smtp with STARTTLS and optional PLAIN auth, or the system mail binary
webhook webhook HTTP POST to the contact's url
command command a top-level command template run as a process, the Nagios model
telegram telegram the Bot API's sendMessage to the contact's chat_id
pagerduty pagerduty the Events API v2 enqueue endpoint, keyed by the contact's routing_key
slack slack an HTTP POST to the contact's webhook_url
incidentio incidentio the alert events HTTP source named by the contact's alert_source_config_id

The command adapter is the primitive: it runs a process the way Nagios's notification commands do, and it is what the email adapter falls back to when no relay is configured. Anything not listed is reached through webhook or command; there is no plugin interface for adapters. The contact side of this table is specified in Contacts and policies.

2. Durability through the log

A notification has three events and a stable identity.

A notification has three events: NotificationRequested is appended before the adapter is called, and either NotificationSent or NotificationFailed is appended afterengine decidesdeliverer attemptslogNotifyDecisionNotificationRequestedappended BEFOREthe attemptadapter.Send(...)NotificationSentappended on successorNotificationFailedappended onpermanent failureThe request is on the log before any adapter runs, so a crash mid-send is recoverable.
Text version
engine decides            deliverer attempts            log
--------------            ------------------            ---
NotifyDecision  ------->  NotificationRequested  ----->  appended BEFORE the attempt
                          adapter.Send(...)
                          NotificationSent       ----->  appended on success
                      or  NotificationFailed     ----->  appended on permanent failure

NotificationRequested is appended before any adapter is called. The deliverer, whichever process that is, folds the log on startup to find every NotificationRequested without a terminal NotificationSent or NotificationFailed and resumes exactly those. The notification ID is derived from the object, the kind, the level and the instant the problem began, never from the current time, so a re-notification of the same problem and a replayed request after a crash produce the same ID and are deduplicated against it.

The guarantee is at least once. If ownership of delivery moves between processes while an attempt is in flight, the new owner may repeat a send that the old one had started; it will never skip one. Retries use backoff and stay inside one delivery shard, so two attempts for one ID never run concurrently.

3. Where delivery runs: -notify-delivery

The engine that owns an object always decides whether to notify and writes that decision to the log. Who performs the side effect is a separate switch.

-notify-delivery Who delivers
embedded (default) the same process that decided, under -target all or -target engine
external one or more -target notifier processes, each pulling its share of open notifications from the log

Under external, several notifiers split the work by object ownership over their own membership bucket; Cluster and coordination describes the split. A -target notifier process requires -notify-delivery external on the engine upstream, or it finds nothing to deliver.

In either mode delivery is sharded by object across -notify-workers workers (default 4), each with a bounded queue of -notify-queue notifications (default 64). A slow receiver fills its shard's queue and then blocks the notifier's log follower; nothing is dropped for being slow. The block is visible as jaque_notifier_blocked_total counting up and jaque_eventlog_consumer_lag{follower="notifier"} growing, and jaque_notifier_pending shows how many notifications sit in the shards.

4. Webhook body

The JSON body posted to a webhook contact's url:

Field Notes
notification_id stable identity, also sent as the Idempotency-Key header
object_id the object the notification is about, e.g. web1/http
kind PROBLEM, RECOVERY, ACKNOWLEDGEMENT, ...
level the escalation level this send belongs to
contact the contact ID
at RFC3339Nano, when the decision to notify was made
status the object's status at decision time: OK, WARNING, CRITICAL or UNKNOWN
output the check's plugin output
perfdata the check's performance data; the key is omitted when there is none
problem_since RFC3339Nano, when the open problem began; "" when the object has no open problem

5. Runtime macros

A command-adapter template (and the mail fallback SMTP uses) can reference these macros at dispatch time, alongside the identity macros already available everywhere: $NOTIFICATIONTYPE$, $NOTIFICATIONNUMBER$, $CONTACTNAME$, $CONTACTEMAIL$. Every other Nagios runtime macro still fails template expansion.

A service or a process contact gets the service vocabulary; a host contact gets the host vocabulary. A process has no macro family of its own, since a process is not part of Nagios's object model and jaque adds no third one for it.

Macro Value
$SERVICESTATE$ OK, WARNING, CRITICAL or UNKNOWN
$SERVICEOUTPUT$ the check's plugin output
$SERVICEPERFDATA$ the check's performance data
$SERVICEDURATION$ time since the open problem began, XdYhZmWs
$LASTSERVICESTATECHANGE$ unix seconds when the open problem began
Macro Value
$HOSTSTATE$ UP when the status is OK, otherwise DOWN
$HOSTOUTPUT$ the check's plugin output
$HOSTPERFDATA$ the check's performance data
$HOSTDURATION$ time since the open problem began, XdYhZmWs
$LASTHOSTSTATECHANGE$ unix seconds when the open problem began

$HOSTSTATE$ never renders UNREACHABLE: reachability tracking is not part of a notification, so a host's state macro is always one of UP or DOWN. Nagios has a third host state, UNREACHABLE; this macro does not carry it.

Both families also get $TIMET$ (unix seconds), $DATE$ (YYYY-MM-DD), $TIME$ (HH:MM:SS), $SHORTDATETIME$ and $LONGDATETIME$, all as of the notification decision.

6. SMTP

-smtp-host, -smtp-port (default 25), -smtp-user, -smtp-pass, -smtp-from

With -smtp-host set, email contacts are delivered over SMTP with STARTTLS; -smtp-user empty disables authentication. With -smtp-host empty, email contacts are delivered by running the system mail binary through the command adapter, which suits a host that already has local mail delivery configured and nothing else.

7. Telegram

A telegram contact carries only a chat_id. The bot token is one per jaque instance and is a process secret, set once via -telegram-token or JAQUE_TELEGRAM_TOKEN, the same posture as SMTP credentials. With the token empty, every telegram contact fails permanently rather than silently.

To obtain a token, talk to @BotFather, send /newbot and keep the token it returns. To find a chat_id, message the bot (or add it to a group), then read https://api.telegram.org/bot<token>/getUpdates: the chat.id on the message you sent is the value for the contact.

8. Message templates

An email is plain text, CRLF-terminated, with a fixed set of headers in a fixed order: From, To, Subject, Date, Message-ID, MIME-Version, Content-Type. Only Subject and the body are templated; the rest are computed from the delivery and the SMTP configuration.

The default subject and body:

[jaque] {{.Kind}} {{.Object}} is {{.Status}}
Object: {{.Object}}
Kind: {{.Kind}}
Status: {{.Status}}
Output: {{.Output}}
Since: {{.ProblemSince.Format "2006-01-02 15:04:05 MST"}} ({{.Duration}})
Level: {{.Level}}
Contact: {{.Contact}}
At: {{.At.Format "2006-01-02 15:04:05 MST"}}

The default Telegram text:

[{{.Kind}}] {{.Object}} is {{.Status}}
{{.Output}}
Since: {{.ProblemSince.Format "2006-01-02 15:04:05 MST"}} ({{.Duration}})
Level {{.Level}}

An email contact overrides either or both with subject_template and body_template; a telegram contact overrides its text with template. Each defaults to "", meaning the default above:

alice: {
	type:             "email"
	address:          "[email protected]"
	subject_template: "[{{.Level}}] {{.Object}} {{.Status}}"
	body_template:    "{{.Output}}\nSince {{.ProblemSince.Format \"15:04:05\"}}\n"
}

Every template renders against the same fields:

Field Meaning
Object the object the notification is about
Kind PROBLEM, RECOVERY, ACKNOWLEDGEMENT, ...
Status the object's status at decision time
Level the escalation level this send belongs to
Contact the contact ID
Output the check's plugin output
Perfdata the check's performance data
At when the decision to notify was made
ProblemSince when the open problem began
Duration time since the open problem began, XdYhZmWs

At and ProblemSince are timestamps with a .Format "layout" method, so a template can call {{.At.Format "2006-01-02 15:04:05 MST"}} or any other layout rather than just printing the zero-argument value.

Every template, default or overridden, is parsed and executed once at startup against a fixed probe value. A broken template or one that names a field not in the table above fails the process at load time rather than at the moment a notification is due.

HTML mail is not supported: the body is always text/plain, and there is no HTML alternative part.

9. Incident platforms: PagerDuty, Slack, incident.io

PagerDuty and incident.io both group a problem's notifications into one open incident that resolves itself when jaque says the problem is over. What makes that possible is an alert key: one key per problem episode, shared by the PROBLEM notification, every re-notification, every escalation and the RECOVERY. The key is derived from the object ID and the instant the problem began, never from the notification ID, which varies by kind and level. An incident platform keyed on the notification ID would open a new incident on every re-notification instead of updating the one already open.

A PROBLEM or STATUS_CHANGE maps to PagerDuty's trigger event action and to incident.io's firing status, a RECOVERY to resolve and resolved, and an ACKNOWLEDGEMENT to PagerDuty's acknowledge (incident.io has no equivalent and receives nothing). The flapping and downtime kinds are not incidents: both adapters report them delivered without posting, so a contact that should not receive them at all lists them in the level's disabled_kinds.

PagerDuty additionally wants a severity, derived from the object's status at decision time:

Status PagerDuty severity
CRITICAL critical
WARNING warning
UNKNOWN error
OK info

The incident.io API token is a process secret the same way the Telegram bot token is: -incidentio-token or JAQUE_INCIDENTIO_TOKEN. With the token empty, every incidentio contact fails permanently rather than silently.

Slack is the exception among the three: a slack contact's webhook_url is itself the credential, since posting to it is the whole authorization Slack asks for. Unlike the PagerDuty routing key and the incident.io token, it is not held back as a process secret. It lives in the config, on the contact.

10. Grouping

Dependency suppression already handles the case where a failure is declared to depend on another: nothing is sent for the child while the parent is the one that is broken. What it cannot handle is the residue: an undeclared dependency nobody wrote down, the race window before a failing parent has itself gone hard DOWN, or a correlated failure with no dependency edge at all, like a bad deploy that breaks forty unrelated checks at once. Grouping is the safety net for that residue: instead of forty messages, a contact gets one digest. It is not a general-purpose alerting router: there is no group_by, no group_interval, no re-notify inside an open group, one window per contact and nothing more.

A digestable contact, one of email, webhook, telegram or slack, may set group_wait, a duration. The first notification buffered for that contact opens a window of that length; every further notification for the same contact arriving before the window closes joins it. When the window closes, everything buffered is delivered as one message, oldest first. group_wait is scoped per contact rather than globally: two contacts with different settings, or one grouped and one not, are unaffected by each other.

command, pagerduty and incidentio contacts do not take group_wait; the config schema rejects it there. The command adapter's contract is one macro line per object: a digest has nowhere to put a second object's macros. PagerDuty and incident.io already group on the platform's own terms (section 9): an alert key ties every notification for one problem episode into a single incident that opens once and resolves once, which is a different and stronger guarantee than a time window, so a second, weaker grouping mechanism underneath it would only compete with it. slack, unlike those two, has no alert-key grouping of its own: it is a plain HTTP POST to a webhook, section 9's exception, so it benefits from group_wait exactly as email and webhook do.

Grouping does not change what lands on the event log. Every object still gets exactly one NotificationRequested, and every notification and contact pair still gets exactly one terminal NotificationSent or NotificationFailed, precisely as section 2 describes. What differs is delivery-side only: a buffered notification is an ordinary open log entry, just viewed from memory until its window closes. A crash mid-window loses nothing but time: the open entries re-fold on restart, re-buffer, and the window restarts from zero, at most delaying delivery by another group_wait. The window itself is not persisted anywhere; buying back that delay would mean writing the window to the log, and the log's job is delivery record-keeping rather than scheduling.

A window that closes with exactly one notification in it is delivered through the ordinary single-message path, never as a one-item digest. Grouping only changes anything once there is something to group. A window with a PROBLEM and its RECOVERY both buffered does not cancel them against each other the way some alerting systems do: both are delivered, in the order they happened, inside the same digest.

The default email digest subject and body:

[jaque] {{.Count}} notifications
{{.Count}} notifications for {{.Contact}}:

{{range .Items}}[{{.Kind}}] {{.Object}} is {{.Status}}: {{.Output}}
  Since: {{.ProblemSince.Format "2006-01-02 15:04:05 MST"}} ({{.Duration}}), level {{.Level}}
{{end}}

The email digest's headers follow section 8's shape with three fields computed differently: Subject is the rendered digest subject above, Date is the last member's At (members are always ordered oldest first), and Message-ID hashes every member's notification ID joined by | together with the contact, rather than a single notification ID.

The default Telegram digest text, also used verbatim for Slack's digest message since a single Slack message is already plain text:

{{.Count}} notifications:
{{range .Items}}[{{.Kind}}] {{.Object}} is {{.Status}}: {{.Output}}
{{end}}

A webhook digest POSTs the same per-notification body section 4 documents, once per member, wrapped in an envelope:

{"notifications": [ <body>, <body>, ... ]}

Digest templates are fixed defaults in this version: there is no CUE field to override a digest subject, body or text the way subject_template, body_template and template override the single-message templates in section 8. That override, along with group_by/label-based grouping, group_interval, and re-notifying inside an already-open group, is out of scope for now.

11. Security considerations

Every credential an adapter needs, the SMTP password, the Telegram bot token, the incident.io token, is a process-level flag or environment variable and never part of the CUE config, so a configuration can be reviewed and shared without carrying a secret. The one exception is a slack contact's webhook_url: it is both address and credential, so a config carrying one is not shareable the way the rest of contacts: is. The command adapter runs processes on the delivering host with the arguments the commands map specifies; that map is trusted configuration. Webhook, Telegram, PagerDuty, Slack and incident.io deliveries are outbound HTTPS to operator-chosen or vendor endpoints; the notifier opens no listener of its own beyond /metrics on -listen.