Monitoring & Events

Watch your fleet instead of polling it. Subscribe to file changes, dispatch a command across many nodes at once, or be told when a machine comes back online — then read the results as they arrive.

Overview

A subscription is a standing request: "tell me when this happens on these nodes". It has three parts, and they're the same whatever you subscribe to:

1. Subscribe — fs_subscribe, command_subscribe, presence_subscribe, or fs_grep_subscribe. Each returns a subscription_id.
2. Read — event_tail on that id. Pass wait_seconds so it blocks until something arrives instead of spinning.
3. Unsubscribe — event_unsubscribe when you're done.

Two properties are worth knowing up front, because they're what make subscriptions cheap:

• Reading keeps a subscription alive. It expires about 10 idle minutes after the last event_tail — and from router 0.4.91 a file or presence subscription can ask for longer with keep_seconds, up to 24 hours.
• Unsubscribing stops the work, not just the delivery. Watchers are torn down on every node, and any command still running under the subscription is killed. An abandoned subscription keeps that work running until it expires — so end it deliberately.

What a node allows

Filesystem, search, and command subscriptions run on the node, so the node's owner has the final say over what runs there. Three keys in the agent's profile config (/etc/nobgp/default.yml on Linux) govern it, and all three are permissive by default — a freshly installed agent serves all of them, so a subscription works without touching the box:

allow-tools:        # capability domains this node serves
  - fs
  - command
allow-roots:        # where the fs domain may reach
  - /
allow-admin: true   # may a caller ask for elevated execution?

• allow-tools is the allowlist of capability domains — fs and command. One answer per node: it governs event sources and the direct tools, so dropping command refuses both command_subscribe and the command tool on that machine. Search rides the fs domain, so dropping fs takes the file watches, the fs_* tools, fs_grep and fs_grep_subscribe with it. An explicit allow-tools: [] serves neither. These are domains, not tool names: a value that is neither (fs_read, commands) is ignored with a log line, and if nothing valid remains the permissive default comes back — a typo can't quietly lock a machine out of its own remote remedy.
• allow-roots bounds where the fs domain may reach. Narrow it to e.g. [/etc, /var/log] and anything outside those trees is refused — that is what stops a single recursive watch on /, or one fleet-wide search, from walking a peer's whole filesystem. On Windows the roots are drive-qualified (C:\ProgramData\app); the whole-filesystem value is / or \, and both are accepted on either platform, so the default reaches everything on a Windows node too. One tree is excluded whatever the value: the agent's own configuration directory (/etc/nobgp, /usr/local/etc/nobgp, C:\ProgramData\nobgp), which holds the node's key and the config these keys are read from.
• allow-admin vetoes elevated execution and elevated file access — the node half of the two gates on elevation, the other being the caller's organization role at the router (Execution identity). With allow-admin: false, anything that would run as uid 0 is refused rather than silently downgraded. Everything else runs as the profile's user — and where that would itself resolve to uid 0, the node refuses the unelevated request too rather than falling back to root (unelevated never means root), which is why allow-admin: false on a node with no usable user is an off switch rather than a narrowing. The fs_* tools are a real privilege drop now, not an authorization check standing in for one: the agent drops privilege before opening the file, so a file written unelevated is owned by that account.

Paths are checked with symlinks resolved on both sides, so a link inside a root can't smuggle a watch out of it. Changes take effect without a restart — the agent watches its own config and reloads itself. To see what a node is applying right now, read the allow block of nobgp status on the box.

They govern every remote path onto the node

All three bound what a node_grant reaches — the MCP tools and the event-bus sources — and, since agent 0.4.33, a published terminal service as well. Services were exempt before then; the exemption meant a service published to run as root handed out root on a node whose owner had set allow-admin: false, so it is gone.

sudo nobgp config --allow-tools= and friends warn at the moment you save when the result leaves a node serving no commands or no file access, and --allow-admin=false asks for confirmation when it would turn remote execution off entirely. Take the warning seriously: without command, the browser terminal is refused too, and the way back is at the machine itself. The same warning appears in the allow block of nobgp status.

allow-admin: false on Windows

A Windows node that captured no install account runs processes only as LocalSystem, so there is no unprivileged identity for a dispatched command to drop to. Since agent 0.4.33 the setting is honoured there as the only answer such a node can give — it refuses remote execution on that node entirely. Earlier agents logged that they could not apply it and kept admin enabled. On a Windows node you want to keep operable, narrow what reaches it with the router-side node_grant instead.

Since agent 0.4.44 a configured user gives such a node a second identity, and from agent 0.4.46 it covers every source: an fs watch acts as that account and a dispatched command runs as it, so allow-admin: false narrows such a node instead of switching it off. On 0.4.44 and 0.4.45 the identity covered file operations only, so the fs sources gained it while the exec source did not.

Renamed from event-sources / event-watch-roots

Earlier agents called these event-sources and event-watch-roots, and both were empty by default. The keys were renamed and the defaults flipped to permissive; an agent upgrading from the old names migrates them without tightening anything the node already served. A refusal now means the owner narrowed something, not that they never opted in.

Presence subscriptions need none of this — the router publishes those, and the nodes run nothing.

matched_nodes is who the work was sent to

Not who accepted it. A node that won't serve a source says so with a refused event naming the reason, rather than going quiet.

Choosing a cohort

Every subscribe tool takes the same node selector, so you address a set of machines once instead of looping over them:

Field	Restricts to
network_id / network_name	The network to act in (one or the other, not both)
node_ids / node_names	Specific nodes, by id or by name
platform / os / arch	What the nodes report about themselves (debian, linux, arm64…). arch takes the same spellings the rest of the product uses — arm64, amd64, armv7, armv6, 386 — and accepts the kernel synonyms (aarch64, x86_64) too
labels	Nodes carrying all of these labels

All fields are ANDed, and a subscription only ever matches nodes that are currently online — an offline machine can't start a watcher or run a dispatched command. Omit every node-narrowing field to cover the whole network.

The durable way to name a cohort is labels: tag once, then scope by meaning.

Label the Raspberry Pis in production with role=edge
Now watch /etc/nginx on everything labelled role=edge

Because filesystem and presence subscriptions re-evaluate their selector continuously, a node you enroll or relabel later joins the running subscription on its own — no re-subscribing. (The two dispatches are the deliberate exception: command_subscribe and fs_grep_subscribe pin their cohort when you subscribe, so a machine that appears a minute later never picks up a migration that was already sent, or a search that has already been handed out.)

Every subscribe call reports what your selector actually matched — matched_nodes plus the resolved nodes list — so you never have to create a throwaway subscription to find out. For a dispatch that list is the fixed cohort; for a file or presence subscription it is a snapshot, since the selector keeps being re-evaluated and later joiners won't appear in it. A selector that matches nothing right now still creates a file or presence subscription (it picks nodes up as they match) but says so plainly rather than returning a success-shaped zero; the dispatches — command_subscribe and fs_grep_subscribe — return not_found instead, because work sent to nobody is nothing.

Watching files

Watch /var/log/app for writes on every node labelled role=edge,
and watch /etc/nginx/nginx.conf for everything

Each entry in watches is one path with its own recursive flag and ops filter (create, write, remove, rename, chmod):

{
  "labels": { "role": "edge" },
  "watches": [
    { "path": "/var/log/app", "recursive": true, "ops": ["write"] },
    { "path": "/etc/nginx/nginx.conf" }
  ]
}

Paths are literal — there is no glob syntax, and a path containing *, ?, or [ is refused. To watch a tree, name the directory and set recursive.

The path has to exist already

A selector keeps being re-evaluated, so a node enrolled or relabelled later joins the subscription on its own. A path does not: a running watch never re-examines a path the node answered no such file or directory for.

How you find out depends on how many paths you declared. If every path was missing, the node has no watch to run and says so — a refused event naming each path and what it answered — and because nothing is running for it, the node tries the whole watch again the next time the router re-sends the subscription, which a reconnect does. If some path was missing, the watch runs on the rest and nothing reports the one that failed: its events simply never arrive, and an empty read looks exactly like a quiet tree.

To be told when something appears, watch the parent directory with recursive and let the create tell you. To watch a path you are about to make, make it first and subscribe after.

Events carry paths and operations, never file content:

{
  "source_id": "9f1c…",
  "at": "2026-07-29T10:15:04Z",
  "fs": { "changes": [{ "path": "/etc/nginx/nginx.conf", "op": "write", "timestamp_ms": 1785060904123 }] }
}

To see what actually changed, read the file with fs_read on the node named by source_id.

Narrow the ops

Filtering happens on the node, so anything you exclude costs nothing. On a busy tree, chmod noise from a package manager can overflow a subscription that only ever cared about writes.

A noBGP write arrives as a create, not a write

The file tools write atomically — fs_write, fs_edit, fs_copy and op=fetch_url all land bytes in a sibling temporary file and rename it onto the destination — so the watcher sees a new file appearing. An ops: ["write"] filter therefore misses every change made by these very tools. Watch for create too.

From agent 0.4.86 the temporary sibling itself is dropped on the node and never reaches you: by the time you read an event about config.yaml.nobgp-write-1274 the name is already gone, so it could be neither read nor acted on and only brought overflow closer. Measured on one node, two writes and a delete produced seven events, four of them siblings. Earlier agents forward them, so on a mixed fleet expect the noise from the nodes that have not upgraded. Only the agent's own scratch names are filtered, and only where the marker ends the filename — a file of your own called report.nobgp-write-final.csv still reports.

Each platform's native watcher has its own habits, and they show through:

• macOS (FSEvents) reports paths with symlinks resolved — a watch on /tmp/x reports events under /private/tmp/x — so don't prefix-match against the path you subscribed. A create followed immediately by writes can also coalesce into the create alone, which means an ops: ["write"] filter may miss a file that was created and written in one go.
• Linux (inotify) reports the distinct operations, on the paths as you gave them.

A node that could not watch any of the paths you gave it reports a refused event rather than sitting there silently, and from agent 0.4.89 that refusal names each path and its own reason — missing, a permissions problem, or the node's 1024-directory watch cap — instead of one sentence covering all three. ⚠ A node that watched some of them runs with partial coverage and does not tell you which ones it dropped, so subscribe to a path you are unsure about on its own if you need to know it is being watched.

Dispatching a command across a fleet

command_subscribe runs a command once on each matching node, detached, and collects a report from every one of them. It returns immediately — nothing holds a session open.

Run the backup script on every node labelled role=db and tell me how each one went

{
  "labels": { "role": "db" },
  "command": "/usr/local/bin/backup.sh",
  "max_seconds": 1800
}

The response lists exactly which nodes the work went to. Each node reports back:

{
  "source_id": "9f1c…",
  "status": "completed",
  "at": "2026-07-29T10:41:22Z",
  "command": {
    "lines": [{ "line": "backup finished: 4.2 GB", "stderr": false }],
    "exit_code": 0,
    "duration_ms": 184320
  }
}

exit_code appears on the event that ends a run — including when it is 0 — and is absent from the streaming output events in between, where there is no exit status yet. A run the node killed reports 128 + the signal number on Linux and macOS (143 for SIGTERM, 137 for SIGKILL) from agent 0.4.38, so a timed-out or cancelled run is distinguishable from one that chose its own exit code; Windows reports 1.

From router 0.4.84, on nodes running agent 0.4.86 or newer, the event also carries command.signal — the name of the signal that ended the run, never a number — so you no longer have to decode the exit code to find out. It is read from the same wait status the single-node tool reads, so the two surfaces cannot name one kill two different ways; a node on an older agent simply omits it, and a cohort can span both. A Windows node never reports one on a dispatch: it has no signals, and a dispatch is torn down with taskkill /T /F, which you never asked for by name — so there is nothing honest to attribute, and the field is absent rather than guessed from the exit code. It is the same field the single-node command tool reports, and it survives the case that needs it most: a run killed at its deadline reports cancelled with exit_code overwritten to -1, where the name is the whole answer.

event_tail also returns a progress block for a dispatch:

{ "dispatched": 12, "accepted": 11, "settled": 12, "outstanding": 0 }

• accepted below dispatched means some nodes never took the work — offline since, or refusing the source.
• outstanding: 0 means every node has finished. That is how you know the dispatch is done.

Every run is bounded (max_seconds, default 300, maximum 3600) and the dispatch always ends. There is no repeating mode: a command that should keep running is a service, not a subscription. event_unsubscribe kills whatever is still running.

Sending a script to a whole fleet

From router 0.4.83, command_subscribe takes the same script field command does — a whole script body, real newlines, no shell quoting — plus shell, workdir and env. Provide command (one line) or script (a body), never both.

{
  "labels": { "role": "db" },
  "script": "set -e\nsystemctl stop app\n/usr/local/bin/backup.sh --full\nsystemctl start app\n",
  "max_seconds": 1800
}

⚠ The body is lowered for each node separately, not once for the cohort — verbatim under the login shell on Unix, a temp batch script on Windows, -EncodedCommand under shell: "powershell". That is what lets one dispatch cross platforms at all, which is the thing this tool most often has to do: before it, a POSIX one-liner sent to a mixed fleet came back as 'printf' is not recognized from the one Windows node, buried among a dozen successes.

Two per-node refusals follow from it, and both arrive as that node's own refused event while the rest of the cohort runs: a node that has never reported its OS (nothing to lower the body for), and a node whose lowered line exceeds its platform's command-line limit — the encodings inflate, so the same body can fit on Linux and not on Windows. Only a cohort where no node can be served fails the call outright. workdir and env are one setting for the whole cohort, so the directory has to exist on every node the selector matched. Their contents are no longer restricted, from router 0.4.144: a workdir may hold any character, an env value may hold any character, and a name only has to avoid an equals sign and a null byte. The old refusal of a double quote, a percent sign, a newline or a null byte described the way noBGP used to fold these into the command line; they now travel to each node as fields of their own, which nothing parses on the way.

From agent 0.4.85 each node applies those two itself, and a node that cannot refuses instead of running: a workdir that is missing or is not a directory there, or an env name that would inject code into the run (LD_PRELOAD, BASH_ENV, NODE_OPTIONS and the rest of that list), arrives as that node's own refused event with the reason while the cohort carries on. ⚠ A node below agent 0.4.85 now ignores both and reports success, from router 0.4.144 — that is the other half of the same change, since the fold that used to carry them to such an agent is what was retired. It only reaches a node that is not taking upgrades, because one that reconnects installs what its channel offers within about a minute. With no workdir at all the command runs in the agent's own working directory rather than a home directory, so write absolute paths.

Who the command runs as

Execution identity matches the command tool field for field: admin, which since router 0.4.83 is a string — "true" or "false" — with a JSON boolean still accepted. Omit it to let each node pick.

• admin omitted — each node runs the work at the identity it offers. A node with a configured account runs it as that account (typically the installing account). A node whose unelevated identity would be the superuser — no account on a root-running agent, an account that no longer resolves, or user: root — runs it elevated. So one dispatch covers a cohort that mixes both shapes. The subscribe response carries admin_used, a map of node id to a boolean. true means that node was asked to run elevated. A node missing from the map was refused before dispatch and ran nothing; its refused event says why. A caller who may not elevate gets a refused event on each node where the work would run elevated, and the other nodes still run. The call fails with forbidden only when every node is refused. On a node whose owner set allow-admin: false, an omitted admin runs unelevated.
• admin: "false" — each node's own configured user. A node whose unelevated identity would be the superuser reports a refused event for that node instead of running the work as root (unelevated never means root). Send "false" when the work must not run as root on any node.
• admin: "true" — the node's ambient identity, and gated twice. The router refuses the whole dispatch with forbidden unless you are an org Owner or Admin (or a node holding a manage grant) — checked once against the network's organization, before any node is selected, so an elevated fan-out is all-or-nothing rather than root on some peers and refused on others. Past that, a node whose owner set allow-admin: false reports refused rather than running the work unprivileged. The node's gate is on the resolved uid 0, not a spelling. See Execution identity.
• admin: "auto" — refused with invalid_args. Omit admin instead.

A cohort is rarely uniform: network_directory reports each node's user, allow_admin and user_is_admin, and the same dispatch can run as a login on one node, a service account on the next, and be refused on a third. A Windows node refuses an unelevated dispatch where it captured no install account, because LocalSystem reports as uid 0 — and, on agents 0.4.44 and 0.4.45, on one with a second identity too, since that identity covered file operations and not execution. Agent 0.4.46 dispatches as the configured account there, so read this as a per-node answer the refused event reports rather than a rule about the platform.

To run as some third account, use su inside the command itself.

Same authority as command

A dispatched command runs through each node's shell with the same authority the command tool gives you. Scope the selector deliberately.

Progress from inside the command

A dispatched command sees $NOBGP_SUBSCRIPTION in its environment, so a long script can publish its own progress back into the same subscription you're reading — with nobgp notify, which reads that variable itself:

#!/bin/sh
nobgp notify phase=start
pg_dump mydb | gzip > /backups/db.sql.gz
nobgp notify phase=done bytes="$(stat -c %s /backups/db.sql.gz)"

key=value arguments become a typed JSON object; a plain string or a JSON document is sent as written. Publishing when nobody is listening exits 0 — a script under the bus shouldn't fail because the reader went away. Publishing nothing is different: no payload and no --status fails with exit 1, because it almost always means a pipe is missing its - (./thing | nobgp notify - reads stdin; ./thing | nobgp notify does not).

Your own client can do the same with event_publish — for example to drop a marker into the stream before starting a phase, or to write down what you concluded once one has ended.

From router 0.4.88 that second one works: a publish succeeds for as long as the subscription does, including after outstanding: 0. Through router 0.4.87 a marker published into a settled dispatch was dropped — the call reported delivered: false and the event was simply not in the next event_tail, even though the subscription was otherwise alive and readable.

On the node itself, nobgp events shows what that machine is publishing, which is the half you can't see from the subscriber side. It's the quickest way to confirm a source is actually reporting.

Searching a fleet

fs_grep_subscribe runs one RE2 regex search across every matching node and delivers the matches as events, instead of making you wait on one big response per machine. It is the async half of fs_grep — same walk, same skipping of binary files and noise directories (.git, node_modules, .venv, target, dist, …), aimed at a cohort.

Search /etc/nginx for "ssl_certificate" on everything labelled role=edge

{
  "labels": { "role": "edge" },
  "pattern": "ssl_certificate",
  "paths": ["/etc/nginx"],
  "max_count": 200
}

It returns immediately with a subscription_id. Each match arrives as its own event, and source_id is the node the match is on — so a fleet-wide search is attributable without a second call:

{
  "source_id": "9f1c…",
  "at": "2026-07-31T09:12:44Z",
  "payload": { "path": "/etc/nginx/sites-enabled/api", "line": 42, "text": "  ssl_certificate /etc/ssl/api.crt;" }
}

Each node then reports once more with a terminal status — completed with a summary of how many matches across how many files, or failed / refused with a reason. Like a command dispatch, the walk always ends: event_tail's progress block reaching outstanding: 0 means every node has finished.

Read that summary before concluding a machine is clean — for two separate reasons, and neither implies the other.

• A file that was there and was too big. An agent skips any file over its 16 MiB per-file cap without reading it, so a node holding a rotated log can report zero matches having never opened the file you were asking about. From agent 0.4.40 the summary says so — 0 match(es) across 0 file(s), 1 file(s) too large to search — and the synchronous fs_grep returns the same count as skipped_files.
• A path the node never looked at. From router 0.4.96 the summary also names paths that could not be read at all, and why — , 1 path(s) NOT SEARCHED: /opt/app (not_found) — counted as unreadable_paths on fs_grep. This is the one that bites a cohort: a path present on some machines and absent on others is the normal case across a fleet, and such a node used to report completed with no matches, indistinguishable from having searched. A node running an agent below 0.4.96 cannot report it, so its silence is still ambiguous.

An empty cohort search is evidence of nothing only on the nodes where both numbers are zero.

max_count is per node (default 1000, hard cap 100 000), and paths must be absolute. Narrow them rather than searching / on a busy fleet — and drain the queue while the walk runs, since only the newest 256 events are held.

From router 0.4.81 the walk takes admin, like fs_grep does. Omitted, a node with a configured account searches as that account — which is how a fleet-wide search comes back short and looks complete wherever the files that matter are root-owned. A node that offers only its elevated identity searches elevated. Send admin: "true" to search elevated on every node. With admin: "true", your authority to elevate is checked once, on the network's organization, so that cohort is never partly elevated. With admin omitted, a caller who may not elevate gets a refused event on each node that would search elevated, and the other nodes search unelevated. Either way, a node whose owner refused elevation locally reports that as its own event and the rest of the search carries on.

Reach for	When
fs_grep	One node, and you want its matches in one response
fs_grep_subscribe	More than one node · a tree big enough that you'd rather read matches as they land · you want to keep working while it runs

It reads, but it isn't marked read-only

The search itself only reads. The call stands up a subscription and starts work on every node in the cohort, which is state — so a client that auto-approves on capability hints is told so rather than waved through.

Waiting for a machine

presence_subscribe reports node lifecycle transitions. The router publishes these, so the nodes run nothing and pay nothing — it's the cheapest thing on the bus and the right way to wait for a machine to come back after a reboot. If the wait is longer than ten minutes — an overnight rebuild, a machine someone will power on tomorrow — set keep_seconds so the subscription is still there when it does.

Tell me when kitchen-pi comes back online

{
  "source_id": "9f1c…",
  "at": "2026-07-29T10:02:00Z",
  "presence": { "transition": "registered", "node_name": "kitchen-pi", "version": "0.4.15", "platform": "raspbian" }
}

transition is one of:

• online — the node's connection came up
• ready — that same connection has held for 5 seconds (router 0.4.84+)
• offline — the node's connection went away
• registered — the node re-registered. This is where version and platform can change, so it's the transition to watch when tracking an agent rollout.

Wait for ready, not online, before you use a node

ready is not a debounced online — online keeps its exact meaning, the bridge came up, which is true again after every reconnect. During a provision it is true for a fraction of a second before the node re-registers and drops: measured, the fleet went registered → online → offline → registered → online in 1.5 seconds, and the doomed bridge behind that first online lived 188 ms.

ready is published only when a session has survived 5 seconds, so it's the transition that says the node will still be there when your next call lands. A session that dies inside the window emits no ready at all; its replacement arms its own. Keep watching online when the question is the bridge itself — a gap, a bounce, a flap count.

Subscribe before you cause the thing you're waiting for

There is no replay. A subscription's queue starts empty and holds only what arrives after it exists — event_tail's offset re-reads what that queue already holds, it doesn't reach back past the subscription. Reboot a machine and then subscribe, and the ready you were waiting for has already gone by: event_tail returns empty for as long as you're willing to wait, and an empty tail looks exactly like a node that hasn't come back yet, so the mistake reads as patience.

The order is always:

1. Subscribe — presence_subscribe, scoped to the node or cohort.
2. Do the disruptive thing — reboot, upgrade, provision, restart the agent.
3. event_tail with wait_seconds — that call is the wait.

If you also need the state before it changes, read network_directory after subscribing rather than before, so nothing can fall between the two calls.

Don't wait with a timer

A guessed sleep, or network_directory polled in a loop, is wrong in both directions — it reports a node up before its session has held, and it burns the whole guess when the machine was back in two seconds. network_directory answers is it up now; presence answers tell me when that changes.

If you did subscribe too late, the directory's online_at (router 0.4.108+) is the fallback: it's re-stamped on every reconnect, so an online_at later than the moment you rebooted the node is the new session rather than the one you just ended. online: true on its own can't tell those apart.

Reading events

event_tail returns pending events oldest first:

{ "subscription_id": "…", "max": 50, "wait_seconds": 30 }

Use wait_seconds (up to 30) when you're waiting on something specific rather than calling in a loop. An empty result is normal. The field is also accepted as plain wait from router 0.4.146 — the name most callers reach for, which used to be refused as an unknown field and left the caller dropping it and polling instead.

Reading no longer destroys what it returns (router 0.4.84). Each response carries a next_offset, and the next call resumes there on its own — so an ordinary loop behaves exactly as it always did. What changed is that a response you never received is recoverable: send its next_offset back as offset and the same events are delivered again. Before this the queue was drained on read, so an event a lost response was carrying was simply gone.

The newest 256 events are retained. An offset older than that is served from the oldest event still held, with truncated: true and dropped_events saying how many were missed — never silence at a different position. An offset past the end is invalid_args: it belongs to a different subscription.

⚠ Re-reading is not replay. The offset reaches back only as far as the subscription's own creation — the queue starts empty, so nothing that happened before you subscribed is in it at any offset. See Subscribe before you cause the thing you're waiting for.

⚠ One subscription has one cursor, and every reader shares it. If two readers tail the same subscription — or one of your own calls succeeds and its response never reaches you — the position moves and your next offset-less call starts later, with nothing flagged: truncated stays false because nothing aged out. Subtract the number of events a response carried from its next_offset to see where that read started, compare it with the next_offset you last held, and ask for the difference back by offset while it is still retained. See Re-reading events for the two limits on that recovery.

Don't read at all — be told

Through a node's local MCP server, a subscription is an MCP resource. Subscribe to nobgp://events/<subscription_id> once and the server pushes notifications/resources/updated as events land, so waiting for a long command or a file change costs no polling and survives between turns. A read carries the progress block too from agent 0.4.121, so you can tell a finished dispatch from a quiet one without a second call. event_tail remains the fallback for clients that don't consume notifications — and it is the one to use on a node whose agent is below 0.4.121, where this path delivers nothing at all.

Each event may carry a status. Absent (or ok) means ordinary data with more to come; anything else is that node's last word — the other nodes on the subscription carry on:

status	Meaning
completed	The work finished cleanly — a command exited 0, or a node's search walk completed and reason summarises what it found
failed	The work finished badly — a non-zero exit, or a search the node could not complete
timeout	The command hit max_seconds and was killed
cancelled	You unsubscribed and the run was torn down
overflow	The node couldn't keep up and stopped — see reason
refused	The node won't run this source at all — see reason

Refusals and overflow

Both are answers, not errors, and both name a reason:

• refused — the node won't serve this source: the kind isn't in its allow-tools, the path is outside its allow-roots, the run asked for elevation its allow-admin: false vetoes, or the kind isn't supported on its platform. A caller who was not entitled to ask for elevation never gets this far — the subscribe call itself returns forbidden and nothing is dispatched. See What a node allows. matched_nodes tells you who the work was sent to, not who accepted it.
• overflow — changes (or output) outran what the node could report, so it stopped rather than delivering a silent fraction. The reason names which side overflowed — the node's own buffer or the kernel's queue. Narrow the watch paths or ops; for a chatty command, filter at the source (grep, tail -n) or redirect to a file and read it with fs_read.

A subscription holds only the newest 256 pending events. Treat events as coarse signals and investigate details with command or fs_read on the node that reported.

Staying subscribed while you are away

A subscription dies about ten idle minutes after the last event_tail, which is fine for a loop and wrong for the thing you most often want a watcher for: tell me if this changes while I am doing something else. From router 0.4.91, fs_subscribe and presence_subscribe take keep_seconds — the idle lifetime, in seconds.

{
  "node_names": ["kitchen-pi"],
  "keep_seconds": 14400
}

Omit it for the default 600 (10 minutes); the maximum is 86400 (24 hours). Reading still resets the clock — this changes how long the subscription tolerates silence, not how it is kept alive.

• A value over the maximum is clamped rather than refused, and ttl_seconds on the subscribe response is the lifetime that was actually applied. That is the only place a clamp shows, so read it back rather than assuming you got what you asked for.
• A value below the default is raised to it. keep_seconds is an extension: sending 30 must not give you a subscription that dies sooner than sending nothing.
• The two dispatches don't take it. command_subscribe and fs_grep_subscribe are bound to work, not to idleness — they close when every node has settled — so there is no idle clock to extend and sending the field is invalid_args.

It keeps the watchers, not the events

The queue still holds the newest 256 events and drops the oldest. A client away for twenty hours comes back to what fits, not to what it missed — event_tail says how many were dropped, but it cannot hand back what the queue no longer holds. If you need every event, keep reading; keep_seconds is what stops the watchers being torn down while you are gone.

Cleaning up

List my subscriptions in production and cancel the file watcher

event_subscriptions lists the subscriptions you created in a network, with each one's id and kind (fs, command, grep, presence). It's the recovery path when an id is lost to a new session or a crashed script — finding the orphan and ending it beats waiting out the idle clock while its watchers keep running.

Limits

Limit	Value
Idle lifetime (refreshed by every event_tail)	10 minutes default · up to 24 hours with keep_seconds on fs_subscribe / presence_subscribe
Pending events held per subscription	newest 256
Watches per fs_subscribe	64
Watch path / command length	4096 characters
Command runtime	300s default, 3600s maximum
event_publish payload	8192 bytes

A network's active subscriptions also share a budget for their declarations, since those travel to each node in a single frame. An oversized or over-budget subscription is refused with invalid_args and tells you to narrow it or unsubscribe something.

Isolated networks

Isolated networks have no event bus — their members must not observe one another.

Next Steps

• MCP Reference - Full schemas for every event tool
• Local MCP Server - Push delivery, and driving a network from one machine's Claude
• Core Concepts - How subscriptions fit with sessions and services
• Publishing Services - For work that should keep running, rather than a bounded dispatch