Monitoring

frost-node exposes Prometheus metrics on metrics_listen. There is no default — set it; 9184 is the conventional port. Keep it bound to localhost and scrape it locally.

curl -s http://127.0.0.1:9184/metrics

What matters most

Ordered by how badly you want to know.

The node halted

Frost is fail-stop by design. It halts on execution-layer errors, on a journal hash mismatch, and on checkpoint-chain staleness. Each of these is a case where continuing would risk building a different chain than the network.

Alert on the process exiting, and alert on restart loops separately. A node that restarts repeatedly is not self-healing, it is telling you something.

A halt on hash mismatch is the most serious signal a Frost node can produce. It means a rebuild did not reproduce the recorded block hash — nondeterministic execution. Do not simply restart. Verify the binary against the recorded digest for the pinned commit, and confirm the consensus parameters compiled into it are the network’s. See Bootstrap and recovery.

Disk

This is the resource that will actually run out.

Post-quantum signatures make history growth substantial, and how much of it you keep depends on client and tier. A pruned node is bounded — a rolling execution-layer window and a consensus store held under its retention window and byte clamp — while the tx-history tier grows with traffic, on the order of gigabytes per day at the fleet’s resident load on 8141-geth and several times less on frost-reth. A node that fills its disk stops.

Alert on free space and on growth rate, not just a static threshold. A sudden change in growth rate usually means the retention window is not pruning what you think it is.

Checkpoint freshness

Checkpoints certify the block-key schedule. If the checkpoint chain stops advancing, the node alarms and then halts.

Alert on the alarm, not the halt — by the time it halts you have lost the window to act. Freshness problems are usually network-wide rather than local, so check whether checkpoints are being certified at all before assuming your node is at fault.

Head progress

Compare your head against another node:

# yours
curl -s http://127.0.0.1:8645 -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}'

# the public network
curl -s https://rpc.frostfi.net -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}'

Block cadence is load-driven — about one a second under ordinary load, several a second under load, one empty heartbeat block per 1,200 blockless commits when idle — so a height gap is a weak signal on its own. Use commit and checkpoint progress (frost_checkpoint_lag_commits, and the stall watchdog on a follower) as the lag signal.

Better still, compare a block hash at a height you have both passed. Identical hashes prove your node is reproducing the network’s chain exactly; matching heights only prove you are both counting.

Block key expiry

Block keys have expiring validity windows. A validator whose key expires cannot propose.

Rotation is routine and scheduled, but alert on time-to-expiry rather than relying on the schedule having run. Rotate well before the window closes.

Also worth watching

Consensus participation — whether your validator is contributing to the DAG and whether its blocks are being referenced by others. A validator that is technically alive but not participating is a quiet failure.

Ingress rejection rate — a spike usually means either an abusive client or a broken one. The rejection reasons distinguish them.

Retention floors — pruning is clamped by consumer progress, the latest certified checkpoint, the durable-execution floor, the verified-upload floor on archiving hosts, and the commit floor under the byte clamp. If pruning is not keeping up with disk growth, one of these floors is holding it back, and that is the thing to investigate rather than the window setting. When the disk guard fires, the commit floor collapses and frost_retention_disk_guard_active goes to 1 — the disk survives, the history window does not.

Archive upload progress, on hosts that export to cold storage. Data may not be pruned until it has been uploaded and hash-verified, so stalled uploads become stalled pruning become a full disk.

Operator signals

The node exposes a small set of operator-facing signals beyond the process metrics:

• frost_nodeHealth, on the ingress, behind the fleet bearer token: the data directory’s disk usage, the latest handled commit, the latest certified checkpoint, and the retention, observer, snapshot-serving and freshness gauges in one read. Without the token it answers like an unknown method.
• frost_dropStats, on the ingress, unauthenticated: the forced-transaction drop counters.

Gauges worth graphing:

Metric	Meaning
frost_checkpoint_lag_commits	Commits folded past the latest certified checkpoint.
frost_checkpoint_freshness_alarms_total	Freshness alarms raised; rising means certification is falling behind.
frost_retention_target_floor	The commit the pruner is currently cutting to.
frost_retention_disk_guard_active	1 while the disk guard has collapsed the commit floor.
frost_retention_disk_free_bytes	Free space on the consensus store’s filesystem, as the guard sees it.
frost_observer_stranded	1 when a follower has no usable peer ahead of it and must re-bootstrap.
frost_follower_stalled_seconds	How long a follower’s handled commit index has not advanced.
frost_archive_cert_gap_sequences	On exporting hosts: checkpoint sequences skipped for missing interior certificates when export was enabled — the accepted conversion gap.
frost_block_key_rotations_applied_total	Root-certified block-key rotations applied to the schedule.

Health checks

A simple liveness check:

curl -sf http://127.0.0.1:8645 -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}' \
  >/dev/null || exit 1

For readiness, additionally check that the head has advanced since the last probe. A node that responds but is not progressing is not serving anyone.

What not to alert on

Individual transaction rejections. The ingress refuses invalid submissions by design, and that is the system working.

Occasional peer disconnects. The consensus DAG tolerates them; that is the point of a DAG.

Head briefly lagging by a block or two. Cadence is load-driven, and an idle chain closes one heartbeat block per 1,200 blockless commits. Alert on commit and checkpoint progress instead, or you will train yourself to ignore the alerts that matter.