Watchman reference

This is the canonical hosted reference for the public Watchman source guides. It keeps installation, command contracts, configuration, machine output, and the observer API together without turning the product guide into a manual.

Install and releases

The verified installer is the fastest route to a release binary. It checks the published checksum and supports macOS and Linux on x86-64 and ARM64.

12curl -fsSL https://tools.yshubham.com/watchman/install.sh | sh
watchman version

Source builds remain first-class. Build with the locked Cargo dependency graph, then install target/release/gpu-watchman as watchman. Container, systemd, shell completion, and release-artifact instructions belong with the source release and are reflected by this hosted installation path.

CLI workflows

Collection is read-only by default. Start with snapshot, ps, top, or doctor; use serve only when you deliberately want a local observer exposing health, metrics, and report endpoints.

12345watchman snapshot --all --details
watchman ps
watchman doctor --probe https://inference.example/metrics
watchman serve --listen 127.0.0.1:9400 \
  --api-token-file /run/secrets/watchman-api-token

Active workflows are explicit. canary sends bounded correctness traffic, benchmark saturation sends a closed-loop concurrency experiment, and capacity is an offline estimate. Use compare, rollout, and benchmark compare only against saved, compatible evidence.

Operational profiles

Profiles are strict and versioned. Manage them with watchman config init, validate, and show; select one through --config PATH, GPU_WATCHMAN_CONFIG, and optionally --profile NAME.

123watchman config init gpu-watchman.toml
watchman --config gpu-watchman.toml config validate
watchman --config gpu-watchman.toml --profile local serve --no-api-auth

Precedence is deterministic: built-in defaults, profile, environment, then CLI. Prefer *_FILE variables for credentials, and treat a no-auth server as a loopback-only debugging exception.

Architecture and development

Watchman separates collection from interpretation. Stable data contracts, telemetry adapters, inference probes, analyses, planning, presentation, and the CLI have distinct boundaries so a new backend or health rule cannot quietly change the machine-readable report.

Extensions need the same evidence discipline. A telemetry backend must say what it observed and what it could not observe; a runtime probe must preserve transport and data boundaries; a rule must be testable without fabricating a healthy signal.

Report contract

Human text, JSON, and NDJSON are views of one versioned record. Hardware reports use schema_version: 3; runtime inspection uses runtime_fingerprint_version: 1; saturation reports use saturation_benchmark_version: 1; capacity planning uses capacity_version: 3.

Incomplete evidence is never silently promoted to a pass. Optional sources record ok, partial, unavailable, or skipped. Runtime, artifact, and benchmark reports preserve their respective boundaries so a deployment gate can distinguish missing evidence from a healthy result.

Prometheus and HTTP API

watchman serve exposes four bounded routes: /livez, /metrics, /healthz, and /api/v1/report. Authentication is required by default, even on loopback; a non-loopback listener also needs explicit remote-listen consent.

Prometheus metrics describe the observer as well as the node. They include collection-source coverage, GPU signals, and inference telemetry. Keep a remote listener behind an authenticated TLS boundary instead of using the observer as a public dashboard.