Producer observability

Purpose

Producer observability is a projection over the scheduler-neutral producer registry. It answers how live execution evidence can be observed without changing what a producer means, where it runs, or which repository owns its semantics.

The semantic authority remains:

• estate/producers.yaml and estate/producers.d/*.yaml for producer identity, authority, scheduler declaration, outputs and freshness meaning;
• evidence:producer-run-receipt@1 for observed run state;
• participating repositories for their own runtime behavior and domain semantics.

estate/producer-observability.yaml adds only the observation strategy.

Observation modes

Mode	Meaning	Typical next move
github_workflow	Workflow outcome is sufficient for the bounded sentinel meaning.	Central observer may reconstruct a receipt without changing the producer repository.
semantic_receipt	Scheduler success is weaker than the producer's real artifact/terminal semantics.	Add a small producer-local receipt emitter from existing evidence.
translated_manifest	Existing run/release manifests already carry most required semantics.	Add a deterministic translator into producer-run-receipt@1.
local_receipt	Manual/systemd/local command needs a small completion wrapper.	Emit a metadata-only receipt at the command boundary.
deferred_external	Producer is governed but its external scheduler cannot yet be reconciled cheaply.	Keep it visible and explicitly unobserved rather than inventing health.

These modes describe observation cost, not producer maturity or authority.

Staleness

stale_after is an optional operational observation threshold. It is deliberately separate from freshness.target.

Rules:

• manual and event-driven producers do not become stale merely because they have not run recently;
• only use stale_after when a meaningful wall-clock expectation exists;
• a stale observation never changes producer authority or lifecycle;
• unknown or deferred observation remains explicit.

Privacy

Observability is metadata-only by default.

Receipts may contain identities, timestamps, stage states, output types/refs, warnings and evidence references. They must not embed private accounting payloads, private documents, chat histories, research datasets, API secrets or other source content merely to make a dashboard convenient.

local_only is available for future producers whose observation metadata itself must not leave the host.

Deterministic projection

Run:

make producer-observability-generate
make producer-observability-check
make producer-observability-summary

The compiler joins the semantic producer registry with the observability policy and writes:

generated/producer-observability.compiled.json

The compiled projection is intended as the stable handoff to an external observability backend. It includes producer identity, owner, authority, scheduler, freshness, outputs and the observation strategy, but it contains no live run health.

make check verifies that:

1. every governed producer has exactly one observability strategy;
2. no unknown producer appears in the observability policy;
3. observation modes are compatible with scheduler type;
4. manual/event-driven producers do not acquire stale clocks implicitly;
5. the generated projection is current.

Control-plane boundary

This repository may compile observation policy and reconcile metadata. It must not become a cross-repository runtime orchestrator.

The intended architecture is:

producer registry + observability policy
                 |
                 v
      compiled producer projection

producer runtime / scheduler
                 |
          run observation
                 |
                 v
    producer-run-receipt@1
                 |
                 v
       observability backend

A future GCP projection may consume the compiled registry and receipts, but GCP remains an operational/UX projection rather than the semantic authority.

Initial retrofit queue

The first compiled census classifies the current producer surface into five implementation queues:

• observer_ready: zero-touch central observation is plausible;
• translator_needed: map existing manifests to the receipt contract;
• emitter_needed: add a small semantically-aware receipt at the producer boundary;
• wrapper_needed: add a local/manual completion wrapper;
• deferred: keep visible but do not block v1.

The queue is generated from current registry evidence; it is not permission to modify participating repositories automatically.

GitHub zero-touch observer

Producer rows classified as github_workflow can be reconciled without changing the participating repository.

The operator command is:

make producer-github-observe

It requires an authenticated GitHub CLI capable of reading the declared repositories and workflows. It is intentionally excluded from ordinary CI network sensing.

The observer:

1. reads only rows whose observation_mode is github_workflow;
2. resolves the repository and workflow path from the declared scheduler ref;
3. reads the latest completed GitHub Actions run;
4. converts only completed run metadata into producer-run-receipt@1;
5. writes immutable receipts under artifacts/producer-receipts/github/;
6. records query/unobserved states in a scan report rather than inventing a run.

Receipt files are ignored by Git. They are an operator-local spool for later reconciliation/export; GitHub remains the evidence source.

For these bounded sentinels a successful workflow supports healthy execution and a valid declared run-evidence output. A failed workflow supports failed execution/artifact state. Cancelled, neutral or skipped runs remain degraded or unknown rather than being promoted to success.

An explicit authority_state=hold is stronger than a green workflow: execution may be healthy, but artifact health remains unknown. This keeps the legacy Ministry Economy updater observable without treating successful execution as proof of trustworthy/current data.

The offline test:

make producer-github-observer-test

covers all nine currently observer-ready producers, success/failure/hold semantics, and byte-identical idempotent receipt writes.

Manifest to receipt translators

The translated_manifest mode is reserved for producers whose existing generated evidence already carries enough run or artifact semantics to support a receipt without creating a second lifecycle system.

The operator-side translator is:

python3 scripts/producer_manifest_translate.py \
  --producer <producer-id> \
  --manifest /path/to/manifest.json \
  [--aux /path/to/secondary-run-record.json] \
  [--started-at 2026-10-04T12:00:00Z] \
  [--finished-at 2026-10-04T12:05:00Z]

The translator never executes a participating repository. It reads only explicitly supplied JSON evidence and reconstructs a fresh producer-run-receipt@1 from allowlisted metadata.

Current adapters

The repository-evidence pass reduced the original eight candidates to five genuine manifest translators:

• producer.event.media-youtube-monitor from media_materialization_run.m1.v1;
• producer.event.welfare-predictive-release from the predictive welfare research-artifact-manifest/v1;
• producer.manual.accounting-report-pack from accounting_report_catalog.v1;
• producer.manual.kb-evidence-selector from the KB Artifacts selection manifest;
• producer.manual.knowledge-inspect-speech from Knowledge Inspect v2 run manifests, optionally paired with its run record.

Three rows were deliberately reclassified after inspecting repository truth:

• People Intel is local_receipt: the current repository does not implement the run-manifest seam previously declared by the estate projection.
• Estate Movement is local_receipt: its manifest describes the observed estate window, not execution start/end.
• Real EPH acceptance is semantic_receipt: its truth spans GitHub scheduler outcome, exact source evidence, four immutable science bundles and comparison outputs. Treating one manifest as the run would overclaim.

Execution timestamps

A translator may use timestamps already present in the source evidence. Otherwise the operator must provide --started-at and/or --finished-at. The translator does not use filesystem mtime or the observation window as a substitute for execution lifecycle.

Privacy boundary

Translation is allowlist-based, not filter-based.

Adapters do not copy arbitrary source-manifest fields and never copy:

• document or transcript bodies;
• accounting values;
• query text or selected evidence bodies;
• email/message content;
• dataset rows;
• credentials or secret-like fields;
• local absolute paths.

Accounting catalog translation proves catalog/report-bundle materialization only. Professional-pack and drilldown outputs remain unknown unless their own evidence is supplied; aggregate artifact state therefore remains degraded rather than being promoted from a catalog alone.

Run the offline privacy/adapter proof with:

make producer-manifest-translator-test

Local receipt wrapper

local_receipt producers use one generic wrapper around an explicitly supplied command. The wrapper is deliberately not a registry-driven orchestrator: it never discovers or chooses commands on behalf of the caller.

Current compiled wrapper classes:

• 4 command + artifact_evidence=none: execution completion is useful, but artifact health remains unknown unless the caller supplies checks;
• 13 command + artifact_evidence=required: execution success alone must not be promoted to artifact health;
• 2 systemd_command + artifact_evidence=required: the wrapper observes ExecStart execution but does not claim timer/scheduler health.

Run:

python3 scripts/producer_local_receipt.py \
  --producer <producer-id> \
  [--cwd <repo-or-working-dir>] \
  [--evidence-exists <path>] \
  [--evidence-nonempty <path>] \
  [--evidence-manifest <path>] \
  [--evidence-json '<path>#/json/pointer=<json-literal>'] \
  [--evidence-ref <semantic-ref>] \
  -- <existing-command> [args...]

Semantics

The wrapped command's exit code is preserved by default.

• command exit 0 + no evidence checks -> execution healthy, artifact unknown;
• command exit 0 + all supplied checks pass -> artifact healthy, outputs valid;
• command exit 0 + any supplied check fails -> execution healthy, artifact failed, outputs invalid;
• command exit nonzero -> execution/artifact failed for that run.

For producers whose policy says artifact_evidence=required, omitting evidence checks is legal but cannot produce a healthy artifact assertion.

--enforce-evidence is opt-in. It returns exit 4 when the wrapped command succeeds but supplied evidence checks fail. Without that flag, observability does not change an otherwise successful producer exit code.

Evidence checks

Checks are intentionally small and declarative:

• --evidence-exists: path exists;
• --evidence-nonempty: file has bytes or directory has at least one entry;
• --evidence-manifest: non-empty JSON object;
• --evidence-json: JSON Pointer equality against an expected JSON scalar;
• --evidence-ref: retain an already-governed semantic ref.

Local paths are used only to perform the check. The receipt stores either the semantic ref supplied by the caller or a content-hash/check ref. Absolute filesystem paths are never projected into the receipt.

Privacy

The wrapper does not record:

• the command or arguments;
• cwd;
• environment variables;
• source evidence bodies;
• local absolute paths.

This prevents API keys, private prompts, accounting values, research rows or other command-line/source payload from entering the future GCP observability plane accidentally.

systemd boundary

For systemd_user and systemd_system producers the wrapper reports scheduler_state=unknown. A successful ExecStart proves execution, not that the timer itself is enabled, punctual or otherwise healthy.

The wrapper can be placed around an existing ExecStart without changing the underlying producer command. Installation of user units remains the owning repository/operator's responsibility; projects does not install, enable, disable or restart systemd units.

Boundary hygiene

The Stage 1D evidence pass also aligned four previously conceptual boundaries to runnable commands:

• Argentina Geography -> make product-fixture;
• bounded intelligence pack -> make intelligence-tse;
• Paper KB corpus -> make corpus-parse-validate;
• Paper KB projections -> make export-review-catalog-records.

The two Paper KB composite targets are thin aliases over the pre-existing operations and do not change paper/corpus semantics.

Run the offline proof with:

make producer-local-receipt-test

The tests cover the complete 19-producer local-receipt classification, execution-only behavior, evidence-required behavior, successful and failed evidence, exit-code preservation, optional fail-closed evidence enforcement, systemd scheduler-state conservatism and receipt privacy.

Gate A readiness

The accepted convergence state is recorded in docs/observability/gate-a-readiness.md.

Current decision: GATE A = PASS / READY FOR OBS-G1.

The final pre-GCP authority, transport, privacy and rollout boundary is docs/observability/gcp-handoff.md.