Capture protocol

Protocol version 1 is the file/process boundary between a consuming development tool and the xpwebapi read-only capture worker. The installed JSON Schemas are the machine-readable contract; the models below are canonical.

Request fields

CaptureRequest contains exactly:

protocol_version, fixed integer 1;
capture_session_id and sortie_id;
correlation with campaign_id, route_profile_id, and scenario_id;
identity_readiness, currently kind="dataref_match", target aircraft FlyJSim Q4XP, and one or more identity refs;
discriminated transport, WebSocket or UDP;
sample_groups and capture refs;
retry policy;
nullable capture_limit_seconds and stop_file.

Each identity ref has id, path, declared_type, nullable encoding, rate_hz, operator, and expected_value. Each sample group has id, rate_hz, and nullable duration_seconds. Each capture ref has id, path, declared_type, availability, sample_group_id, and nullable encoding.

The retry object has initial_attempts, reconnect_attempts, backoff_seconds, backoff_max_seconds, subscription_timeout_seconds, aircraft_identity_timeout_seconds, first_values_timeout_seconds, max_disconnect_seconds, stale_after_seconds, poll_interval_seconds, and shutdown_timeout_seconds.

Identity subscription metadata, send, and feedback operations are always bounded by subscription_timeout_seconds. The separate aircraft_identity_timeout_seconds may be a positive number or null; null waits for matching aircraft identity until explicit stop, interruption, transport failure, or an owning reconnect deadline. UDP reports awaiting_first_identity_packet for the full finite or unbounded identity wait.

load_capture_request() reads bytes once. The exact same bytes are parsed and hashed, producing the request_sha256 passed explicitly into CaptureRunner. SourceProvenance is also resolved by the CLI and passed explicitly; the runner never fabricates either value or reads it from writer internals.

Event JSONL

Every row contains protocol_version, event, sequence, capture_session_id, sortie_id, timestamp_utc, and elapsed_seconds. Sequence starts at 1 and strictly increases. Elapsed time is non-decreasing, so multiple events may share a monotonic instant.

Event-specific fields are exact:

capture_started: request_sha256, correlation, transport, provenance.
transport_state: state, attempt, nullable reason, capabilities.
transport_ready: connection_generation, endpoint, read_only, package_version, nullable git_revision, nullable git_dirty, capabilities.
subscription_result: purpose, accepted_ref_ids, rejected, nullable request_id.
aircraft_ready: connection_generation, target_aircraft, identity_observations, required_ref_ids, optional_missing_ref_ids, ready_elapsed_seconds.
sample: sample_group_id, ref_id, path, declared_type, status, nullable value, nullable source_observed_elapsed_seconds, nullable source_age_seconds.
gap_started: reason, affected_ref_ids, gap_start_elapsed_seconds.
gap_ended: the gap-start fields plus gap_duration_seconds and skipped_slot_count.
retry: phase, attempt, maximum_attempts, delay_seconds, reason.
capture_stopped: termination, sample_count, gap_count, retry_count, preceding_sha256.
capture_failed: reason, the three counters, preceding_sha256.
capture_interrupted: signal, the three counters, preceding_sha256.

sample.status is sampled, missing, stale, disconnected, unsupported, or invalid. Only sampled has a non-null value. missing has both source-time fields null; other statuses may retain both source times. The terminal preceding_sha256 covers every complete row before the terminal row.

Status JSON

Every status document contains:

protocol_version, capture_session_id, sortie_id, state;
updated_at_utc, elapsed_seconds, events_path, request_sha256;
transport, transport_connection_state, connection_generation;
nullable transport_ready_at_utc and aircraft_ready_at_utc;
target_aircraft, identity_ref_count, matched_identity_ref_count;
required_capture_ref_count, observed_required_ref_count;
counters containing sample_count, gap_count, retry_count, accepted_ref_count, and rejected_ref_count;
attempt_phase, current_attempt, maximum_attempts, nullable reason.

Nonterminal states are starting, connecting, transport_ready, awaiting_aircraft, subscribing, awaiting_first_values, aircraft_ready, capturing, reconnecting, and finalizing. Every nonterminal state can enter finalizing for a pre-readiness requested stop or another clean stop. Only reconnecting has a non-null nonterminal reason.

Terminal status adds events_sha256 and events_size_bytes. complete has a reason of stop_file, capture_limit, requested, or groups_complete, exit_code=0, and clean_shutdown=true. failed has a diagnostic reason, exit_code=3, and clean_shutdown=false. interrupted has reason SIGINT or SIGTERM, exit_code=130, and a boolean clean_shutdown.

Terminal event/status publication is a two-phase event/status commit: durable terminal JSONL is the first commit point, and atomic terminal status replace is the second. A complete-but-unclean runner outcome has process exit 3. When the terminal event committed but status did not, JSONL is authoritative and status may remain finalizing; this is deliberate status residue, not permission to rewrite the event stream.

A full terminal-row write closes the event stream logically and prevents any further append, retry, or abandon mutation. The committed hash and size become available only after flush and fsync succeed. Flush/fsync failure leaves raw unproven evidence with finalizing status and a failed, unclean outcome; close failure after successful fsync is post-commit.

Signals and cleanup

CaptureInterruption retains the first SIGINT or SIGTERM identity across the CLI signal handler and runner. Programmatic stop sets only the stop event and is recorded as requested.

The runner passes one absolute cleanup deadline to the transport. WebSocket and UDP adapters assign per-stage transport shutdown budgets to unsubscribe, listener stop, channel close, and HTTP-session or beacon-owner close. All stages are attempted while the first cleanup failure remains authoritative.

Transport safety

The transport factory always constructs read-only clients. Capture adapters expose only open, subscribe, connected, liveness_state, and close; no general API or raw socket escapes. WebSocket capture sends only subscribe/unsubscribe value messages. UDP sends only validated RREF\0 packets. Reservation collision handling uses exclusive creation and no cross-path rollback so a path replacement race cannot cause the worker to delete another process's file.

Installed JSON Schemas

The four protocol-v1 schemas ship in the xpwebapi.schemas package:

capture-event-v1.schema.json
capture-request-v1.schema.json
capture-status-v1.schema.json
capture-version-v1.schema.json

from importlib.resources import files

schema_text = (
    files("xpwebapi.schemas")
    .joinpath("capture-request-v1.schema.json")
    .read_text(encoding="utf-8")
)

API declarations

capture_protocol

Strict version-1 request protocol for the read-only capture worker.

Classes:

AircraftIdentityRef –

One DataRef assertion proving the loaded aircraft is Q4XP.
CaptureCorrelation –

Cross-repository campaign, route, and scenario identifiers.
CaptureRef –

One read-only DataRef captured in a sample group.
CaptureRequest –

Canonical strict version-1 capture request.
CaptureRetryPolicy –

Bounded retry and lifecycle timeout policy.
CaptureSampleGroup –

Fixed-cadence sampling group.
DatarefMatchIdentityReadiness –

Q4XP identity readiness established from matching DataRefs.
LoadedCaptureRequest –

Validated request paired with the SHA-256 of its exact source bytes.
StrictModel –

Base for immutable protocol models without input coercion.
UdpCaptureConfig –

Bounded beacon-discovered UDP capture settings.
WebsocketCaptureConfig –

Bounded WebSocket capture connection settings.

Functions:

load_capture_request –

Read, decode, validate, and hash one exact request byte buffer.
resolve_stop_file –

Resolve the request/CLI stop-file contract and reject pre-existing stops.

AircraftIdentityRef


              flowchart TD
              xpwebapi.capture_protocol.AircraftIdentityRef[AircraftIdentityRef]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.AircraftIdentityRef
                


              click xpwebapi.capture_protocol.AircraftIdentityRef href "" "xpwebapi.capture_protocol.AircraftIdentityRef"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

One DataRef assertion proving the loaded aircraft is Q4XP.

CaptureCorrelation


              flowchart TD
              xpwebapi.capture_protocol.CaptureCorrelation[CaptureCorrelation]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.CaptureCorrelation
                


              click xpwebapi.capture_protocol.CaptureCorrelation href "" "xpwebapi.capture_protocol.CaptureCorrelation"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Cross-repository campaign, route, and scenario identifiers.

CaptureRef


              flowchart TD
              xpwebapi.capture_protocol.CaptureRef[CaptureRef]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.CaptureRef
                


              click xpwebapi.capture_protocol.CaptureRef href "" "xpwebapi.capture_protocol.CaptureRef"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

One read-only DataRef captured in a sample group.

CaptureRequest


              flowchart TD
              xpwebapi.capture_protocol.CaptureRequest[CaptureRequest]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.CaptureRequest
                


              click xpwebapi.capture_protocol.CaptureRequest href "" "xpwebapi.capture_protocol.CaptureRequest"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Canonical strict version-1 capture request.

CaptureRetryPolicy


              flowchart TD
              xpwebapi.capture_protocol.CaptureRetryPolicy[CaptureRetryPolicy]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.CaptureRetryPolicy
                


              click xpwebapi.capture_protocol.CaptureRetryPolicy href "" "xpwebapi.capture_protocol.CaptureRetryPolicy"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Bounded retry and lifecycle timeout policy.

CaptureSampleGroup


              flowchart TD
              xpwebapi.capture_protocol.CaptureSampleGroup[CaptureSampleGroup]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.CaptureSampleGroup
                


              click xpwebapi.capture_protocol.CaptureSampleGroup href "" "xpwebapi.capture_protocol.CaptureSampleGroup"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Fixed-cadence sampling group.

DatarefMatchIdentityReadiness


              flowchart TD
              xpwebapi.capture_protocol.DatarefMatchIdentityReadiness[DatarefMatchIdentityReadiness]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.DatarefMatchIdentityReadiness
                


              click xpwebapi.capture_protocol.DatarefMatchIdentityReadiness href "" "xpwebapi.capture_protocol.DatarefMatchIdentityReadiness"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Q4XP identity readiness established from matching DataRefs.

LoadedCaptureRequest `dataclass`

LoadedCaptureRequest(request: CaptureRequest, request_sha256: str)

Validated request paired with the SHA-256 of its exact source bytes.

StrictModel


              flowchart TD
              xpwebapi.capture_protocol.StrictModel[StrictModel]

              

              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Base for immutable protocol models without input coercion.

UdpCaptureConfig


              flowchart TD
              xpwebapi.capture_protocol.UdpCaptureConfig[UdpCaptureConfig]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.UdpCaptureConfig
                


              click xpwebapi.capture_protocol.UdpCaptureConfig href "" "xpwebapi.capture_protocol.UdpCaptureConfig"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Bounded beacon-discovered UDP capture settings.

WebsocketCaptureConfig


              flowchart TD
              xpwebapi.capture_protocol.WebsocketCaptureConfig[WebsocketCaptureConfig]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_protocol.WebsocketCaptureConfig
                


              click xpwebapi.capture_protocol.WebsocketCaptureConfig href "" "xpwebapi.capture_protocol.WebsocketCaptureConfig"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Bounded WebSocket capture connection settings.

load_capture_request

load_capture_request(path: Path) -> LoadedCaptureRequest

Read, decode, validate, and hash one exact request byte buffer.

resolve_stop_file

resolve_stop_file(
    request_path: Path, request_value: Path | None, cli_value: Path | None
) -> Path | None

Resolve the request/CLI stop-file contract and reject pre-existing stops.

capture_events

Strict event, status, and version models for capture protocol version 1.

Classes:

CaptureCounters –

Cumulative capture lifecycle counters.
CaptureEventIdentity –

Immutable identity copied into every event envelope.
CaptureEventKind –

All protocol-v1 JSONL event discriminators.
CaptureStatusState –

All protocol-v1 lifecycle states.
EventEnvelope –

Fields common to every JSONL event row.
IdentityObservation –

One observation used to establish Q4XP aircraft readiness.
SampleStatus –

All protocol-v1 sample validity states.
SourceProvenance –

Source package and checkout identity embedded in capture evidence.
StatusBase –

Fields common to terminal and nonterminal polling documents.
TransportCapabilities –

Read-only values supported by one transport endpoint.
VersionJsonDocument –

Machine-readable worker version and source provenance.

CaptureCounters


              flowchart TD
              xpwebapi.capture_events.CaptureCounters[CaptureCounters]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.CaptureCounters
                


              click xpwebapi.capture_events.CaptureCounters href "" "xpwebapi.capture_events.CaptureCounters"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Cumulative capture lifecycle counters.

CaptureEventIdentity


              flowchart TD
              xpwebapi.capture_events.CaptureEventIdentity[CaptureEventIdentity]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.CaptureEventIdentity
                


              click xpwebapi.capture_events.CaptureEventIdentity href "" "xpwebapi.capture_events.CaptureEventIdentity"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Immutable identity copied into every event envelope.

CaptureEventKind


              flowchart TD
              xpwebapi.capture_events.CaptureEventKind[CaptureEventKind]

              

              click xpwebapi.capture_events.CaptureEventKind href "" "xpwebapi.capture_events.CaptureEventKind"

All protocol-v1 JSONL event discriminators.

CaptureStatusState


              flowchart TD
              xpwebapi.capture_events.CaptureStatusState[CaptureStatusState]

              

              click xpwebapi.capture_events.CaptureStatusState href "" "xpwebapi.capture_events.CaptureStatusState"

All protocol-v1 lifecycle states.

EventEnvelope


              flowchart TD
              xpwebapi.capture_events.EventEnvelope[EventEnvelope]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.EventEnvelope
                


              click xpwebapi.capture_events.EventEnvelope href "" "xpwebapi.capture_events.EventEnvelope"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Fields common to every JSONL event row.

IdentityObservation


              flowchart TD
              xpwebapi.capture_events.IdentityObservation[IdentityObservation]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.IdentityObservation
                


              click xpwebapi.capture_events.IdentityObservation href "" "xpwebapi.capture_events.IdentityObservation"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

One observation used to establish Q4XP aircraft readiness.

SampleStatus


              flowchart TD
              xpwebapi.capture_events.SampleStatus[SampleStatus]

              

              click xpwebapi.capture_events.SampleStatus href "" "xpwebapi.capture_events.SampleStatus"

All protocol-v1 sample validity states.

SourceProvenance


              flowchart TD
              xpwebapi.capture_events.SourceProvenance[SourceProvenance]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.SourceProvenance
                


              click xpwebapi.capture_events.SourceProvenance href "" "xpwebapi.capture_events.SourceProvenance"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Source package and checkout identity embedded in capture evidence.

StatusBase


              flowchart TD
              xpwebapi.capture_events.StatusBase[StatusBase]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.StatusBase
                


              click xpwebapi.capture_events.StatusBase href "" "xpwebapi.capture_events.StatusBase"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Fields common to terminal and nonterminal polling documents.

TransportCapabilities


              flowchart TD
              xpwebapi.capture_events.TransportCapabilities[TransportCapabilities]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.TransportCapabilities
                


              click xpwebapi.capture_events.TransportCapabilities href "" "xpwebapi.capture_events.TransportCapabilities"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Read-only values supported by one transport endpoint.

VersionJsonDocument


              flowchart TD
              xpwebapi.capture_events.VersionJsonDocument[VersionJsonDocument]
              xpwebapi.capture_protocol.StrictModel[StrictModel]

                              xpwebapi.capture_protocol.StrictModel --> xpwebapi.capture_events.VersionJsonDocument
                


              click xpwebapi.capture_events.VersionJsonDocument href "" "xpwebapi.capture_events.VersionJsonDocument"
              click xpwebapi.capture_protocol.StrictModel href "" "xpwebapi.capture_protocol.StrictModel"

Machine-readable worker version and source provenance.

capture_transport

Deadline-bounded, read-only transport adapters for capture workers.

Classes:

CaptureTransport –

Narrow network surface exposed to the capture runner.
Observation –

One transport callback mapped back to its request ref id.
SubscriptionResult –

Accepted and rejected refs from one purpose-specific subscription.

Functions:

create_capture_transport –

Create a fresh transport and client owner for one connection generation.

CaptureTransport


              flowchart TD
              xpwebapi.capture_transport.CaptureTransport[CaptureTransport]

              

              click xpwebapi.capture_transport.CaptureTransport href "" "xpwebapi.capture_transport.CaptureTransport"

Narrow network surface exposed to the capture runner.

Methods:

arm_identity_wait –

Record the full identity-readiness window after bounded subscription.

arm_identity_wait

arm_identity_wait(deadline: float | None) -> None

Record the full identity-readiness window after bounded subscription.

Observation `dataclass`

Observation(ref_id: str, path: str, value: object, observed_monotonic: float)

One transport callback mapped back to its request ref id.

SubscriptionResult `dataclass`

SubscriptionResult(
    purpose: Purpose,
    accepted_ref_ids: tuple[str, ...],
    rejected: Mapping[str, str],
    request_id: int | None,
)

Accepted and rejected refs from one purpose-specific subscription.

create_capture_transport

create_capture_transport(
    request: CaptureRequest,
    *,
    client_factory: ClientFactory | None = None,
    beacon_factory: ClientFactory | None = None,
    clock: Clock = monotonic,
) -> CaptureTransport

Create a fresh transport and client owner for one connection generation.

Capture protocol

Request fields

Event JSONL

Status JSON

Signals and cleanup

Transport safety

Installed JSON Schemas

API declarations

capture_protocol

AircraftIdentityRef

CaptureCorrelation

CaptureRef

CaptureRequest

CaptureRetryPolicy

CaptureSampleGroup

DatarefMatchIdentityReadiness

LoadedCaptureRequest dataclass

StrictModel

UdpCaptureConfig

WebsocketCaptureConfig

load_capture_request

resolve_stop_file

capture_events

CaptureCounters

CaptureEventIdentity

CaptureEventKind

CaptureStatusState

EventEnvelope

IdentityObservation

SampleStatus

SourceProvenance

StatusBase

TransportCapabilities

VersionJsonDocument

capture_transport

CaptureTransport

arm_identity_wait

Observation dataclass

SubscriptionResult dataclass

create_capture_transport

LoadedCaptureRequest `dataclass`

Observation `dataclass`

SubscriptionResult `dataclass`