1 realtime timebases tick ordering and clock ownership
forgejo-actions edited this page 2026-07-15 12:43:13 +00:00

Realtime Timebases, Tick Ordering, And Clock Ownership

Purpose

Realtime packets carry several values that look like time or sequence numbers but have different owners, scopes, and comparison rules. This page is the active contract for choosing and consuming those values.

The central rule is: compare values only inside the domain and lifetime that owns them. Wall time is for absolute timestamps and diagnostics, zone ticks describe simulation progress, network time schedules client presentation, snapshot IDs order one session's replication stream, and movement sequences order one client's inputs.

Time And Ordering Domains

Domain Fields and types Producer and scope Correct use Incorrect use
Server wall-clock UTC server_time, ChatMessage.sent_at, protobuf Timestamp The authoritative server process, using UTC Human-readable logs, audit timestamps, cross-service timestamps, and operator correlation Snapshot ordering, interpolation, timeout durations, or datagram freshness
Server wall-clock milliseconds Server-produced now_ms, *_at_ms, *_until_ms, expires_at_ms, cooldown and replicated-motion millisecond values The authority that owns the gameplay state, normally from utc_now_ms() Expiry, cooldown, persisted absolute times, and authoritative action or motion state Replacing network_time_ms, ordering snapshots, or trusting an unresolved client proposal as authority
Zone simulation tick server_tick_id One hosted zone tick loop; it starts at 1 for that loop and advances once per tick Simulation chronology, tick cadence, replication scheduling, and rejecting older authoritative movement samples A globally unique ID, durable time, elapsed milliseconds, or per-session delivery order
Protocol monotonic time network_time_ms, started_at_network_time_ms The server derives it from the zone tick and tick interval; the client advances the last received anchor with a local monotonic Instant Snapshot buffer delay, reorder-grace expiry, movement interpolation and bounded extrapolation, delayed presentation, and cutscene playheads Calendar time, persistence timestamps, or independently deriving it from local wall time
Replication order snapshot_id, snapshot receipts, missing snapshot IDs One active session's repair buffer; live IDs start at 1 Deduplication, contiguous apply order, gap detection, targeted repair, and the acknowledgement frontier Simulation elapsed time, ordering another session, or assuming one snapshot is emitted per zone tick
Movement input order MoveInput.sequence, MoveInputBatch.first_sequence, acknowledged_sequence The client session; normal sequencing starts at 1 Reordering inside the negotiated window, stale/duplicate rejection, prediction replay, and movement acknowledgement Measuring time, comparing with snapshot IDs, or ordering non-movement actions
Request and event order Gameplay request IDs, trigger intent_sequence, authoritative event revisions, cutscene sequence_id The subsystem or session named by the field Idempotency, replay, intent ordering, or identifying a cutscene run Clock arithmetic or cross-domain ordering
Local monotonic duration Rust Instant, Tokio deadlines, elapsed durations The local client or server process only RTT, heartbeat age, timeout deadlines, fallback delay, reconnect delay, and performance measurement Serialization, persistence, or comparison between processes
Capture observation time timestamp_unix_ms in packet debug JSONL The capturing process Approximate correlation between capture files Proof of protocol causality when host clocks may differ

SceneEnvironmentState.time_of_day_hours is replicated world state, not a transport clock. Farming tick_id and last_growth_tick_id are domain scheduler identifiers, not server_tick_id. Their names do not permit cross-comparison.

Clock Ownership

The game server owns all authoritative clock anchors sent in live snapshots and heartbeats:

  1. The zone loop waits for the configured tick interval, increments its server_tick_id, and runs authoritative simulation.
  2. Movement inputs are drained in client sequence order during that simulation tick.
  3. AOI projection allocates a session snapshot_id only when that session emits a snapshot. An unchanged or rate-shaped session can therefore skip zone ticks without creating snapshot gaps.
  4. The snapshot carries the zone server_tick_id, the per-session snapshot_id, network_time_ms, server UTC, and an optional acknowledged movement sequence.
  5. A due heartbeat carries the same zone tick/time anchors but no replication snapshot ID.

The current game-server tick interval is 100 ms, and network_time_ms_for_tick uses saturating multiplication of the tick ID by that interval. This implementation relationship does not transfer ownership to clients. Consumers must use the transmitted network_time_ms; they must not hard-code 100 ms or reconstruct network time from a tick ID.

Initial world bootstrap is a deliberate exception. It is a full snapshot with snapshot_id = 0, server_tick_id = 0, and network_time_ms = 0. Snapshot ID zero is a bootstrap sentinel, not the first live replication ID. A client may apply it before the live snapshot floor advances and discards it if it arrives late.

Client Consumption

The client observes a server clock anchor from each heartbeat and state snapshot. It stores the last tick, UTC timestamp, and network time, then pairs network_time_ms with a local Instant. Estimated network time is:

received network_time_ms + local monotonic elapsed milliseconds

No wall-clock offset calculation is involved. A local system-clock adjustment therefore must not move interpolation or reorder deadlines.

State snapshots are staged in a map keyed by snapshot_id. The client discards IDs at or below its applied floor, buffers newer IDs, and applies them in numeric order. network_time_ms controls when a buffered snapshot is ready and when a missing-ID gap has outlived the negotiated reorder grace. If timing helpers have no clock estimate, they fail open to immediate application or gap expiry rather than inventing a wall-clock conversion.

Authoritative movement samples reject a sample whose server_tick_id does not advance. Their interpolation interval and sample age use network_time_ms. Pending local moves remain ordered by client movement sequence and are removed only when a movement correction or snapshot acknowledgement advances that sequence.

Ordering Example

Suppose zone tick 420 produces network_time_ms = 42000. Session A emits snapshot 37, session B emits snapshot 24, and A's snapshot acknowledges movement sequence 88.

  • 420 orders simulation samples inside the current zone-loop lifetime.
  • 42000 schedules presentation and snapshot timing on either client.
  • A compares 37 only with A's earlier and later snapshot IDs.
  • B compares 24 only with B's snapshot IDs.
  • A compares movement acknowledgement 88 only with A's movement inputs.
  • The UTC server_time explains when operators observed the tick but cannot override any of those order decisions.

If snapshot 39 arrives before 38, the client buffers by snapshot ID. It waits until 38 arrives or until the network-time reorder grace expires, then requests repair using snapshot IDs. It does not sort the packets by UTC or packet-capture timestamp.

Invariants

  • server_tick_id is monotonic only within its hosted zone-loop lifetime. Restarting or replacing that authority can reset the domain.
  • A live session's nonzero snapshot_id is monotonic and unique within that session's replication stream. Zero remains reserved for bootstrap.
  • Snapshot receipt acknowledgements never regress, never exceed the server's issued window, and list missing IDs in strictly increasing order above the acknowledgement.
  • network_time_ms is monotonic protocol time. Its transmitted value, not a locally assumed tick duration, is authoritative for presentation timing.
  • server_time and capture timestamps may move or skew without changing protocol order.
  • Movement sequences use ordinary increasing u32 comparisons and saturating increments. The current contract has no modular wrap-around semantics.
  • Saturation prevents arithmetic wrap; it does not create a new epoch or make repeated maximum values distinct.
  • Client-authored timing fields are prediction or intent until server validation and authoritative state resolve them.
  • Values from different sessions, zones, processes, or domains are not comparable unless a higher-level contract explicitly supplies a shared scope.

Compatibility Rules

clock_sync_v1 is stable, default-enabled, and required by the current compatibility policy. Existing field meanings are permanent inside this protocol generation.

  • Never reinterpret server_time, server_tick_id, network_time_ms, or snapshot_id to mean another domain.
  • Never renumber protobuf fields. Add a new tagged field for a new timebase or ordering key.
  • Ship receive-side support before a producer starts sending a new required field or relying on new semantics.
  • Use a new capability or protocol version when an older peer cannot safely ignore the change.
  • Keep bootstrap zero behavior and live nonzero ordering compatible while old and new peers coexist.
  • Treat a tick-interval change as a simulation and presentation change. Continue sending explicit network time and test clients across the transition.

Extension Workflow

Before adding a realtime time or sequence field, state its owner, unit, epoch or zero sentinel, monotonicity, reset boundary, overflow behavior, and allowed comparisons.

Then update the protocol as one change:

  1. Add the protobuf field with a fresh tag and document its unit in the owning world/wire type.
  2. Add or update the common-proto codec and RealtimeTimeSource/field-schema guidance.
  3. Set the value at the authoritative producer; do not synthesize authority in a transport adapter.
  4. Update every consumer's ordering, buffering, interpolation, persistence, and reset behavior.
  5. Add validation for client-authored values and explicit tests for zero, stale, duplicate, future, overflow, restart, and out-of-order cases that apply.
  6. Update the realtime registry and capability negotiation when packet availability or compatibility changes, then regenerate derived artifacts.

When registry metadata changes, verify it with:

cargo run -p aether-common-proto --bin aether-realtime-registry -- --check

Focused contract tests include:

cargo test -p aether-common-proto server_heartbeat_round_trips_through_typed_helper
cargo test -p aether-common-proto snapshot_timing_policy_gates_apply_and_gap_expiry
cargo test -p aether-game-server allocates_monotonic_snapshot_ids_from_one
cargo test -p aether apply_state_snapshot_datagram_orders_out_of_order_snapshots_before_release

Debugging And Packet Capture

Enable metadata-only capture before starting the normal client, server, or smoke process:

$env:AETHER_PACKET_DEBUG_CAPTURE_DIR = "artifacts/packet-capture"

After reproducing, select the timing-bearing packets:

Get-Content artifacts/packet-capture/*.jsonl |
  ForEach-Object { $_ | ConvertFrom-Json } |
  Where-Object { $_.packet -in @("server_state_snapshot", "server_heartbeat") } |
  Select-Object timestamp_unix_ms, executable, direction, transport, packet, message_id, payload_len

Capture records do not contain payload bytes, so they cannot reveal snapshot_id, server_tick_id, network_time_ms, or disconnect/advisory fields. Correlate them with client snapshot logs, server AOI logs, movement diagnostics, snapshot repair metrics, and the multiplayer diagnostics panel.

For an ordering incident, record all applicable values rather than one generic "tick": session token, zone and instance, snapshot_id, server_tick_id, network_time_ms, movement sequence/acknowledgement, full-sync flag, packet transport, and whether the sample came from bootstrap, heartbeat, datagram, or reliable repair.