Session Handoff And Transfer
Session handoff moves an authenticated character from login allocation into a game-server realtime session. Zone transfer reuses the same handoff machinery when a live session must move to another zone or game-server instance.
State Owners
- Login server owns character selection, realm checks, initial spawn resolution, game-instance allocation requests, and handoff-token creation.
- Control-plane owns game-instance allocation, handoff records, handoff consumed markers, redirect records, transfer activation confirmation, service registry lookups, and Redis-backed presence coordination.
- Game server owns pending realtime sessions, active realtime sessions, admission reservations, transfer activation commit records, resume ledger writes, and live player/session mutation.
- Postgres owns durable character location and account/character data. The legacy
handoff_tokenstable can still participate in reconciliation when rows exist. - Redis owns short-lived handoff records, consumed handoff markers, presence, transfer activation confirmation markers, resume ledger records, and session lifecycle work queues.
The shared wire contract lives in crates/common-proto/src/coordination.rs. Runtime payload versions are explicit for handoff records and transfer activation payloads, and unknown versions are rejected.
Login Handoff
- Character select validates the bearer login session, realm availability, character ownership, and realm match.
- The login server resolves the selected character, start zone, placement, and account claims.
- It asks control-plane
assign/gamefor a zone-specific game instance, falling back to configured default game host only on allocation unavailability. - It creates a short-lived
handoff-{uuid}token. Login-created handoff tokens expire after 1 minute. - It builds a
HandoffCacheRecordwith account ID, trusted account claims, character ID, realm ID, target zone, target placement, target instance, and optional dungeon run ID. - It calls control-plane
handoffs/prepare. - The client receives the token and target endpoint in the gameplay handoff response.
Control-plane stores the handoff under a preferred reconciliation scope. Current scopes support exact instance targeting, placement-level lookup, and legacy port lookup so game servers can preflight and consume the same logical token during rollout or redirects.
Control-Plane Handoff Records
Prepared handoffs live in Redis. Preflight reads a token without consuming it. Consume uses an atomic take and then writes a consumed marker so replayed tokens return AlreadyConsumed instead of being accepted twice.
Important Redis key families include:
login:handoff:*for prepared handoff payloads;login:handoff-consumed:*for replay protection;transfer:activated:*for confirmed cross-instance transfer activation;- presence keys by account and character;
realtime:resume:{session_token}for resume ledger records;- session lifecycle journal queue, processing, and dead-letter keys.
Startup reconciliation can rebuild Redis handoff cache entries from Postgres handoff rows if those rows are populated, expire stale pending sessions, roll back expired transfers, repair presence, and refresh control-plane instance registration.
Game-Server Handshake
The target game server handles the handoff token through a preflight/consume handshake:
- Rate-limit the handoff-token attempt, require QUIC, and negotiate realtime compatibility.
- Call control-plane
handoffs/preflightfor this instance, placement, and port. - Reject
MissingorAlreadyConsumedresponses. - Validate the handoff payload version and target zone.
- Reserve local admission for the target zone.
- If the instance cannot admit the player, try control-plane allocation plus
handoffs/redirect. - Call control-plane
handoffs/consume. - On consume failure, roll back the admission reservation.
- Build the player snapshot or apply transfer activation.
- Register a pending realtime session and write presence.
Pending registration returns a session ticket and activation payload. The session becomes active when the realtime stream claims the ticket and promotes the pending entry into the active session directory. If presence write or transfer confirmation fails during acceptance, the server rolls back pending registration.
Pending, Active, Resume
The game server tracks sessions in three main states:
- admission reservations: short-lived capacity holds for a zone;
- pending realtime sessions: accepted handshakes waiting for realtime activation;
- active realtime sessions: live session actors indexed by token and zone.
Handshake registration commits the admission reservation, inserts a PendingRealtimeSession, and persists a resume ledger record. Realtime stream activation restores pending state from the resume ledger if needed, takes the pending entry, builds an ActiveRealtimeSession, persists the activated resume state, and inserts the handle into the active directory.
The resume ledger gives reconnect and recovery paths a durable snapshot. Disconnect cleanup is journaled through the session lifecycle queue so presence, resume-ledger persistence/deletion, and location cleanup can be retried or dead-lettered instead of being lost on a transient Redis/Postgres failure.
The session lifecycle journal exposes dead-letter inspection and requeue endpoints. Its worker recovers in-flight entries on startup, drains pending work, skips stale cleanup when a session is active again, and deletes the resume ledger after confirmed transfer activation.
In-Process Zone Transfer
An in-process transfer keeps the player on the same game-server instance.
- Trigger authority resolves the target zone and tile.
- The active session is mutated to relocate the player and update movement state.
- Character location is persisted to Postgres with a zone-transfer source.
- Dungeon run state is carried or hydrated when needed.
- The active session directory moves the session from the previous zone index to the next zone index.
- Presence is rewritten for the new zone and runtime profile.
Trigger authority rejects new trigger work while a session has a pending transfer, and movement authority drains queued movement input while transfer is pending. Treat DB location, active-session zone index, dungeon runtime state, and presence as one consistency set when debugging an in-process transfer.
Cross-Instance Zone Transfer
A cross-instance transfer creates a new handoff token with transfer activation metadata:
- The source game server snapshots account claims, character ID, realm ID, source zone, pending dungeon run, and target tile.
- It freezes the source session into a transfer-pending state and generates a transfer token.
- It asks control-plane allocation for a target instance and prepares a handoff record for that target.
- The handoff record includes
TransferActivation: source zone, target zone, target tile, and payload version. - The source active session stores the pending transfer token and target endpoint.
- The source tries to persist the active session to the resume ledger before returning the transfer packet.
- The client connects to the target game server with the transfer token.
- Target handshake consumes the handoff, applies transfer activation, persists the target character location, registers a pending session, writes presence, and confirms transfer activation through control-plane.
The target game server tracks activation commit records with phases:
PendingRegistered: admission was consumed and pending registration exists locally.Confirmed: control-plane transfer activation confirmation succeeded.RolledBack: local pending registration was removed after presence or confirmation failed.RecoveryRequired: admission or registration failed after a transfer commit attempt started.
These records make mixed-state failures visible instead of hiding them behind a generic handshake error. Expired or unconfirmed source transfers roll back source transfer state, refresh presence, persist the resume ledger, and can send transfer refresh packets.
Failure And Recovery
Check these state stores together when a player reports a failed handoff, resume, or transfer:
- Control-plane handoff state: token scope, TTL, consumed marker, and redirect target.
- Game-server admission: reservation exists, zone is available, server is not draining, and capacity is not exhausted.
- Pending session state: session ticket exists until realtime stream activation or rollback.
- Active session state: token appears in the active directory and the zone index matches the player snapshot.
- Postgres character location: zone/tile matches the last committed authority action.
- Redis presence: character/session points at the same instance, zone, and runtime profile as the server state.
- Transfer activation marker: confirmed for completed cross-instance transfers.
- Resume ledger: exists for reconnectable sessions and is deleted after confirmed transfer activation.
- Session lifecycle dead letters: failed cleanup or resume-ledger work may need requeue.
Persistence during transfer activation is durable-retry aware. If target location persistence fails, the game server queues a transfer-activation persistence retry when possible and returns a service-unavailable handshake response rather than silently accepting a mixed location.
Control-panel cleanup tools can clear stale presence, handoff, and registry keys, but use them after identifying the owner that stopped updating the state. Blind cleanup can hide the authority failure that caused the mismatch.