Admin Actions And Internal Auth
This page documents the operator-facing admin actions and the signed internal request contract used by control surfaces when they talk to services. It covers the actions that are routed today and the broader game-server admin action handler that is compiled ahead of its HTTP routing rollout.
Operator Surfaces
The control panel has three distinct admin paths:
- Recovery actions in the admin panel run directly against Redis or Postgres from the panel process: clear presence, clear login handoffs, clear service registry keys, and revoke login sessions.
- Account actions run through the login-server account admin repository from the panel process: refresh, create account, toggle admin, seed starter characters, and delete account.
- Game-server operations use signed internal HTTP endpoints when routed. Today the game server wires
POST /v1/internal/chat/system, session-lifecycle dead-letter inspect/requeue endpoints, and persistence-retry dead-letter inspect/requeue endpoints.
The broader world::AdminActionRequest handler in services/game-server/src/admin_actions.rs supports system chat, chat mute/unmute, kick, inspect, and encounter-calendar actions, but that handler is not routed as a public HTTP endpoint yet. Treat it as the server-side policy contract for the next routed admin action surface, not as an already callable panel route.
Dry Runs
Panel recovery actions support AdminActionExecutionMode::DryRun and AdminActionExecutionMode::Execute. Dry run formats the same target set the execute path would touch and explicitly reports that no Redis keys or database rows were changed. The UI exposes dry run separately from execute and keeps the destructive execute path behind typed confirmation.
Dry-run support currently belongs to the panel recovery actions. Account admin operations and the signed game-server system chat endpoint do not carry a dry-run field. The staged world::AdminActionRequest model also does not define dry-run semantics, so do not assume a rejected or unauthorized admin action is a dry run.
Recovery Actions
Redis recovery actions scan descriptor-owned key families and delete the matches:
- Presence cleanup targets
presence:accounts,presence:account:*, andpresence:character:*. - Handoff cleanup targets
login:handoff:*andlogin:handoff-consumed:*. - Registry cleanup targets
registry:service:*andregistry:instance:*.
Each report includes the number of deleted keys, matched keys, target count, and scan iteration count. A cancelled scan reports the cancellation with whatever partial work was already completed.
Session revocation updates login sessions with:
UPDATE login_sessions SET revoked_at = NOW() WHERE revoked_at IS NULL
That makes the operation naturally repeatable: a second execute should affect only sessions that were still active after the first run.
Account Operations
Login account admin is repository-backed and schema-gated. Before any operation, the repository verifies that login migrations have been recorded and that the accounts.is_admin column exists with the expected BOOLEAN NOT NULL DEFAULT FALSE shape.
Supported operations are:
Refresh: reload account summaries.Create: normalize username, reject blank passwords, hash the password, insert the account with the requested admin claim, and seed starter characters in the same transaction.SetAdmin: updateaccounts.is_adminfor an existing username.SeedStarterCharacters: seed default characters for an existing account.Delete: delete the account and rely on database cascades for related characters and sessions.
Create is all-or-nothing with starter character seeding. If seeding fails, the account insert rolls back. Delete is intentionally destructive; recover it from backup or a recreated account, not from a local undo path.
Game-Server Admin Operations
The currently routed game-server operation is system chat. SystemChatBroadcastRequest accepts a message and optional zone_id, rejects empty messages with 400 invalid_request, and returns the number of delivered sessions.
The staged AdminActionRequest handler additionally defines these server-owned actions:
SystemChat: broadcast a server-authored chat message to a zone or all zones.MuteChatandUnmuteChat: update server chat moderation state for a character.KickSession: remove an active session and send a typed disconnect with the supplied reason oradmin_kick.InspectSession: return online character, zone, and admin-claim facts for a session token.- Encounter-calendar inspect, announce, and refresh actions.
For staged admin actions, the server resolves operator claims from an active session token before authorization. Client-supplied operator fields are not trusted. If session lookup is missing or fails, the server forces operator.is_admin = false and rejects with operator_claim_untrusted. If the session is trusted but not admin, it rejects with operator_not_admin.
Audit Logs
Panel recovery actions append to control-panel-admin-audit.log under the configured workspace root. Each line includes timestamp, mode (dry-run or execute), action label, and a newline-collapsed result. If the action succeeds but audit writing fails, the panel returns the success text plus an Audit log failed warning. If the action fails, the failure is still audited when possible.
The staged game-server admin action path builds an AdminActionAuditRecord, emits an authoritative_audit event, and persists request and response payload JSON into admin_action_audit_log. Audit records include request id, operator account id, operator username, trusted admin flag, action kind, target, reason, accepted flag, failure reason, and creation time.
Account admin operations return refreshed account summaries to the panel, but they are not backed by the game-server admin action audit table. When account changes are part of an incident, preserve panel logs, database migration state, and the operator command record alongside any database backup or restore notes.
Signed Internal Requests
Internal service requests use these headers:
x-aether-internal-servicex-aether-internal-key-idx-aether-internal-timestampx-aether-internal-noncex-aether-internal-content-sha256x-aether-internal-signature
sign_request signs the shared secret, uppercase method, path and query, service identity, timestamp, nonce, and SHA-256 body hash. The timestamp must be within 300 seconds of the verifier clock. Signature comparison is length-checked and byte-compared without early success.
Known service identities are login-server, game-server, and control-panel. Game-server internal routes currently allow the control panel identity and use a 1 MiB body limit. Control-plane and game-server protected routes share the same verifier shape.
Use AETHER_CONTROL_AUTH_SECRET for the shared secret. The development default secret is rejected for remote control-plane use unless insecure local behavior is explicitly allowed. Keep AETHER_CONTROL_ALLOW_INSECURE_HTTP limited to local or LAN workflows where that tradeoff is intentional.
Nonces And Replay Protection
The verifier rejects missing or disallowed service identities, stale timestamps, content-hash mismatches, bad signatures, and replayed nonces. Nonces are registered in Redis with SET ... EX ... NX under:
{scope}:internal-request-nonce:{service}:{nonce}
The nonce scope separates services such as control-plane and game-server. A duplicate nonce returns unauthorized. A Redis failure while registering a nonce is a service error because replay protection could not be proven.
Partial Failure Recovery
Prefer dry run before destructive recovery actions, then execute only after the target list matches the incident. Redis cleanup and session revocation are repeatable, but a cancelled Redis scan can leave a partially cleaned keyspace; rerun dry run to inspect the remaining targets before executing again.
For account operations, use schema readiness errors as a stop sign. Start the login server, run migrations, or repair drift before retrying the operation. Account creation rolls back on starter-character seeding failure, while account delete depends on database backup or a new account creation path.
For signed system chat, an ambiguous timeout can mean the message was already delivered. Check game-server logs and player reports before retrying to avoid duplicate announcements.
For staged game-server admin actions, side effects execute before durable audit persistence. If the handler returns an error after executing the action, treat it as "possibly applied, audit failed." Inspect authoritative audit logs, admin_action_audit_log, and current runtime state before replaying a kick, mute, unmute, or system chat.
Session lifecycle dead letters are inspected and requeued through signed internal endpoints. Inspect defaults to a bounded page, and requeue requires a non-empty dead-letter id. Requeue only after confirming the failed lifecycle work is still desired for the affected session.