Runbook — Upgrade and Rollback

Before upgrade

1. Name the current and target full commits/releases; never upgrade to an unnamed “latest.”
2. Read upstream schema, protocol, and client-data changes.
3. Capture filesystem, configuration, database, extracted assets, and service state.
4. Verify the backup by restoring it into an isolated location.
5. Generate a new working tree; do not mutate the only known-good baseline.

Upgrade

1. Fetch the target from the preservation mirror and verify the object ID.
2. Apply the ordered, versioned local patch series.
3. Rebuild with the pinned target toolchain.
4. Apply database migrations once and record their exact identifiers.
5. Regenerate configuration and client-derived assets only when declared by the target baseline.
6. Run the full smoke-test ladder before widening network exposure.

Rollback

1. Stop the target services and preserve failure evidence.
2. Restore the previous server working copy and its matching database snapshot as one unit.
3. Restore the matching client-derived asset set and configuration.
4. Run the previous baseline smoke test.
5. Record whether rollback was complete; open a destructive-risk issue for any irreversible migration.

Never reuse a database that has been migrated forward with an older server unless that exact downgrade is documented and tested.