MMO Preservation

Local, auditable tooling for reconstructing legacy MMO clients and servers. Each game or component gets its own Git repository under https://gitty.ryuum3gum1n.de/MMO-Preservation; this repository contains the reusable pipeline.

The pipeline deliberately combines conventional decompilers with a local code model. Ghidra, ILSpy, or CFR recovers the evidence-backed structure first. The model then proposes readable names, types, and C/C++ candidates while recording its confidence, assumptions, and suspected bugs. Generated source is never labeled as original source.

The unattended campaign design is inspired by swstegall/decomp-agents: persistent atomic claims, bounded attempts, isolated per-function work, independent grading, and crash recovery. This implementation is original and adapted to Ghidra JSONL plus local Ollama models; it does not copy the upstream AGPL implementation or its meteor-decomp/Claude-specific runner.

For native binaries, the pinned Ghidra MCP integration also lets the local model investigate call graphs, xrefs, strings, P-code, memory, and individual functions interactively. The integration exposes a read-only allowlist and writes an append-only JSONL provenance journal for every session.

flowchart LR
    A["Original artifact"] --> B["Hash + identify"]
    B --> C{"Artifact type"}
    C -->|"Native PE / ELF / Mach-O"| D["Ghidra headless"]
    C -->|".NET"| E["ILSpy"]
    C -->|"JVM"| F["CFR"]
    D --> G["Pseudocode + disassembly"]
    G --> N["Evidence facts + Win32 type/symbol closure"]
    N --> R{"Deterministic recognition"}
    R -->|"stub / thunk / wrapper"| W["Mechanical wrapper fast path"]
    R -->|"bounded function"| H["Skeleton pass, then readability skin"]
    R -->|"large / disassembly only"| S["Hierarchical slice + synthesis"]
    W --> I["Candidate + provenance"]
    H --> I
    S --> I
    I --> M["Parallel Clang ABI + normalized object matching"]
    M --> P["Stateful, relocation-aware P-code oracle"]
    P --> X["Independent isolated-byte oracle (Triton / angr / bounded fallback)"]
    X -->|"counterexample"| H
    X --> J["Consensus-gated source trees"]
    E --> L["Deterministic managed source"]
    F --> L
    L --> J
    J --> K["Dependency modules + unresolved-symbol ledger"]
    K --> T["Protocol, server integration + human tests"]
    T --> Q["Verified local corpus + metrics dashboard"]

Recommended interactive 4090 model

Use qwen2.5-coder:7b through Ollama. Its Q4 package is about 4.7 GB and this pipeline caps it at a 16,384-token context with 2,048 output tokens. That leaves enough GPU headroom for Windows and other applications instead of treating "weights fit in VRAM" as sufficient.

qwen2.5-coder:14b is an optional higher-quality pass when the GPU is otherwise idle. qwen3-coder:30b is opt-in only: its 18-19 GB weights plus KV cache and CUDA workspace can exhaust a 24 GB card or spill heavily into system memory when anything else is using the GPU.

Do not use Qwen3-Coder-Next as the single-GPU default. Its official Q4 package is roughly 48 GB, so an 80B model gains quality at the cost of substantial CPU/RAM offload. Specialized models such as LLM4Decompile are useful experiments for Linux x86-64 functions, but their supported binary formats are too narrow for a general MMO client/server pipeline. See docs/MODELS.md for the selection and future fine-tuning plan.

Unattended single-GPU campaigns

For a 4090, use several specialized roles sequentially rather than loading several models at once:

• qwen2.5-coder:3b proposes small, straightforward functions;
• qwen2.5-coder:7b handles medium functions and bounded escalation;
• codegemma:7b-instruct-v1.1-q4_0 independently compares each candidate with Ghidra pseudocode and disassembly.

CodeGemma is deliberately the reviewer, not the primary decompiler: its instruction model offers an independent model family, while its 8K context is less suitable than Qwen's 32K context for larger reconstruction prompts. The runner loads one role at a time and explicitly unloads it before switching, which avoids the operating-system stalls caused by oversized or concurrent models.

Install the two additional role models:

ollama pull qwen2.5-coder:3b
ollama pull codegemma:7b-instruct-v1.1-q4_0

Preview classification without invoking a model, run a bounded smoke batch, then leave the queue running:

.\preserve.ps1 grind "..\example-mmo-client" --dry-run
.\preserve.ps1 grind "..\example-mmo-client" --max-tasks 10
.\preserve.ps1 grind "..\example-mmo-client"

grind stores its local SQLite/WAL queue under analysis/ai/, checkpoints every function, recovers interrupted refinement, slicing, grading, and review claims after a reboot, and never retries a function indefinitely. A process lock prevents concurrent runners from duplicating work. Each model-produced function uses two constrained requests: a behavioral skeleton preserves raw control flow, calls, constants, offsets, and ABI evidence; a readability skin may improve its presentation without changing those properties. Both remain hypotheses until the normal mechanical and behavioral gates pass. Small, medium, and escalation tiers default to 4,500, 9,000, and 12,000 input characters. Larger functions and functions with disassembly but no pseudocode are split into resumable 4,000-character evidence slices. The escalation model records each slice before a bounded synthesis pass, avoiding the giant prompt and KV-cache spike that stalled the workstation. Use --filter, --max-tasks, --max-attempts, --slice-chars, and the model/size flags to tune a campaign. --no-hierarchical restores the older deferred_large behavior. When an integration slice manifest is present, bounded lanes default to a configurable 70-percent active-slice, 20-percent shared-function, and 10-percent exploration split. Missing callees that unlock more slice callers rank first. The evidence DAG selectively expires stale grade, review, and differential receipts after source, type, callee, or tool hashes change.

The terminal states distinguish mechanically verified candidates from review failures, uncertainty, exhausted model attempts, and explicit blockers. A model-review pass is still only an automated screen. “Coverage complete” means that the queue has no runnable entries; it never claims that the recovered source is proven equivalent. PE imports and ordinal/IAT stubs, transfer thunks, MSVC exception funclets, and recognized MSVC runtime helpers are retained in the evidence and call graph as cataloged_non_model; they never enter an LLM lane. This avoids generating duplicate source for compiler bookkeeping while keeping its addresses and dependency relationships auditable. Projects may record source-hash-pinned maintainer decisions in analysis/ai/human-review.json; human_rejected and human_screened always override an automated verdict.

For continuous operation, supervise runs indexing, bounded reconstruction, native grading, fail-closed differential tests, source assembly, health checks, and optional scoped Git checkpoints in a loop:

.\preserve.ps1 supervise "..\example-mmo-client" `
  --grade-workers 4 --git-checkpoint --git-push
.\preserve.ps1 status "..\example-mmo-client"
.\scripts\install-supervisor-task.ps1 `
  -Project "..\example-mmo-client" -GitCheckpoint -GitPush

The scheduled task runs its PowerShell host in the background and restarts after login/reboot. It loads only one small model role at a time, while native compiler workers run in parallel. It pauses for low disk space or a configured GPU temperature or VRAM ceiling and refuses to include pre-staged human changes in an automated commit. Deterministic, model, slice, grade, and review work have independent per-cycle quotas, so a large cheap backlog cannot starve model repair or validation. The heartbeat records stage age and structured progress. A parent watchdog isolates the restart-safe loop in a disposable process, terminates its complete process tree after a no-progress or absolute-stage deadline, and applies bounded per-stage retries before opening a visible circuit. The Windows task uses both startup and logon triggers, restart retries, and IgnoreNew instance handling. Each healthy cycle also refreshes the mechanical, exact, and differential assemblies, dependency modules, local training corpus, and observability report. See docs/AUTONOMOUS.md for the generated evidence and acceptance rules.

Native Rust monitor

Open the native dashboard to inspect the current stage, heartbeat, per-lane backlog and ETA, active-slice progress, quality gates, machine health, watchdog history, and structured activity:

.\scripts\start-decomp-monitor.ps1

The monitor is read-only apart from explicit Start and confirmed Stop controls for the configured Windows scheduled task. Start launches the supervisor in the background and leaves an existing instance alone. Its Rust crate forbids unsafe code and never writes to campaign SQLite, recovered source, evidence, or Git. See gui/decomp-monitor/README.md for configuration and ETA semantics.

Each producer attempt is timed and tied to its exact candidate hash. Later mechanical, review, exact, and differential verdicts update that append-only attempt ledger. After a model has enough completed samples, untouched tasks may be reassigned using verified promotion value per measured GPU-second; before that threshold the conservative 3B/7B size roles remain unchanged. Counterexample-driven retries always retain the escalation model.

Prerequisites

• Python 3.11 or newer
• Ollama for Windows
• Native code: Ghidra and the JDK version its current release requires
• Native validation: LLVM Clang and llvm-objdump (the Windows LLVM installer is supported)
• .NET: ILSpy command line
• JVM: CFR, configured only for Java projects
• InstallShield media: Unshield; apply the large-cabinet patch when a volume exceeds 4 GiB

Install the Python command and model:

py -3 -m venv .venv
.\.venv\Scripts\python -m pip install -e .
ollama pull qwen2.5-coder:7b
mmo-preserve doctor

Install the pinned Ghidra MCP extension and authenticated bridge once:

.\scripts\install-ghidra-mcp.ps1

The installer verifies all upstream release hashes, isolates the bridge dependencies, disables Ghidra scripts, enforces explicit program selectors, scopes file access to the organization directory, and keeps the server bound to loopback. See docs/GHIDRA_MCP.md for the exact pin, compatibility patch, and workflow.

The dependency-free preserve.ps1 wrapper also works when Python is available without an activated virtual environment.

Per-project tool paths belong in the ignored preservation.local.toml, for example:

[paths]
input = "D:\\MMO\\Example\\game.exe"

[tools]
ghidra_home = "C:\\Tools\\ghidra_12.1_PUBLIC"
cfr_jar = "C:\\Tools\\cfr-0.152.jar"

Create one repository per decompilation

.\preserve.ps1 new "Example MMO Client" `
  --kind client `
  --input "D:\MMO\Example\game.exe" `
  --output "..\example-mmo-client"

This operation:

• hashes and identifies the input;
• creates the evidence, analysis, recovered-source, test, and documentation layout;
• stores the machine-local input path in ignored preservation.local.toml;
• initializes a main Git repository; and
• configures origin as https://gitty.ryuum3gum1n.de/MMO-Preservation/example-mmo-client.git.

It does not create the remote Gitty repository or push. Create the empty repository in Gitty, then review, commit, and push the scaffold. Original binaries are ignored by default; only their SHA-256 evidence record is committed until redistribution rights and large-file storage policy are explicit.

Extract and reconstruct

# Check the requirements for this exact artifact type.
.\preserve.ps1 doctor --project "..\example-mmo-client"

# Auto-select Ghidra, ILSpy, or CFR and verify the input hash.
.\preserve.ps1 extract "..\example-mmo-client"

# Build shared evidence, then use the persistent single-GPU queue.
.\preserve.ps1 index "..\example-mmo-client"
.\preserve.ps1 grind "..\example-mmo-client" --dry-run
.\preserve.ps1 grind "..\example-mmo-client" --max-tasks 10

# Independently validate and package accepted candidates.
.\preserve.ps1 grade "..\example-mmo-client" --workers 4
# Retry difficult compiler mismatches through the bounded Clang optimization/ABI matrix.
.\preserve.ps1 grade "..\example-mmo-client" `
  --filter "login|packet" --workers 4 --exhaustive-match --force
.\preserve.ps1 abi-infer "..\example-mmo-client"
.\preserve.ps1 toolchain-detect "..\example-mmo-client"
.\preserve.ps1 toolchain-receipt "..\example-mmo-client"
.\preserve.ps1 compiler-profiles "..\example-mmo-client"
.\preserve.ps1 differential-specs "..\example-mmo-client"
.\preserve.ps1 differential "..\example-mmo-client"
.\preserve.ps1 assemble "..\example-mmo-client" --acceptance exact
.\preserve.ps1 assemble "..\example-mmo-client" --acceptance differential
.\preserve.ps1 modules "..\example-mmo-client" `
  --acceptance differential --maximum-cluster-size 256
.\preserve.ps1 corpus "..\example-mmo-client"
.\preserve.ps1 corpus "..\example-mmo-client" --exact-only
.\preserve.ps1 metrics "..\example-mmo-client"
.\preserve.ps1 integration-init "..\example-mmo-client" `
  --server-repository "..\server-example-mmo"
.\preserve.ps1 integration-test "..\example-mmo-client"
.\preserve.ps1 protocol-slice "..\example-mmo-client"

# Plan organization-wide matching, specialist worktrees, and local-model research.
.\preserve.ps1 bsim-auto "..\example-mmo-client"
.\preserve.ps1 agents "..\example-mmo-client"
.\preserve.ps1 model-lab "..\example-mmo-client"
.\preserve.ps1 model-calibrate "..\example-mmo-client" --examples 256

# Or keep all bounded stages running unattended.
.\preserve.ps1 supervise "..\example-mmo-client" --git-checkpoint

# Ask a bounded question against a live, read-only Ghidra MCP session.
.\scripts\start-ghidra-mcp.ps1 -InputFile "D:\MMO\Example\game.exe"
.\preserve.ps1 investigate "..\example-mmo-client" `
  --goal "Trace packet authentication from the receive handler and identify unchecked lengths"
.\scripts\stop-ghidra-mcp.ps1

# Audit a cohesive recovered tree, or the managed decompiler output when no cohesive tree exists.
.\preserve.ps1 audit "..\example-mmo-client"

Refinement is resumable. An unchanged function/model pair is skipped, and --force explicitly regenerates it. Use --filter "login|packet|00401a20" to focus on a subsystem or address. Every candidate in recovered/functions/ has a matching metadata record containing its exact input hash, model, confidence, assumptions, and risks.

index also builds an evidence-attributed fact store and a conservative win32-x86-msvc compatibility archive. Function-specific declaration closure supplies supported callee prototypes, imports, globals, Ghidra scalar aliases, Win32/Winsock types, and x86 SEH scaffolding to grading. Conflicting exclusive declarations are recorded and omitted from the generated translation unit instead of being chosen silently.

abi-infer proposes review-only Win32 x86 calling conventions, stack/register parameters, structure fields, vtables, classes, and globals. It records conflicting hypotheses rather than accepting one. compiler-profiles learns a bounded, tool-hash-pinned ranking from existing grade matrices and may recommend profiles to call-graph or BSim neighbors; every target still requires its own compile, object, and behavioral validation.

toolchain-receipt checks the configured artifact hash, pins locally installed compiler/linker identities, records an explicit block when the evidence-backed historical MSVC generation is not installed, and compiles a fixed synthetic C probe. It never passes legacy bytes or source to a compiler, never executes or retains the probe objects, and never treats modern MSVC or Clang compatibility mode as historical equivalence.

The mechanical grader normalizes instruction syntax and relocation-sensitive addresses, records EXACT, RELOCATION_EQUIVALENT, SEMANTIC_MATCH, PARTIAL, or MISMATCH, and checks ABI shape for deterministic wrappers. Before blaming a candidate for a compiler diagnostic, it classifies missing callees, imports, types, and declarations as synthetic-harness dependencies, retries with bounded generated declarations, and records unresolved edges in the campaign database. Callers wait in pending_dependencies, and resolvable callee tasks receive dependency priority; actual syntax/ABI errors alone are returned to the model repair lane. Empty stubs and constant returns that pass their applicable checks bypass the independent model reviewer; they still need differential or integration evidence for behavioral acceptance.

The differential specification schema supports deterministic object, heap, stack, and raw-memory fixtures, register bindings, pointer normalization, termination, stack-delta, call trace, observed memory, flags, branch/coverage-guided vectors, replay, and counterexample minimization. Non-leaf functions are eligible only when every direct call fits a bounded verified-internal or deterministic import model; unknown calls, out-of-fixture effects, filesystem access, and real networking block the comparison. Ghidra also exports a hash-pinned contiguous function body for a second executor. Pinned Unicorn is preferred for Win32 x86 leaf functions; Triton and angr remain optional. The small deterministic x86 interpreter is diagnostic-only and cannot satisfy promotion. Unsupported execution is blocked, not accepted. Both semantic oracles must pass. Neither path launches the preserved program.

modules packages accepted functions into bounded, call-graph-grouped subsystem object libraries and writes an unresolved-symbol ledger. corpus exports verified examples and failed repair counterexamples under analysis/training/ without copying binary bytes. metrics writes both analysis/observability/status.json and a self-refreshing analysis/observability/index.html dashboard.

Promotion-grade independent execution now prefers the pinned Unicorn Win32 x86 leaf adapter. The built-in instruction subset is diagnostic-only and a missing external engine fails closed. See docs/SEMANTIC-ORACLES.md for installation, receipts, and safety bounds.

grade-history reclassifies retained mechanical receipts with the current v5 classifier, invalidates stale versions and hash mismatches, and clusters repeated declaration failures. type-recovery correlates those clusters with RTTI/vtable, layout, and call-site evidence into a review-only shared declaration plan; it never injects hypotheses directly into a build.

The v0.7 BSim workflow uses bsim-auto for corpus updates, one pinned headless query per analyzed repository, result enrichment, conservative clustering, and candidate-only propagation. Planning is the default; --execute-corpus and --execute-queries are explicit. corpus-materialize can create immutable detached release worktrees directly from canonical Git blobs without hooks, checkout filters, LFS hydration, submodules, installers, or preserved-program execution. signature-execute runs only regenerated hash-pinned Ghidra plans and local signature inputs with an asserted rights class and expected SHA-256. The automatic client corpus plan never downloads, copies, checks out, installs, or executes preserved binaries, and the target query list reserves capacity for active-slice functions. See docs/BSIM.md for the lock, provenance, and per-target validation rules.

type-constraints converts RTTI, vtables, callsite prototypes, class layouts, and shared grade-dependency clusters into a global review graph. Conflicting values require explicit accept/reject decisions. Applying a reviewed graph expires only dependent grade, review, and differential evidence; it never promotes candidates. typed-ir validates a versioned typed CFG against pinned CFG/P-code facts and emits deterministic C++. Readable identifiers remain a separate behavior-receipt-pinned artifact. cluster-plan gives cyclic call groups atomic SCC work units while leaves remain one-function claims, with exclusive leases for shared interfaces.

agents derives ABI-review, differential-repair, and protocol jobs into an atomic SQLite lease queue. It is plan-only by default. --execute requires a clean checkout and gives each worker an isolated Git worktree, path scope, validation commands, and commit; GPU tasks share one lock. Merging and pushing remain separate --merge and --push opt-ins.

protocol-slice verifies an explicit Twin/client/server preservation manifest through static repository/evidence contracts and optional loopback-only HTTP/TCP services. Static success is reported separately from a live matched-login claim. model-lab verifies local corpus hashes, keeps each artifact hash in one split, writes a held-out benchmark, and emits a QLoRA plan only after its example/artifact gates pass. Neither command executes a legacy client or starts model training. model-calibrate runs a resumable, isolated 3B/7B crossover over identical hash-pinned function evidence and reports mechanically viable candidates per GPU-hour without replacing or promoting reconstruction files. Specialist checkpoints with a mismatched published target remain benchmark-only. A 512-attempt-or-larger sweep is now blocked until golden-calibration has 30–50 target-compatible, artifact-family-isolated examples and complete paired results containing at least one mechanical signal.

whole-pe-status records the pinned Qiling capability and can hash a closed read-only Windows rootfs. A Qiling result is only one independent component and still requires Ghidra P-code agreement. path-vectors uses bounded angr or Triton exploration only to create replay inputs. trace-plan and trace-import require hash-pinned disposable-VM, snapshot, network, collector, artifact, and output receipts. Runtime traces are test-generation evidence, never source equivalence. See docs/EVIDENCE-V07.md.

For an OpenAI-compatible local runtime such as llama.cpp instead of Ollama:

.\preserve.ps1 refine "..\example-mmo-client" `
  --provider openai_compatible `
  --endpoint http://127.0.0.1:8080/v1 `
  --model local-qwen3-coder

What “fully decompiled” means here

Compilation permanently removes information: comments, many names and types, original file boundaries, macros, and sometimes whole code paths. No model can recover those exactly. A project is considered reconstructed when maintainers have assembled a cohesive source tree, can build it, and have verified important behavior against the preserved binary or protocol captures. See docs/RECONSTRUCTION.md for the staged process.

Only analyze software you are authorized to preserve. Keep clean-room notes where required, do not bypass live-service access controls, and treat all model-generated code and bug reports as hypotheses that need tests.

Development

Run its tests with:

python -m unittest discover -s tests -v