# WolfPanel Agent — Rollback Guide

## How automatic rollback works

`update.self_update()` (in `src/update.py`) performs, in order:

1. download → verify size → verify SHA-256 → verify signature
2. extract to `/opt/wolfpanel/versions/<new-version>/`
3. **verify the extracted package's own `VERSION` file matches the manifest
   version** (the downgrade/tamper guard — if this fails, the update is
   cancelled here: the symlink is never touched and the service is never
   restarted)
4. back up the currently-active version to
   `/opt/wolfpanel/versions/<old-version>.backup`
5. atomically flip `/opt/wolfpanel/current` to the new version
6. write `/var/lib/wolfpanel/update_state.json` describing how to roll back
7. spawn a **rollback watcher** and restart the service so the new version
   starts running
8. once the new version comes up and successfully sends a heartbeat,
   `heartbeat.send_once()` deletes `update_state.json` and reports success —
   this is the "update confirmed good" signal

If step 8 never happens within `WOLFPANEL_UPDATE_TIMEOUT` seconds (default
60s) — the new version crashed, hung, or can't reach the Central API — the
watcher spawned in step 7 finds `update_state.json` still present, restores
`current` to point at the `.backup` directory, and restarts the service
again.

## The systemd cgroup problem (found and fixed during this hardening pass)

The watcher used to be spawned as a plain detached subprocess
(`subprocess.Popen(..., start_new_session=True)`) and the agent then called
`systemctl restart wolfpanel-agent` to start the new version. Under the
service's default `KillMode=control-group`, that restart kills **every**
process in the unit's cgroup — and `start_new_session=True` only detaches a
process from its controlling terminal/session, it does **not** move it out
of the systemd cgroup. In practice, this meant the watcher was very likely
to be killed by the same restart it was supposed to be watching over,
silently disabling automatic rollback in the one environment (systemd) this
agent actually ships for.

**Fix applied**: when running under systemd, the watcher is now launched via
`systemd-run --unit <name> --collect -- <watcher>`, which creates a sibling
transient unit outside `wolfpanel-agent.service`'s cgroup. `systemctl
restart wolfpanel-agent` cannot touch it. The plain
`start_new_session=True` spawn is kept as a fallback for non-systemd
(standalone) runs, where no cgroup exists to kill it in the first place.

## What is verified, and what needs a real VM

This was fixed and reasoned through statically (cgroup/KillMode semantics
are documented systemd behavior); **it has not been exercised against a real
systemd unit in this environment**, because this hardening pass was done on
a Windows box with no Linux/systemd available. Before trusting it in
production, run the test below on a disposable Linux VM.

### VM test: prove the watcher survives the restart

```bash
# 1. Install the agent for real (mock API is fine).
sudo ./install.sh --token dev-token --api-url http://localhost:9999
echo "WOLFPANEL_API_MOCK=1" | sudo tee -a /etc/wolfpanel/agent.conf
sudo systemctl restart wolfpanel-agent

# 2. Trigger a self-update to a version whose main.py deliberately never
#    sends a heartbeat (simulates "new version is broken"), so we exercise
#    the timeout path, not the happy path.
sudo /opt/wolfpanel/current/wolfpanel-agent update --yes

# 3. Immediately check that a transient rollback unit exists and is NOT in
#    wolfpanel-agent.service's cgroup:
systemctl list-units 'wolfpanel-agent-rollback-*'
systemctl status wolfpanel-agent-rollback-<job-id>-<timestamp>

# 4. Wait past WOLFPANEL_UPDATE_TIMEOUT (default 60s), then confirm:
readlink /opt/wolfpanel/current        # should point back at the old version
systemctl is-active wolfpanel-agent    # should be "active" again
journalctl -u wolfpanel-agent --since "2 min ago" | grep -i rollback
```

Expected: the transient unit is still alive immediately after the restart
(step 3), and `current` has flipped back to the pre-update version by step 4.
If the transient unit is gone by step 3, `systemd-run` is not available or
not behaving as expected on that distro, and the watcher is still exposed to
the original race — investigate before shipping to that platform.

### Manual rollback (always available, no watcher required)

```bash
sudo ln -sfn /opt/wolfpanel/versions/<desired-version> /opt/wolfpanel/current
sudo systemctl restart wolfpanel-agent
```

`update.clean_old_versions()` keeps the last `WOLFPANEL_UPDATE_RETENTION`
(default 3) version directories in `/opt/wolfpanel/versions/`, plus whatever
is currently active, so a manual rollback target is normally still on disk;
`*.backup` directories are also excluded from that cleanup.