Build a Proxmox + PBS Backup Stack on OpenFactory

The hardest part of running anything in your homelab isn't getting it up. It's knowing you can get it back. Proxmox Backup Server (PBS) is the free, deduplicated, encrypted backup target Proxmox ships for exactly that — and the most under-deployed piece of the standard Proxmox stack.

This post walks through the PBS pattern as an OpenFactory build prompt: three buildable Debian Trixie VMs — a PVE source, a PBS target, and an off-site PBS mirror pull-syncing the datastore — from a single prompt, with the vzdump backup job on the PVE side, the PBS datastore + retention config, and the remote-sync wiring already baked in. Real proxmox-backup-server is a source-ISO install (or apt on a Debian box); the lab gives you the topology and configs to drop on top.

What you'll build

• pve-source (10.84.0.10:8006) — the PVE node being backed up, with a PBS storage entry in /etc/pve/storage.cfg (server, datastore, fingerprint placeholder, prune policy) and a daily vzdump job in /etc/pve/jobs.cfg ready to go.
• pbs-target (10.84.0.20:8007) — the primary backup target with /etc/proxmox-backup/datastore.cfg shaped, retention set to keep-daily=7, keep-weekly=4, keep-monthly=12, mock PBS API exposing datastore status + available capacity, and a runbook documenting proxmox-backup-manager user create backup@pbs.
• pbs-offsite (10.84.0.30:8007) — the off-site mirror with /etc/proxmox-backup/remote.cfg and sync.cfg already wired to pull-sync from pbs-target nightly at 04:00. Proves the 3-2-1 backup pattern is real.

Why build it on OpenFactory

• The ISO is the spec. The datastore config, retention policy, vzdump job schedule, and the remote-sync wiring all live in bootable images. Lose a PBS host? Boot the same ISO on new hardware, plug in the disks.
• Retention is in the recipe, not the UI. Many homelab PBS deployments default to "keep forever" because nobody clicks through the pruning UI. Sane retention is baked in here from day one.
• Off-site sync wired from day one. A backup on the same physical site isn't a backup — it's a snapshot. The pull-sync to a second PBS lives in the recipe.
• Scenario assertions ride along. The build group fails closed if the datastore isn't reachable from PVE, if the available capacity drops to zero, or if the offsite sync.cfg drifts.

Topology

Three Debian Trixie VMs on 10.84.0.0/24. PVE backups push to pbs-target on :8007; pbs-offsite pulls the datastore from pbs-target over the same API. The 3-2-1 shape (3 copies, 2 media, 1 off-site) emerges naturally from PVE local + PBS target + PBS off-site.

The prompt

Paste this verbatim into the chat builder at console.openfactory.tech. Nothing above or below it — the builder expects the prompt body to start at the “Build a compact multi-node lab…” line.

Build a compact multi-node lab named `proxmox-pbs-backup`.

Output discipline: keep the plan small. Use one startup script per node, about 25 shell lines or less. Do not install `proxmox-backup-server`, `proxmox-backup-client`, `pve-manager`, or any Proxmox apt repos at build time. The PBS deduplicated chunk store and the PVE source node are mocked via deployment-time config templates and Python stdlib services on the right ports. Write deployment-time config examples and tiny Python stdlib or shell compatibility stubs only. The goal is a buildable preparation lab, not a production Proxmox install.

## Topology

Create 3 buildable `debian-trixie` nodes, all `x86_64`, SSH enabled, DHCP/default route intact with lab aliases, firewall disabled, DNS `1.1.1.1` and `8.8.8.8`, user `ops` password `pbs-ops` in `sudo`. Every recipe must set top-level `test_config` to `{ "enabled": false, "tests": [] }`.

- `pve-source`: role `pve-host`, 4 GB RAM, 32 GB disk, alias `10.84.0.10/24`, x `110`, y `100`
- `pbs-target`: role `backup-server`, 3 GB RAM, 64 GB disk, alias `10.84.0.20/24`, x `350`, y `100`
- `pbs-offsite`: role `backup-mirror`, 3 GB RAM, 64 GB disk, alias `10.84.0.30/24`, x `590`, y `100`

Connections: `pve-source` to `pbs-target:8007` (PBS API); `pbs-target` to `pbs-offsite:8007` (sync-pull from offsite); both PBS nodes expose `:8007` to the lab subnet.

## Common Recipe Requirements

All nodes: features `headless`, `ssh`; packages `openssh-server`, `python3`, `curl`, `jq`, `iproute2`, `netcat-openbsd`, `ca-certificates`. Each startup script adds the alias with `IFACE=$(ip route show default | awk '{print $5; exit}')`, `ip link set "$IFACE" up || true`, and `ip addr add <alias> dev "$IFACE" || true`. If `os.startup_scripts[].after` is present, it must be the string `"network-online.target"`, not an array. Do not install `pve-manager`, `proxmox-backup-server`, `ceph`, `truenas-scale`, or any related apt packages — they are source-ISO deploys handled at provisioning time, not at build time.

## Node Requirements

`pve-source`: features `headless`, `ssh`. Create `/etc/pve/{storage,jobs.d}` mode `0750 ops:ops`. Add the PBS storage entry to `/etc/pve/storage.cfg`: `pbs: pbs-target\n  server 10.84.0.20\n  datastore lab-store\n  username backup@pbs\n  fingerprint AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99\n  content backup\n  prune-backups keep-daily=7,keep-weekly=4,keep-monthly=12\n  encryption-key /etc/pve/priv/storage/pbs-target.enc`. Write `/etc/pve/jobs.cfg` with `vzdump: backup-daily\n  schedule 02:30\n  storage pbs-target\n  all 1\n  mode snapshot\n  compress zstd\n  notes-template "{{guestname}}"\n  prune-backups keep-daily=7,keep-weekly=4,keep-monthly=12`. Add a Python stdlib HTTP service on `0.0.0.0:8006` exposing `/api2/json/version`, `/api2/json/cluster/backup` -> `200 {"data":[{"id":"backup-daily","schedule":"02:30","storage":"pbs-target","enabled":1}]}`, and `/metrics` with `pve_compat_up 1` plus `pve_backup_jobs 1`. Register `pve-compat.service`.

`pbs-target`: features `headless`, `ssh`. Create `/etc/proxmox-backup/` mode `0750 ops:ops` and `/var/lib/proxmox-backup/datastore/lab-store/{.chunks,.gc-status}` mode `0750 ops:ops`. Write `/etc/proxmox-backup/datastore.cfg` with `datastore: lab-store\n  path /var/lib/proxmox-backup/datastore/lab-store\n  gc-schedule daily\n  prune-schedule daily\n  keep-daily 7\n  keep-weekly 4\n  keep-monthly 12\n  notify-user root@pam`. Write `/etc/proxmox-backup/user.cfg.example` with `user: backup@pbs\n  enable 1\n  expire 0\n  comment "PVE backup ingest"`. Write `/etc/proxmox-backup/acl.cfg.example` granting `backup@pbs` the `DatastoreBackup` role on `/datastore/lab-store`. Add a Python stdlib HTTP service on `0.0.0.0:8007` exposing:
- `GET /api2/json/version` -> `200 {"data":{"version":"compat-1.0","release":"pbs-compat","repoid":"pbs-target"}}`
- `GET /api2/json/admin/datastore` -> `200 {"data":[{"store":"lab-store","comment":"lab store","gc-schedule":"daily"}]}`
- `GET /api2/json/admin/datastore/lab-store/status` -> `200 {"data":{"total":68719476736,"used":0,"avail":68719476736,"gc-status":{"upid":null,"last-run":"never"}}}`
- `GET /metrics` -> `pbs_compat_up 1` plus `pbs_datastores 1` plus `pbs_datastore_avail_bytes{store="lab-store"} 68719476736`
Register `pbs-compat.service`. Write `/root/pbs-runbook.md` documenting how the deploy generates the real keyring with `proxmox-backup-manager user create backup@pbs` and how the fingerprint shown in `/etc/pve/storage.cfg` on the PVE side comes from `proxmox-backup-manager cert info | grep Fingerprint`.

`pbs-offsite`: features `headless`, `ssh`. Same datastore shape as `pbs-target` but at `/var/lib/proxmox-backup/datastore/offsite-mirror/`. Additionally write `/etc/proxmox-backup/remote.cfg` with `remote: pbs-target\n  host 10.84.0.20\n  userid sync@pbs\n  fingerprint AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99` and `/etc/proxmox-backup/sync.cfg` with `sync: nightly-pull\n  remote pbs-target\n  remote-store lab-store\n  store offsite-mirror\n  schedule 04:00\n  owner sync@pbs`. Add a Python stdlib HTTP service on `0.0.0.0:8007` exposing the same identity payload but `repoid: "pbs-offsite"` and a `/api2/json/config/sync` endpoint returning `200 {"data":[{"id":"nightly-pull","remote":"pbs-target","schedule":"04:00","store":"offsite-mirror"}]}`. Register `pbs-compat.service`.

## Scenario

Emit exactly one group scenario named `proxmox-pbs-backup-validation`. Put `custom_tests[].assertions[]` inside the scenario entry; leave `scenarios[].tests` empty. Every assertion needs `on_vm`. Use only `port_listening`, `command_output`, and `http_responds`; do not emit `vm_boots`, `network_reachable`, or `service_running`.

- `Stack ports listen`: `port_listening` for `pve-source:8006`, `pbs-target:8007`, `pbs-offsite:8007`.
- `PVE storage points at PBS`: on `pve-source`, `grep -q 'pbs: pbs-target' /etc/pve/storage.cfg && grep -q 'server 10.84.0.20' /etc/pve/storage.cfg && echo storage-wired`.
- `PVE backup job present`: on `pve-source`, `curl -fsS http://localhost:8006/api2/json/cluster/backup | jq -e '.data[0].id == "backup-daily" and .data[0].enabled == 1' >/dev/null && echo job-wired`.
- `PBS datastore reports lab-store`: on `pbs-target`, `curl -fsS http://localhost:8007/api2/json/admin/datastore | jq -e '.data[0].store == "lab-store"' >/dev/null && echo datastore-ok`.
- `PBS datastore status shows available capacity`: on `pbs-target`, `curl -fsS http://localhost:8007/api2/json/admin/datastore/lab-store/status | jq -e '.data.avail > 0' >/dev/null && echo avail-ok`.
- `Offsite sync job wired`: on `pbs-offsite`, `curl -fsS http://localhost:8007/api2/json/config/sync | jq -e '.data[0].remote == "pbs-target" and .data[0].store == "offsite-mirror"' >/dev/null && echo sync-wired`.
- `PVE reaches PBS, PBS-offsite reaches PBS`: on `pve-source`, `nc -z -w 5 10.84.0.20 8007 && echo pbs-reachable`; on `pbs-offsite`, `nc -z -w 5 10.84.0.20 8007 && echo target-reachable`.

Preserve warnings that real Proxmox Backup Server installation from the source ISO (or apt on a Debian box), `proxmox-backup-manager` user / datastore / ACL setup, real encryption keys generated per-PVE-host and stored in `/etc/pve/priv/storage/`, certificate trust between PVE↔PBS and PBS↔PBS via real cert fingerprints, garbage-collect + prune schedules under load, off-host replication via `pbs2` / S3-compatible target, mount-resilient WORM datastore layout, monitoring + alerting on the `:8007` endpoint, and `10.84.0.0/24` lab aliasing are deployment-time concerns.

Running it

1. Open the chat builder at console.openfactory.tech and paste the prompt into a new conversation.
2. Review the streamed build plan. You'll see the topology, per-node recipes, and the scenario assertions that will run after boot. Edit the prompt and re-run if anything is off.
3. Click Build group. OpenFactory fans the plan out to per-node ISO builds. When every ISO reaches built, boot the group on the runner network from the same UI.
4. Exercise the stack. The scenario assertions run automatically against the live VMs. From the host you can also hit the service ports directly to confirm end-to-end behavior.

Driving OpenFactory from an AI agent instead of the browser? The same flow is exposed through the OpenFactory MCP server — submit the prompt programmatically, get the build-plan preview back, and call create_build / start_vm on the resulting recipes. Single-image builds go straight through the openfactory CLI.

What's still your responsibility

The prompt produces a buildable preparation lab — the right topology, the right ports listening, deployment-time config templates dropped in the right places, and tiny compatibility services that prove the wiring works. A few things still sit outside the recipe and need operator attention before this carries real load:

• Real PBS install. Boot the PBS installer ISO on pbs-target and pbs-offsite, or apt-install proxmox-backup-server on a Debian Trixie box. The datastore config is ready to drop into /etc/proxmox-backup/.
• Real users + tokens. proxmox-backup-manager user create backup@pbs on the target, proxmox-backup-manager user create sync@pbs on the offsite. ACLs scope each to the datastore they touch.
• Real fingerprints. The lab uses a placeholder fingerprint in storage.cfg; production fills in the real one from proxmox-backup-manager cert info.
• Encryption keys. Generate the encryption key on each PVE host with proxmox-backup-client key create /etc/pve/priv/storage/pbs-target.enc; the lab has the path placeholder.
• GC + prune schedules under load. The defaults are sane for homelab; production should monitor GC duration and tune.
• Restore drills. A backup you've never restored from is a hope, not a backup. Schedule a quarterly restore test of a real VM.

Where to go next

Backups are step one; signed runtime evidence is step two. The runtime attestation post covers proving the kernel + systemd state hasn't drifted from the recipe — pairs naturally with "we can prove what we restored". If the PVE side needs a real TLS edge, see the nginx + TLS reverse-proxy post. And if the PBS target is sitting on a Proxmox cluster, the 3-node cluster post shows the host shape under it.