Trusted Orchestration Host — TazPod LXC Provisioning Runbook

Level 2 (Topic) — where TazPod LXC provisioning (create.sh) must run, why it is pinned to tazlab, and the exact acceptance checks. Decision and runbook from TAZ-19 (follow-up of TAZ-12; design option O3 in the TAZ-10 memo), bootstrap committed at tazzo/tazpod@e38dc17.

Decision

The trusted orchestration host is tazlab — the Proxmox VE node at 192.168.1.200. Admin workstations (desktop, macbook-fedora) stay interactive operator clients; they do not run provisioning directly.

Why tazlab and not a workstation — the from= pin forces it

TAZ-12 authorised the provisioning key on CT106 with a source-address pin:

restrict,from="192.168.1.200" <tazpod-provision public key>

from="192.168.1.200" means sshd on CT106 accepts that key only when the TCP connection originates from 192.168.1.200 — i.e. from tazlab. A workstation on a different address is rejected no matter which key it holds. So even when the operator sits at desktop, the Ansible run must be executed on tazlab (or through it: ssh tazlab). Copying the private key onto a workstation to dodge the pin would re-widen the exposure TAZ-12 removed; that is explicitly out of scope.

tazlab is also the only host that can carry out create.sh’s root@192.168.1.200 phases (pct exec, raw LXC config, admin-key injection) locally.

What the control host must provide

RequirementWhy
ansible-core (ansible, ansible-playbook)phase 4 baseline + the TAZ-19 acceptance check
terraformphase 1 (terraform init/apply) and terraform-pet/
gopass + the store GPG keyphase 5 Tailscale OAuth transfer (bootstrap/tailscale/*)
git, openssh-clientthe checkouts and the SSH phases
/root/.ssh/tazpod-provision + .pubthe only Ansible/SSH control key (root-only)
tazpod checkoutruntimes/lxc/create.sh, inventories, playbooks
tazlab-secrets checkoutadmin public keys (infra/ssh-client-keys/*/id_ed25519.pub)
runtimes/lxc/configs/runtime.envPROXMOX_VE_API_TOKEN (secret, operator-only)
the container’s ansible.cfgoverlayfs/ACL fallback, see runtimes/lxc/ansible/ansible.cfg

Optional: go (phase 3b builds the tazpod binary; the playbook skips the install if it is absent).

Provisioning key (/root/.ssh/tazpod-provision)

  • Location: /root/.ssh/tazpod-provision, .pub beside it. Mode 0600, root:root.
  • Source of truth: generated on tazlab for TAZ-12; it never exists inside CT106’s agent-readable home. The agent key ~/.ssh/id_ed25519 is revoked (TAZ-12).
  • Authorisation on CT106’s root: restrict,from="192.168.1.200".
  • inventory.ini and terraform/outputs.tf point at the absolute path /root/.ssh/tazpod-provision, which is why the run must happen on the host that owns that path.
  • create.sh also needs <key>.pub for the Terraform phase (the new CT is seeded with this public key).

The bootstrap verifies presence, ownership (root:root) and mode (0600); it prints the public-key fingerprint only, never private key material.

Host bootstrap

The repo ships an idempotent installer at runtimes/lxc/scripts/install-orchestration-host.sh (added in tazzo/tazpod@e38dc17, “lxc: add the trusted orchestration-host bootstrap (TAZ-19)”):

# on tazlab, as root
GITHUB_TOKEN="$(...)" /root/tazpod/runtimes/lxc/scripts/install-orchestration-host.sh

It installs ansible-core/terraform/gopass/git, clones tazpod and tazlab-secrets under /root, and verifies the keypair’s presence/ownership/mode. It prints the acceptance commands and never prints private key material. Terraform comes from the HashiCorp apt repo; if ansible-core is absent from the distro, the script falls back to a venv at /opt/ansible. The clone is skipped when <dir>/.git already exists, so re-runs are safe.

Acceptance verification (operator, on tazlab)

The bootstrap writes the checkouts to /root/tazpod and /root/tazlab-secrets; set TAZLAB_SECRETS=/root/tazlab-secrets before a create.sh run. First run the check:

cd /root/tazpod/runtimes/lxc/ansible
ANSIBLE_CONFIG=$PWD/ansible.cfg ansible -i inventory.ini tazpod -m ping
ANSIBLE_CONFIG=$PWD/ansible.cfg ansible-playbook -i inventory.ini tazpod-baseline.yml --check --diff

Expected: pong for the ping; a completed --check with failed=0 unreachable=0. Record the summary line in the issue.

create.sh prerequisites (full run)

cd /root/tazpod/runtimes/lxc
export TAZLAB_SECRETS=/root/tazlab-secrets
# configs/runtime.env must exist here (PROXMOX_VE_API_TOKEN) — secret, do not commit
./create.sh 192.168.1.206 106

Phase order: 0 pet → 1 terraform → 2 raw LXC config → 2b admin keys → 3 wait SSH → 3b build binary → 4 Ansible baseline → 5 Tailscale OAuth from gopass → 6 Tailscale up → 7 verify.

Notes and open items

  • Unlocking gopass on tazlab requires the store GPG key; provision it out of band (Secrets & PKI Engineer / operator), never in the repo.
  • runtime.env move from CT106 to tazlab is a one-time secret transfer; delete the CT106 copy if it is no longer needed there.
  • root@192.168.1.200 phases run over SSH from tazlab to itself; confirm the provisioning key (or local root auth) is authorised for root@192.168.1.200 before a full create.sh run.
  • The access-path residue from the TAZ-12 agent run (/root/security/taz-10-backup/) is root-only and not agent-readable; the operator may delete it once satisfied.
  • Confirmed 2026-09-26 from the agent sandbox: /root/.ssh/tazpod-provision is not readable, /home/tazpod/.ssh/id_ed25519 is absent, and no private-key material exists under agent-readable paths.
  • The privileged install + verification on tazlab is an operator action; the wiki records the decision and prerequisites, not a executed result.

See Also