Entity: Hermes Agent (AI Agent Service)

This page is the repository hub for the Hermes Agent Proxmox KVM deployment.

⚠️ DECOMMISSIONED (2026-09-26) — VM 501 hermes was destroyed; it had been unused. The data disk was preserved on purpose as local-lvm:vm-501-data-orphan (the whole /home/hermes state), and ./create.sh can recreate the VM and reattach it with zero re-onboarding. Everything below is the historical architecture; see Removed: VM 501 decommissioned for what was deleted and what survives.

Overview

Hermes Agent ran as a dedicated KVM VM (VM 501 hermes) on the Proxmox node tazlab, migrated from the old LXC container (CT 105) on 2026-08-12 via the CRISP project hermes-agent-redesign and decommissioned on 2026-09-26. The deployment lived in a standalone repository (github.com/tazzo/hermes-agent) with Terraform + Ansible + shell orchestrators — no longer part of ephemeral-castle.

Data persistence followed the pet/cattle principle as in the old LXC deployment, adapted to VM semantics: the root disk was ephemeral (destroyed with the VM), while a dedicated 20G data disk survives destroy/create cycles via an lvrename out-of-convention trick. That is exactly the mechanism the decommission relied on.

Removed: VM 501 decommissioned (2026-09-26)

The board destroyed the unused Hermes Agent VM on 2026-09-26 via destroy.sh (run log: /workspace/hermes-agent/logs/destroy_20260926_075300.log).

Evidence from the run:

[phase2] update VM 501: -delete virtio1          → unused0: local-lvm:vm-501-disk-2
[phase3] Renamed "vm-501-disk-2" to "vm-501-data-orphan"
         Logical volume "vm-501-cloudinit" removed / "vm-501-disk-1" removed / "vm-501-disk-0" removed
[phase3.5] Successfully removed 1 resource instance(s)   (terraform state)
[phase4] DATA DISK PRESERVED: local-lvm:vm-501-disk-2 (as vm-501-data-orphan)

Preserved on purpose: the data disk local-lvm:vm-501-disk-2 (20 GB), renamed out of the vm-501-* naming convention to local-lvm:vm-501-data-orphan. It holds the whole /home/hermes state (Hermes code and sessions, operator settings, the refurb-collector workspace, the research site). destroy.sh never deletes it, and the orphan name is deliberately outside the convention that qm destroy sweeps.

Recreate with zero re-onboarding: ./create.sh in /workspace/hermes-agent renames vm-501-data-orphan back to vm-501-disk-2, re-creates the VM, reattaches the disk and re-runs Ansible — the pet volume is the same, so no restore or onboarding is needed.

Live verification (2026-09-26, run from CT106 tazpod-proxmox):

ssh root@192.168.1.200 'qm list | grep -E "^\s*501\s"'   # → no output: VM 501 NOT PRESENT
ssh root@192.168.1.200 'lvs | grep vm-501'
# → vm-501-data-orphan pve Vwi-a-tz-- 20.00g data 28.59

Historical context worth keeping: this is a clean, reversible retire, not a data loss — the point of the lvrename orphan pattern is precisely that a destroy leaves the state volume intact. The prior LXC deployment (CT 105) had been destroyed on 2026-08-12 and was not recoverable; VM 501 is.

Architecture

Proxmox node "tazlab" (192.168.1.200)
└── VM 501 "hermes" (KVM, Ubuntu 26.04 LTS "resolute")
    ├── 4 vCPU, 8 GB RAM, static IP 192.168.1.205/24 (gw 192.168.1.1)
    ├── virtio0: root disk 30 GB (local-lvm) — EPHEMERAL, destroyed with VM
    ├── virtio1: data disk 20 GB (local-lvm:vm-501-disk-2) — PERSISTENT pet
    │   └── /home/hermes (whole home = Hermes state + code, option A)
    ├── Hermes v0.20.0 (bare-metal via install.sh)
    │   ├── hermes-gateway.service
    │   ├── hermes-dashboard.service (port 9119, bind 0.0.0.0 + basic auth)
    │   └── hermes-restart.service (root oneshot — restarts gateway+dashboard, Hermes self-restart helper)
    ├── Post-2026-08-12 additions (user space: systemctl --user + loginctl enable-linger hermes)
    │   ├── hermes-research.service (Caddy static publisher, :8080, VPN-only)
    │   ├── hermes-chrome.service (headless Chrome, CDP 127.0.0.1:9222 — Playwright connect_over_cdp only)
    │   ├── /mnt/shared (CIFS from //192.168.1.207/shared, CT 107 lxc-storage)
    │   └── Refurb collector (VM-side scripts in /home/hermes, cron every 2 days — NOT in IaC)
    └── Users: bootstrap (admin, sudo NOPASSWD), hermes (UID 10000, no sudo)

Repository Structure

hermes-agent/                     # github.com/tazzo/hermes-agent
├── create.sh                     # 8-phase orchestrator (enterprise logging)
├── destroy.sh                    # unlink data disk → lvrename → qm destroy
├── terraform/                    # VM 501 resource (root disk ONLY — data disk outside state)
├── ansible/                      # 4 playbooks + roles (baseline, agent, configure, verify)
├── configs/
│   └── runtime.env               # VM_ID, IP, CORES, MEMORY, DISK sizes
└── logs/                         # Timestamped, rotated (keep 20)

Quick Facts

Canonical Starting Pages for Agents (historical — see Removed section)

Deployment & Lifecycle

• VM Deployment Architecture — KVM architecture, persistence model, lifecycle, key discoveries
• LXC Deployment Architecture — SUPERSEDED (see hermes-lxc-deployment.md banner, historical only)

CRISP Project (source of truth)

• Project: skills/crisp/projects/hermes-agent-redesign/ (RESEARCH/DESIGN/PLAN/tasks/retrospective)
• Report: skills/memory/reports/2026-08-12_hermes-agent-vm-redesign.md

Secrets (gopass)

• infra/hermes-vm/dashboard-password — dashboard basic auth
• infra/hermes-vm/bootstrap-password + infra/hermes-vm/hermes-password — SSH passwords
• infra/ssh-host-keys/hermes-agent/ed25519 — SSH host key (injected post-boot via Ansible)
• bootstrap/proxmox/token-id + bootstrap/proxmox/token-secret — Proxmox API for create/destroy
• infra/hermes-vm/samba-username + infra/hermes-vm/samba-password — CIFS credentials (/etc/cifs-credentials-hermes, injected via env SAMBA_USERNAME/SAMBA_PASSWORD at playbook run)

Key Discoveries (2026-08-12, destructive tests)

• qm destroy deletes by naming convention: removes ALL vm-501-* volumes even unused, even with --destroy-unreferenced-disks 0. Fix: lvrename the data volume to vm-501-data-orphan before destroy, rename back on create.
• Data disk OUTSIDE Terraform state: if declared in .tf, terraform destroy deletes it. Managed via pvesm alloc + qm set/qm unlink.
• gopass multi-line: gopass show -o returns only first line (35B vs 411B for SSH keys) — use gopass cat.
• Ansible 2.21 become non-root bug: sudo -u hermes pattern instead of become_method: su.
• Dashboard auth: scrypt hash must be STANDARD base64 with padding + maxmem=0 (urlsafe fails). Hash computed locally where gopass exists.
• agent.wait_for_ip.disabled = true: skips IP wait (static via cloud-init) → apply drops from ~15 min to <1 min.

Known Issues / Technical Debt

Relationships

• Infrastructure: ephemeral-castle — former home; ephemeral-castle/hermes/ removed (commit 504f353); now standalone repo
• GitOps cluster: tazlab-k8s — talks to the cluster via LLM provider
• Semantic memory: mnemosyne-mcp-server
• Context governance: Skills System — operational memory and rules

See Also

• Wiki Index