TazLab K8s: tazlab-db Detail

Level 3 (Detail) — PostgreSQL cluster configuration, databases, users, backup.

Concept

The cluster runs a single shared PostgreSQL instance managed by Crunchy PostgreSQL Operator (PGO). It serves all stateful workloads: Mnemosyne (vector memory), Grafana (dashboards), TazLab test/validation, and pgAdmin.

PostgresCluster

File: infrastructure/instances/tazlab-db/cluster.yaml

Databases and Users

Initialization SQL

File: infrastructure/instances/tazlab-db/init-sql.yaml

Applied as ConfigMap tazlab-db-init-sql. On first cluster creation, it:

mnemosyne database

• Creates vector and uuid-ossp extensions
• Creates memories table with UUID PK, VECTOR(3072) embedding column
• Creates archived_files table for session archival tracking

tazlab database

• Creates a memories table (different schema — serial PK, JSONB metadata/tags)

tazlab-test database

• Creates rebirth_test table
• Inserts a row with timestamp for rebirth validation

grafana database

• Grants all privileges to grafana user

S3 Backup

File: infrastructure/configs/tazlab-db/s3-external-secret.yaml

PGBackrest configuration:

Credentials are injected via ExternalSecret s3-backrest-creds which renders the PGBackrest config file format.

External Service

File: infrastructure/instances/tazlab-db/service.yaml

Exposes PostgreSQL externally through MetalLB. Selector targets the primary instance (role: master).

Users Secret Format

PGO generates a Secret for each user in the format <cluster-name>-pguser-<username> (e.g., tazlab-db-pguser-mnemosyne). Each Secret contains:

• user, password, host, port, dbname, uri, jdbc-uri

Applications reference these secrets via secretKeyRef in their env vars.

Server TLS (Vault PKI)

File: infrastructure/instances/tazlab-db/cluster.yaml (PostgresCluster spec)

Both the Postgres server and replication traffic use the same Vault PKI secret, provisioned via a VaultPKISecret resource that requests a server certificate from the Vault PKI engine. This replaces the default PGO-generated self-signed cert with a trusted CA-signed certificate.

PGO CA Trust Bundle

PGO’s operator-managed root CA and the Vault PKI intermediate CA are combined into a single trusted CA bundle. This ensures PostgreSQL accepts both:

• Certificates signed by the PGO internal CA (operator-internal communication)
• Certificates signed by the Vault PKI intermediate (client mTLS connections)

The bundle is injected into the PostgresCluster’s customTLSSecret configuration so the server presents the Vault-issued cert while trusting clients signed by either CA.

Client VaultPKISecrets

Four VaultPKISecret resources issue client certificates for mTLS authentication to the database:

Each secret requests a client certificate from the Vault PKI intermediate, enabling certificate-based authentication (pg_hba cert method) for those users.

Application mTLS Authentication

All three cluster applications connecting to PostgreSQL use client certificate authentication via pg_hba cert rules. The Vault Database Engine (database/) has been removed — no dynamic credentials are needed.

File: infrastructure/instances/tazlab-db/cluster.yaml (PostgresCluster patroni.dynamicConfiguration)

The pg_hba rules place hostssl cert-authentication entries before the default md5 rules. Each app also has a reject fallback after cert to prevent password-based auth from matching the generic host all all ... md5 rule:

hostssl postgres mnemosyne 0.0.0.0/0 cert
host postgres mnemosyne 0.0.0.0/0 reject
hostssl grafana grafana 0.0.0.0/0 cert
host postgres grafana 0.0.0.0/0 reject
hostssl postgres pgadmin 0.0.0.0/0 cert
host postgres pgadmin 0.0.0.0/0 reject

Grafana

Certificate: db-client-grafana-tls (CN=grafana). Mounted via extraInitContainer that copies from VaultPKISecret. grafana.ini.database sets ssl_mode: require, ca_cert_path, client_cert_path, client_key_path.

mnemosyne

Certificate: db-client-mnemosyne-tls (CN=mnemosyne). Go code refactored in 12-mnemosyne-mtls-refactor to support sslmode=verify-full. DB_PASS and md5 rules removed. Verified via pg_stat_ssl: ssl=t, client_dn=/CN=mnemosyne.

pgAdmin

Certificate: db-client-pgadmin-tls (CN=pgadmin). Pre-configured servers.json ConfigMap with cert paths pre-filled. rolloutRestartTargets added for cert rotation.

Vault Database Engine — Removed

The database/ engine has been disabled (vault secrets disable database, 2026-07-09). All apps use mTLS — no dynamic credentials needed. TD-044 closed. The Terraform configuration remains in the repository for re-creation if needed.

DAG Position

operators-data (Level 0, installs PGO)
→ configs (Level 2, creates s3-backrest-creds ExternalSecret)
→ instances (Level 3, creates PostgresCluster + service)
→ apps-data (Level 3, mnemosyne-mcp depends on DB being ready)

Known Issues

• The memories table is created with VECTOR(3072) but gemini-embedding-001 produces 768-dim vectors (TD-002). This works because the init SQL runs only on first creation and the dimension mismatch is handled by the auto-detection fallback.
• Only 1 replica — no HA for the database. Acceptable for lab, inadequate for production.

See Also

• Parent topics: Repository Mapping, Operators Inventory
• Sibling details: External Secrets Detail, Mnemosyne Deployment Detail
• Reference: ExternalSecret Example