TazLab K8s: teaching Detail

Level 3 (Detail) — Didactics application at teaching.tazlab.net: namespace, workload, exposure, Vault-delivered LLM key, and image automation.

Concept

teaching serves the seed-reproducible math/physics exercise generator at teaching.tazlab.net. It reuses the wiki/blog GitOps lane end to end (same Flux image automation, same cert-manager issuance, same Traefik ingress class), but it is not a static nginx site: it is a Python (FastAPI + SymPy) service on port 8080 with probes, resource limits, a read-only rootfs, and one Vault-delivered runtime secret. Its LLM key is delivered from Vault through VSO, the same pattern Hindsight and mnemosyne-mcp use.

Source repository: github.com/tazzo/teaching.tazlab.net (public, created 2026-09-24). Design record: CRISP project teaching-tazlab-net (DESIGN.md §2.1-§2.3, §2.9, §2.11).

Manifests

Cluster manifests (tazlab-k8s)

FilePurpose
apps/base/teaching/teaching.yamlDeployment + ClusterIP Service + TLS Ingress
apps/base/teaching/redirect-ingress.yamlSeparate web-only Ingress carrying the HTTP→HTTPS redirect
apps/base/teaching/middlewares.yamlTraefik redirectScheme Middleware
apps/base/teaching/certificate.yamlcert-manager Certificate for teaching.tazlab.net
apps/base/teaching/kustomization.yamlBase resource list
apps/cluster/teaching/kustomization.yamlOverlay → resources: ../../base/teaching (no patches)
infrastructure/operators/teaching/namespace.yamlNamespace teaching (registered in infrastructure/operators/core/kustomization.yaml)
infrastructure/operators/vso/sa-vso-auth-sa-teaching.yamlServiceAccount vso-auth-sa in teaching (VSO requirement)
infrastructure/operators/vso/vaultstaticsecret-teaching-opencode-key.yamlVaultStaticSecret → Secret teaching-opencode-key
infrastructure/automation/teaching/automation.yamlImageRepository + ImagePolicy + ImageUpdateAutomation
clusters/tazlab-k8s/apps-teaching.yamlThe Flux Kustomization — this file alone registers the app with Flux

Registration points (the app is invisible to Flux without them): infrastructure/operators/core/kustomization.yaml (../teaching), infrastructure/operators/vso/kustomization.yaml (the VSO ServiceAccount), infrastructure/operators/vso/vaultauth.yaml (the allow-list, below), infrastructure/operators/vso-secrets/kustomization.yaml (the VaultStaticSecret), infrastructure/cluster-instances/kustomization.yaml (../automation/teaching).

Deployment

FieldValue
Imagetazzo/tazlab-teaching:teaching-<N>-<sha>
Container port8080 (name: http)
Replicas1
StrategymaxSurge: 1, maxUnavailable: 0
Graceful shutdownterminationGracePeriodSeconds: 45 + preStop: sleep 5
Security contextnon-root uid/gid 10001, readOnlyRootFilesystem: true, all capabilities dropped, no privilege escalation
Writable scratchemptyDir mounted at /tmp (PDF renderer + Python), with HOME=/tmp and TMPDIR=/tmp
Requests / limits200m CPU / 256Mi · 2 CPU / 512Mi

Image line with the flux setter marker (the tag is rewritten by the ImageUpdateAutomation on every publication, so only the marker string is stable):

image: tazzo/tazlab-teaching:teaching-4-49afcaa45229d60726c8d8ba987ad8d2894a4438 # {"$imagepolicy": "flux-system:teaching"}

Environment contract

VariableSource / value
OPENCODE_API_KEYsecretKeyRef → teaching-opencode-key key value, optional: true
TEACHING_LLM_MODELdeepseek-v4.1-flash
TEACHING_LLM_REASONING_EFFORTnone
TEACHING_LLM_TIMEOUT20
TEACHING_LLM_MAX_CONCURRENCY4
TEACHING_LOG_LEVELINFO
HOME / TMPDIR/tmp

optional: true is deliberate: a missing key must not block the rollout, but it also means a broken Vault delivery is silent at the pod level — the failure surfaces only as degraded=true responses plus the startup WARN. The delivery chain below is therefore the thing to check first when variants stop working.

Probes

All three hit GET /api/healthz on the named port http; the endpoint is async and never touches SymPy, so it cannot be starved by the threadpool.

ProbePeriodFailure thresholdBudget
startup2s15~30s to become ready before liveness starts
readiness5s2~10s to be pulled from the Service
liveness20s5~100s before a restart

Service

ClusterIP, port: 80 → targetPort: http (8080). Nothing else in the cluster routes to it except the two Ingresses.

Exposure

ItemValue
Ingress classtraefik (cluster default, MetalLB VIP 192.168.1.240)
Main Ingressteaching-ingress — entrypoints web, websecure, router.tls: "true", host teaching.tazlab.net, TLS secret teaching-tazlab.net-tls
Redirect Ingressteaching-redirect-ingress — entrypoints web only, no spec.tls, middleware teaching-teaching-redirect-to-https@kubernetescrd, router.priority: "1"
Middlewareteaching-redirect-to-https — redirectScheme {scheme: https, permanent: true}
Certificateteaching-tazlab.net-tls, ClusterIssuer letsencrypt-issuer (HTTP-01 via Traefik), commonName + dnsNames teaching.tazlab.net, RSA 2048, rotationPolicy: Always
DNSnone required: *.tazlab.net is wildcard-resolved to the house IP, so the hostname already answered at the edge before the Certificate existed

Live: http://teaching.tazlab.net/ → 301 to https://teaching.tazlab.net/; https://teaching.tazlab.net/ → 200 with a valid chain; Certificate Ready=True (“up to date and has not expired”).

Why the redirect Ingress is a separate object

The site pattern (hugo-wiki, hugo-blog) carries router.entrypoints: web, websecure together with router.tls: "true", which produces an HTTPS-only router: cleartext HTTP never reaches the workload and Traefik answers 404 page not found (verified live on both sites). A redirectScheme middleware attached to that same router would therefore be a no-op — the router it is attached to does not exist on the web entrypoint for that host.

The redirect therefore requires a second Ingress on the web entrypoint alone, with the Middleware attached to it. The blog already runs a second Ingress for its apex hosts, but that one is a web, websecure TLS router carrying a redirectRegex to blog.tazlab.net; this one is web-only and carries no TLS section. The separate object is what makes the redirect reachable at all; it is not stylistic.

router.priority: "1" keeps it ACME-safe. cert-manager v1.16.2 creates a separate per-challenge solver Ingress (cm-acme-http-solver-*, HTTP-only, single /.well-known/acme-challenge/<token> path). Traefik v3 derives router priority from rule length, so the longer solver rule wins over the pinned application rule and HTTP-01 is served rather than redirected. An entrypoint-level redirect (Traefik’s internal router, priority MaxInt-1) would have outranked the solver and broken issuance — which is why the redirect is expressed as a router-level middleware instead.

Homepage tile

infrastructure/instances/homepage/homepage.yaml lists a Didattica tile under Developer (href: https://teaching.tazlab.net, icon mdi-school). It is a static services.yaml entry, not an Ingress annotation.

Vault-delivered OPENCODE_API_KEY

The only runtime secret; delivered by Vault Secrets Operator (VSO), not External Secrets (the repository contains zero ExternalSecret resources and the live cluster returns none cluster-wide).

File: infrastructure/operators/vso/vaultstaticsecret-teaching-opencode-key.yaml

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: teaching-opencode-key
  namespace: teaching
spec:
  vaultAuthRef: vso-system/vso-jwt-auth
  mount: secret
  type: kv-v2
  path: tazlab-k8s/static/apps/teaching/OPENCODE_API_KEY
  refreshAfter: 3600s
  destination:
    create: true
    name: teaching-opencode-key
    overwrite: true
  rolloutRestartTargets:
    - kind: Deployment
      name: teaching

The path follows the ecosystem convention for LLM keys (tazlab-k8s/static/apps/<consumer>/OPENCODE_API_KEY, as used by Hindsight). rolloutRestartTargets is what makes a rotated key take effect — the Deployment has no reloader annotation.

The vaultauth allow-list requirement

VaultAuth/vso-jwt-auth (namespace vso-system) carries an explicit spec.allowedNamespaces list; teaching was added to it:

allowedNamespaces:
  - monitoring
  - auth
  - dex
  - flux-system
  - cert-manager
  - longhorn-system
  - tazlab-db
  - cloudflare-ddns
  - hindsight
  - teaching

Without that entry the VaultStaticSecret is rejected with “namespace not allowed to use this auth method”, the Secret is never created, and — because the Deployment reads the key with optional: true — the pod starts normally and silently serves degraded responses. Two further prerequisites are already satisfied by construction: the JWT role binds bound_claims.sub = "system:serviceaccount:*:vso-auth-sa" (namespace glob, declared in ephemeral-castle), and the namespace needs its own ServiceAccount/vso-auth-sa.

Live evidence: VaultStaticSecret teaching-opencode-key → SYNCED=True HEALTHY=True READY=True; Secret teaching-opencode-key (Opaque, key value) exists in namespace teaching.

Runtime contract

ItemValue
Serveruvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 2
Proxy headers--forwarded-allow-ips 10.244.0.0/16 (the live pod CIDR — never *)
Shutdown--timeout-graceful-shutdown 25 (uvicorn’s default is to wait forever)
Access log--no-access-log (the app emits its own structured line per request)
Concurrency capnot set today: DESIGN §2.3 specifies --limit-concurrency 48 (503 on overload instead of queueing), but the image CMD omits it — overload protection currently rests on the 2 uvicorn workers alone
HealthGET /api/healthz → {"status":"ok","version":…}; never touches SymPy
Buildstage 1 node:24-bookworm (Vite bundle), stage 2 python:3.14-slim; the Node binary is copied into the runtime image because KaTeX renders server-side for the PDF export
Statestateless: no PVC, no database, no server-side session

The graph-reading page (cinematica-grafici-lettura)

Changed shape on 2026-09-30 (commit 74dadcc, the re-land of 1572410 onto the polish(web) frontend refresh). The page is no longer a form the operator configures and then prints: it is a random graph mill. Every configurator choice defaults to random, one click produces exactly one graph, and the generator itself draws the reading the motion can answer — the teacher asks the questions in person, from the graph.

Topic route: #/fisica/cinematica/cinematica-grafici-lettura; page id cinematica-grafici-lettura, kind graph_reading, backing topic physics.kinematics.segments. The registry entry (app/pages.py) pins it with configurable=True, count=1, show_statement=False.

What the configurator serves

Served from the topic’s own configurer() and rendered generically by the client:

ControlKindValuesDefault
Quanti tratti (config.segment_count)select3 / 4 / 5 segments3
Tipo di moto, tratto per tratto (config.segments)segment_kinds, count_from: countCasuale / uniform / accelerate / decelerate, one selector per segmentall Casuale
Grandezza sul grafico (config.quantity)selectCasuale / velocity–time / position–timeCasuale
Sistema di misura (config.units)selectCasuale / SI / km·h⁻¹·km / cm·s⁻¹·cmCasuale

The seed is a page-level control, not part of the descriptor: web/src/app.js renders a Casuale/Numero select next to the seed number. Casuale (the default) draws a new seed on every press of Genera; a number pins it, and the same number always returns the same exercise.

What is gone

  • No “what must the student read” control. The generator draws it (_draw_ask): velocity at an instant or total distance always, plus acceleration only when a drawn segment is not uniform. The option survives in the API for callers that name one — and is refused with a machine reason when an acceleration is asked of an all-uniform motion.
  • No Numero di esercizi field. The page pins count=1, and the client omits the count input when the page pins it: one click, one graph.
  • No statement above the graph. show_statement=False makes web/src/pages.js skip it — with no student to state the problem to, it would be noise. The statement is still generated and still verified, so the item stays a normal exercise for the PDF export and the tests.

The card is the graph, its legend, the collapsible steps and the answer.

Verified live on 2026-09-30 (GET /api/pages, then the page in a browser): the descriptor serves exactly the four controls above with count: 1 and show_statement: false; the DOM carries #seed-mode (Casuale/Numero) with #seed disabled while random, Genera, and no count input; one press yields one card, one <svg>, no .statement, one Passaggi block, and a second press with the random seed yields a different graph.

The descriptor contract — three files, or it is a silent no-op

The configurator descriptor and its renderer are a contract across three files:

FileRole
app/generators/physics/segments.pyemits the control dicts (configurer())
app/api/schemas.pydocuments them (ConfigControl: kind is "select" or "segment_kinds", count_from names the control that sizes a list)
web/src/config.jsrenders them — an unknown kind is skipped with a console warning, not an error

A new control kind must be added in all three. Nothing fails when the third is missing.

Same commit, four generator/steps defects

The random draw exposed bugs the deterministic configurations had hidden: a fully uniform motion tripped the “nothing is moving” guard and moved only its first segment (a 3-segment motion came out discontinuous); consecutive decelerating segments raised the speed mid-draw and moved a trace boundary, so a deceleration now gives up at most 3 m/s and half the speed; a pinned deceleration could run the motion to a stop or below and draw a negative speed, so the prefix is lifted by one constant (no Δv, therefore no change of kind or acceleration); and a velocity read off a converted position graph printed the SI slope under the axis’ own unit (3.5 m/s labelled km/h) — the conversion step now carries both.

Unit systems are a table of exact scales

The unit system is not a boolean any more: si, kmh (1 m/s = 18/5 km/h) and cm (×100 both) are two exact Fraction scales plus the labels the axes and the answer carry. A converted variant is therefore the same motion written differently, never a redrawn one, and every conversion in the module is one multiply. The acceleration answer stays in m/s², so a converted variant makes the student convert before dividing — and the steps show it.

Frontend test gate — web/src/ is tested, not only compiled

web/ has its own vitest suite, run inside the same test job as pytest. web/package.json declares "test": "vitest run"; web/vitest.config.js sets environment: "jsdom", include: ["test/**/*.test.js"], css: false, restoreMocks: true — the runner goes through Vite’s pipeline because config.js reaches KaTeX’s stylesheet and JSXGraph through pages.js, and jsdom is the environment the renderer is written for.

SuiteTestsWhat it drives
web/test/config.test.js9renderConfigurator against the descriptor in test/fixtures/pages.json, with the strings from app/i18n/it.json
web/test/app.test.js12main booted end to end on a jsdom document and a stubbed fetch: the shell, a configurator page, the graph-filling page, a page with no configurator

The assertions are behavioural, not snapshots — a snapshot passes just as happily on the wrong resolution of a merge conflict, which is the failure this gate exists for (TD-110). What they pin: one field per control and one option per choice, the served defaults preselected, one segment selector per segment as count_from says, a read() payload the generator accepts, an unknown control kind skipped rather than rendered broken, Genera sending the teacher’s choices (difficulty=easy, count=1, figure=full) and drawing the card back, the graph-filling page asking twice (hidden grid, then the model at the same seed), and a failed generation reported as invalid_options instead of swallowed.

The fixtures cannot drift from the server. tests/test_web_fixture.py (3 tests) asserts web/test/fixtures/pages.json equals GET /api/pages and that every items.json entry equals the response its recorded URL returns, so the suite cannot certify a page nobody serves; a third test proves the strings the DOM hands back are ones the generator accepts (a kinds list one shorter than count is refused 400 invalid_options). Regenerate with python web/test/refresh_fixtures.py when the catalogue moves.

Counts from run 36757786129 (commit df8a7b7, merged as c96ddd1, 2026-09-30): Test Files 2 passed (2) / Tests 21 passed (21) in 1.92s, then python -m pytest tests -q at 482 collected, 2 skipped. The test job ran 1m31s and build-and-push 1m26s.

What the gate still does not see: KaTeX’s stylesheet is not loaded (css: false), and figure.js and formula.js are exercised only through pages.js, never directly.

Logging contract

DESIGN §2.11 (“i log devono essere professionali, se qualcosa non va i log devono cantare”). The formatter, request-id plumbing and startup banner live in app/logging_setup.py; the request/response middleware, the quiet-path set and the X-Forwarded-For trust check live in app/main.py.

RuleDetail
Formatone JSON object per line on stdout; every line carries ts (RFC3339), level, logger, msg, request_id
Correlationa middleware assigns a request_id per request (contextvar), so one request is reconstructible from the stream
Request completionmethod, path, status, duration_ms, client IP — the IP is taken from X-Forwarded-For only for the trusted pod CIDR
Startupa banner with version, baked git sha and the resolved configuration; a missing OPENCODE_API_KEY logs a loud WARN naming the check (VaultStaticSecret) instead of failing silently
Generationper-item outcome (accepted / discarded + machine-readable reason); a discarded item is a WARN — it means a generator produced something unverifiable
LLMINFO on the provider call, WARN with the reason when the result is degraded (provider error class, or N specs rejected)
Errorsunhandled exceptions ERROR with class, message and stack trace; 4xx at WARN
Noise control/api/healthz at DEBUG; uvicorn loggers pinned to WARNING and uvicorn.access disabled; httpx/httpcore/weasyprint/fontTools at WARNING
Secretsthe provider key, the Docker Hub password and full request bodies never appear in any line

The acceptance test for this contract is behavioural: induce a failure and confirm the log line names the cause.

Image Automation

File: infrastructure/automation/teaching/automation.yaml

ResourceNameDetails
ImageRepositoryteachingtazzo/tazlab-teaching, poll 1m, no secretRef (public image)
ImagePolicyteachingTag ^teaching-(?P<value>[0-9]+)-.*$, extract: '$value', numerical asc
ImageUpdateAutomationteachingCommits to ./apps/base/teaching, strategy Setters, branch master, author fluxcdbot <fluxcdbot@tazlab.net>, message chore: self-update teaching image [skip ci]

Live: scan found 4 tags, latestRef = the running tag, ImageUpdateAutomation repository up-to-date. latest is published for humans only and is never consumed — the Policy regex requires the immutable <prefix>-<number>-<sha> shape.

Delivery Chain

teaching.tazlab.net (git push to master)
→ GitHub Action .github/workflows/publish.yml
   job `test` (container python:3.14-slim — the interpreter the service itself runs on):
     Pango/cairo/gdk-pixbuf + Node → pip install -e '.[dev]' → npm ci (KaTeX + dev deps) → npm test (vitest, web/) → python -m pytest tests -q
   job `build-and-push` (needs: test): checkout@v7, login-action@v4 (username roberto.tazzoli@gmail.com + secret DOCKER_PASSWORD), build-push-action@v7, GIT_SHA build-arg
→ Docker Hub tazzo/tazlab-teaching:teaching-<run_number>-<sha>  (+ :latest, humans only)
→ Flux ImageRepository (poll 1m) → ImagePolicy (numerical asc) → ImageUpdateAutomation (commit to master)
→ Flux Kustomization apps-teaching (path ./apps/cluster/teaching, interval 15m, prune, wait)
→ Deployment rollout (maxUnavailable 0, probes gate the Service)

The image job cannot run on a red suite (needs: test, TD-077 closed 2026-09-25): four rendering defects had reached the live site through this pipeline before the gate existed, and each would have been caught by it. The test job runs in the same Python major as production (a different interpreter could pass while production fails) and installs the rendering path’s real dependencies (WeasyPrint’s Pango/cairo stack and the server-side KaTeX copy), not a stub of them.

That gate now covers both halves of the service: “a green pipeline means the exercises are correct” holds for the generators, the API and the server-side render, and since 2026-09-30 (TD-110) for the page that renders them — see §Frontend test gate. The JS step sits between npm ci and pytest, so the dev dependencies are already on disk when npm test runs and a red frontend stops the publish before the image job.

The workflow adopts the current action majors (site workflows still pin @v4/@v3/@v5); the Docker Hub username is hardcoded as in the sibling pipelines (a past failure used the namespace name instead), the password exists only as a per-repository GitHub secret.

DAG Position

infrastructure-operators-core  (creates Namespace teaching)
infrastructure-operators-vso   (installs VSO; allowedNamespaces must list teaching)
infrastructure-operators-vso-secrets (creates Secret teaching-opencode-key)
infrastructure-configs         (ClusterIssuer letsencrypt-issuer; no ExternalSecrets exist)
→ apps-teaching (Level 3, deploys teaching: Deployment + Service + 2 Ingresses + Certificate)

apps-teaching dependsOn: infrastructure-configs, interval: 15m, prune: true, postBuild.substituteFrom: cluster-vars, blocking (default wait: true). Image automation is not a Kustomization: it is registered in infrastructure/cluster-instances/kustomization.yaml alongside the other four pipelines.

Sequence note: until the Certificate is Ready, the hostname presents the in-cluster Vault-PKI wildcard CN=*.tazlab.net (untrusted CA), so HTTPS verification must happen after certificate readiness, not at rollout time.

Deviations From the Site Lane

Aspecthugo-wiki / hugo-blogteaching
Runtimenginx, port 80uvicorn, port 8080
Probes / limitsnone declaredstartup+readiness+liveness, explicit requests/limits
Cleartext HTTP404 (HTTPS-only router)301 → HTTPS (second, web-only Ingress)
Runtime secretnoneOPENCODE_API_KEY via VSO from Vault
Filesystemdefaultread-only rootfs + emptyDir at /tmp
TLSown Certificate per siteown Certificate, own secretName

See Also