TazLab K8s: teaching Detail
Level 3 (Detail) — Didactics application at teaching.tazlab.net: namespace, workload, exposure, Vault-delivered LLM key, and image automation.
Concept
teaching serves the seed-reproducible math/physics exercise generator at teaching.tazlab.net. It reuses the wiki/blog GitOps lane end to end (same Flux image automation, same cert-manager issuance, same Traefik ingress class), but it is not a static nginx site: it is a Python (FastAPI + SymPy) service on port 8080 with probes, resource limits, a read-only rootfs, and one Vault-delivered runtime secret. Its LLM key is delivered from Vault through VSO, the same pattern Hindsight and mnemosyne-mcp use.
Source repository: github.com/tazzo/teaching.tazlab.net (public, created 2026-09-24). Design record: CRISP project teaching-tazlab-net (DESIGN.md §2.1-§2.3, §2.9, §2.11).
Manifests
Cluster manifests (tazlab-k8s)
| File | Purpose |
|---|---|
apps/base/teaching/teaching.yaml | Deployment + ClusterIP Service + TLS Ingress |
apps/base/teaching/redirect-ingress.yaml | Separate web-only Ingress carrying the HTTP→HTTPS redirect |
apps/base/teaching/middlewares.yaml | Traefik redirectScheme Middleware |
apps/base/teaching/certificate.yaml | cert-manager Certificate for teaching.tazlab.net |
apps/base/teaching/kustomization.yaml | Base resource list |
apps/cluster/teaching/kustomization.yaml | Overlay → resources: ../../base/teaching (no patches) |
infrastructure/operators/teaching/namespace.yaml | Namespace teaching (registered in infrastructure/operators/core/kustomization.yaml) |
infrastructure/operators/vso/sa-vso-auth-sa-teaching.yaml | ServiceAccount vso-auth-sa in teaching (VSO requirement) |
infrastructure/operators/vso/vaultstaticsecret-teaching-opencode-key.yaml | VaultStaticSecret → Secret teaching-opencode-key |
infrastructure/automation/teaching/automation.yaml | ImageRepository + ImagePolicy + ImageUpdateAutomation |
clusters/tazlab-k8s/apps-teaching.yaml | The Flux Kustomization — this file alone registers the app with Flux |
Registration points (the app is invisible to Flux without them): infrastructure/operators/core/kustomization.yaml (../teaching), infrastructure/operators/vso/kustomization.yaml (the VSO ServiceAccount), infrastructure/operators/vso/vaultauth.yaml (the allow-list, below), infrastructure/operators/vso-secrets/kustomization.yaml (the VaultStaticSecret), infrastructure/cluster-instances/kustomization.yaml (../automation/teaching).
Deployment
| Field | Value |
|---|---|
| Image | tazzo/tazlab-teaching:teaching-<N>-<sha> |
| Container port | 8080 (name: http) |
| Replicas | 1 |
| Strategy | maxSurge: 1, maxUnavailable: 0 |
| Graceful shutdown | terminationGracePeriodSeconds: 45 + preStop: sleep 5 |
| Security context | non-root uid/gid 10001, readOnlyRootFilesystem: true, all capabilities dropped, no privilege escalation |
| Writable scratch | emptyDir mounted at /tmp (PDF renderer + Python), with HOME=/tmp and TMPDIR=/tmp |
| Requests / limits | 200m CPU / 256Mi · 2 CPU / 512Mi |
Image line with the flux setter marker (the tag is rewritten by the ImageUpdateAutomation on every publication, so only the marker string is stable):
image: tazzo/tazlab-teaching:teaching-4-49afcaa45229d60726c8d8ba987ad8d2894a4438 # {"$imagepolicy": "flux-system:teaching"}
Environment contract
| Variable | Source / value |
|---|---|
OPENCODE_API_KEY | secretKeyRef → teaching-opencode-key key value, optional: true |
TEACHING_LLM_MODEL | deepseek-v4.1-flash |
TEACHING_LLM_REASONING_EFFORT | none |
TEACHING_LLM_TIMEOUT | 20 |
TEACHING_LLM_MAX_CONCURRENCY | 4 |
TEACHING_LOG_LEVEL | INFO |
HOME / TMPDIR | /tmp |
optional: true is deliberate: a missing key must not block the rollout, but it also means a broken Vault delivery is silent at the pod level — the failure surfaces only as degraded=true responses plus the startup WARN. The delivery chain below is therefore the thing to check first when variants stop working.
Probes
All three hit GET /api/healthz on the named port http; the endpoint is async and never touches SymPy, so it cannot be starved by the threadpool.
| Probe | Period | Failure threshold | Budget |
|---|---|---|---|
| startup | 2s | 15 | ~30s to become ready before liveness starts |
| readiness | 5s | 2 | ~10s to be pulled from the Service |
| liveness | 20s | 5 | ~100s before a restart |
Service
ClusterIP, port: 80 → targetPort: http (8080). Nothing else in the cluster routes to it except the two Ingresses.
Exposure
| Item | Value |
|---|---|
| Ingress class | traefik (cluster default, MetalLB VIP 192.168.1.240) |
| Main Ingress | teaching-ingress — entrypoints web, websecure, router.tls: "true", host teaching.tazlab.net, TLS secret teaching-tazlab.net-tls |
| Redirect Ingress | teaching-redirect-ingress — entrypoints web only, no spec.tls, middleware teaching-teaching-redirect-to-https@kubernetescrd, router.priority: "1" |
| Middleware | teaching-redirect-to-https — redirectScheme {scheme: https, permanent: true} |
| Certificate | teaching-tazlab.net-tls, ClusterIssuer letsencrypt-issuer (HTTP-01 via Traefik), commonName + dnsNames teaching.tazlab.net, RSA 2048, rotationPolicy: Always |
| DNS | none required: *.tazlab.net is wildcard-resolved to the house IP, so the hostname already answered at the edge before the Certificate existed |
Live: http://teaching.tazlab.net/ → 301 to https://teaching.tazlab.net/; https://teaching.tazlab.net/ → 200 with a valid chain; Certificate Ready=True (“up to date and has not expired”).
Why the redirect Ingress is a separate object
The site pattern (hugo-wiki, hugo-blog) carries router.entrypoints: web, websecure together with router.tls: "true", which produces an HTTPS-only router: cleartext HTTP never reaches the workload and Traefik answers 404 page not found (verified live on both sites). A redirectScheme middleware attached to that same router would therefore be a no-op — the router it is attached to does not exist on the web entrypoint for that host.
The redirect therefore requires a second Ingress on the web entrypoint alone, with the Middleware attached to it. The blog already runs a second Ingress for its apex hosts, but that one is a web, websecure TLS router carrying a redirectRegex to blog.tazlab.net; this one is web-only and carries no TLS section. The separate object is what makes the redirect reachable at all; it is not stylistic.
router.priority: "1" keeps it ACME-safe. cert-manager v1.16.2 creates a separate per-challenge solver Ingress (cm-acme-http-solver-*, HTTP-only, single /.well-known/acme-challenge/<token> path). Traefik v3 derives router priority from rule length, so the longer solver rule wins over the pinned application rule and HTTP-01 is served rather than redirected. An entrypoint-level redirect (Traefik’s internal router, priority MaxInt-1) would have outranked the solver and broken issuance — which is why the redirect is expressed as a router-level middleware instead.
Homepage tile
infrastructure/instances/homepage/homepage.yaml lists a Didattica tile under Developer (href: https://teaching.tazlab.net, icon mdi-school). It is a static services.yaml entry, not an Ingress annotation.
Vault-delivered OPENCODE_API_KEY
The only runtime secret; delivered by Vault Secrets Operator (VSO), not External Secrets (the repository contains zero ExternalSecret resources and the live cluster returns none cluster-wide).
File: infrastructure/operators/vso/vaultstaticsecret-teaching-opencode-key.yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: teaching-opencode-key
namespace: teaching
spec:
vaultAuthRef: vso-system/vso-jwt-auth
mount: secret
type: kv-v2
path: tazlab-k8s/static/apps/teaching/OPENCODE_API_KEY
refreshAfter: 3600s
destination:
create: true
name: teaching-opencode-key
overwrite: true
rolloutRestartTargets:
- kind: Deployment
name: teaching
The path follows the ecosystem convention for LLM keys (tazlab-k8s/static/apps/<consumer>/OPENCODE_API_KEY, as used by Hindsight). rolloutRestartTargets is what makes a rotated key take effect — the Deployment has no reloader annotation.
The vaultauth allow-list requirement
VaultAuth/vso-jwt-auth (namespace vso-system) carries an explicit spec.allowedNamespaces list; teaching was added to it:
allowedNamespaces:
- monitoring
- auth
- dex
- flux-system
- cert-manager
- longhorn-system
- tazlab-db
- cloudflare-ddns
- hindsight
- teaching
Without that entry the VaultStaticSecret is rejected with “namespace not allowed to use this auth method”, the Secret is never created, and — because the Deployment reads the key with optional: true — the pod starts normally and silently serves degraded responses. Two further prerequisites are already satisfied by construction: the JWT role binds bound_claims.sub = "system:serviceaccount:*:vso-auth-sa" (namespace glob, declared in ephemeral-castle), and the namespace needs its own ServiceAccount/vso-auth-sa.
Live evidence: VaultStaticSecret teaching-opencode-key → SYNCED=True HEALTHY=True READY=True; Secret teaching-opencode-key (Opaque, key value) exists in namespace teaching.
Runtime contract
| Item | Value |
|---|---|
| Server | uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 2 |
| Proxy headers | --forwarded-allow-ips 10.244.0.0/16 (the live pod CIDR — never *) |
| Shutdown | --timeout-graceful-shutdown 25 (uvicorn’s default is to wait forever) |
| Access log | --no-access-log (the app emits its own structured line per request) |
| Concurrency cap | not set today: DESIGN §2.3 specifies --limit-concurrency 48 (503 on overload instead of queueing), but the image CMD omits it — overload protection currently rests on the 2 uvicorn workers alone |
| Health | GET /api/healthz → {"status":"ok","version":…}; never touches SymPy |
| Build | stage 1 node:24-bookworm (Vite bundle), stage 2 python:3.14-slim; the Node binary is copied into the runtime image because KaTeX renders server-side for the PDF export |
| State | stateless: no PVC, no database, no server-side session |
The graph-reading page (cinematica-grafici-lettura)
Changed shape on 2026-09-30 (commit 74dadcc, the re-land of 1572410 onto the polish(web)
frontend refresh). The page is no longer a form the operator configures and then prints: it is
a random graph mill. Every configurator choice defaults to random, one click produces
exactly one graph, and the generator itself draws the reading the motion can answer — the
teacher asks the questions in person, from the graph.
Topic route: #/fisica/cinematica/cinematica-grafici-lettura; page id
cinematica-grafici-lettura, kind graph_reading, backing topic
physics.kinematics.segments. The registry entry (app/pages.py) pins it with
configurable=True, count=1, show_statement=False.
What the configurator serves
Served from the topic’s own configurer() and rendered generically by the client:
| Control | Kind | Values | Default |
|---|---|---|---|
Quanti tratti (config.segment_count) | select | 3 / 4 / 5 segments | 3 |
Tipo di moto, tratto per tratto (config.segments) | segment_kinds, count_from: count | Casuale / uniform / accelerate / decelerate, one selector per segment | all Casuale |
Grandezza sul grafico (config.quantity) | select | Casuale / velocity–time / position–time | Casuale |
Sistema di misura (config.units) | select | Casuale / SI / km·h⁻¹·km / cm·s⁻¹·cm | Casuale |
The seed is a page-level control, not part of the descriptor: web/src/app.js renders a
Casuale/Numero select next to the seed number. Casuale (the default) draws a new seed
on every press of Genera; a number pins it, and the same number always returns the same
exercise.
What is gone
- No “what must the student read” control. The generator draws it (
_draw_ask): velocity at an instant or total distance always, plus acceleration only when a drawn segment is not uniform. The option survives in the API for callers that name one — and is refused with a machine reason when an acceleration is asked of an all-uniform motion. - No
Numero di esercizifield. The page pinscount=1, and the client omits the count input when the page pins it: one click, one graph. - No statement above the graph.
show_statement=Falsemakesweb/src/pages.jsskip it — with no student to state the problem to, it would be noise. The statement is still generated and still verified, so the item stays a normal exercise for the PDF export and the tests.
The card is the graph, its legend, the collapsible steps and the answer.
Verified live on 2026-09-30 (GET /api/pages, then the page in a browser): the descriptor
serves exactly the four controls above with count: 1 and show_statement: false; the DOM
carries #seed-mode (Casuale/Numero) with #seed disabled while random, Genera, and
no count input; one press yields one card, one <svg>, no .statement, one Passaggi
block, and a second press with the random seed yields a different graph.
The descriptor contract — three files, or it is a silent no-op
The configurator descriptor and its renderer are a contract across three files:
| File | Role |
|---|---|
app/generators/physics/segments.py | emits the control dicts (configurer()) |
app/api/schemas.py | documents them (ConfigControl: kind is "select" or "segment_kinds", count_from names the control that sizes a list) |
web/src/config.js | renders them — an unknown kind is skipped with a console warning, not an error |
A new control kind must be added in all three. Nothing fails when the third is missing.
Same commit, four generator/steps defects
The random draw exposed bugs the deterministic configurations had hidden: a fully uniform motion tripped the “nothing is moving” guard and moved only its first segment (a 3-segment motion came out discontinuous); consecutive decelerating segments raised the speed mid-draw and moved a trace boundary, so a deceleration now gives up at most 3 m/s and half the speed; a pinned deceleration could run the motion to a stop or below and draw a negative speed, so the prefix is lifted by one constant (no Δv, therefore no change of kind or acceleration); and a velocity read off a converted position graph printed the SI slope under the axis’ own unit (3.5 m/s labelled km/h) — the conversion step now carries both.
Unit systems are a table of exact scales
The unit system is not a boolean any more: si, kmh (1 m/s = 18/5 km/h) and cm
(×100 both) are two exact Fraction scales plus the labels the axes and the answer carry.
A converted variant is therefore the same motion written differently, never a redrawn one,
and every conversion in the module is one multiply. The acceleration answer stays in m/s²,
so a converted variant makes the student convert before dividing — and the steps show it.
Frontend test gate — web/src/ is tested, not only compiled
web/ has its own vitest suite, run inside the same test job as pytest. web/package.json
declares "test": "vitest run"; web/vitest.config.js sets environment: "jsdom",
include: ["test/**/*.test.js"], css: false, restoreMocks: true — the runner goes through
Vite’s pipeline because config.js reaches KaTeX’s stylesheet and JSXGraph through pages.js,
and jsdom is the environment the renderer is written for.
| Suite | Tests | What it drives |
|---|---|---|
web/test/config.test.js | 9 | renderConfigurator against the descriptor in test/fixtures/pages.json, with the strings from app/i18n/it.json |
web/test/app.test.js | 12 | main booted end to end on a jsdom document and a stubbed fetch: the shell, a configurator page, the graph-filling page, a page with no configurator |
The assertions are behavioural, not snapshots — a snapshot passes just as happily on the wrong
resolution of a merge conflict, which is the failure this gate exists for (TD-110). What they
pin: one field per control and one option per choice, the served defaults preselected, one
segment selector per segment as count_from says, a read() payload the generator accepts,
an unknown control kind skipped rather than rendered broken, Genera sending the teacher’s
choices (difficulty=easy, count=1, figure=full) and drawing the card back, the
graph-filling page asking twice (hidden grid, then the model at the same seed), and a failed
generation reported as invalid_options instead of swallowed.
The fixtures cannot drift from the server. tests/test_web_fixture.py (3 tests) asserts
web/test/fixtures/pages.json equals GET /api/pages and that every items.json entry equals
the response its recorded URL returns, so the suite cannot certify a page nobody serves; a
third test proves the strings the DOM hands back are ones the generator accepts (a kinds
list one shorter than count is refused 400 invalid_options). Regenerate with
python web/test/refresh_fixtures.py when the catalogue moves.
Counts from run 36757786129 (commit df8a7b7, merged as c96ddd1, 2026-09-30):
Test Files 2 passed (2) / Tests 21 passed (21) in 1.92s, then python -m pytest tests -q
at 482 collected, 2 skipped. The test job ran 1m31s and build-and-push 1m26s.
What the gate still does not see: KaTeX’s stylesheet is not loaded (css: false), and
figure.js and formula.js are exercised only through pages.js, never directly.
Logging contract
DESIGN §2.11 (“i log devono essere professionali, se qualcosa non va i log devono cantare”). The formatter, request-id plumbing and startup banner live in app/logging_setup.py; the request/response middleware, the quiet-path set and the X-Forwarded-For trust check live in app/main.py.
| Rule | Detail |
|---|---|
| Format | one JSON object per line on stdout; every line carries ts (RFC3339), level, logger, msg, request_id |
| Correlation | a middleware assigns a request_id per request (contextvar), so one request is reconstructible from the stream |
| Request completion | method, path, status, duration_ms, client IP — the IP is taken from X-Forwarded-For only for the trusted pod CIDR |
| Startup | a banner with version, baked git sha and the resolved configuration; a missing OPENCODE_API_KEY logs a loud WARN naming the check (VaultStaticSecret) instead of failing silently |
| Generation | per-item outcome (accepted / discarded + machine-readable reason); a discarded item is a WARN — it means a generator produced something unverifiable |
| LLM | INFO on the provider call, WARN with the reason when the result is degraded (provider error class, or N specs rejected) |
| Errors | unhandled exceptions ERROR with class, message and stack trace; 4xx at WARN |
| Noise control | /api/healthz at DEBUG; uvicorn loggers pinned to WARNING and uvicorn.access disabled; httpx/httpcore/weasyprint/fontTools at WARNING |
| Secrets | the provider key, the Docker Hub password and full request bodies never appear in any line |
The acceptance test for this contract is behavioural: induce a failure and confirm the log line names the cause.
Image Automation
File: infrastructure/automation/teaching/automation.yaml
| Resource | Name | Details |
|---|---|---|
| ImageRepository | teaching | tazzo/tazlab-teaching, poll 1m, no secretRef (public image) |
| ImagePolicy | teaching | Tag ^teaching-(?P<value>[0-9]+)-.*$, extract: '$value', numerical asc |
| ImageUpdateAutomation | teaching | Commits to ./apps/base/teaching, strategy Setters, branch master, author fluxcdbot <fluxcdbot@tazlab.net>, message chore: self-update teaching image [skip ci] |
Live: scan found 4 tags, latestRef = the running tag, ImageUpdateAutomation repository up-to-date. latest is published for humans only and is never consumed — the Policy regex requires the immutable <prefix>-<number>-<sha> shape.
Delivery Chain
teaching.tazlab.net (git push to master)
→ GitHub Action .github/workflows/publish.yml
job `test` (container python:3.14-slim — the interpreter the service itself runs on):
Pango/cairo/gdk-pixbuf + Node → pip install -e '.[dev]' → npm ci (KaTeX + dev deps) → npm test (vitest, web/) → python -m pytest tests -q
job `build-and-push` (needs: test): checkout@v7, login-action@v4 (username roberto.tazzoli@gmail.com + secret DOCKER_PASSWORD), build-push-action@v7, GIT_SHA build-arg
→ Docker Hub tazzo/tazlab-teaching:teaching-<run_number>-<sha> (+ :latest, humans only)
→ Flux ImageRepository (poll 1m) → ImagePolicy (numerical asc) → ImageUpdateAutomation (commit to master)
→ Flux Kustomization apps-teaching (path ./apps/cluster/teaching, interval 15m, prune, wait)
→ Deployment rollout (maxUnavailable 0, probes gate the Service)
The image job cannot run on a red suite (needs: test, TD-077 closed 2026-09-25): four rendering defects had reached the live site through this pipeline before the gate existed, and each would have been caught by it. The test job runs in the same Python major as production (a different interpreter could pass while production fails) and installs the rendering path’s real dependencies (WeasyPrint’s Pango/cairo stack and the server-side KaTeX copy), not a stub of them.
That gate now covers both halves of the service: “a green pipeline means the exercises are correct” holds for the generators, the API and the server-side render, and since 2026-09-30 (TD-110) for the page that renders them — see §Frontend test gate. The JS step sits between npm ci and pytest, so the dev dependencies are already on disk when npm test runs and a red frontend stops the publish before the image job.
The workflow adopts the current action majors (site workflows still pin @v4/@v3/@v5); the Docker Hub username is hardcoded as in the sibling pipelines (a past failure used the namespace name instead), the password exists only as a per-repository GitHub secret.
DAG Position
infrastructure-operators-core (creates Namespace teaching)
infrastructure-operators-vso (installs VSO; allowedNamespaces must list teaching)
infrastructure-operators-vso-secrets (creates Secret teaching-opencode-key)
infrastructure-configs (ClusterIssuer letsencrypt-issuer; no ExternalSecrets exist)
→ apps-teaching (Level 3, deploys teaching: Deployment + Service + 2 Ingresses + Certificate)
apps-teaching dependsOn: infrastructure-configs, interval: 15m, prune: true, postBuild.substituteFrom: cluster-vars, blocking (default wait: true). Image automation is not a Kustomization: it is registered in infrastructure/cluster-instances/kustomization.yaml alongside the other four pipelines.
Sequence note: until the Certificate is Ready, the hostname presents the in-cluster Vault-PKI wildcard CN=*.tazlab.net (untrusted CA), so HTTPS verification must happen after certificate readiness, not at rollout time.
Deviations From the Site Lane
| Aspect | hugo-wiki / hugo-blog | teaching |
|---|---|---|
| Runtime | nginx, port 80 | uvicorn, port 8080 |
| Probes / limits | none declared | startup+readiness+liveness, explicit requests/limits |
| Cleartext HTTP | 404 (HTTPS-only router) | 301 → HTTPS (second, web-only Ingress) |
| Runtime secret | none | OPENCODE_API_KEY via VSO from Vault |
| Filesystem | default | read-only rootfs + emptyDir at /tmp |
| TLS | own Certificate per site | own Certificate, own secretName |
See Also
- Parent hub: tazlab-k8s
- Sibling details: hugo-wiki Detail, hugo-blog Detail, Flux Kustomizations Detail, Image Automation Detail, cert-manager Detail, Traefik Detail
- Secret delivery: Secrets Mapping
- Source repository: github.com/tazzo/teaching.tazlab.net