Files
ZitPlane/project/CI-ARCHITECTURE.md
T

22 KiB
Raw Blame History

ZitPlane P05-CI-01 - Gitea Actions verification architecture

Status: P05-CI-01 Run #81 overall FAIL; P05-CI-02 Run #83 3 PASS / 1 FAIL (Account, Login, Security pass; backend proxy host-port collision). CI-only portless source correction after Run #83 remains unverified. Full OIDC and runtime acceptance pending. Approved baseline: 7ecf53169776cc7f08af3dde6fe4b4492acf3ea1 (2026-11-22).

Exact job map

Job Runner / source What PASS means Not covered
account ubuntu-latest, node:22-bookworm; users/ui/ Structural guard + npm ci, tsc --noEmit, Next standalone production build Real login/redirect, live Account Portal
login ubuntu-latest, node:22-bookworm; auth/ui/v2/ Frozen pnpm install, Nx proto/client/login build and original Login unit suite Interactive admin/password/MFA/SSO integration
security ubuntu-latest, node:22-bookworm NPM and pnpm production dependency advisories at HIGH+ thresholds triaged by runner registry License/legal or full application security proof
backend ubuntu-latest, node:22-alpine; deployment/docker/compose.backend.dev.yaml read-only Docker daemon preflight, Python helper contracts, disposable ZITADEL stack, readiness, OIDC discovery and JWKS Portal OIDC callback, MFA, live account session, production TLS

All jobs use the previously seen Gitea standard ubuntu-latest runner label; no separate runner is created or installed. A failed runner capability is a BLOCKED gate, never a PASS. Docker job must not run from an untrusted pull request; the workflow is workflow_dispatch only.

Dependency/root cause fixes

  • auth/ui/v2/pnpm-workspace.yaml has only the 3 actually imported packages. P03's original pnpm-lock.yaml retains exact packages:/snapshots: resolution graph; 4 missing-upstream importers were removed, 4 present importers retained.
  • Account Portal package-lock v3 generated using local npm install --package-lock-only --ignore-scripts --no-audit --no-fund --engine-strict, not an install/test/build. Root package manifest matches lock manifest.
  • Login CI sets up pinned pnpm 10.30.3, runs --frozen-lockfile and the upstream Nx dependency graph; no upstream UI/auth logic was modified.
  • Account CI uses npm ci only from committed lock, plus TS + production Next build. No automatically loosened package versions or npm install fallback.
  • Security jobs audit dependency advisory metadata on the runner. Any failure is reported verbatim with severity/package/version; no blind package updates.

Disposable backend safety and behavior

The P02 Compose has a fixed named network zitplane-backend-dev-internal and loopback-only 127.0.0.1:8080. CI refuses to start if that network already exists. It generates independent random .env in runner checkout and uses unique Compose project name zitplane-p05-${GITHUB_RUN_ID}. A marker is written only after the network absence check; cleanup uses that exact Compose project, deletes its ephemeral volumes, and removes its temporary credentials. It never connects to auth.vib.tools, account.vib.tools or any production database.

tests/ci/oidc_discovery.py probes ready/discovery/JWKS inside disposable Docker network only, avoiding exposing admin PATs. OIDC full user-session acceptance is explicitly NOT VERIFIED: P05 does not provision a disposable confidential OIDC app, fake user or Session API script. A 200 readiness/discovery alone does not constitute user login, MFA, PKCE callback, protected account route or logout acceptance.

Future gates

  1. Document actual Gitea runner job results and exact failure/line in notes/P05-CI-01.md and docs/P05-CI-01.md.
  2. If any job fails, keep immutable logged evidence; fix only reproduced narrow defects with separate approval if outside P05 Scope Lock.
  3. Provide disposable OIDC client/user provisioning and Account Portal auth/session lifecycle tests in a narrow separately approved integration phase.
  4. Production Docker/Coolify/TLS and DNS are out of scope.

Reference: notes/P05-CI-01.md, docs/PHASES.md, .gitea/workflows/p05-ci.yml, tests/ci/.

P05-CI-02 Narrow Auth.js and CI additions

  • users/ui/src/auth.ts callbacks.session: const subject: string = typeof token.zitadelSubject === "string" ? token.zitadelSubject : ""; explicitly narrows the JWT claim. Session still sets id only when subject nonempty and OIDC access-token expiry is in the future. users/ui/src/types/next-auth.d.ts was not changed. This addresses Run #81's TS2345 (user.id: {}) without a cast or altered PKCE/state/nonce.
  • tests/ci/account_negative.py + Account job: anonymous identity API 401/no-store and protected route redirect; no fake logged-in account. tests/ci/oidc_discovery.py: issuer/origin/JWKS negative contracts plus actual disposable ready/discovery/JWKS. Tests are CI-only.
  • tests/ci/advisory_triage.py: parse live npm/pnpm registry JSON into advisory IDs, package, affected range, patch availability. Prior high/critical failure threshold maintained; no dependency versions changed.
  • Backend CI: after safe Docker/network preflight and startup, write one marker in disposable Postgres database postgres, restart its container, verify marker persists and ZITADEL becomes ready, then test OIDC discovery/JWKS. Always cleanup only the current CI Compose project and its volumes; failure must remain visible.
  • Corrected Alpine Git checkout from P05-CI-01 is exercised by the new run. Fixed name zitplane-backend-dev-internal is a known isolation limitation: existing network triggers fail-closed instead of reuse.
  • No full interactive user login callback, MFA, Admin Console user session or production TLS verification; separate future scope.

Verified Gitea Run #81 and source follow-up

Initial dispatched commit 42b79362c13b9dbb07f85a58ad278742d91b1f5b. Two PASS: Login V2 frozen reproducible install, production build and 1219/1219 upstream unit tests; dependency advisory gate at high/critical only (18 moderate findings outstanding). Two FAIL: Account Portal TS2345 Auth.js session callback type mismatch; Alpine backend actions/checkout fallback Not found because Git missing before checkout. The Docker runtime and full interactive OIDC flow were not executed.

Pre-checkout Git install inserted in the backend workflow after this failed run; this is only a source fix until a future run actually passes. Reference docs/P05-CI-01.md for job numbers and notes/P05-CI-01.md for the next phase defect IDs. There was no second CI run or local test.

P05-CI-02 Run #83 final evidence and post-run portless exception

Gitea Run #83 on SHA c27fe3d3234d16375b962b88014cdd8733ab2fb4: three job PASS (Account npm ci/typecheck/build/anonymous 401 and protected redirect; Login V2 frozen/Nx build + 1219 tests; HIGH/CRITICAL security audit with 18 Moderate Login records and 1 Moderate Account package); Docker job FAIL after Postgres, ZITADEL API and prebuilt Login V2 reported healthy. Traefik proxy attempted to bind occupied runner TCP 8080. Always cleanup removed only job-owned containers, network and volumes; persistence and OIDC discovery were not reached.

Only after that run, a P05-scoped CI overlay at tests/ci/compose.portless.override.yaml set proxy.ports: !reset [], as documented by Docker Compose: https://docs.docker.com/reference/compose-file/merge/#reset-value. Every backend workflow Compose call now includes this override; a fail-closed effective-config probe rejects any inherited host port. The P02 Compose and application logic remain untouched. Post-run override has not been CI tested. New exact Gitea main commit is documentation/source only and must not be described as the tested Run #83 SHA.

Unresolved: one separately approved isolated backend re-verification cycle for the portless override, Postgres restart persistence and live OIDC/JWKS; distinct disposable authenticated OIDC/client/user + MFA E2E; targeted dependency-only security remediation of Moderate findings (no upgrades in P05-CI-02).

P05-CI-03 — backend-only disposable acceptance

This is a separate .gitea/workflows/p05-backend-acceptance.yml triggered manually on Gitea main. It does not schedule the previous Account/ Login V2/ security jobs again.

  • A short Alpine Git prerequisite precedes actions/checkout@v4. After checkout, only Python 3 stdlib and Docker CLI/Compose are installed in CI (not on the developer PC).
  • tests/ci/backend_acceptance.sh accepts only a numeric GITHUB_RUN_ID; derives a unique Compose project zitplane-p05-<run-id>; refuses existing hardcoded P02 network, same project containers or volumes. An existing local .env or ownership marker also causes fail-closed.
  • The original deployment/docker/compose.backend.dev.yaml is read-only; tests/ci/compose.portless.override.yaml removes proxy.ports and JSON merged configuration explicitly rejects host publishing. This isolates the shared Actions Docker host from an occupied TCP 8080.
  • After source-level P02 unit contracts (CI only) and fresh ignored .env creation, a marker .p05-ci-owned is created only after all collision/config checks. Then docker compose up -d --wait runs official pinned PostgreSQL, ZITADEL API, Login V2 and Traefik images. Postgres/API/Login need healthy, proxy needs running.
  • A synthetic marker row in disposable Postgres default postgres DB is written then validated after a database container restart and Compose health recovery.
  • Existing tests/ci/oidc_discovery.py runs in a throwaway python:3.12-alpine container on the owned internal network, validates negative issuer/JWKS fixtures, real /debug/ready, exact OIDC issuer/metadata and a nonempty public JWKS.
  • tests/ci/backend_cleanup.sh is run by an always step: it must find an ownership marker before any Docker teardown; only its Compose project is removed with down -v. Cleanup then checks that no owned containers, volumes or the fixed network remain. A missing marker means Docker was never started by this run and cleanup is not required; not a backend PASS.

This phase can verify local disposable backend runtime only. It cannot certify real account login, Admin Console session, MFA, global SSO logout, production TLS/Coolify or open Moderate package vulnerabilities. See docs/P05-CI-03.md for actual run evidence and notes/P05-CI-03.md for approval/scope.

P05-CI-03 verified runtime outcome — Gitea Run #86

Gitea manual backend-only Run #86, job #184, tested 03fb58954dec886128a4011b6121754cbcdd79ff; SUCCESS. The Git checkout, isolated acceptance and always-run cleanup steps all succeeded. P02 3/3 backend contract tests, combined portless config, four service health, disposable Postgres write/restart/read marker, local ZITADEL readiness, exact issuer metadata, OIDC discovery, nonempty JWKS and malformed-input contract checks each produced real PASS markers. The CI-owned network, containers and named volumes were removed; no host TCP 8080 binding remained. Docker v29.9.3 and Compose v5.7.2 used on the runner.

This confirms disposable backend acceptance, not ZITADEL Console user login, Account Portal positive/authenticated callback, real MFA, global SSO, production routing or Moderate advisory remediation. The original P02 Compose and official source/lock versions remain untouched. No repeated Account/Login build/test jobs ran in P05-CI-03. Runtime outcome recorded in notes/P05-CI-03.md and docs/P05-CI-03.md; final docs-only commit is not a new tested binary/SHA.

P05-SEC-01 — Independent frozen-build and strict dependency audit gates

Manual Gitea-only .gitea/workflows/p05-security-acceptance.yml has four independent jobs: Account npm ci/TypeScript/Next.js build plus anonymous negative route checks; official Login V2 pnpm@10.30.3 install --frozen-lockfile, Nx proto/client/standalone build and upstream Vitest regression; standalone Account production registry audit; standalone Login workspace production registry audit. Independent jobs prevent a remaining Moderate advisory from hiding build/test results.

Each registry job first runs npm audit --omit=dev --audit-level=high or pnpm audit --prod --audit-level high under set -eu to fail-closed on High/Critical. Then nonmutating JSON is collected, tests/ci/advisory_triage.py prints exact IDs/package/range/fix, and tests/ci/security_advisory_gate.py requires complete/count-consistent JSON and zero Moderate/High/Critical. Any Moderate is an actual failed gate, not a warning disguised as acceptance.

Only approved manifest and lockfile metadata were patched; original Login/auth/API/UI source, pnpm-workspace.yaml, P02 backend, Compose, real credentials and production unchanged. Two transitive glob/minimatch risks may remain because the root PNPM override configuration is outside Scope Lock and those require semver-major upgrades. See project/DEPENDENCY-SECURITY.md for the original 18+1 records and exact immediate parents; source state is not proof of advisory resolution until actual Gitea run.

P05-SEC-01 Run #87 terminal job results

Manual Run #87 tested 752d243. Account job #185 PASS; Login job #186 PASS (frozen pnpm, Nx Next 16.4.6 build and 1219 unit cases); Account security job #187 PASS at all levels with 0 findings; Login security job #188 FAIL on two Moderate glob/minimatch advisories (0 High/Critical). The strict tests/ci/security_advisory_gate.py did not suppress them, so overall run FAILURE is expected and correct. Full source/security acceptance remains blocked pending a separate narrow transitive dependency phase; no repeated run or application source change.

P05-SEC-02 — focused Login-only CI acceptance mode

Existing manual .gitea/workflows/p05-security-acceptance.yml now offers workflow_dispatch.inputs.login_only (default false retains the prior full Account/Login jobs). For this phase dispatch login_only=true, which conditionally skips account and security_account jobs; only login and security_login should execute. No production secrets or remote identity client.

login starts with CI-only python3 ../../../tests/ci/transitive_override_contract.py for the exact seven owned parent overrides and no affected glob/minimatch lock versions, then pinned PNPM frozen install, upstream proto/client/Login standalone build and full vitest --run regressions. security_login uses existing production PNPM HIGH/CRITICAL audit and independent JSON detail + zero Moderate+ gate; no suppression. Both jobs must conclude SUCCESS before declaring P05-SEC-02 accepted. The historical Account zero-vulnerability proof is Gitea Run #87, not retested in this focused phase.

A single manual run is authorized; Git source commit/push precedes dispatch, and final evidence-only documentation commit/push follows without a second run. Source-only audit and actual runner results are distinct; see notes/P05-SEC-02.md and docs/P05-SEC-02.md.

Run #88 and CI helper parsing defect — acceptance remains blocked

Gitea Run #88 source 66cfed5, exact tree 9a1d2abd, intentionally skipped Account jobs #189/#191. Login security #192 PASS (zero Moderate/High/Critical). Login build #190 FAIL before frozen install in transitive_override_contract.py due misclassification of existing >= version-range keys as parent-scoped overrides. Build/unit steps were skipped, not failed application regressions. Narrow post-run regex correction now explicitly accepts only >glob / >minimatch; unverified until a newly approved run. One focused run only; follow-up P05-SEC-02-A01 should run only Login frozen/build/unit with security gate, under a separately approved baseline.

P05-SEC-02-A01 — final frozen-build acceptance scope

Run #88 (66cfed5) proved strict Login production vulnerability audit 0 but did not execute frozen PNPM install, Nx standalone build or Vitest: new CI-only overridden() parser previously misread axios@>= etc. Corrected suffix-only >glob/>minimatch version is already committed at baseline 7655dbd; no further source/CI changes are planned.

One manual p05-security-acceptance.yml dispatch with login_only=true tests this existing code; Account build/security jobs intentionally skipped and previously accepted (Run #87). Login job: exact seven parent→child contract, PNPM v10.30.3 frozen install, Nx original proto/client/Login standalone build and upstream Vitest; Login security job: fail-closed live registry zero Moderate/High/Critical. No local tests/build/Docker or production usage; exact CI job outcomes recorded under notes/P05-SEC-02-A01.md. If a gate fails, this acceptance remains BLOCKED and no unapproved repeat run occurs.

P05-SEC-02-A01 terminal acceptance — Gitea Run #89

One manual Gitea Run #89 tested SHA ff4c58ed861474061c8f045be3a9f0752caf6ca0, 2026-11-22 16:30:28–16:34:54 UTC, overall SUCCESS. login_only=true skipped Account build/security jobs #193/#195, previously accepted in Run #87. Login job #194 SUCCESS: corrected exact seven-parent override source contract/lock matched (ZITPLANE_EXACT_SEVEN_PARENT_OVERRIDES_PASS, ZITPLANE_GLOB_MINIMATCH_PATCHED_LOCK_CONTRACT_PASS), PNPM 10.30.3 frozen installation (Lockfile is up to date, 1121 packages) PASS, official Nx proto/client/Login 16.4.6 standalone build/artifact PASS, upstream Vitest 84/84 test files, 1219/1219 unit tests PASS. Login security job #196 SUCCESS: fail-closed HIGH/CRITICAL gate PASS, production registry pnpm audit --prod 0 Moderate/High/Critical, ZITPLANE_PNPM_ZERO_MODERATE_HIGH_CRITICAL_PASS.

No new CI/auth/application/manifest/lock changes in A01; all checks were against the existing corrected validator. This closes the dependency-security/frozen-build acceptance only, not human OIDC login/callback, real MFA or production integration. See notes/P05-SEC-02-A01.md for exact markers/timestamps.

P05-OIDC-01 — Gitea-only real OIDC browser acceptance

New manual workflow .gitea/workflows/p05-oidc-acceptance.yml executes one isolated Docker runner job. tests/ci/oidc_e2e/run.sh creates the unchanged P02 stack under exact numeric run ownership and no host ports; a CI-only FirstInstance IAM_OWNER machine PAT and official API provision a disposable OIDC client/user. A pinned Chromium image runs the unmodified Account Portal in isolated CI development mode, driving official prebuilt Login V2 and real Auth.js PKCE/state/nonce, callback, authenticated Account API/page, negatives and local logout. An always() step calls tests/ci/oidc_e2e/cleanup.sh to remove only owned browser/forwarder/secrets and invokes audited backend owned-data cleanup. The run must fail if cleanup is incomplete; no browser secrets are logged/uploaded.

Important: The original backend external issuer is HTTP localhost, while Account Portal production mode correctly requires HTTPS. This CI development-origin browser acceptance does not certify production TLS, full self-hosted login source wiring, MFA or Admin Console human login. See project/OIDC-E2E-ARCHITECTURE.md for individual modules and notes/P05-OIDC-01.md for the actual run status.

P05-OIDC-01 real CI evidence: Manual Gitea Run #90 (tested cbcd60e) had healthy 4-service portless backend PASS, CI-only localhost:8080 namespace forwarder readiness FAIL, no disposable OIDC project/client/user or Chromium browser launched; always() cleanup job step PASS, no leftover CI resources. Overall job FAILED; not an authenticated login test PASS. Next narrowly scoped P05-OIDC-02 must instrument/repair private forwarder lifecycle and reverify the full browser acceptance, without changing original application or backend code.

P05-OIDC-02 scoped follow-up

New source in tests/ci/oidc_e2e/run.sh: test namespace-local Traefik proxy Host route before creating sidecar, resolve published alpine/socat:1.8.1.3 to immutable digest, validate prebuilt socat entrypoint, create no-host-port loopback sidecar, check running state/exit plus bounded private forwarded HTTP 200. provision.py now treats /debug/ready as HTTP status-only; official management provisioning and all browser/auth assertions unchanged. Existing manual Gitea workflow and always-run owned cleanup unchanged. One separate Gitea acceptance run needed; never mark a static stage check as an actual real OIDC browser PASS.

P05-OIDC-02 Gitea Run #91 result: tested 4831165, job #198 overall FAIL. All 4 services healthy PASS, no runner host port PASS, private Traefik:80 Host-route HTTP 200 PASS, prebuilt socat/digest verification PASS. The run-owned socat sidecar exited 1 before forwarded 8080 check, so no project/client/user or Chromium/Auth.js checks executed. Mandatory cleanup PASS, no remaining owned resources. Source-only status readiness fix in provision.py was not exercised. Root cause inside sidecar unproven without redacted startup error diagnostics. Next separately approved P05-OIDC-03 limited to private CI process/diagnostics.

P05-OIDC-03 approved CI-only source: The existing manual workflow is unchanged. Run #91's verified socat exit 1 justifies a single-run diagnostic attempt plus a one-time deterministic, digest-pinned python:3.12-alpine stdlib private forwarder fallback. Only run-labeled containers, 127.0.0.1:8080 → private Traefik:80, bounded error-class diagnostics and original proxy Host HTTP 200 gate are permitted. The diagnostic/fallback happens before real provisioning; no authentication, browser or cleanup source changed. A new single Gitea run must prove all real client/user/browser/callback/session and mandatory owned cleanup checks or remain BLOCKED.

P05-OIDC-03 queued CI evidence: The only approved Gitea Run #93 (source 7b80f411b2981e2274d171c12596301ed757b768) was still QUEUED at last inspection; job #206 has not started, so neither the socat diagnostic/Python fallback nor the browser/provisioning/cleanup ran. Do not mark technical failure/success; runner scheduling is a pending dependency, not a proven outage. Source audit passed, and no second dispatch occurred. Follow-up should examine the same Run #93 terminal steps/actual cleanup and update docs, not rerun prior suites.