fix(ci): pin and diagnose private OIDC forwarder readiness

This commit is contained in:
victorsteele428 committed 2026-11-22 17:45:36 +00:00
1 parent b78270e9d4
commit 4831165548
22 files changed
+231 -27

No files matched your search

+1
View File
@@ -16,6 +16,7 @@ This is the **developer-focused implementation history**. `notes/CHANGELOG.md` i
| 2026-11-22 | P05-SEC-02 | Seven parent-targeted glob/minimatch patched resolutions, PNPM lockfile metadata-only, manual Login-only CI and post-run validator fix | Run #88: strict Login audit 0 findings PASS, Login build job blocked before install by parser false-positive; post-run fix untested. `notes/P05-SEC-02.md` |
| 2026-11-22 | P05-SEC-02-A01 | Verify existing corrected helper, exact seven PNPM scoped overrides, frozen Login Nx build and upstream Vitest; no source/lock changes | **Gitea Run #89 SUCCESS** on `ff4c58e`, 84/84 files, 1219/1219 tests, Login audit 0 Moderate/High/Critical; `notes/P05-SEC-02-A01.md` |
| 2026-11-22 | P05-OIDC-01 | Added manual disposable IAM_OWNER/client/human/browser OIDC acceptance harness, secret isolation and cleanup | Run #90 FAIL before provisioning: private loopback forwarder unready; 4 backend services healthy, owned cleanup PASS. `notes/P05-OIDC-01.md` |
| 2026-11-22 | P05-OIDC-02 | CI-only prebuilt socat tag+immutable digest startup, proxy/private forwarder HTTP readiness/exit diagnostics, status-only "/debug/ready" provisioner | Source prepared, one Gitea acceptance pending; `notes/P05-OIDC-02.md` |
## Required entry for each future change
+5
View File
@@ -134,3 +134,8 @@ New manual workflow `.gitea/workflows/p05-oidc-acceptance.yml` executes **one**
**P05-OIDC-01 real CI evidence:** Manual Gitea Run #90 (tested `cbcd60e`) had healthy 4-service portless backend PASS, **CI-only localhost:8080 namespace forwarder readiness FAIL**, no disposable OIDC project/client/user or Chromium browser launched; `always()` cleanup job step PASS, no leftover CI resources. Overall job FAILED; **not** an authenticated login test PASS. Next narrowly scoped P05-OIDC-02 must instrument/repair private forwarder lifecycle and reverify the full browser acceptance, without changing original application or backend code.
## P05-OIDC-02 scoped follow-up
New source in `tests/ci/oidc_e2e/run.sh`: test namespace-local Traefik proxy Host route before creating sidecar, resolve published `alpine/socat:1.8.1.3` to immutable digest, validate prebuilt `socat` entrypoint, create no-host-port loopback sidecar, check running state/exit plus bounded private forwarded HTTP 200. `provision.py` now treats `/debug/ready` as HTTP status-only; official management provisioning and all browser/auth assertions unchanged. Existing manual Gitea workflow and always-run owned cleanup unchanged. One separate Gitea acceptance run needed; never mark a static stage check as an actual real OIDC browser PASS.
+3
View File
@@ -168,3 +168,6 @@ Official Login V2 user/auth source, user Account Portal, backend Compose and pro
## P05-OIDC-01 — disposable real OIDC acceptance source map
`.gitea/workflows/p05-oidc-acceptance.yml`: one manual Gitea isolated Docker job and mandatory `always()` cleanup. `tests/ci/oidc_e2e/compose.owner.override.yaml`: only ephemeral IAM_OWNER PAT bootstrap; original P02 Compose untouched. `tests/ci/oidc_e2e/run.sh`: ownership preflight, portless backend startup, private loopback routing and original-app browser harness. `tests/ci/oidc_e2e/provision.py`: official ZITADEL API client/project/verified human creation and 0600 temporary secret fixture. `tests/ci/oidc_e2e/browser.sh`: CI-only pinned Playwright/Account npm dependencies. `tests/ci/oidc_e2e/browser.cjs`: actual Chromium Auth.js/OIDC/password/callback/session and fail-closed checks. `tests/ci/oidc_e2e/cleanup.sh`: owned-only containers, volume, bootstrap/database data cleanup. New directory `README.md` and `MINIFEST.md` document all modules. Detailed boundaries: `project/OIDC-E2E-ARCHITECTURE.md`; actual test outcomes pending Gitea.
**P05-OIDC-02 updated CI functions only:** `tests/ci/oidc_e2e/run.sh` stage between four-service readiness and provisioning now explicitly validates proxy Host/HTTP status, pulls version-pinned prebuilt socat, checks its entrypoint/immutable image digest and checks the private 8080 listener plus sidecar running/exit state. `tests/ci/oidc_e2e/provision.py::wait_until_ready()` checks HTTP 200 without JSON decoding. `cleanup.sh`, browser scripts and all application/backend/auth source remain read-only.
+3
View File
@@ -130,3 +130,6 @@ Existing `tests/ci/security_advisory_gate.py` remains the independent **live reg
**P05-OIDC-01 runtime evidence:** `run.sh` reached four-service readiness in Run #90, then `provision.py::wait_until_ready()` timed out on private `localhost:8080` before `request()` could provision any client, project or user. Thus `browser.cjs::main()` and its positive/negative real login/session checks **were not executed**. The only other completed operation was `cleanup.sh`, which verified removal of owned state. The sidecar's precise root cause is unknown; avoid runtime/PASS claims for unexecuted browser functions.
**P05-OIDC-02 narrow source audit:** Shell orchestration in `tests/ci/oidc_e2e/run.sh` now probes private Traefik `/debug/ready` with exact `Host: localhost:8080`, verifies prebuilt socat image entrypoint/digest, and stops before credential provisioning when the loopback forwarder state, socket or HTTP response is not ready. Python `provision.py::wait_until_ready()` now checks only `HTTP 200` for `/debug/ready` without interpreting an arbitrary body as JSON. No user API, token, session, IAM policy, Auth.js or official Login V2 code changed.
+7
View File
@@ -50,3 +50,10 @@ Gitea manual [Run #90](https://dev.ivib.cc/vibtools/ZitPlane/actions/runs/90), t
The mandatory `cleanup.sh` step #3 succeeded and printed `ZITPLANE_BACKEND_CLEANUP_VERIFIED_PASS` + `ZITPLANE_OIDC_ALL_RESOURCES_CLEANED_PASS`. It removed only run #90-owned proxy namespace forwarder, separate ephemeral secrets volume, Compose containers, Postgres/bootstrap volumes and fixed CI network. **Backend bootstrap/cleanup PASS; full end-to-end OIDC acceptance BLOCKED.**
Next P05-OIDC-02 should narrowly instrument/check forwarder startup before provisioning, use an approved pinned prebuilt image to avoid silent runtime package installation, verify exact **private** loopback listener/Host routing without any host port exposure, preserve original Go backend/Compose/Login/Auth.js and credentials isolation, then run **one separately authorized** Gitea acceptance cycle. No CI-only patch in P05-OIDC-01 was executed after Run #90.
## P05-OIDC-02 — isolated private forwarder reliability (source prepared)
The previous Gitea Run #90 verified a healthy, portless P02 stack but did not capture the private socat sidecar's exit status or verify the 127.0.0.1:80 proxy independently; the true forwarder failure cause is **unknown**. The new narrow `run.sh` checks the private Traefik:80 `/debug/ready` with `Host: localhost:8080` first, then uses the published version-specific `alpine/socat:1.8.1.3` image (preinstalled `socat`), checks `Entrypoint == ["socat"]`, resolves the pulled tag to its immutable sha256 registry digest and **runs by digest**. There is no `apk add` at runtime. Container state/exit code and bounded local forwarded HTTP 200 readiness are checked **before official OIDC project/user provisioning**. Only safe stage/status markers are printed on failure; no HTTP bodies, cookies, callbacks or passwords.
`provision.py::wait_until_ready()` now independently verifies HTTP 200 rather than attempting to parse the `/debug/ready` response as JSON; **all official management/client/human creation requests remain strictly parsed JSON and authenticated with the ephemeral IAM_OWNER PAT**. No other protected code, developer Compose, original Login V2 or Account auth source changed. The existing `cleanup.sh` continues to remove only run-labeled/owned resources. **New Gitea real-browser acceptance pending; this is not a verified login yet**. Exact plan/evidence: `notes/P05-OIDC-02.md`.
+3
View File
@@ -84,3 +84,6 @@ These are **major** dependency migrations underneath older parents, not an upstr
## P05-OIDC-01 — isolated browser test tooling and why
CI only: official ZITADEL v4.20.4, official Login V2 v4.20.4, PostgreSQL 17.10/Traefik 3.7.7 from P02; existing portless Compose override; new `alpine:3.21` namespace-local `socat` loopback forwarder, `python:3.12-alpine` stdlib API provisioner, pinned `mcr.microsoft.com/playwright:v1.58.2-noble` (Chromium), pinned `playwright@1.58.2` and the Account Portal's existing `npm ci` lock inside an ephemeral browser container. No manifests or lockfiles changed, no local PC builds/tests, and no host HTTP port published. Dynamic IAM_OWNER bootstrap PAT expires in two hours and all token/password/client data are destroyed with owned volumes after the run. P02 issuer HTTP localhost requires Account `NODE_ENV=development` **inside CI only**; production HTTPS remains unverified. Detailed topology and cleanup: `project/OIDC-E2E-ARCHITECTURE.md`.
**P05-OIDC-02 CI forwarding dependency:** Uses the published, version-selected `alpine/socat:1.8.1.3` image rather than installing packages inside `alpine:3.21` at container startup. The disposable runner checks the image has `["socat"]` entrypoint, resolves `RepoDigests` and launches the immutable pulled sha256 reference; never `latest`. Existing `python:3.12-alpine` probe containers check no-host-port proxy and private forwarding readiness. This is Gitea-only and must not be run on developer PC or production. No package manifests/lockfiles modified.
+18 -16
View File
@@ -1,27 +1,27 @@
# ZitPlane ? exact tracked source inventory
**Snapshot:** 2026-11-22, P05-OIDC-01 source tested on Gitea Run #90 at `cbcd60e76ec909df4f9c5826adf7a3e2a899c73e`: backend/portless PASS, private localhost forwarder BLOCKED; no client/user/browser E2E executed, cleanup PASS. Final results-only documentation commit is NOT a new runtime test.
**Snapshot:** 2026-11-22, P05-OIDC-02 focused source candidate on approved baseline `b78270e`. Run #90 backend/cleanup PASS, private loopback FAIL; **new OIDC acceptance CI not yet executed**.
**Git-tracked file paths: 746.** Untracked `AI_INSTRUCTIONS.md` and ignored local secrets excluded.
**Git-tracked file paths: 748.** Untracked `AI_INSTRUCTIONS.md` and ignored local secrets excluded.
This is an exact Git file-path locator, not a claim of source-built Login runtime, browser authentication or deployment. See `notes/P05-OIDC-01.md` and `project/OIDC-E2E-ARCHITECTURE.md`.
This inventory records exact Git paths, not live browser authentication. See `notes/P05-OIDC-02.md` and `project/OIDC-E2E-ARCHITECTURE.md`.
| Group | Paths | Responsibility |
| --- | ---: | --- |
| `(repository root)` | 6 | Tracked repository root |
| `.gitea` | 8 | Manual Gitea CI-only workflows |
| `auth` | 639 | Original official ZITADEL Login V2 source/locked dependencies |
| `core` | 2 | Local backend helper |
| `.gitea` | 8 | Manual Gitea CI-only verification workflows |
| `auth` | 639 | Pinned official Login V2 source and dependency lock |
| `core` | 2 | Development backend |
| `deployment` | 2 | Read-only developer Compose |
| `docs` | 19 | User/operator guides |
| `notes` | 11 | Prechange plans/evidence |
| `packages` | 1 | Placeholder |
| `project` | 12 | Developer architecture/source/security maps |
| `scripts` | 1 | Placeholder |
| `tests` | 21 | CI verification scripts and disposable OIDC browser harness |
| `users` | 24 | Read-only Account Portal |
| `docs` | 20 | Operator and user guides |
| `notes` | 12 | Phase plans and real CI evidence |
| `packages` | 1 | First-party placeholder |
| `project` | 12 | Architecture, function maps, security and source inventory |
| `scripts` | 1 | First-party placeholder |
| `tests` | 21 | Gitea CI-only contracts and browser harness |
| `users` | 24 | Read-only Account Portal source |
## Exact Git-tracked paths
## Exact tracked paths
### (repository root) ? 6 files
@@ -695,7 +695,7 @@ This is an exact Git file-path locator, not a claim of source-built Login runtim
- `deployment/README.md`
- `deployment/docker/compose.backend.dev.yaml`
### docs ? 19 files
### docs ? 20 files
- `docs/ARCHITECTURE.md`
- `docs/DEVELOPMENT.md`
@@ -708,6 +708,7 @@ This is an exact Git file-path locator, not a claim of source-built Login runtim
- `docs/P05-CI-02.md`
- `docs/P05-CI-03.md`
- `docs/P05-OIDC-01.md`
- `docs/P05-OIDC-02.md`
- `docs/P05-SEC-01.md`
- `docs/P05-SEC-02-A01.md`
- `docs/P05-SEC-02.md`
@@ -717,7 +718,7 @@ This is an exact Git file-path locator, not a claim of source-built Login runtim
- `docs/TROUBLESHOOTING.md`
- `docs/USER-GUIDE.md`
### notes ? 11 files
### notes ? 12 files
- `notes/CHANGELOG.md`
- `notes/DOCS-01.md`
@@ -726,6 +727,7 @@ This is an exact Git file-path locator, not a claim of source-built Login runtim
- `notes/P05-CI-02.md`
- `notes/P05-CI-03.md`
- `notes/P05-OIDC-01.md`
- `notes/P05-OIDC-02.md`
- `notes/P05-SEC-01.md`
- `notes/P05-SEC-02-A01.md`
- `notes/P05-SEC-02.md`