Files
DataStack/MEMORY.md
T

324 lines
46 KiB
Markdown
Raw Normal View History

# MEMORY.md — DataStack Cross-IDE Persistent Project Memory
> **Last Synchronized**: 2026-11-19
> **Status**: **V2-Phase-01 Complete & Verified (`BASELINE-2026-11-19-V2`) — `1 / 4` V2 Phases Done (Storage `.csv.gz` Auto-Compression, S3/R2 Off-Site Archive with AES-256-GCM Secret Vault, Per-Client `storageQuotaMb` & VPS Disk Pause Guard `HTTP 507`) — Verified (`89/89` TS tests, `10/10` Python tests, ESLint-enabled build & lint `0` errors / `0` warnings).**
> **Scope Rule**: Maintain this file accurately across VS Code, Google Antigravity, Cursor, and any other IDE after every modification, feature update, or bug fix (see `.ai-rules` and `AGENTS.md`).
---
## 1. Project Identity & Core Principles
- **Project**: **DataStack** — Self-hosted, real-time telemetry ingestion, admin governance (`/vcon/*`), and partitioned CSV storage/export platform.
- **Runtime Split (CRITICAL)**:
- **Port `3000` (`web`)**: Next.js 15 App Router — public landing (`/`), PWA lifecycle, `/vcon/*` Admin Console, REST ingestion (`POST /api/events`), metrics/settings/client APIs, and streaming exports (`GET /api/export`).
- **Port `4000` (`ingestion`)**: Dedicated Node.js Express 5 + Socket.IO server (`server/index.js`) — WebSocket handshakes, JWT client auth, rate limiting, idempotency deduplication, Redis/RAM batching, and CSV flush worker.
- Socket.IO connections NEVER run inside Next.js serverless routes; they run strictly on Port `4000`.
- **Storage & Database Boundary (CRITICAL)**:
- **PostgreSQL (`pg` + `prisma/schema.prisma`)**: Stores strictly control-plane metadata (`admin_users`, `admin_login_attempts`, `client_sdks`, `deleted_clients`, `admin_settings`). **Never insert per-row telemetry events into PostgreSQL.**
- **Partitioned CSV Storage (`/storage/data/{clientId}/{YYYY-MM-DD}.csv`)**: All ingested telemetry events flush in batches to daily partitioned CSV files with cooperative file locking, configurable size rotation (`fileRotationMb`, e.g., `-part2.csv`), and retention pruning (`autoPruneDays`).
- **Redis (`ioredis`)**: Shared cross-process buffer queue (`datastack:buffer:events`), recent event cache (`datastack:recent:events`), and distributed flush lock (`datastack:flush:lock`). Persisted in Docker via `--appendonly yes` (`redisdata:/data`). Falls back gracefully to local RAM when `REDIS_URL` is omitted.
---
## 2. Tech Stack & Local Setup
### Stack
- **Frontend / Web API**: Next.js `15.4.9` (App Router, `output: "standalone"`), React `19.2.1`, TypeScript `5.9.3`, Tailwind CSS v4 (`@tailwindcss/postcss`), `lucide-react`, `motion`.
- **Ingestion Server**: Node.js `22` / Express `5.3.0`, Socket.IO `4.8.4`.
- **Persistence & Caching**: `pg` `8.23.1` (PostgreSQL 16 in Docker), `ioredis` `6.0.0` (Redis 7 AOF in Docker), `csv-parse` `7.0.3`, `json2csv` `6.0.0-alpha.2`.
- **Auth & Security**: `jsonwebtoken` `9.0.3`, Node `crypto.scrypt` password hashing, WebCrypto HMAC-SHA256 admin session cookies (`middleware.ts`), CSRF `Origin`/`Referer` vs `Host`/`X-Forwarded-Host` checks, `DATASTACK_INTERNAL_SECRET` timing-safe bridge auth between Port `3000` ↔ `4000`.
- **Client SDKs**: Official pip-installable Python SDK in `sdk/python/datastack/` (`datastack`) + backward-compatible wrapper `sdk-python/` (`datastack-sdk`).
### Environment & Setup Commands
1. **Configure Environment**: Copy `.env.example` to `.env` (in production `NODE_ENV=production`, all 4 secrets MUST be 32+ chars: `DATASTACK_ADMIN_SESSION_SECRET`, `DATASTACK_JWT_SECRET`, `DATASTACK_ADMIN_SETUP_KEY`, `DATASTACK_INTERNAL_SECRET`; optional `DATABASE_URL` and `REDIS_URL`).
2. **Install Dependencies**: `npm ci` (or `npm install`) using root `package.json` + `package-lock.json`.
3. **Run Locally (Two Terminals)**:
- Terminal 1 (Port `3000`): `npm run dev`
- Terminal 2 (Port `4000`): `npm run server`
4. **Build, Lint & Test Verification**:
- Build: `npm run build`
- Lint: `npm run lint` (`eslint .`)
- TS Test Suite (Phases 01–03, 06, 07): `npx --yes tsx --test tests/phase01-auth.test.ts tests/phase02-sync.test.ts tests/phase03-storage-buffer.test.ts tests/phase06-pwa-landing.test.ts tests/phase07-deployment.test.ts`
- Python SDK Tests: `python3 -m unittest tests/phase05_sdk_test.py`
---
## 3. Folder Structure & File Architecture
```text
datastack/
├── .ai-rules # Universal cross-IDE rules (enforces MEMORY.md sync)
├── AGENTS.md # Engineering rules, UI single-source-of-truth & memory sync
├── MEMORY.md # Persistent cross-IDE project memory (this file)
├── package.json / package-lock.json
├── middleware.ts # /vcon/* auth guard & Origin/X-Forwarded-Host CSRF enforcement
├── app/
│ ├── (landing)/page.tsx # Public landing page
│ ├── globals.css # Tailwind v4 + centralized CSS variables (--radius: 0rem, --shadow: none)
│ ├── layout.tsx / not-found.tsx / error.tsx
│ ├── api/
│ │ ├── auth/ # Admin login, logout, setup (/vcon/setup), profile update
│ │ ├── clients/ # CRUD clients, token regeneration, revocation
│ │ ├── data/ # Paginated CSV search/query across partitions
│ │ ├── events/ # Authenticated HTTP ingestion (single/batch/file records) & recent list
│ │ ├── export/ # Streaming CSV / JSON filtered export (UTC dates, optional flush=false)
│ │ ├── health/ # Web health + client Bearer token probe + async buffer depth
│ │ ├── metrics/ # Aggregated Port 3000 + Port 4000 + DB + CSV metrics
│ │ ├── settings/ # Allowlisted runtime config get/post + live Port 4000 sync
│ │ └── storage/ # CSV partition listing & manual cross-process buffer flush
│ └── vcon/ # Protected Admin Console pages:
│ ├── dashboard/ clients/ data/ export/ login/ profile/ sdk/ settings/ setup/ simulator/ storage/
├── components/
│ ├── ui/ # Single-source-of-truth UI primitives (Button, Input, Table, Modal, Panel, etc.)
│ ├── layout/ # MainLayout, Navbar, Sidebar, VconShell, PageShell, AdminPageHeader, Footer
│ ├── sections/ # Modular feature sections (DashboardOverview, ClientsTable, DataBrowser, ExportWorkspace, SettingsWorkspace, etc.)
│ └── pwa/ # PwaLifecycle.tsx service worker registration
├── lib/
│ ├── adminAuth.ts # Uniform API session guard (requireAdminApiSession) + proxy-safe extractClientIp()
│ ├── vconCache.ts # In-memory SWR cache + clearAuthCaches() for logout/SW cleanup
│ └── vconUiHelpers.ts # Pure Admin UI sync helpers (settings bounds, bridge sync, UTC presets, timestamps)
├── core/
│ ├── adminSettings/ # settingsManager.ts (DB hydration, Port 4000 sync, unhydrated prune guard)
│ ├── apiEngine/ # apiService.ts (dashboard metrics & Port 4000 internal bridge queries)
│ ├── auth/ # adminUsers.ts, adminSession.ts (scrypt, brute-force throttle, tokenVersion), jwt.ts
│ ├── data/ # buffer.ts (Redis + RAM queue, distributed lock, graceful SIGTERM/SIGINT flush)
│ ├── Database/ # database.ts (pg.Pool + idempotent DDL + additive ALTER TABLE guards)
│ ├── sdk/ # sdkManager.ts (DB JWT hydration, client lifecycle, deleted_clients tombstone)
│ ├── Security/ # security.ts (rate limit, maxPayloadSizeKb, idempotency, internal headers)
│ └── Storage/ # csvStorage.ts (cooperative lock, rotation, cached streaming row count, prune)
├── scripts/
│ ├── dev-all.js # Local concurrent Next.js (3000) + Ingestion (4000) dev runner
│ └── start-all.js # Production concurrent Next.js (`next start`) + Ingestion (`server/index.js`) runner
├── server/
│ ├── index.js # Express 5 + Socket.IO (Port 4000), /health, /internal/* bridge, PG sync, graceful exit
│ ├── socket/handlers.js # Handshake JWT auth, ping_verify, sync_data, batch_sync
│ └── workers/csvWriter.js # Partitioned CSV batch writer & rotation/pruning worker
├── sdk/python/datastack/ # Official pip-installable Python SDK (`datastack` v2.2.0)
│ ├── datastack/ # client.py, config.py, heartbeat.py, spool.py, exceptions.py
│ ├── config.datastack.example.py
│ ├── pyproject.toml / setup.py
│ ├── README.md # SDK overview, installation, quickstart, docs index
│ └── docs/ # Production SDK Documentation Suite (LicenX Standard):
│ ├── README.md # SDK Documentation Hub & architecture overview
│ ├── INSTALLATION.md # Python runtimes, pip commands, and verification
│ ├── QUICKSTART.md # 5-minute zero-boilerplate onboarding guide
│ ├── API_REFERENCE.md # Complete DataStack class & method specifications
│ ├── CONFIGURATION.md # config.datastack.py & DATASTACK_* environment reference
│ ├── EXAMPLES.md # IoT streaming, FastAPI, batching, and file ingestion
│ ├── ERROR_HANDLING.md # Exception hierarchy, 401/403/413/429 codes & retries
│ └── LIFECYCLE.md # 5-state machine, zero-CPU IDLE mode & offline spooling
├── sdk-python/ # Backward-compatible re-export package (`datastack-sdk`)
├── docker/
│ ├── docker-compose.yml # web, ingestion, proxy (Nginx), postgres (pgdata), redis (redisdata AOF)
│ ├── Dockerfile.web # Multi-stage standalone Next.js build + safe /app/storage ownership check
│ ├── Dockerfile.server # Multi-stage Port 4000 ingestion server + safe /app/storage ownership check
│ ├── Dockerfile.proxy # Nginx reverse proxy container
│ └── coolify-nginx.conf # Routes `/health` & `/socket.io/` -> `ingestion:4000`, `/` -> `web:3000`
├── nginx/default.conf # Local Nginx proxy config matching coolify-nginx.conf
├── prisma/schema.prisma # Reference PostgreSQL metadata schema
├── public/ # PWA manifest.json, sw.js, icons (192, 512, maskable, apple-touch)
├── docs/ # Comprehensive Production Documentation Suite:
│ ├── index.md # Primary documentation hub and portal navigation index
│ ├── MANIFEST.md # Master documentation inventory & authoring governance
│ ├── audit-coverage-report.md # Forensic codebase audit matrix & coverage report
│ ├── users/ # End-User & Operator Guides:
│ │ ├── index.md # User portal overview & reading pathways
│ │ ├── getting-started.md # 5-minute fast-track operator onboarding guide
│ │ ├── installation.md # Production Docker Compose, Coolify & volume guide
│ │ ├── configuration.md # Environment variables & platform runtime settings
│ │ ├── features/ # Deep-dive guides: admin-console, client-management,
│ │ │ # data-simulator, partitioned-storage, data-export
│ │ ├── workflows.md # End-to-end operator journeys & disaster recovery
│ │ ├── troubleshooting.md # Forensic error recovery, status codes & diagnostics
│ │ └── faq.md # Storage architecture & operational questions
│ ├── dev/ # Developer & Contributor Guides:
│ │ ├── index.md # Developer hub index & architectural principles
│ │ ├── architecture.md # Dual-runtime blueprint, service bridge & storage
│ │ ├── codebase-structure.md # File-by-file module, route & component inventory
│ │ ├── local-setup.md # Local workstation setup & dual-terminal workflow
│ │ ├── api-reference.md # Exhaustive REST API & Socket.IO specification
│ │ ├── database-schema.md # PostgreSQL control plane & DDL migrations
│ │ ├── security.md # Security architecture, scrypt, sessions & CSRF
│ │ ├── building-packaging.md # Next.js standalone build & multi-stage Dockerfiles
│ │ ├── testing.md # Automated test suites guide (node:test + unittest)
│ │ └── contributing.md # Architectural rules, memory sync & PR checklist
│ └── reference/ # Technical Reference Portal:
│ ├── index.md # Technical reference index
│ ├── configuration-reference.md # Complete environment variable & setting bounds
│ ├── dependencies.md # Runtime & dev dependencies audit with rationales
│ └── glossary.md # Technical domain terminology & concept glossary
├── notes/ # Production Hardening Governance & Scope-Lock Suite (`BASELINE-2026-11-19-R2`):
│ ├── README.md # Master governance index & 6-step scope-locked execution protocol
│ ├── FORENSIC_AUDIT_REPORT.md # 100% raw-code verified audit (18 bugs: 0C/6H/8M/4L), Demo vs Prod & UI-vs-Reality matrices
│ ├── BASELINE_FREEZE.md # Frozen baseline snapshot (`BASELINE-2026-11-19-R2`) & 1:1 bug alignment
│ ├── PRODUCTION_READINESS_SCOPE_LOCK.md # (Doc 001) Exact bug fix contracts & per-phase Scope Locks (`Phase-01`..`Phase-04`)
│ ├── PRODUCTION_ROADMAP.md # (Doc 002) Dependency-ordered A-Z 4-phase roadmap & burn-down tracker
│ ├── PHASE_EXECUTION_LOG.md # (Doc 003) Live phase completion log (`4/4` complete) & zero-regression guardrails
│ ├── ERROR_HANDLING_MATRIX.md # (Doc 004) Active error-handling catalog & 9 planned hardening additions
│ └── ACTUAL_IMPLEMENTATION_STATUS.md # Ground-truth working vs completed-per-phase vs remaining ledger
└── tests/ # Automated verification suites:
├── phase01-auth.test.ts # Auth fail-closed, tokenVersion, brute-force lockout, Bearer revocation, CSRF
├── phase02-sync.test.ts # Internal bridge auth, settings sync, JWT rotation kick, rate/security metrics
├── phase03-storage-buffer.test.ts # Redis durability, cooperative CSV locks, streaming row count, export filters
├── phase04-ui-sync.test.ts # UI/API settings bounds, bridge status, UTC presets, timestamp & folder matching
├── phase05-packaging.test.ts # Python/Node packaging parity, PEP 561 py.typed, Node engines & start-all exit
├── phase05_sdk_test.py # Python SDK WebSocket/HTTP fallback, offline spool, file streaming, edge hardening
├── phase06-pwa-landing.test.ts # PWA manifest/icons/sw + navigation routes
└── phase07-deployment.test.ts # Compose/Redis AOF, Nginx /health, lockfile, prune guard, shutdown flush
```
---
## 4. Active Features & Development Status
All planned hardening & deployment phases (**Phase-01 through Phase-07**) are **Complete & Verified**:
1. **Admin Authentication & CSRF Hardening (Completed — Phase-01)**
- Fail-closed secret enforcement in `NODE_ENV=production` (`/vcon/setup` bootstrap via `DATASTACK_ADMIN_SETUP_KEY`).
- Password hashing via `crypto.scrypt`, session invalidation via DB-backed `tokenVersion`, composite IP+email login lockout (`admin_login_attempts`), and strict `Origin`/`Referer` vs `Host`/`X-Forwarded-Host` CSRF verification.
2. **Real-Time Dual-Runtime Sync (Completed — Phase-02)**
- Authenticated `/internal/*` bridge between Port `3000` and Port `4000` (`DATASTACK_INTERNAL_SECRET`, backward-compatible Coolify fallback).
- Live settings & client updates (create/rotate/revoke/delete) propagate immediately; JWT secret rotation re-signs stored client tokens and disconnects active sockets safely.
3. **Redis Durability & Cooperative CSV Storage Engine (Completed — Phase-03)**
- Shared Redis queue (`datastack:buffer:events`) with distributed flush lock (`NX PX`) where `LTRIM` executes strictly AFTER CSV write confirmation; automatic RAM fallback if Redis disconnects.
- Cooperative partition file lockfiles (with stale-lock recovery), mtime/size-cached RFC 4180 streaming row counter, and filtered CSV/JSON exports (`GET /api/export`).
4. **Admin Console UI & Data/Export Workspace (Completed — Phase-04)**
- Built with shared `components/ui/` and `components/layout/` primitives (`0rem` radius, zero shadows, Fluent/Segoe dark & light tokens).
5. **Production Python Client SDK (Completed — Phase-05)**
- Supports single events (`send()`/`sync()`), auto-splitting/rate-limit-paced batches (`send_batch()`/`batch_sync()`), and direct file ingestion (`send_file()` for `.csv`, `.json`, `.jsonl`, `.ndjson` with SHA-256 deterministic `eventId`).
- Automatic `config.datastack.py` discovery, WebSocket primary transport -> Heartbeat HTTP fallback -> `.datastack_spool/spool.jsonl` offline spool & auto-drain.
6. **PWA & Landing Integrity (Completed — Phase-06)**
- Complete `public/manifest.json`, icons, `public/sw.js`, and root-relative navigation links.
7. **Docker / Coolify Deployment & Persistence Hardening (Completed — Phase-07)**
- Redis AOF persistence (`redisdata:/data`), deterministic `package-lock.json` for `npm ci`, Nginx `location = /health` → `ingestion:4000`, `/api/health` container probes, non-blocking startup ownership check in Dockerfiles, unhydrated-DB `autoPruneDays` safety guard in `SettingsManager`, DB JWT hydration in `SDKManager`, graceful `SIGTERM`/`SIGINT` flush in `DataBufferEngine`, and non-destructive `ADD COLUMN IF NOT EXISTS` DDL guards.
- **Current Remediation Status (2026-11-18)**:
- **Phase-01 Complete & Verified** (`BUG-A01`, `BUG-A02`, `BUG-A03`, `BUG-A04`, `BUG-A05`, `BUG-E02`):
- Removed spoofable header/UA CSRF bypasses in `middleware.ts` and rejected `sec-fetch-site: cross-site`.
- Enforced DB-backed `tokenVersion` session validation across `/vcon/*` layout (`app/vcon/layout.tsx` + `components/layout/VconShell.tsx`) and read/export APIs (`/api/data`, `GET /api/events`, `/api/export`, `/api/metrics`, `/api/storage`).
- Prevented transient DB errors from triggering false `/vcon/setup` redirects and prevented swallowed `NEXT_REDIRECT` errors on `/`, `/vcon/login`, and `/vcon/setup`.
- Standardized `SDKManager.normalizeClientId`, blocked deleted/unregistered clients on `/api/health` and Admin `POST /api/events`, hardened `POST /api/clients` & `PATCH /api/clients/[id]` input validation, and excluded `/vcon/*` from Service Worker caching with `CLEAR_AUTH_CACHE` on sign-out.
- Verified: `26/26` TS tests, `3/3` Python tests, `npm run build` clean, `eslint .` clean, Playwright browser auth/logout E2E clean.
- **Phase-02 Complete & Verified** (`BUG-B01`–`BUG-B05`):
- Enforced fail-closed client verification in `server/index.js` (`verifyClientInDb`) backed by PG, deletion tombstones, Redis `datastack:clients:*` mirrors, and normalized `clientId` keys across both runtimes.
- Added cross-runtime Redis idempotency (`datastack:idem:*`) and fixed-window rate limiting (`datastack:rl:*`) in `core/Security/security.ts` (`validateEventAsync`) and `server/socket/handlers.js` (`validateSingleEventAsync`) with automatic local RAM fallback.
- Surfaced bridge sync telemetry (`BridgeSyncResult`, `SettingsBridgeSyncStatus`, `runtime.ingestionBridgeStatus`, `POST /api/settings` `bridgeSync`) and normalized `SDKManager.incrementEventCountAsync()`.
- Fixed Socket `sync_data` flat payload preservation (`extractSingleEventData`), reconciled `PORT_SOCKET`/`PORT` precedence and live `/internal/metrics` ports, and added concurrent production `scripts/start-all.js` for `npm start`.
- Verified: `29/29` TS tests, `3/3` Python tests, `npm run build` clean, `eslint .` clean, Playwright browser check (`/vcon/dashboard`, `/vcon/settings`, `/api/metrics`) clean with `0` console errors.
- **Phase-03 Complete & Verified** (`BUG-C01`–`BUG-C06`):
- Hardened Redis flush lock TTL (`30000ms`), added in-process `activeFlushPromise` coalescing with a `flushDone` identity guard (`core/data/buffer.ts`, `server/index.js`), isolated `redis.rpush` from best-effort `REDIS_RECENT_KEY`, and logged malformed buffered JSON warnings.
- Enforced `sanitizeClientId()` path containment and `lstat` symlink exclusion, unique lock owner tokens (`${pid}:${ts}:${uuid}`), atomic `rename` stale-lock takeover, owner-verified lock unlock, and structured `JSON.stringify([clientId, date])` partition grouping in `core/Storage/csvStorage.ts` and `server/workers/csvWriter.js`.
- Fixed trailing newline blank-row prevention (`fileEndsWithNewline`), proactive size-boundary rotation (`-partN.csv`), `StringDecoder('utf8')` streaming row counting, cooperatively locked `pruneExpiredPartitions`, and 1-hour `PRUNE_COOLDOWN_MS` in `core/adminSettings/settingsManager.ts`.
- Enforced `parser`/`fileStream` destruction in `finally` blocks across `/api/data`, `GET /api/events`, and `/api/export`, error propagation inside `makeExportStream`, `502` surfacing on Port `4000` flush failure in `/api/export`, normalized `clientId` filtering, numeric timestamp sorting (`parseTimestampMs`), and `?format=json` / `?includeHeader=false` parity for `?filename=` exports.
- Verified: `32/32` TS tests, `3/3` Python tests, `npm run build` clean, `eslint .` clean, Playwright browser check (`/vcon/dashboard`, `/vcon/data`, `/vcon/storage`, `/vcon/export`, `/api/data`, `/api/storage`, `/api/export`) clean with `0` console errors.
- **Phase-04 Complete & Verified** (`BUG-D01`–`BUG-D07`):
- Extracted pure UI synchronization helpers in `lib/vconUiHelpers.ts` (`SETTINGS_LIMITS`, `normalizeEditableSettings`, `describeSettingsBridgeSync`, `mapBridgeStatusToSocketUiState`, `parseEventTimestamp`, `formatEventTimeLabel`, `getUtcDateRangeForPreset`, `matchesSelectedClientFolder`, `extractContentDispositionFilename`).
- Updated `components/sections/SettingsWorkspace.tsx` (`BUG-D01`) to probe `/api/metrics` honestly (removing false `/api/health` fallback), surface `bridgeSync` warning/error toasts on save, align integer bounds with `/api/settings` (`1..100000`, `1..10240`, `0..3650`), and eliminate the startup `serverPort` double-fetch.
- Updated `app/vcon/storage/page.tsx` (`BUG-D02`) and `components/sections/ExportWorkspace.tsx` (`BUG-D03`) with `cache: 'no-store'`, `401` redirects, UTC preset ranges, normalized client folder filters, and fetch+Blob downloads with `502`/JSON error surfacing.
- Updated `components/sections/DashboardOverview.tsx` (`BUG-D04`) and `components/sections/DataBrowser.tsx` (`BUG-D05`) with `ingestionBridgeStatus` display, `auth_error` alert banner, safe ISO/epoch-ms timestamp formatting, invalid date-range guard, and Enter-key filter submission.
- Updated `components/sections/NewClientModal.tsx`, `components/sections/ClientsTable.tsx`, `app/vcon/simulator/page.tsx`, `app/vcon/sdk/page.tsx`, `components/sections/ProfileWorkspace.tsx`, and `components/layout/Sidebar.tsx` (`BUG-D06`, `BUG-D07`) with `1..100000` rate limit parity, live `runtime.serverPort` sync, real HTTP batch ingestion in Simulator, password bounds check, and `datastack:profile-updated` live Sidebar sync.
- Verified: `37/37` TS tests (`tests/phase04-ui-sync.test.ts`), `3/3` Python tests, `npm run build` clean, `eslint .` clean, Playwright browser check across all Admin Console routes clean with `0` console errors.
- **Phase-05 Complete & Verified** (`BUG-E01`, `BUG-E03`, `BUG-E04`):
- Hardened canonical Python SDK (`sdk/python/datastack/datastack/*`): normalized `CLIENT_ID` comparison (`_normalize_client_id`), sanitized ragged CSV extra columns (`restkey="_extra"`), accurate `send_file()` `success` + `unprocessedItems`, partial-batch `rateLimitedEventIds` retry exception fallback, `DataStackConfigError` wrapping for malformed numeric env/config values (`config.py`), concurrent `stop()` vs `set_idle_connected()` race fix + clean thread join (`heartbeat.py`), `os.fsync` + orphan `.tmp` unlink + warning logs (`spool.py`), and PEP 561 `py.typed` marker.
- Aligned packaging metadata across `sdk/python/datastack/pyproject.toml`, `sdk/python/datastack/setup.py`, and `sdk-python/setup.py`, coordinated child-exit shutdown in `scripts/start-all.js`, and added `engines.node >=20.0.0` in `package.json`.
- Verified: `39/39` TS tests (`tests/phase05-packaging.test.ts`), `6/6` Python tests (`tests/phase05_sdk_test.py`), `py_compile`, `npm run build`, and `eslint .` clean.
- **Previous Remediation Cycle (2026-11-18)**: Completed Phases 01–05 (`39/39` TS tests, `6/6` Python tests).
- **New Production Hardening Cycle (`BASELINE-2026-11-19`)**:
- Re-audited entire codebase from raw source code (`notes/FORENSIC_AUDIT_REPORT.md`): **29** verified bugs/mismatches (`4` Critical, `8` High, `13` Medium, `4` Low).
- **Phase-01 Complete & Verified (`2026-11-19`)** (`BUG-01`, `BUG-04`, `BUG-05`, `BUG-06`):
- Blocked volatile RAM admin creation in production without `DATABASE_URL` (`AdminDatabaseRequiredError`), surfaced production DB warning on `/vcon/setup`, and mapped pre-setup login to `409 SETUP_REQUIRED`.
- Added brute-force attempt throttling (`429` + `Retry-After`) and completion-first ordering to `POST /api/auth/setup`.
- Added server-side `token_version` revocation (`revokeAdminSessions()`) on `POST /api/auth/logout`, removed client `Origin` trust from cookie `Secure` flag, and added `sec-fetch-site: same-origin` CSRF fallback.
- Prevented empty production `jwt_secret` persistence in PG and replaced silent DB write `catch {}` swallows in `SettingsManager` and `SDKManager` with `DATABASE_WRITE_FAILED` propagation and safe RAM rollback.
- Verified: `43/43` TS tests, `6/6` Python tests, `npm run build`, `eslint .`, and Playwright login/logout revocation check clean.
- **Phase-02 Complete & Verified (`2026-11-19`)** (`BUG-02`, `BUG-03`, `BUG-17`, `BUG-22`):
- Protected Port 4000's `inMemoryClientRegistry` from empty Port 3000 `bulk_sync` wipes (`!forceReplace`), exposed `inMemoryClients` on `/internal/metrics`, added non-destructive hydration in `SDKManager` (`listClientsAsync`, `getClientAsync`, `syncAllClientsToIngestionAsync`), and auto-reconciled Port 4000 cold starts in `ApiEngine.getDashboardMetrics()`.
- Ordered `SettingsManager.updateSettingsAsync()` JWT rotation to sync Port 4000 *before* committing Port 3000 RAM/PG/re-sign, aborting fail-closed (`JWT_ROTATION_BRIDGE_FAILED`) on bridge rejection/prod unreachable, rolling back Port 4000 on PG/re-sign error, and adding a 60s dual-secret grace window in `server/index.js` `requireInternalAuth()`.
- Added authenticated `POST /internal/security-check` on Port 4000 (backed by zero-copy `validateSingleEventLocal`) so Port 3000 HTTP `SecurityEngine.validateEventAsync()` shares rate limits & idempotency with Port 4000 WS even without Redis, and exposed explicit `securityMode` (`redis_shared` | `bridge_coordinated` | `degraded_local`) in `/api/metrics`.
- Replaced per-event PG `UPDATE client_sdks` queries on both Port 3000 (`SDKManager`) and Port 4000 (`server/index.js`) with 1s coalescing in-memory deltas flushed via batched `UNNEST` SQL updates (flushed on reads and graceful shutdown).
- Verified: `47/47` TS tests, `6/6` Python tests, `npm run build`, and `eslint .` clean.
- **Phase-03 Complete & Verified (`2026-11-19`)** (`BUG-18`, `BUG-19`, `BUG-20`, `BUG-29`):
- Added bounded Redis flush lock wait/retry, periodic Lua `PEXPIRE` TTL renewal with ownership fencing before `LTRIM`, and explicit `FLUSH_LOCK_TIMEOUT` (`503` on `/internal/flush`) across both `core/data/buffer.ts` and `server/index.js`.
- Added full deleted-client buffer purge (`DataBufferEngine.purgeClientEventsAsync`, Port 4000 `purgeClientFromLocalBuffer`) plus flush-time and recent-read tombstone filtering so deleted client directories never resurrect on subsequent flushes; ordered `SDKManager.deleteClientAsync()` to purge buffers and sync Port 4000 before deleting disk files.
- Converted `server/index.js` `onBufferPush` to `async` and awaited it before firing Socket.IO ACK callbacks in `server/socket/handlers.js` (`sync_data` and `batch_sync`), returning `BUFFER_PERSIST_FAILED` on persistence error.
- Unified canonical CSV storage root resolution via `DATASTACK_STORAGE_DIR` (`resolveDefaultStorageDir()`) across `core/Storage/csvStorage.ts`, `server/workers/csvWriter.js`, `core/sdk/sdkManager.ts`, and `.env.example`.
- Verified: `51/51` TS tests, `6/6` Python tests, `npm run build`, and `eslint .` clean.
- **Phase-04 Complete & Verified (`2026-11-19`)** (`BUG-21`, `BUG-23`, `BUG-24`):
- Aligned HTTP `POST /api/events` single-event envelope stripping 1:1 with Port 4000 WS `sync_data` and eliminated fabricated `{ sample, rand }` telemetry on empty `{}`.
- Converted `GET /api/data` directory/stat scans to non-blocking async `node:fs/promises` with newest-first date-partition pruning, and scaled `REDIS_RECENT_LIMIT` boundedly (`500..5000`) via `resolveRedisRecentLimit()` / `DATASTACK_RECENT_LIMIT` across Port 3000 & Port 4000.
- Replaced all `Math.random()` client secrets (`crypto.randomBytes(16)`) and fallback `eventId`s (`crypto.randomBytes(6)`) with CSPRNG across `core/sdk/sdkManager.ts`, `app/api/events/route.ts`, and `server/socket/handlers.js`.
- Unified canonical PostgreSQL metadata DDL (`CORE_SCHEMA_TABLE_DDLS` + `CORE_SCHEMA_ALTER_DDLS` + `ensureCoreSchemaOnPool()`) in `core/Database/database.ts`, delegated `core/auth/adminUsers.ts` to it, mirrored all 5 tables + 3 additive `ALTER TABLE` guards 1:1 in standalone `server/index.js`, and documented canonical `pg` ownership in `prisma/schema.prisma`.
- Verified: `55/55` TS tests (`tests/phase04-ingestion-hygiene.test.ts` added), `6/6` Python tests, `npm run build`, and `eslint .` clean.
- **Phase-05 Complete & Verified (`2026-11-19`)** (`BUG-07`, `BUG-08`, `BUG-09`, `BUG-14`):
- Fixed Simulator HTTP `Burst 10` to send canonical `{ clientId, batch }` (with `events` batch alias & explicit `accepted` count in `POST /api/events`) and summarize exact server-confirmed counts via `summarizeHttpBatchResult()` without `res.ok ? count : 0` falsification.
- Fixed `parseEventTimestamp()` & added `getEventUtcDate()` in `lib/vconUiHelpers.ts` for integer and fractional epoch-seconds (`< 1e11 * 1000`) 1:1 with backend CSV parsers.
- Added pending-queue-only `getPendingBufferedSummary()` (`?bufferedOnly=true` / `?includeBufferedSummary=true`) and fair multi-client partition scanning (`selectRecentPartitionFiles()`, lifting the global 5-file cap) in `GET /api/events`.
- Updated `ExportWorkspace.tsx` to match buffered events strictly against active client/date filters via `countMatchingBufferedEvents()` (preventing empty filtered header downloads) and refresh `/api/storage`, `/api/metrics`, and `vconCache` immediately post-export; updated `DashboardOverview.tsx` to attribute strictly UTC-today buffered events to `Events today` and call `await refresh()` after Quick Export.
- Verified: `58/58` TS tests, `6/6` Python tests, `npm run build`, `eslint .`, and Playwright E2E browser check (`/vcon/simulator`, `/vcon/export`, `/vcon/dashboard`, `0` console errors/warnings) clean.
- **Phase-06 Complete & Verified (`2026-11-19`)** (`BUG-10`, `BUG-11`, `BUG-12`, `BUG-13`, `BUG-15`, `BUG-16`):
- Returned explicit `SESSION_EXPIRED` / `AUTH_UNAVAILABLE` vs `INVALID_PASSWORD` codes in `/api/auth/profile`, replaced dead `401 && !currentPassword` redirect with `shouldRedirectProfileUnauthorized()`, and formatted `Session Expires` with calendar date + 24h time (`formatSessionExpiryLabel()`).
- Added opt-in `?purgeFiles=true` checkbox in `ClientsTable.tsx` client deletion modal and removed the misleading unused `Secret key` row from `CredentialsDisplay`.
- Refactored `SettingsWorkspace.tsx` numeric inputs to string drafts (`NumericSettingDrafts`) validated inline via `validateEditableSettingsDraft()` (`noValidate` submit + tab jump to first invalid field, zero snap-to-`0`, zero silent clamping) and clarified `Socket port (PORT_SOCKET)` as read-only runtime config.
- Unified public endpoint resolution (`resolvePublicEndpoints()`) across `SettingsWorkspace`, `/vcon/sdk`, and `/vcon/simulator` (consistent on port-80 `localhost` proxies) and updated `SdkSection.tsx` + `/vcon/sdk` with external path/wheel `pip install` guidance.
- Added pre-hydration `datastack-theme` bootstrap script + `suppressHydrationWarning` in `app/layout.tsx` (zero light-mode FOUC) and theme restoration/toggle in `AdminSetup.tsx`.
- Upgraded `public/sw.js` (`datastack-cache-v3`) with an uncached `503` HTML offline navigation fallback shell (`buildOfflineShellResponse()`, `Cache-Control: no-store`) for `/vcon/dashboard` and public navigations.
- Verified: `64/64` TS tests, `6/6` Python tests, `npm run build`, `eslint .`, and Playwright Chromium E2E check (`/vcon/profile`, `/vcon/clients`, `/vcon/settings`, `/vcon/sdk`, Light Mode reload, `0` console errors/warnings) clean.
- **Fresh Post-Phase-07 Forensic Audit & Baseline Freeze (`BASELINE-2026-11-19-R2`)**:
- Previous `Phase-07` hardening (`BUG-25`–`BUG-28`) landed in Python SDK (`client.py`), legacy `sdk-python`, `scripts/start-all.js` / `dev-all.js`, `nginx/default.conf`, `server/index.js`, `docker/docker-compose.yml`, and `.env.example`.
- Conducted a brand-new 100% raw-source-code audit (`notes/FORENSIC_AUDIT_REPORT.md`) identifying **18 verified remaining bugs** (`0` Critical, `6` High, `8` Medium, `4` Low).
- Frozen baseline (`BASELINE-2026-11-19-R2`) and created the complete 7-document governance & Scope-Lock suite in `notes/` (`README.md`, `BASELINE_FREEZE.md`, `PRODUCTION_READINESS_SCOPE_LOCK.md`, `PRODUCTION_ROADMAP.md`, `PHASE_EXECUTION_LOG.md`, `ERROR_HANDLING_MATRIX.md`, `ACTUAL_IMPLEMENTATION_STATUS.md`).
- **R2 Phase-01 Complete & Verified (`2026-11-19`)** (`BUG-01`, `BUG-02`):
- Moved same-origin CSRF enforcement in `middleware.ts` ahead of the public-route early return so `POST /api/auth/login` and `POST /api/auth/setup` reject cross-origin/missing-origin requests with `403` while preserving same-origin browser submissions, `sec-fetch-site: same-origin` fallback, safe `GET` routes, and Bearer `POST /api/events`.
- Extracted shared reverse-proxy-safe `extractClientIp()` in `lib/adminAuth.ts` (trusting `x-real-ip` first, then rightmost non-empty `x-forwarded-for` hop) and wired it into both `app/api/auth/login/route.ts` and `app/api/auth/setup/route.ts` to block spoofed-XFF brute-force lockout bypasses.
- Verified: `14/14` `tests/phase01-auth.test.ts` pass, `npm run build`, `npm run lint`, live HTTP `403`/`401` CSRF probes, and Chromium `/vcon/login` → `/vcon/dashboard` E2E (`0` errors).
- **R2 Phase-02 Complete & Verified (`2026-11-19`)** (`BUG-05`, `BUG-06`, `BUG-07`, `BUG-08`, `BUG-09`, `BUG-10`):
- Attached `.on('error', ...)` listeners to `DatabaseEngine.getPool()` (`max: 10`) and `server/index.js` `dbPool`, and consolidated `core/auth/adminUsers.ts` onto the shared `DatabaseEngine.getPool()`.
- Replaced non-atomic Redis `LTRIM`+`LPUSH` client deletion purging in `core/data/buffer.ts` with atomic Lua script `PURGE_CLIENT_LIST_LUA_SCRIPT` (`cjson.decode` + chunked `RPUSH`), and replaced non-atomic Redis `INCR`+`PEXPIRE` rate limiting with atomic Lua script `REDIS_RATE_LIMIT_LUA_SCRIPT` across both `core/Security/security.ts` and `server/socket/handlers.js`.
- Enforced `throwOnLockTimeout: true` locally and remotely on `POST /internal/flush` in `GET /api/export` and `POST /api/storage`, surfacing explicit `503 FLUSH_LOCK_TIMEOUT`.
- Added `DataBufferEngine.registerShutdownHook()` and registered `SDKManager.flushPendingClientActivityAsync()` so Port `3000` flushes coalesced `total_events` deltas on `SIGTERM`/`SIGINT`.
- Wrapped `GET`, `PATCH` (`regenerate_token` & update), and `DELETE` in `app/api/clients/[id]/route.ts` with structured JSON `try/catch` error responses.
- Verified: `14/14` `tests/phase02-sync.test.ts` pass, `3/3` `R2 PHASE-02` buffer/storage tests pass, `npm run build` pass, `npm run lint` pass (`0` errors).
- **R2 Phase-03 Complete & Verified (`2026-11-19`)** (`BUG-03`, `BUG-04`, `BUG-11`, `BUG-12`, `BUG-13`, `BUG-15`):
- Unified `SettingsBridgeSyncStatus` & `POST /api/settings` `bridgeSync` response (`attempted`, `ok`, `synced`, `status`, `error`, `checkedAt`) and updated `describeSettingsBridgeSync()` + `SettingsWorkspace.tsx` so Port `4000` warnings/online state work reliably.
- Counted regular non-symlink `.csv` files in `SDKManager.purgeClientStorageDir()` (`deleteClientWithStatsAsync()`) and returned exact `purgedFiles` from `DELETE /api/clients/[id]?purgeFiles=true`.
- Aligned flat batch item metadata stripping (`eventId`, `timestamp`, `clientId`) 1:1 with single-event stripping across HTTP `POST /api/events`, WS `batch_sync`, and Python SDK `send_batch()`/`send_file()`.
- Exposed `DataBufferEngine.getLocalBufferSnapshot()` for exact Port `3000` RAM pending summary in `getPendingBufferedSummary()`.
- Added non-blocking `pruneExpiredPartitionsAsync()` off live event loops in `core/Storage/csvStorage.ts` & `server/workers/csvWriter.js` (wired into `settingsManager.ts` & `server/index.js`) and deduplicated `GET /api/storage` to a single `listAllFilesAsync()` scan with accurate client-scoped `stats`.
- Cleared `selectedClientToken` on `/vcon/sdk` client switch (with loading placeholder + disabled `Copy Config`), added `400 INVALID_DATE_RANGE` for `from > to` in `GET /api/export`, and normalized `clientId` filtering in `ApiEngine.getExportFiles*` & Port `4000` `/internal/recent`.
- Verified: `21/21` `phase04-*` tests pass, `2/2` `R2 PHASE-03` storage tests pass, `9/9` Python SDK tests pass, Chromium E2E across `/vcon/settings`, `/vcon/clients`, `/vcon/sdk`, `/vcon/storage` (`0` console errors), `npm run build` & `npm run lint` (`0` errors).
- **R2 Phase-04 Complete & Verified (`2026-11-19`)** (`BUG-14`, `BUG-16`, `BUG-17`, `BUG-18`):
- Added `from __future__ import annotations`, imported `Tuple`, and replaced PEP 585 `tuple[int, Dict[str, Any]]` with Python `3.8`-safe `Tuple[int, Dict[str, Any]]` in `sdk/python/datastack/datastack/client.py`, verified by a full-package Python `3.8` AST scan test in `tests/phase05_sdk_test.py`.
- Added unbuffered streaming `location = /api/export` (`proxy_buffering off;`, `proxy_read_timeout 300s;`) targeting `http://nextjs_upstream` in both `nginx/default.conf` and `docker/coolify-nginx.conf`.
- Removed `eslint: { ignoreDuringBuilds: true }` from `next.config.ts`, set `package.json` `"name": "datastack"`, pruned unused template packages (`@google/genai`, `firebase-tools`), and synchronized `package-lock.json`.
- Isolated `DATASTACK_DISABLE_LISTEN=1` and live Port `4000` coupling across `tests/phase01-auth.test.ts`, `tests/phase02-sync.test.ts`, `tests/phase03-storage-buffer.test.ts`, `tests/phase04-ingestion-hygiene.test.ts`, and `tests/phase07-deployment.test.ts`.
- Verified: `83/83` full TS suite pass (with live Port `4000` running, without manual env flags), `10/10` Python SDK tests + `py_compile` pass, `npm run build` (with ESLint active) pass, `npm run lint` (`0` errors, `0` warnings).
- **Progress**: `4 / 4` phases complete (`18 / 18` bugs resolved — **100% Production Ready**).
- **Pre-Release Docker & Coolify Deployment Hardening Complete & Verified (`2026-11-19`)** (`DEPLOY-BUG-01`–`05`, audited in `notes/DEPLOYMENT_COOLIFY_AUDIT_REPORT.md`):
- **`DEPLOY-BUG-01`**: Replaced static `upstream` blocks in `docker/coolify-nginx.conf` with Docker embedded DNS `resolver 127.0.0.11 valid=10s ipv6=off;` and variable `proxy_pass` targets (`$nextjs_upstream` = `http://web:3000`, `$socketio_upstream` = `http://ingestion:4000`), preventing `502 Bad Gateway` stale container IP outages after container restarts/redeploys.
- **`DEPLOY-BUG-02`**: Refactored Port `4000` startup in `server/index.js` into `startIngestionServer()` awaiting bounded `syncSettingsFromDatabase()` before `server.listen(PORT)` so rotated PG `jwt_secret` and rate-limit settings hydrate prior to `/health: 200` and `/internal/*` bridge traffic.
- **`DEPLOY-BUG-03`**: Memoized `ensureDatabaseSchema()` per active `dbPool` instance in `server/index.js` (resetting only on pool swap or DDL failure) so 3-second `syncSettingsFromDatabase()` ticks no longer issue 8 `CREATE/ALTER TABLE` DDLs repeatedly.
- **`DEPLOY-BUG-04`**: Reordered `gracefulShutdown()` in `server/index.js` to close Socket.IO/HTTP ingress first (`closeServerGracefully()`), drain `pendingRedisWrites` and `flushBuffer()`/`flushPendingClientActivity()` with bounded timeouts (`2000ms`/`6000ms`), and close Redis (`quit()` with `2000ms` timeout + `.disconnect()` fallback) and PostgreSQL (`dbPool.end()` bounded `2000ms`); added `stop_grace_period: 20s` to both `web` and `ingestion` in `docker/docker-compose.yml`.
- **`DEPLOY-BUG-05`**: Clarified Coolify `Base Directory: /` and `Docker Compose Location: /docker/docker-compose.yml` in `docs/users/installation.md` and `README.md`.
- Verified: `85/85` full TS suite pass (`11/11` in `tests/phase07-deployment.test.ts`), `10/10` Python SDK tests pass, `npm run build` clean, `npm run lint` (`0` errors, `0` warnings).
- **V2 Baseline Freeze & Governance Planning Complete (`BASELINE-2026-11-19-V2`)**:
- Froze the clean, Deployment-Hardened runtime state (`85/85` TS tests, `10/10` Python tests, clean `npm run build` & `eslint .`, zero runtime code changes) as **`BASELINE-2026-11-19-V2`**.
- Created raw-source verified V2 planning & contract documents in `notes/`:
- `notes/V2_FEATURE_GAP_AUDIT.md` (`V2-FEAT-01`–`V2-FEAT-07` gap analysis + full `/vcon/*` control mapping).
- `notes/V2_DATABASE_AND_SETTINGS_CONTRACT.md` (zero-loss additive PostgreSQL DDL rules preserving `admin_users` `id = 1 CHECK (id = 1)`, AES-256-GCM DB secret encryption + masked `GET /api/settings`, Port `3000`↔`4000` live sync).
- `notes/BASELINE_FREEZE.md`, `notes/PRODUCTION_READINESS_SCOPE_LOCK.md` (Doc `001` Scope Locks for `V2-Phase-01`–`04`), `notes/PRODUCTION_ROADMAP.md` (Doc `002`), `notes/PHASE_EXECUTION_LOG.md` (Doc `003`, `0 / 4` V2 phases done), `notes/ERROR_HANDLING_MATRIX.md` (Doc `004`, including `V2-E01`–`V2-E12`), `notes/ACTUAL_IMPLEMENTATION_STATUS.md`, and `notes/README.md`.
- Roadmap grouped into 4 Scope-Locked V2 Phases (`V2-Phase-01` Archival/Gzip + Quota/Disk Guard; `V2-Phase-02` Webhooks/Alerts + IP Allowlist/Expiry + Prometheus/NDJSON; `V2-Phase-03` Multi-Admin RBAC + Audit Log; `V2-Phase-04` Node.js/TypeScript SDK + `/vcon/sdk` guide).
- **`V2-Phase-01` Complete & Verified (`2026-11-19`)** (`V2-FEAT-01`, `V2-FEAT-03`, `V2-E01`–`V2-E05`):
- **AES-256-GCM Secret Vault (`core/Security/secretVault.ts`)**: Deterministic HKDF-SHA256 key derivation (`encryptSecret`, `decryptSecret`, `getMaskedHintFromEncrypted`, `SecretVaultError`) so `s3_secret_access_key_enc` is encrypted at rest in PostgreSQL and never leaked in plaintext by `GET /api/settings` (`hasS3SecretKey` + `s3SecretKeyMask: "••••xxxx"`).
- **S3 / Cloudflare R2 / MinIO SigV4 Client (`core/Storage/s3Archive.ts`)**: Pure Node `node:crypto` + `fetch` AWS Signature v4 client (`testS3Connection`, `uploadFileToS3`, `buildS3ObjectKey`, `normalizeS3EndpointUrl`) with `POST /api/storage/s3-test` and `POST /api/storage/archive`.
- **Transparent `.csv.gz` Auto-Compression & Safe Archive-Before-Prune (`core/Storage/csvStorage.ts`, `server/workers/csvWriter.js`)**: Cooperatively locked atomic `.csv` → `.csv.gz` compression (`compressOldPartitionsAsync` / `compressOldPartitions`) excluding today's active UTC partition; transparent `.csv` + `.csv.gz` reading/streaming across `GET /api/data` (surfacing `500 Corrupt .csv.gz partition` on corrupt gzip — `V2-E03`), `GET /api/events`, and `GET /api/export` (`application/gzip` for raw single `.csv.gz` download); fail-safe `beforeDeleteFile` S3 upload hook (`V2-E02`) that never deletes local partitions if S3 upload fails (`502 S3_ARCHIVE_FAILED`).
- **Per-Client Storage Quota (`storageQuotaMb`) & VPS Disk Pause Guard (`diskPauseThresholdPct`)**: Additive PostgreSQL columns (`client_sdks.storage_quota_mb` + 9 `admin_settings` columns in `core/Database/database.ts`, synced in `server/index.js` while preserving `ddlCount === 8`), `30s` TTL cached `statfs` + directory byte counters, and HTTP `507 Insufficient Storage` (`DISK_SPACE_LOW` / `STORAGE_QUOTA_EXCEEDED` — `V2-E04`, `V2-E05`) on `POST /api/events` and Port `4000` `sync_data`/`batch_sync` while keeping all Admin read/export/settings routes online.
- **Admin UI Controls (`SettingsWorkspace.tsx`, `NewClientModal.tsx`, `ClientsTable.tsx`, `lib/vconUiHelpers.ts`)**: Added live VPS disk usage bar, `Auto-compress (.csv → .csv.gz)`, `Disk pause threshold (0 or 50–99%)`, full `S3 / R2 off-site archive` configuration with `Test S3 Connection` & `Archive Now`, and per-client `Storage quota (MB, 0 = unlimited)` in modal and table.
- **Verified**: `89/89` TS tests (`4/4` in `tests/v2-phase01-storage-quotas.test.ts`), `10/10` Python SDK tests, `npm run build` clean, `npm run lint` (`0` errors / `0` warnings), and Chromium E2E across `/vcon/settings` (Storage tab) and `/vcon/clients` (`0` console errors).
---
## 5. Known Constraints & Operational Notes for Future Contributors
- Never add per-row telemetry `INSERT` queries to PostgreSQL or add `DROP`/`TRUNCATE` auto-migrations.
- Never move Socket.IO handlers into Next.js `app/api/*` routes.
- When deploying updates to an existing Coolify VPS, always preserve existing `SERVICE_*` environment variables and avoid `docker compose down -v` so `pgdata`, `datastack-data`, and `redisdata` volumes remain intact.
- After any code change, verify with `npm run build`, `npm run lint`, the TS `tests/phase*.test.ts` suite (`npx --yes tsx --test`), and `python3 -m unittest tests/phase05_sdk_test.py`, then synchronize this `MEMORY.md`.