feat(auth): implement RBAC audit logging and session security
Integrate granular audit logging for admin actions, transition to role-based API access via `requireAdminApiSession`, and update session tokens to persist user roles and IDs for consistent multi-admin management.
This commit is contained in:
1 parent
2873b9904a
commit
5c977d6977
37 files changed
+5939
-349
No files matched your search
@@ -1,7 +1,7 @@
|
||||
# MEMORY.md — DataStack Cross-IDE Persistent Project Memory
|
||||
|
||||
> **Last Synchronized**: 2026-11-19
|
||||
> **Status**: **V2-Phase-01 Complete & Verified (`BASELINE-2026-11-19-V2`) — `1 / 4` V2 Phases Done (Storage `.csv.gz` Auto-Compression, S3/R2 Off-Site Archive with AES-256-GCM Secret Vault, Per-Client `storageQuotaMb` & VPS Disk Pause Guard `HTTP 507`) — Verified (`89/89` TS tests, `10/10` Python tests, ESLint-enabled build & lint `0` errors / `0` warnings).**
|
||||
> **Last Synchronized**: 2026-11-20
|
||||
> **Status**: **V2-Phase-04 & `V2-Deploy-Fix-01` Complete & Verified (`BASELINE-2026-11-19-V2`) — `4 / 4` V2 Phases + Final Coolify Redeploy Hardening Done (`V2-Phase-01` Storage `.csv.gz` Compression, S3/R2 Off-Site Archive & Per-Client/Disk Quotas; `V2-Phase-02` Outbound Webhook Alerting, Per-Client IP/CIDR Allowlist & Token Expiry, Prometheus `/api/prometheus` Metrics Endpoint & Streaming `.ndjson` Export; `V2-Phase-03` Multi-Admin RBAC `owner`/`admin`/`operator`/`viewer` via `admin_team_members` & Immutable Admin Audit Log `admin_audit_logs`; `V2-Phase-04` Official Node.js / TypeScript Client SDK `sdk/node/datastack` & `/vcon/sdk` Multi-Language Guide; `V2-Deploy-Fix-01` `DEPLOY-V2-BUG-01..03`) — Verified (`102/102` TS tests, `10/10` Python tests, ESLint-enabled build & lint `0` errors / `0` warnings).**
|
||||
> **Scope Rule**: Maintain this file accurately across VS Code, Google Antigravity, Cursor, and any other IDE after every modification, feature update, or bug fix (see `.ai-rules` and `AGENTS.md`).
|
||||
|
||||
---
|
||||
@@ -314,6 +314,22 @@ All planned hardening & deployment phases (**Phase-01 through Phase-07**) are **
|
||||
- **Per-Client Storage Quota (`storageQuotaMb`) & VPS Disk Pause Guard (`diskPauseThresholdPct`)**: Additive PostgreSQL columns (`client_sdks.storage_quota_mb` + 9 `admin_settings` columns in `core/Database/database.ts`, synced in `server/index.js` while preserving `ddlCount === 8`), `30s` TTL cached `statfs` + directory byte counters, and HTTP `507 Insufficient Storage` (`DISK_SPACE_LOW` / `STORAGE_QUOTA_EXCEEDED` — `V2-E04`, `V2-E05`) on `POST /api/events` and Port `4000` `sync_data`/`batch_sync` while keeping all Admin read/export/settings routes online.
|
||||
- **Admin UI Controls (`SettingsWorkspace.tsx`, `NewClientModal.tsx`, `ClientsTable.tsx`, `lib/vconUiHelpers.ts`)**: Added live VPS disk usage bar, `Auto-compress (.csv → .csv.gz)`, `Disk pause threshold (0 or 50–99%)`, full `S3 / R2 off-site archive` configuration with `Test S3 Connection` & `Archive Now`, and per-client `Storage quota (MB, 0 = unlimited)` in modal and table.
|
||||
- **Verified**: `89/89` TS tests (`4/4` in `tests/v2-phase01-storage-quotas.test.ts`), `10/10` Python SDK tests, `npm run build` clean, `npm run lint` (`0` errors / `0` warnings), and Chromium E2E across `/vcon/settings` (Storage tab) and `/vcon/clients` (`0` console errors).
|
||||
- **`V2-Phase-02` Complete & Verified (`2026-11-19`)** (`V2-FEAT-02`, `V2-FEAT-04`, `V2-FEAT-06`, `V2-E06`–`V2-E09`):
|
||||
- **Per-Client IP / CIDR Allowlist & Token Expiration (`core/Security/ipAllowlist.ts`, `core/sdk/sdkManager.ts`, `app/api/clients/**`, `app/api/events/route.ts`, `server/index.js`, `server/socket/handlers.js`)**: Validates and normalizes comma-separated IPv4, IPv4 CIDR (`/0..32`), IPv6, IPv6 CIDR (`/0..128`), IPv4-mapped IPv6 (`::ffff:`), loopback equivalence (`127.0.0.1` ↔ `::1`), and wildcard `*`; validates ISO `expiresAt` timestamps. Enforced across both Port `3000` `POST /api/events` and Port `4000` Socket.IO handshakes/events (`403 CLIENT_IP_FORBIDDEN` — `V2-E07`; `401 CLIENT_TOKEN_EXPIRED` — `V2-E08`), synced live via `/internal/client-sync` and additive PostgreSQL columns (`client_sdks.allowed_ips`, `client_sdks.expires_at`).
|
||||
- **Outbound Webhook Alerting (`core/alerts/webhookNotifier.ts`, `core/adminSettings/settingsManager.ts`, `app/api/settings/webhook-test/route.ts`)**: Sends Slack (`text`), Discord (`content`), and structured JSON webhook payloads signed with optional HMAC-SHA256 (`X-DataStack-Signature: sha256=<hex>`) and deduplicated per `event:clientId` via configurable `webhookCooldownSec` (`10..86400s`, default `300s`). Triggers non-blocking alerts on `DISK_SPACE_LOW`, `STORAGE_QUOTA_EXCEEDED`, `S3_ARCHIVE_FAILED`, `CLIENT_IP_FORBIDDEN`, and `CLIENT_TOKEN_EXPIRED`, plus on-demand `POST /api/settings/webhook-test` (`200` on delivery, `502 WEBHOOK_DELIVERY_FAILED` on upstream error — `V2-E06`). `webhook_secret_enc` is encrypted via AES-256-GCM Secret Vault and masked in `GET /api/settings` (`hasWebhookSecret`, `webhookSecretMask`).
|
||||
- **Prometheus Scrape Endpoint (`app/api/prometheus/route.ts`)**: `GET /api/prometheus` outputs Prometheus 0.0.4 text exposition format (`datastack_up`, `datastack_ingestion_bridge_up`, `datastack_active_sockets`, `datastack_events_total`, `datastack_buffer_events_total`, `datastack_storage_files_total`, `datastack_storage_bytes_total`, `datastack_storage_rows_total`, `datastack_disk_used_percent`, `datastack_disk_free_bytes`, `datastack_disk_total_bytes`, `datastack_clients_total`, `datastack_client_events_total`, `datastack_client_storage_bytes`). Protected by constant-time SHA-256 Bearer token comparison (`prometheus_metrics_token_enc` encrypted at rest) or active Admin Console session cookie (`401 PROMETHEUS_UNAUTHORIZED` — `V2-E09`).
|
||||
- **Streaming `.ndjson` Export (`app/api/export/route.ts`, `components/sections/ExportWorkspace.tsx`)**: Added `format=ndjson` (`Content-Type: application/x-ndjson; charset=utf-8`, `.ndjson` extension) streaming newline-delimited JSON objects across both `.csv` and `.csv.gz` partitions, with `NDJSON` format radio selector in `/vcon/export`.
|
||||
- **Admin UI Controls (`SettingsWorkspace.tsx`, `NewClientModal.tsx`, `ClientsTable.tsx`, `lib/vconUiHelpers.ts`)**: Added `Outbound webhook alerts` and `Prometheus metrics (/api/prometheus)` groups under `/vcon/settings` → **Security** tab, and `Allowed IPs / CIDR` + `Token expiration` controls in `/vcon/clients` New/Edit modals and table badges.
|
||||
- **Verified**: `93/93` TS tests (`4/4` in `tests/v2-phase02-alerts-ip-prometheus.test.ts`), `10/10` Python SDK tests, `npm run build` clean, `npm run lint` (`0` errors / `0` warnings), and Chromium E2E across `/vcon/settings` (`0` console errors).
|
||||
- **`V2-Phase-03` Complete & Verified (`2026-11-19`)** (`V2-FEAT-05`, `V2-E10`–`V2-E11`):
|
||||
- **Multi-Admin Role-Based Access Control (`core/Database/database.ts`, `core/auth/adminSession.ts`, `core/auth/adminUsers.ts`, `lib/adminAuth.ts`, `app/api/team/route.ts`, `app/api/team/[id]/route.ts`)**: Added canonical `admin_team_members` table in PostgreSQL (plus in-memory fallback) while preserving `admin_users` (`id = 1 CHECK (id = 1)`) as the immutable Primary Owner (`owner-1`, protected by `400 CANNOT_REVOKE_PRIMARY_OWNER` — `V2-E11`). Supports roles `owner` (rank 3 — full platform settings, secrets & team management), `admin` / `operator` (rank 2 — manage SDK clients, rotate tokens, manual buffer flushes, S3 test/archive), and `viewer` (rank 1 — read-only dashboards, events, clients, settings & exports). Enforced centrally via `requireAdminApiSession(req, { minRole })` returning `403 INSUFFICIENT_ROLE_PERMISSION` (`V2-E10`). Role/status/password changes on any team member immediately bump `token_version` to revoke existing sessions.
|
||||
- **Immutable Admin Audit Log (`core/auth/auditLog.ts`, `app/api/audit/route.ts`)**: Added canonical `admin_audit_logs` table in PostgreSQL (plus bounded 1,000-entry in-memory fallback) with automatic secret key redaction. Records governance events (`ADMIN_LOGIN`, `ADMIN_PROFILE_UPDATED`, `ADMIN_PASSWORD_UPDATED`, `TEAM_MEMBER_CREATED`, `TEAM_MEMBER_UPDATED`, `TEAM_MEMBER_REVOKED`, `TEAM_MEMBER_DELETED`, `CLIENT_CREATED`, `CLIENT_UPDATED`, `CLIENT_TOKEN_ROTATED`, `CLIENT_DELETED`, `SETTINGS_UPDATED`, `S3_ARCHIVE_TRIGGERED`, `WEBHOOK_TEST_TRIGGERED`) without ever inserting telemetry events into PostgreSQL. Exposed filterable `GET /api/audit` (`?action=`, `?actor=`, `?q=`, `?limit=`).
|
||||
- **Admin Console UI (`components/sections/TeamWorkspace.tsx`, `components/sections/AuditLogViewer.tsx`, `app/vcon/team/page.tsx`, `app/vcon/audit/page.tsx`, `components/layout/Sidebar.tsx`)**: Added `/vcon/team` (Team & RBAC directory, add member modal, role selector, session revocation & deletion) and `/vcon/audit` (filterable governance trail viewer) linked from `Sidebar.tsx`.
|
||||
- **Verified**: `97/97` TS tests (`4/4` in `tests/v2-phase03-rbac-audit.test.ts`), `10/10` Python SDK tests, `npm run build` clean, `npm run lint` (`0` errors / `0` warnings), and Chromium E2E across `/vcon/team` and `/vcon/audit` (`0` console errors).
|
||||
- **`V2-Phase-04` Complete & Verified (`2026-11-20`)** (`V2-FEAT-07`, `V2-E12`):
|
||||
- **Official Node.js / TypeScript Client SDK (`sdk/node/datastack/` — `@datastack/sdk` v2.2.0)**: Created `sdk/node/datastack/package.json`, `types.ts`, `spool.ts`, `index.ts`, and `README.md`. Supports `datastack.config.json` / `DATASTACK_*` env / constructor options with strict `normalizeClientId` vs JWT `clientId`/`sub` verification (`DataStackConfigError` — `V2-E12`), Socket.IO primary streaming (`connect()`, `send()`/`sync()`, `sendBatch()`/`batchSync()`), automatic HTTP `POST /api/events` fallback, recursive `413 PAYLOAD_TOO_LARGE` batch splitting, partial-batch `429 RATE_LIMIT_EXCEEDED` retry, typed terminal error mapping (`401 CLIENT_TOKEN_EXPIRED` & `403 CLIENT_IP_FORBIDDEN` -> `DataStackAuthError`, `507 STORAGE_QUOTA_EXCEEDED` & `DISK_SPACE_LOW` -> `DataStackQuotaExceededError`), durable `.datastack_spool/spool.jsonl` local disk spooling (`DiskSpooler`) & `flushSpool()`, direct `.csv`/`.json`/`.jsonl`/`.ndjson` file ingestion (`sendFile()` with deterministic SHA-256 `eventId` via `makeDeterministicFileEventId`), and graceful `await client.shutdown(timeoutMs)` drain.
|
||||
- **Multi-Language SDK Guide (`app/vcon/sdk/page.tsx`)**: Added interactive language tabs (`Python SDK (v2.2.0)`, `Node.js / TypeScript SDK (v2.2.0)`, `HTTP / cURL`) with live client JWT token population and copy-ready `config.datastack.py` / `datastack.config.json` snippets.
|
||||
- **Verified**: `101/101` TS tests (`4/4` in `tests/v2-phase04-node-sdk.test.ts`), `10/10` Python SDK tests, `npm run build` clean, and `npm run lint` (`0` errors / `0` warnings).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
import { listAdminAuditLogsAsync } from "@/core/auth/auditLog";
|
||||
import { requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function GET(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req, { minRole: "viewer" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
try {
|
||||
const { searchParams } = new URL(req.url);
|
||||
const rawLimit = Number(searchParams.get("limit") || "100");
|
||||
const limit = Number.isFinite(rawLimit)
|
||||
? Math.min(Math.max(Math.floor(rawLimit), 1), 500)
|
||||
: 100;
|
||||
const action = searchParams.get("action") || undefined;
|
||||
const actorEmail =
|
||||
searchParams.get("actor") ||
|
||||
searchParams.get("actorEmail") ||
|
||||
undefined;
|
||||
const search =
|
||||
searchParams.get("q") || searchParams.get("search") || undefined;
|
||||
|
||||
const logs = await listAdminAuditLogsAsync({
|
||||
limit,
|
||||
action,
|
||||
actorEmail,
|
||||
search,
|
||||
});
|
||||
|
||||
return NextResponse.json({
|
||||
logs,
|
||||
total: logs.length,
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Unable to fetch audit logs.";
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import {
|
||||
type AdminRole,
|
||||
createAdminSessionToken,
|
||||
getAdminSessionSecret,
|
||||
getSessionCookieOptions,
|
||||
@@ -9,6 +10,7 @@ import {
|
||||
clearAdminLoginAttempts,
|
||||
consumeAdminLoginAttempt,
|
||||
} from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { extractClientIp } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
@@ -79,7 +81,13 @@ export async function POST(request: NextRequest) {
|
||||
);
|
||||
}
|
||||
|
||||
let authResult: { authenticated: boolean; tokenVersion: number };
|
||||
let authResult: {
|
||||
authenticated: boolean;
|
||||
tokenVersion: number;
|
||||
role: AdminRole;
|
||||
memberId: string;
|
||||
email: string;
|
||||
};
|
||||
try {
|
||||
authResult = await authenticateAdminCredentials(email, password);
|
||||
} catch (err) {
|
||||
@@ -117,13 +125,29 @@ export async function POST(request: NextRequest) {
|
||||
const remember = body.remember === true;
|
||||
const maxAge = remember ? REMEMBERED_SESSION_SECONDS : SESSION_SECONDS;
|
||||
const expiresAt = Date.now() + maxAge * 1000;
|
||||
const effectiveEmail = authResult.email || email;
|
||||
const token = await createAdminSessionToken(
|
||||
email,
|
||||
effectiveEmail,
|
||||
sessionSecret,
|
||||
expiresAt,
|
||||
authResult.tokenVersion,
|
||||
authResult.role,
|
||||
authResult.memberId,
|
||||
);
|
||||
const response = NextResponse.json({ success: true });
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: effectiveEmail,
|
||||
actorRole: authResult.role || "owner",
|
||||
action: "ADMIN_LOGIN",
|
||||
target: effectiveEmail,
|
||||
ipAddress: clientIp,
|
||||
metadata: { remember },
|
||||
});
|
||||
|
||||
const response = NextResponse.json({
|
||||
success: true,
|
||||
role: authResult.role || "owner",
|
||||
});
|
||||
|
||||
response.cookies.set(
|
||||
SESSION_COOKIE,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import {
|
||||
createAdminSessionToken,
|
||||
formatAdminRoleLabel,
|
||||
getSessionCookieOptions,
|
||||
} from "@/core/auth/adminSession";
|
||||
import {
|
||||
@@ -7,6 +8,8 @@ import {
|
||||
updateAdminAccount,
|
||||
verifyRequestAdminSession,
|
||||
} from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { extractClientIp } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
const SESSION_COOKIE = "datastack_admin_session";
|
||||
@@ -33,10 +36,13 @@ export async function GET(request: NextRequest) {
|
||||
|
||||
try {
|
||||
const account = await getAdminAccountInfo(auth.session.email);
|
||||
const effectiveRole = auth.session.role || account.role || "owner";
|
||||
return NextResponse.json({
|
||||
profile: {
|
||||
email: account.email,
|
||||
role: "Owner",
|
||||
role: formatAdminRoleLabel(effectiveRole),
|
||||
roleKey: effectiveRole,
|
||||
memberId: auth.session.memberId || account.memberId || "owner-1",
|
||||
storageMode: account.storageMode,
|
||||
createdAt: account.createdAt,
|
||||
sessionExpiresAt: new Date(auth.session.expiresAt).toISOString(),
|
||||
@@ -125,24 +131,42 @@ export async function PUT(request: NextRequest) {
|
||||
);
|
||||
}
|
||||
|
||||
// Preserve remaining session duration (e.g., 30-day "Remember me" session)
|
||||
const remainingSeconds = Math.max(
|
||||
SESSION_SECONDS,
|
||||
Math.ceil((auth.session.expiresAt - Date.now()) / 1000),
|
||||
);
|
||||
const expiresAt = Date.now() + remainingSeconds * 1000;
|
||||
const effectiveRole = result.role || auth.session.role || "owner";
|
||||
const effectiveMemberId =
|
||||
result.memberId || auth.session.memberId || "owner-1";
|
||||
const nextToken = await createAdminSessionToken(
|
||||
result.email,
|
||||
auth.sessionSecret,
|
||||
expiresAt,
|
||||
result.tokenVersion,
|
||||
effectiveRole,
|
||||
effectiveMemberId,
|
||||
);
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: result.email,
|
||||
actorRole: effectiveRole,
|
||||
action: newPassword ? "ADMIN_PASSWORD_UPDATED" : "ADMIN_PROFILE_UPDATED",
|
||||
target: result.email,
|
||||
ipAddress: extractClientIp(request),
|
||||
metadata: {
|
||||
emailChanged: result.email !== account.email,
|
||||
passwordRotated: Boolean(newPassword),
|
||||
},
|
||||
});
|
||||
|
||||
const response = NextResponse.json({
|
||||
success: true,
|
||||
profile: {
|
||||
email: result.email,
|
||||
role: "Owner",
|
||||
role: formatAdminRoleLabel(effectiveRole),
|
||||
roleKey: effectiveRole,
|
||||
memberId: effectiveMemberId,
|
||||
storageMode: account.storageMode,
|
||||
createdAt: account.createdAt,
|
||||
sessionExpiresAt: new Date(expiresAt).toISOString(),
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
import { verifyRequestAdminSession } from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { SDKManager } from "@/core/sdk/sdkManager";
|
||||
import {
|
||||
validateAllowedIpsInput,
|
||||
validateClientExpiresAtInput,
|
||||
} from "@/core/Security/ipAllowlist";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
const SESSION_COOKIE = "datastack_admin_session";
|
||||
const ALLOWED_PATCH_KEYS = new Set([
|
||||
"action",
|
||||
"name",
|
||||
@@ -21,14 +21,12 @@ export async function GET(
|
||||
req: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> },
|
||||
) {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const includeToken = req.nextUrl.searchParams.get("includeToken") === "true";
|
||||
const auth = await requireAdminApiSession(req, {
|
||||
minRole: includeToken ? "operator" : "viewer",
|
||||
});
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: "Authentication required." },
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
const { id } = await params;
|
||||
@@ -40,7 +38,6 @@ export async function GET(
|
||||
}
|
||||
const { secret, token, ...clientSummary } = client;
|
||||
void secret;
|
||||
const includeToken = req.nextUrl.searchParams.get("includeToken") === "true";
|
||||
return NextResponse.json({
|
||||
client: {
|
||||
...clientSummary,
|
||||
@@ -65,14 +62,9 @@ export async function PATCH(
|
||||
req: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> },
|
||||
) {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "operator" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: "Authentication required." },
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
const { id } = await params;
|
||||
@@ -90,7 +82,10 @@ export async function PATCH(
|
||||
|
||||
if (!rawBody || typeof rawBody !== "object" || Array.isArray(rawBody)) {
|
||||
return NextResponse.json(
|
||||
{ error: "Update payload must be a JSON object.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "Update payload must be a JSON object.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -120,8 +115,23 @@ export async function PATCH(
|
||||
try {
|
||||
const refreshed = await SDKManager.regenerateTokenAsync(normalizedId);
|
||||
if (!refreshed) {
|
||||
return NextResponse.json({ error: "Client not found" }, { status: 404 });
|
||||
return NextResponse.json(
|
||||
{ error: "Client not found" },
|
||||
{ status: 404 },
|
||||
);
|
||||
}
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "CLIENT_TOKEN_ROTATED",
|
||||
target: refreshed.clientId,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
clientId: refreshed.clientId,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({ client: refreshed });
|
||||
} catch (err: unknown) {
|
||||
const message =
|
||||
@@ -130,7 +140,9 @@ export async function PATCH(
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: message,
|
||||
code: isDbWrite ? "DATABASE_WRITE_FAILED" : "TOKEN_REGENERATION_FAILED",
|
||||
code: isDbWrite
|
||||
? "DATABASE_WRITE_FAILED"
|
||||
: "TOKEN_REGENERATION_FAILED",
|
||||
},
|
||||
{ status: isDbWrite ? 503 : 500 },
|
||||
);
|
||||
@@ -152,7 +164,10 @@ export async function PATCH(
|
||||
body.name.trim().length > 120
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "Client name must be a non-empty string (max 120 characters).", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "Client name must be a non-empty string (max 120 characters).",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -166,7 +181,10 @@ export async function PATCH(
|
||||
body.rateLimit > 100000
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "rateLimit must be an integer between 1 and 100000.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "rateLimit must be an integer between 1 and 100000.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -180,7 +198,10 @@ export async function PATCH(
|
||||
body.storageQuotaMb > 1048576
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "storageQuotaMb must be an integer between 0 and 1048576.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "storageQuotaMb must be an integer between 0 and 1048576.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -196,7 +217,10 @@ export async function PATCH(
|
||||
const ipValidation = validateAllowedIpsInput(body.allowedIps);
|
||||
if (!ipValidation.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: ipValidation.error || "Invalid allowedIps.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: ipValidation.error || "Invalid allowedIps.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -206,16 +230,22 @@ export async function PATCH(
|
||||
const expValidation = validateClientExpiresAtInput(body.expiresAt);
|
||||
if (!expValidation.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: expValidation.error || "Invalid expiresAt.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: expValidation.error || "Invalid expiresAt.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
updates.expiresAt = expValidation.normalized;
|
||||
updates.expiresAt = expValidation.iso ?? null;
|
||||
}
|
||||
if (body.status !== undefined) {
|
||||
if (body.status !== "active" && body.status !== "revoked") {
|
||||
return NextResponse.json(
|
||||
{ error: "status must be 'active' or 'revoked'.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "status must be 'active' or 'revoked'.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -224,7 +254,10 @@ export async function PATCH(
|
||||
|
||||
if (Object.keys(updates).length === 0) {
|
||||
return NextResponse.json(
|
||||
{ error: "No valid fields provided for update.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "No valid fields provided for update.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -234,6 +267,16 @@ export async function PATCH(
|
||||
if (!updated) {
|
||||
return NextResponse.json({ error: "Client not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "CLIENT_UPDATED",
|
||||
target: updated.clientId,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: updates,
|
||||
});
|
||||
|
||||
return NextResponse.json({ client: updated });
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : "Update failed";
|
||||
@@ -252,14 +295,9 @@ export async function DELETE(
|
||||
req: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> },
|
||||
) {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "operator" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: "Authentication required." },
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
const { id } = await params;
|
||||
@@ -272,13 +310,27 @@ export async function DELETE(
|
||||
if (!result.deleted) {
|
||||
return NextResponse.json({ error: "Client not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "CLIENT_DELETED",
|
||||
target: normalizedId,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
purgeFiles,
|
||||
purgedFiles: result.purgedFiles,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
message: `Client ${normalizedId} deleted`,
|
||||
purgedFiles: result.purgedFiles,
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : "Client deletion failed";
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Client deletion failed";
|
||||
const isDbWrite = message.includes("DATABASE_WRITE_FAILED");
|
||||
return NextResponse.json(
|
||||
{
|
||||
|
||||
+60
-27
@@ -1,22 +1,16 @@
|
||||
import { verifyRequestAdminSession } from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { SDKManager } from "@/core/sdk/sdkManager";
|
||||
import {
|
||||
validateAllowedIpsInput,
|
||||
validateClientExpiresAtInput,
|
||||
} from "@/core/Security/ipAllowlist";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
const SESSION_COOKIE = "datastack_admin_session";
|
||||
|
||||
export async function GET(req: NextRequest) {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "viewer" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: "Authentication required." },
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -37,14 +31,9 @@ export async function GET(req: NextRequest) {
|
||||
}
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "operator" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: "Authentication required." },
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
let body: Record<string, unknown>;
|
||||
@@ -71,9 +60,17 @@ export async function POST(req: NextRequest) {
|
||||
expiresAt,
|
||||
} = body;
|
||||
|
||||
if (typeof clientId !== "string" || typeof name !== "string" || !clientId.trim() || !name.trim()) {
|
||||
if (
|
||||
typeof clientId !== "string" ||
|
||||
typeof name !== "string" ||
|
||||
!clientId.trim() ||
|
||||
!name.trim()
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "clientId and name are required strings", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "clientId and name are required strings",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -87,17 +84,32 @@ export async function POST(req: NextRequest) {
|
||||
}
|
||||
|
||||
const parsedRateLimit = rateLimit === undefined ? 100 : Number(rateLimit);
|
||||
if (!Number.isInteger(parsedRateLimit) || parsedRateLimit < 1 || parsedRateLimit > 100000) {
|
||||
if (
|
||||
!Number.isInteger(parsedRateLimit) ||
|
||||
parsedRateLimit < 1 ||
|
||||
parsedRateLimit > 100000
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "rateLimit must be an integer between 1 and 100000.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "rateLimit must be an integer between 1 and 100000.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
|
||||
const parsedQuotaMb = storageQuotaMb === undefined ? 0 : Number(storageQuotaMb);
|
||||
if (!Number.isInteger(parsedQuotaMb) || parsedQuotaMb < 0 || parsedQuotaMb > 1048576) {
|
||||
const parsedQuotaMb =
|
||||
storageQuotaMb === undefined ? 0 : Number(storageQuotaMb);
|
||||
if (
|
||||
!Number.isInteger(parsedQuotaMb) ||
|
||||
parsedQuotaMb < 0 ||
|
||||
parsedQuotaMb > 1048576
|
||||
) {
|
||||
return NextResponse.json(
|
||||
{ error: "storageQuotaMb must be an integer between 0 and 1048576.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: "storageQuotaMb must be an integer between 0 and 1048576.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -113,7 +125,10 @@ export async function POST(req: NextRequest) {
|
||||
const ipValidation = validateAllowedIpsInput(allowedIps);
|
||||
if (!ipValidation.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: ipValidation.error || "Invalid allowedIps.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: ipValidation.error || "Invalid allowedIps.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
@@ -125,11 +140,14 @@ export async function POST(req: NextRequest) {
|
||||
const expValidation = validateClientExpiresAtInput(expiresAt);
|
||||
if (!expValidation.valid) {
|
||||
return NextResponse.json(
|
||||
{ error: expValidation.error || "Invalid expiresAt.", code: "VALIDATION_ERROR" },
|
||||
{
|
||||
error: expValidation.error || "Invalid expiresAt.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
normalizedExpiresAt = expValidation.normalized;
|
||||
normalizedExpiresAt = expValidation.iso ?? null;
|
||||
}
|
||||
|
||||
const existing = await SDKManager.getClientAsync(safeClientId);
|
||||
@@ -149,6 +167,21 @@ export async function POST(req: NextRequest) {
|
||||
expiresAt: normalizedExpiresAt,
|
||||
});
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "CLIENT_CREATED",
|
||||
target: newClient.clientId,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
name: newClient.name,
|
||||
rateLimit: newClient.rateLimit,
|
||||
storageQuotaMb: newClient.storageQuotaMb,
|
||||
allowedIps: newClient.allowedIps,
|
||||
expiresAt: newClient.expiresAt,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({ client: newClient }, { status: 201 });
|
||||
} catch (err: unknown) {
|
||||
const message =
|
||||
|
||||
@@ -73,12 +73,13 @@ export async function GET(req: NextRequest) {
|
||||
);
|
||||
}
|
||||
|
||||
const [clients, storageStats, diskHealth, bufferStats] = await Promise.all([
|
||||
const [clients, allFiles, diskHealth, bufferStats] = await Promise.all([
|
||||
SDKManager.listClientsAsync(),
|
||||
CSVStorageManager.getStatsAsync(),
|
||||
CSVStorageManager.getDiskSpaceHealthAsync(),
|
||||
CSVStorageManager.listAllFilesAsync(),
|
||||
CSVStorageManager.getDiskUsageAsync(),
|
||||
DataBufferEngine.getBufferStatsAsync(),
|
||||
]);
|
||||
const storageStats = CSVStorageManager.formatStorageStats(allFiles);
|
||||
|
||||
let activeSockets = 0;
|
||||
let bridgeUp = 0;
|
||||
@@ -117,7 +118,9 @@ export async function GET(req: NextRequest) {
|
||||
|
||||
let activeClientsCount = 0;
|
||||
let revokedClientsCount = 0;
|
||||
let totalEventsCounter = 0;
|
||||
for (const c of clients) {
|
||||
totalEventsCounter += Math.max(0, Number(c.totalEvents) || 0);
|
||||
if (c.status === "revoked") {
|
||||
revokedClientsCount += 1;
|
||||
} else {
|
||||
@@ -135,6 +138,9 @@ export async function GET(req: NextRequest) {
|
||||
"# HELP datastack_active_sockets Currently connected Socket.IO SDK clients on Port 4000.",
|
||||
"# TYPE datastack_active_sockets gauge",
|
||||
`datastack_active_sockets ${activeSockets}`,
|
||||
"# HELP datastack_events_total Cumulative events ingested across all registered clients.",
|
||||
"# TYPE datastack_events_total counter",
|
||||
`datastack_events_total ${Math.max(totalEventsCounter, storageStats.totalRows)}`,
|
||||
"# HELP datastack_buffer_events_total Total events currently queued in memory or Redis awaiting CSV flush.",
|
||||
"# TYPE datastack_buffer_events_total gauge",
|
||||
`datastack_buffer_events_total ${totalBufferedEvents}`,
|
||||
@@ -143,7 +149,7 @@ export async function GET(req: NextRequest) {
|
||||
`datastack_storage_files_total ${storageStats.totalFiles}`,
|
||||
"# HELP datastack_storage_bytes_total Total bytes occupied by CSV and gzipped CSV partition files on disk.",
|
||||
"# TYPE datastack_storage_bytes_total gauge",
|
||||
`datastack_storage_bytes_total ${storageStats.totalSizeBytes}`,
|
||||
`datastack_storage_bytes_total ${storageStats.totalBytes}`,
|
||||
"# HELP datastack_storage_rows_total Total telemetry rows persisted across all partition files.",
|
||||
"# TYPE datastack_storage_rows_total gauge",
|
||||
`datastack_storage_rows_total ${storageStats.totalRows}`,
|
||||
@@ -152,7 +158,7 @@ export async function GET(req: NextRequest) {
|
||||
`datastack_disk_used_percent ${diskHealth.usedPct}`,
|
||||
"# HELP datastack_disk_free_bytes Free bytes available on the underlying storage volume.",
|
||||
"# TYPE datastack_disk_free_bytes gauge",
|
||||
`datastack_disk_free_bytes ${diskHealth.freeBytes}`,
|
||||
`datastack_disk_free_bytes ${diskHealth.availableBytes}`,
|
||||
"# HELP datastack_disk_total_bytes Total capacity in bytes of the underlying storage volume.",
|
||||
"# TYPE datastack_disk_total_bytes gauge",
|
||||
`datastack_disk_total_bytes ${diskHealth.totalBytes}`,
|
||||
@@ -177,7 +183,7 @@ export async function GET(req: NextRequest) {
|
||||
);
|
||||
|
||||
const perClientBytes = new Map<string, number>();
|
||||
for (const f of storageStats.files) {
|
||||
for (const f of allFiles) {
|
||||
perClientBytes.set(
|
||||
f.clientId,
|
||||
(perClientBytes.get(f.clientId) || 0) + f.sizeBytes,
|
||||
|
||||
+16
-27
@@ -2,12 +2,11 @@ import {
|
||||
SettingsManager,
|
||||
type AdminSettingsUpdateInput,
|
||||
} from "@/core/adminSettings/settingsManager";
|
||||
import { verifyRequestAdminSession } from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { SecretVaultError } from "@/core/Security/secretVault";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
const SESSION_COOKIE = "datastack_admin_session";
|
||||
|
||||
const ALLOWED_SETTING_KEYS = new Set([
|
||||
"maxEventsPerSec",
|
||||
"maxPayloadSizeKb",
|
||||
@@ -69,19 +68,9 @@ function parseOptionalString(
|
||||
|
||||
export async function GET(req: NextRequest) {
|
||||
try {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "viewer" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error:
|
||||
auth.reason === "UNAUTHORIZED"
|
||||
? "Authentication required."
|
||||
: "Admin authentication is temporarily unavailable.",
|
||||
},
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
const settings = await SettingsManager.getPublicSettingsAsync();
|
||||
@@ -94,19 +83,9 @@ export async function GET(req: NextRequest) {
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
try {
|
||||
const auth = await verifyRequestAdminSession(
|
||||
req.cookies.get(SESSION_COOKIE)?.value,
|
||||
);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "owner" });
|
||||
if (!auth.valid) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error:
|
||||
auth.reason === "UNAUTHORIZED"
|
||||
? "Authentication required."
|
||||
: "Admin authentication is temporarily unavailable.",
|
||||
},
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
);
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
let rawBody: unknown;
|
||||
@@ -368,6 +347,16 @@ export async function POST(req: NextRequest) {
|
||||
}
|
||||
|
||||
await SettingsManager.updateSettingsAsync(validated);
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "SETTINGS_UPDATED",
|
||||
target: "admin_settings",
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
updatedFields: Object.keys(validated),
|
||||
},
|
||||
});
|
||||
const settings = await SettingsManager.getPublicSettingsAsync();
|
||||
const bridgeSync = SettingsManager.getLastBridgeSyncStatus();
|
||||
return NextResponse.json({
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import { SettingsManager } from "@/core/adminSettings/settingsManager";
|
||||
import { dispatchWebhookAlertAsync } from "@/core/alerts/webhookNotifier";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { SecretVaultError } from "@/core/Security/secretVault";
|
||||
import { requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "owner" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
@@ -73,7 +74,7 @@ export async function POST(req: NextRequest) {
|
||||
title: "Webhook Verification",
|
||||
message: "Test alert dispatched from DataStack Admin Console.",
|
||||
details: {
|
||||
triggeredBy: auth.auth.user?.email || "admin",
|
||||
triggeredBy: auth.auth.session.email || "admin",
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -85,6 +86,18 @@ export async function POST(req: NextRequest) {
|
||||
{ bypassCooldown: true },
|
||||
);
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "WEBHOOK_TEST_TRIGGERED",
|
||||
target: webhookUrl,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
ok: result.ok,
|
||||
status: result.status,
|
||||
},
|
||||
});
|
||||
|
||||
if (!result.ok) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { SettingsManager } from "@/core/adminSettings/settingsManager";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { SecretVaultError } from "@/core/Security/secretVault";
|
||||
import { CSVStorageManager } from "@/core/Storage/csvStorage";
|
||||
import {
|
||||
@@ -6,11 +7,11 @@ import {
|
||||
normalizeS3EndpointUrl,
|
||||
uploadFileToS3,
|
||||
} from "@/core/Storage/s3Archive";
|
||||
import { requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "operator" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
@@ -135,6 +136,20 @@ export async function POST(req: NextRequest) {
|
||||
});
|
||||
}
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "S3_ARCHIVE_TRIGGERED",
|
||||
target: s3Config.bucket,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
archivedCount: archiveResult.archivedCount,
|
||||
prunedCount: archiveResult.prunedCount,
|
||||
compressedCount,
|
||||
pruneAfterUpload,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({
|
||||
success: archiveResult.errors.length === 0,
|
||||
compressedCount,
|
||||
|
||||
@@ -50,7 +50,7 @@ export async function GET(req: NextRequest) {
|
||||
}
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req);
|
||||
const auth = await requireAdminApiSession(req, { minRole: 'operator' });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ import { requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req);
|
||||
const auth = await requireAdminApiSession(req, { minRole: "operator" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
import {
|
||||
deleteAdminTeamMemberAsync,
|
||||
PrimaryOwnerProtectedError,
|
||||
TeamMemberValidationError,
|
||||
updateAdminTeamMemberAsync,
|
||||
} from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function PATCH(
|
||||
req: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> },
|
||||
) {
|
||||
const auth = await requireAdminApiSession(req, { minRole: "owner" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
let body: Record<string, unknown>;
|
||||
try {
|
||||
const parsed = await req.json();
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: "Team member update payload must be a JSON object.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
body = parsed as Record<string, unknown>;
|
||||
} catch {
|
||||
return NextResponse.json(
|
||||
{ error: "Invalid JSON request payload.", code: "INVALID_JSON" },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const { id } = await params;
|
||||
const updated = await updateAdminTeamMemberAsync(id, {
|
||||
role: typeof body.role === "string" ? body.role : undefined,
|
||||
status: typeof body.status === "string" ? body.status : undefined,
|
||||
password: typeof body.password === "string" ? body.password : undefined,
|
||||
revokeSessions: body.revokeSessions === true,
|
||||
});
|
||||
|
||||
const action =
|
||||
body.status === "disabled" || body.revokeSessions === true
|
||||
? "TEAM_MEMBER_REVOKED"
|
||||
: "TEAM_MEMBER_UPDATED";
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action,
|
||||
target: updated.email,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
memberId: updated.id,
|
||||
role: updated.role,
|
||||
status: updated.status,
|
||||
passwordReset: Boolean(body.password),
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({ success: true, member: updated });
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof PrimaryOwnerProtectedError) {
|
||||
return NextResponse.json(
|
||||
{ error: err.message, code: err.code },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
if (err instanceof TeamMemberValidationError) {
|
||||
const status = err.code === "TEAM_MEMBER_NOT_FOUND" ? 404 : 400;
|
||||
return NextResponse.json(
|
||||
{ error: err.message, code: err.code },
|
||||
{ status },
|
||||
);
|
||||
}
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Unable to update team member.";
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
export async function DELETE(
|
||||
req: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> },
|
||||
) {
|
||||
const auth = await requireAdminApiSession(req, { minRole: "owner" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
try {
|
||||
const { id } = await params;
|
||||
const deleted = await deleteAdminTeamMemberAsync(id);
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "TEAM_MEMBER_DELETED",
|
||||
target: deleted.email,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
memberId: deleted.id,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
deletedId: deleted.id,
|
||||
email: deleted.email,
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof PrimaryOwnerProtectedError) {
|
||||
return NextResponse.json(
|
||||
{ error: err.message, code: err.code },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
if (err instanceof TeamMemberValidationError) {
|
||||
const status = err.code === "TEAM_MEMBER_NOT_FOUND" ? 404 : 400;
|
||||
return NextResponse.json(
|
||||
{ error: err.message, code: err.code },
|
||||
{ status },
|
||||
);
|
||||
}
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Unable to delete team member.";
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
import {
|
||||
createAdminTeamMemberAsync,
|
||||
listAdminTeamMembersAsync,
|
||||
TeamMemberValidationError,
|
||||
} from "@/core/auth/adminUsers";
|
||||
import { recordAdminAuditLogAsync } from "@/core/auth/auditLog";
|
||||
import { extractClientIp, requireAdminApiSession } from "@/lib/adminAuth";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export async function GET(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req, { minRole: "viewer" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
try {
|
||||
const members = await listAdminTeamMembersAsync();
|
||||
return NextResponse.json({
|
||||
members,
|
||||
currentRole: auth.auth.session.role || "owner",
|
||||
currentEmail: auth.auth.session.email,
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Unable to list team members.";
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
export async function POST(req: NextRequest) {
|
||||
const auth = await requireAdminApiSession(req, { minRole: "owner" });
|
||||
if (!auth.valid) {
|
||||
return auth.response;
|
||||
}
|
||||
|
||||
let body: Record<string, unknown>;
|
||||
try {
|
||||
const parsed = await req.json();
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: "Team member payload must be a JSON object.",
|
||||
code: "VALIDATION_ERROR",
|
||||
},
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
body = parsed as Record<string, unknown>;
|
||||
} catch {
|
||||
return NextResponse.json(
|
||||
{ error: "Invalid JSON request payload.", code: "INVALID_JSON" },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const email = typeof body.email === "string" ? body.email : "";
|
||||
const password = typeof body.password === "string" ? body.password : "";
|
||||
const role = typeof body.role === "string" ? body.role : "viewer";
|
||||
|
||||
const created = await createAdminTeamMemberAsync({
|
||||
email,
|
||||
password,
|
||||
role,
|
||||
invitedBy: auth.auth.session.email,
|
||||
});
|
||||
|
||||
await recordAdminAuditLogAsync({
|
||||
actorEmail: auth.auth.session.email,
|
||||
actorRole: auth.auth.session.role || "owner",
|
||||
action: "TEAM_MEMBER_CREATED",
|
||||
target: created.email,
|
||||
ipAddress: extractClientIp(req),
|
||||
metadata: {
|
||||
memberId: created.id,
|
||||
role: created.role,
|
||||
},
|
||||
});
|
||||
|
||||
return NextResponse.json(
|
||||
{ success: true, member: created },
|
||||
{ status: 201 },
|
||||
);
|
||||
} catch (err: unknown) {
|
||||
if (err instanceof TeamMemberValidationError) {
|
||||
const status = err.code === "TEAM_MEMBER_EXISTS" ? 409 : 400;
|
||||
return NextResponse.json(
|
||||
{ error: err.message, code: err.code },
|
||||
{ status },
|
||||
);
|
||||
}
|
||||
const message =
|
||||
err instanceof Error ? err.message : "Unable to create team member.";
|
||||
return NextResponse.json({ error: message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { AuditLogViewer } from '@/components/sections/AuditLogViewer';
|
||||
|
||||
export default function AuditPage() {
|
||||
return <AuditLogViewer />;
|
||||
}
|
||||
+220
-37
@@ -20,8 +20,11 @@ interface SdkClientItem {
|
||||
token?: string;
|
||||
}
|
||||
|
||||
type SdkLanguageTab = 'python' | 'node' | 'curl';
|
||||
|
||||
export default function SdkGuidePage() {
|
||||
const { success, error } = useToast();
|
||||
const [activeTab, setActiveTab] = useState<SdkLanguageTab>('python');
|
||||
const [clients, setClients] = useState<SdkClientItem[]>(() =>
|
||||
getCached<SdkClientItem[]>('vcon:clients', [])
|
||||
);
|
||||
@@ -128,13 +131,14 @@ export default function SdkGuidePage() {
|
||||
const sampleToken = loadingToken
|
||||
? 'LOADING_CLIENT_JWT_TOKEN...'
|
||||
: resolvedToken || 'YOUR_SIGNED_CLIENT_JWT_TOKEN';
|
||||
const clientIdValue = activeClient?.clientId || '';
|
||||
|
||||
const configSnippet = `# config.datastack.py — Place in your Python project root
|
||||
const pythonConfigSnippet = `# config.datastack.py — Place in your Python project root
|
||||
DATASTACK_CONFIG = {
|
||||
"SERVER_URL": "${serverUrl}",
|
||||
"API_URL": "${apiUrl}",
|
||||
"CLIENT_TOKEN": "${sampleToken}",
|
||||
"CLIENT_ID": "${activeClient?.clientId || ''}",
|
||||
"CLIENT_ID": "${clientIdValue}",
|
||||
"AUTO_CONNECT": True,
|
||||
"ENABLE_HTTP_FALLBACK": True,
|
||||
"HEARTBEAT_INTERVAL_SEC": 5.0,
|
||||
@@ -173,6 +177,109 @@ print("SDK Status:", client.status())
|
||||
|
||||
client.disconnect()`;
|
||||
|
||||
const nodeConfigSnippet = `// datastack.config.json — Place in your Node.js / TypeScript project root
|
||||
{
|
||||
"serverUrl": "${serverUrl}",
|
||||
"apiUrl": "${apiUrl}",
|
||||
"clientToken": "${sampleToken}",
|
||||
"clientId": "${clientIdValue}",
|
||||
"autoConnect": true,
|
||||
"enableHttpFallback": true,
|
||||
"heartbeatIntervalMs": 5000,
|
||||
"maxRetries": 3,
|
||||
"batchChunkSize": 250,
|
||||
"offlineSpoolEnabled": true,
|
||||
"offlineSpoolDir": ".datastack_spool"
|
||||
}`;
|
||||
|
||||
const nodeSnippet = `import { DataStack } from '@datastack/sdk';
|
||||
|
||||
// Loads datastack.config.json or explicit constructor options
|
||||
const client = new DataStack({
|
||||
serverUrl: '${serverUrl}',
|
||||
apiUrl: '${apiUrl}',
|
||||
token: '${sampleToken}',
|
||||
clientId: '${clientIdValue}',
|
||||
autoConnect: true,
|
||||
});
|
||||
|
||||
// 1. Single Event (Socket.IO Primary + Auto HTTP Fallback + Offline Spool)
|
||||
const res = await client.send({
|
||||
event_type: 'checkout_completed',
|
||||
order_id: 'ord_9942',
|
||||
amount_usd: 149.5,
|
||||
});
|
||||
console.log('Event response:', res);
|
||||
|
||||
// 2. Batch Events (Auto-chunked & 413/429 resilient)
|
||||
const batchRes = await client.sendBatch([
|
||||
{ sensor: 'temp', value: 24.5, unit: 'C' },
|
||||
{ sensor: 'humidity', value: 61.2, unit: '%' },
|
||||
]);
|
||||
console.log('Batch response:', batchRes);
|
||||
|
||||
// 3. Direct File Ingestion (.csv, .json, .jsonl, .ndjson with SHA-256 deterministic eventId)
|
||||
const fileRes = await client.sendFile('./events_dataset.csv', { chunkSize: 250 });
|
||||
console.log('File ingestion:', fileRes);
|
||||
|
||||
// 4. Graceful Shutdown Drain (awaits in-flight + drains offline spool)
|
||||
await client.shutdown(5000);`;
|
||||
|
||||
const curlConfigSnippet = `# Environment exports for HTTP / cURL ingestion
|
||||
export DATASTACK_API_URL="${apiUrl}"
|
||||
export DATASTACK_CLIENT_ID="${clientIdValue}"
|
||||
export DATASTACK_CLIENT_TOKEN="${sampleToken}"`;
|
||||
|
||||
const curlSnippet = `# 1. Single Event Ingestion via HTTP POST /api/events
|
||||
curl -X POST "${apiUrl}/api/events" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
-H "Authorization: Bearer ${sampleToken}" \\
|
||||
-d '{
|
||||
"clientId": "${clientIdValue}",
|
||||
"data": {
|
||||
"event_type": "deployment_finished",
|
||||
"service": "billing-api",
|
||||
"status": "healthy"
|
||||
}
|
||||
}'
|
||||
|
||||
# 2. Batch Event Ingestion via HTTP POST /api/events
|
||||
curl -X POST "${apiUrl}/api/events" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
-H "Authorization: Bearer ${sampleToken}" \\
|
||||
-d '{
|
||||
"clientId": "${clientIdValue}",
|
||||
"batch": [
|
||||
{ "data": { "metric": "cpu_pct", "value": 31.4 } },
|
||||
{ "data": { "metric": "mem_pct", "value": 58.2 } }
|
||||
]
|
||||
}'`;
|
||||
|
||||
const activeConfigSnippet =
|
||||
activeTab === 'node'
|
||||
? nodeConfigSnippet
|
||||
: activeTab === 'curl'
|
||||
? curlConfigSnippet
|
||||
: pythonConfigSnippet;
|
||||
const activeCodeSnippet =
|
||||
activeTab === 'node'
|
||||
? nodeSnippet
|
||||
: activeTab === 'curl'
|
||||
? curlSnippet
|
||||
: pythonSnippet;
|
||||
const activeConfigLabel =
|
||||
activeTab === 'node'
|
||||
? 'datastack.config.json'
|
||||
: activeTab === 'curl'
|
||||
? 'Environment Variables'
|
||||
: 'config.datastack.py';
|
||||
const activeCodeLabel =
|
||||
activeTab === 'node'
|
||||
? 'Node.js / TypeScript SDK snippet'
|
||||
: activeTab === 'curl'
|
||||
? 'cURL REST snippet'
|
||||
: 'Python SDK snippet';
|
||||
|
||||
const copySnippet = async (text: string, label: string) => {
|
||||
try {
|
||||
if (!navigator?.clipboard?.writeText) {
|
||||
@@ -192,7 +299,7 @@ client.disconnect()`;
|
||||
|
||||
return (
|
||||
<PageShell
|
||||
title="Python SDK"
|
||||
title="Client SDKs (Python & Node.js / TypeScript)"
|
||||
actions={
|
||||
<>
|
||||
{resolvedToken && (
|
||||
@@ -211,7 +318,7 @@ client.disconnect()`;
|
||||
variant="secondary"
|
||||
icon="FileCode"
|
||||
disabled={loadingToken}
|
||||
onClick={() => void copySnippet(configSnippet, 'config.datastack.py')}
|
||||
onClick={() => void copySnippet(activeConfigSnippet, activeConfigLabel)}
|
||||
>
|
||||
Copy Config
|
||||
</Button>
|
||||
@@ -219,7 +326,7 @@ client.disconnect()`;
|
||||
size="xs"
|
||||
variant="secondary"
|
||||
icon="Copy"
|
||||
onClick={() => void copySnippet(pythonSnippet, 'Python SDK snippet')}
|
||||
onClick={() => void copySnippet(activeCodeSnippet, activeCodeLabel)}
|
||||
>
|
||||
Copy Code
|
||||
</Button>
|
||||
@@ -228,7 +335,16 @@ client.disconnect()`;
|
||||
>
|
||||
<StatStrip
|
||||
items={[
|
||||
{ label: 'SDK package', value: 'datastack (v2.2.0)', dot: 'ok' },
|
||||
{
|
||||
label: 'SDK packages',
|
||||
value:
|
||||
activeTab === 'node'
|
||||
? '@datastack/sdk (v2.2.0)'
|
||||
: activeTab === 'curl'
|
||||
? 'REST HTTP/1.1 & HTTP/2'
|
||||
: 'datastack (v2.2.0)',
|
||||
dot: 'ok',
|
||||
},
|
||||
{ label: 'Transport', value: 'WS + HTTP Fallback + Offline Spool' },
|
||||
{ label: 'Server endpoint', value: serverUrl },
|
||||
{
|
||||
@@ -243,36 +359,103 @@ client.disconnect()`;
|
||||
]}
|
||||
/>
|
||||
|
||||
<Card
|
||||
compact
|
||||
className="flex w-full flex-col gap-1.5 bg-[var(--surface)] px-3 py-2.5"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-2 font-mono text-[11.5px]">
|
||||
<span className="truncate text-[var(--accent)]">
|
||||
pip install /path/to/datastack/sdk/python/datastack
|
||||
</span>
|
||||
<div className="mb-3 flex flex-wrap items-center justify-between gap-2 border border-[var(--border)] bg-[var(--surface)] px-3 py-2">
|
||||
<div className="flex items-center gap-1.5" role="tablist" aria-label="SDK Language Selector">
|
||||
<Button
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
icon="Copy"
|
||||
aria-label="Copy pip install command"
|
||||
onClick={() =>
|
||||
void copySnippet(
|
||||
'pip install /path/to/datastack/sdk/python/datastack',
|
||||
'Pip command'
|
||||
)
|
||||
}
|
||||
/>
|
||||
variant={activeTab === 'python' ? 'primary' : 'secondary'}
|
||||
onClick={() => setActiveTab('python')}
|
||||
>
|
||||
Python SDK (v2.2.0)
|
||||
</Button>
|
||||
<Button
|
||||
size="xs"
|
||||
variant={activeTab === 'node' ? 'primary' : 'secondary'}
|
||||
onClick={() => setActiveTab('node')}
|
||||
>
|
||||
Node.js / TypeScript SDK (v2.2.0)
|
||||
</Button>
|
||||
<Button
|
||||
size="xs"
|
||||
variant={activeTab === 'curl' ? 'primary' : 'secondary'}
|
||||
onClick={() => setActiveTab('curl')}
|
||||
>
|
||||
HTTP / cURL
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-[11px] text-[var(--text-faint)]">
|
||||
Self-hosted package bundled in <code className="font-mono text-[var(--text-muted)]">sdk/python/datastack</code>. From inside the repo run{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">pip install ./sdk/python/datastack</code>, or build a portable wheel with{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">pip wheel ./sdk/python/datastack -w dist/</code> for external projects.
|
||||
</p>
|
||||
</Card>
|
||||
<span className="font-mono text-[10.5px] text-[var(--text-faint)]">
|
||||
{activeTab === 'node'
|
||||
? 'sdk/node/datastack'
|
||||
: activeTab === 'curl'
|
||||
? 'POST /api/events'
|
||||
: 'sdk/python/datastack'}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{activeTab === 'python' && (
|
||||
<Card
|
||||
compact
|
||||
className="flex w-full flex-col gap-1.5 bg-[var(--surface)] px-3 py-2.5"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-2 font-mono text-[11.5px]">
|
||||
<span className="truncate text-[var(--accent)]">
|
||||
pip install /path/to/datastack/sdk/python/datastack
|
||||
</span>
|
||||
<Button
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
icon="Copy"
|
||||
aria-label="Copy pip install command"
|
||||
onClick={() =>
|
||||
void copySnippet(
|
||||
'pip install /path/to/datastack/sdk/python/datastack',
|
||||
'Pip command'
|
||||
)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<p className="text-[11px] text-[var(--text-faint)]">
|
||||
Self-hosted package bundled in <code className="font-mono text-[var(--text-muted)]">sdk/python/datastack</code>. From inside the repo run{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">pip install ./sdk/python/datastack</code>, or build a portable wheel with{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">pip wheel ./sdk/python/datastack -w dist/</code> for external projects.
|
||||
</p>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{activeTab === 'node' && (
|
||||
<Card
|
||||
compact
|
||||
className="flex w-full flex-col gap-1.5 bg-[var(--surface)] px-3 py-2.5"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-2 font-mono text-[11.5px]">
|
||||
<span className="truncate text-[var(--accent)]">
|
||||
npm install /path/to/datastack/sdk/node/datastack
|
||||
</span>
|
||||
<Button
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
icon="Copy"
|
||||
aria-label="Copy npm install command"
|
||||
onClick={() =>
|
||||
void copySnippet(
|
||||
'npm install /path/to/datastack/sdk/node/datastack',
|
||||
'NPM command'
|
||||
)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<p className="text-[11px] text-[var(--text-faint)]">
|
||||
Official Node.js & TypeScript package bundled in{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">sdk/node/datastack</code> (Node >= 20). Supports Socket.IO primary streaming, automatic{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">POST /api/events</code> fallback, deterministic{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">sendFile()</code> SHA-256 deduplication, offline disk spooling, and graceful{' '}
|
||||
<code className="font-mono text-[var(--text-muted)]">await client.shutdown()</code>.
|
||||
</p>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
<Panel
|
||||
title="Step 1 — config.datastack.py"
|
||||
title={`Step 1 — ${activeConfigLabel}`}
|
||||
className="w-full"
|
||||
action={
|
||||
<div className="flex items-center gap-2">
|
||||
@@ -296,14 +479,14 @@ client.disconnect()`;
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
icon="Copy"
|
||||
aria-label="Copy config.datastack.py"
|
||||
aria-label={`Copy ${activeConfigLabel}`}
|
||||
disabled={loadingToken}
|
||||
onClick={() => void copySnippet(configSnippet, 'config.datastack.py')}
|
||||
onClick={() => void copySnippet(activeConfigSnippet, activeConfigLabel)}
|
||||
/>
|
||||
</div>
|
||||
}
|
||||
>
|
||||
<CodeBlock>{configSnippet}</CodeBlock>
|
||||
<CodeBlock>{activeConfigSnippet}</CodeBlock>
|
||||
</Panel>
|
||||
|
||||
<Panel
|
||||
@@ -314,12 +497,12 @@ client.disconnect()`;
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
icon="Copy"
|
||||
aria-label="Copy Python SDK snippet"
|
||||
onClick={() => void copySnippet(pythonSnippet, 'Python SDK snippet')}
|
||||
aria-label={`Copy ${activeCodeLabel}`}
|
||||
onClick={() => void copySnippet(activeCodeSnippet, activeCodeLabel)}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<CodeBlock>{pythonSnippet}</CodeBlock>
|
||||
<CodeBlock>{activeCodeSnippet}</CodeBlock>
|
||||
</Panel>
|
||||
</PageShell>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
import { TeamWorkspace } from '@/components/sections/TeamWorkspace';
|
||||
|
||||
export default function TeamPage() {
|
||||
return <TeamWorkspace />;
|
||||
}
|
||||
@@ -34,6 +34,8 @@ const NAVIGATION_SECTIONS = [
|
||||
items: [
|
||||
{ name: 'Python SDK', href: '/vcon/sdk', icon: 'FileCode2' },
|
||||
{ name: 'Simulator', href: '/vcon/simulator', icon: 'Play' },
|
||||
{ name: 'Team & RBAC', href: '/vcon/team', icon: 'Users' },
|
||||
{ name: 'Audit Log', href: '/vcon/audit', icon: 'FileText' },
|
||||
{ name: 'Settings', href: '/vcon/settings', icon: 'Sliders' },
|
||||
{ name: 'Profile', href: '/vcon/profile', icon: 'User' },
|
||||
],
|
||||
|
||||
@@ -0,0 +1,236 @@
|
||||
'use client';
|
||||
|
||||
import { PageShell } from '@/components/layout/PageShell';
|
||||
import { Button } from '@/components/ui/Button';
|
||||
import { Input, Select } from '@/components/ui/Input';
|
||||
import { Panel } from '@/components/ui/Panel';
|
||||
import { StatStrip } from '@/components/ui/StatStrip';
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableEmpty,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
TableWrap,
|
||||
} from '@/components/ui/Table';
|
||||
import { useToast } from '@/components/ui/Toast';
|
||||
import Link from 'next/link';
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
|
||||
interface AuditLogItem {
|
||||
id: string;
|
||||
actorEmail: string;
|
||||
actorRole: string;
|
||||
action: string;
|
||||
target: string;
|
||||
ipAddress: string;
|
||||
metadata: Record<string, unknown>;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
const ACTION_FILTER_OPTIONS = [
|
||||
{ value: 'ALL', label: 'All Governance Actions' },
|
||||
{ value: 'ADMIN_LOGIN', label: 'ADMIN_LOGIN' },
|
||||
{ value: 'ADMIN_PASSWORD_UPDATED', label: 'ADMIN_PASSWORD_UPDATED' },
|
||||
{ value: 'TEAM_MEMBER_CREATED', label: 'TEAM_MEMBER_CREATED' },
|
||||
{ value: 'TEAM_MEMBER_UPDATED', label: 'TEAM_MEMBER_UPDATED' },
|
||||
{ value: 'TEAM_MEMBER_REVOKED', label: 'TEAM_MEMBER_REVOKED' },
|
||||
{ value: 'TEAM_MEMBER_DELETED', label: 'TEAM_MEMBER_DELETED' },
|
||||
{ value: 'CLIENT_CREATED', label: 'CLIENT_CREATED' },
|
||||
{ value: 'CLIENT_UPDATED', label: 'CLIENT_UPDATED' },
|
||||
{ value: 'CLIENT_TOKEN_ROTATED', label: 'CLIENT_TOKEN_ROTATED' },
|
||||
{ value: 'CLIENT_DELETED', label: 'CLIENT_DELETED' },
|
||||
{ value: 'SETTINGS_UPDATED', label: 'SETTINGS_UPDATED' },
|
||||
{ value: 'S3_ARCHIVE_TRIGGERED', label: 'S3_ARCHIVE_TRIGGERED' },
|
||||
{ value: 'WEBHOOK_TEST_TRIGGERED', label: 'WEBHOOK_TEST_TRIGGERED' },
|
||||
];
|
||||
|
||||
export function AuditLogViewer() {
|
||||
const { error } = useToast();
|
||||
const [logs, setLogs] = useState<AuditLogItem[]>([]);
|
||||
const [loading, setLoading] = useState<boolean>(true);
|
||||
const [actionFilter, setActionFilter] = useState<string>('ALL');
|
||||
const [searchQuery, setSearchQuery] = useState<string>('');
|
||||
|
||||
const fetchAuditLogs = useCallback(async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
const params = new URLSearchParams();
|
||||
params.set('limit', '150');
|
||||
if (actionFilter && actionFilter !== 'ALL') {
|
||||
params.set('action', actionFilter);
|
||||
}
|
||||
if (searchQuery.trim()) {
|
||||
params.set('q', searchQuery.trim());
|
||||
}
|
||||
const res = await fetch(`/api/audit?${params.toString()}`, {
|
||||
credentials: 'include',
|
||||
cache: 'no-store',
|
||||
});
|
||||
if (res.status === 401) {
|
||||
window.location.assign('/vcon/login');
|
||||
return;
|
||||
}
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Failed to load audit logs.');
|
||||
}
|
||||
setLogs(Array.isArray(data.logs) ? data.logs : []);
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to load audit logs.');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, [actionFilter, searchQuery, error]);
|
||||
|
||||
useEffect(() => {
|
||||
void fetchAuditLogs();
|
||||
}, [fetchAuditLogs]);
|
||||
|
||||
const uniqueActors = new Set(logs.map((l) => l.actorEmail)).size;
|
||||
const mutationCount = logs.filter((l) => l.action !== 'ADMIN_LOGIN').length;
|
||||
|
||||
return (
|
||||
<PageShell
|
||||
title="Admin Audit Log"
|
||||
actions={
|
||||
<div className="flex items-center gap-1.5">
|
||||
<Link href="/vcon/team">
|
||||
<Button variant="secondary" size="sm" icon="Users">
|
||||
Team & RBAC
|
||||
</Button>
|
||||
</Link>
|
||||
<Button
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
icon="RefreshCw"
|
||||
onClick={() => void fetchAuditLogs()}
|
||||
loading={loading}
|
||||
>
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-3">
|
||||
<StatStrip
|
||||
items={[
|
||||
{
|
||||
label: 'Recorded Events',
|
||||
value: String(logs.length),
|
||||
dot: 'ok',
|
||||
},
|
||||
{
|
||||
label: 'Distinct Actors',
|
||||
value: String(uniqueActors),
|
||||
},
|
||||
{
|
||||
label: 'Config & Client Mutations',
|
||||
value: String(mutationCount),
|
||||
},
|
||||
{
|
||||
label: 'Telemetry Separation',
|
||||
value: 'Strict (0 CSV Rows)',
|
||||
dot: 'ok',
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<Panel
|
||||
title="Governance Event Trail"
|
||||
meta={`${logs.length} ENTRIES`}
|
||||
>
|
||||
<div className="p-3 pb-2 grid grid-cols-1 gap-2 sm:grid-cols-3">
|
||||
<Select
|
||||
label="Filter by Action"
|
||||
value={actionFilter}
|
||||
onChange={(e) => setActionFilter(e.target.value)}
|
||||
options={ACTION_FILTER_OPTIONS}
|
||||
/>
|
||||
<div className="sm:col-span-2">
|
||||
<Input
|
||||
label="Search Actor, Target, or IP"
|
||||
placeholder="Search email, client_id, action, or IP..."
|
||||
value={searchQuery}
|
||||
onChange={(e) => setSearchQuery(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<TableWrap>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Timestamp (UTC)</TableHead>
|
||||
<TableHead>Actor</TableHead>
|
||||
<TableHead>Action</TableHead>
|
||||
<TableHead>Target Resource</TableHead>
|
||||
<TableHead>Source IP</TableHead>
|
||||
<TableHead>Metadata</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{logs.length === 0 ? (
|
||||
<TableEmpty
|
||||
colSpan={6}
|
||||
message={
|
||||
loading
|
||||
? 'Loading audit trail...'
|
||||
: 'No audit log entries match the current filters.'
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
logs.map((row) => (
|
||||
<TableRow key={row.id}>
|
||||
<TableCell>
|
||||
<span className="font-mono text-[11px] text-[var(--text-muted)]">
|
||||
{row.createdAt
|
||||
? row.createdAt.replace('T', ' ').slice(0, 19)
|
||||
: '—'}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<span className="font-mono text-[11.5px] text-[var(--text)]">
|
||||
{row.actorEmail}
|
||||
</span>
|
||||
<span className="border border-[var(--border)] bg-[var(--surface-2)] px-1 py-0.2 text-[9.5px] uppercase text-[var(--text-muted)]">
|
||||
{row.actorRole}
|
||||
</span>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="border border-[var(--border-strong)] bg-[var(--surface-2)] px-1.5 py-0.5 font-mono text-[10.5px] font-medium text-[var(--accent)]">
|
||||
{row.action}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="font-mono text-[11px] text-[var(--text)]">
|
||||
{row.target}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="font-mono text-[11px] text-[var(--text-muted)]">
|
||||
{row.ipAddress || 'unknown'}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="block max-w-[260px] truncate font-mono text-[10.5px] text-[var(--text-faint)]">
|
||||
{JSON.stringify(row.metadata || {})}
|
||||
</span>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
))
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</TableWrap>
|
||||
</Panel>
|
||||
</div>
|
||||
</PageShell>
|
||||
);
|
||||
}
|
||||
|
||||
export default AuditLogViewer;
|
||||
@@ -0,0 +1,447 @@
|
||||
'use client';
|
||||
|
||||
import { PageShell } from '@/components/layout/PageShell';
|
||||
import { Button } from '@/components/ui/Button';
|
||||
import { Input, Select } from '@/components/ui/Input';
|
||||
import { Modal } from '@/components/ui/Modal';
|
||||
import { Panel } from '@/components/ui/Panel';
|
||||
import { StatStrip } from '@/components/ui/StatStrip';
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableEmpty,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
TableWrap,
|
||||
} from '@/components/ui/Table';
|
||||
import { useToast } from '@/components/ui/Toast';
|
||||
import Link from 'next/link';
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
|
||||
interface TeamMemberItem {
|
||||
id: string;
|
||||
email: string;
|
||||
role: 'owner' | 'admin' | 'operator' | 'viewer';
|
||||
status: 'active' | 'disabled';
|
||||
tokenVersion: number;
|
||||
invitedBy: string;
|
||||
isPrimaryOwner: boolean;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export function TeamWorkspace() {
|
||||
const { success, error } = useToast();
|
||||
const [members, setMembers] = useState<TeamMemberItem[]>([]);
|
||||
const [currentRole, setCurrentRole] = useState<string>('owner');
|
||||
const [loading, setLoading] = useState<boolean>(true);
|
||||
|
||||
const [createOpen, setCreateOpen] = useState<boolean>(false);
|
||||
const [newEmail, setNewEmail] = useState<string>('');
|
||||
const [newPassword, setNewPassword] = useState<string>('');
|
||||
const [newRole, setNewRole] = useState<string>('viewer');
|
||||
const [creating, setCreating] = useState<boolean>(false);
|
||||
|
||||
const [busyId, setBusyId] = useState<string | null>(null);
|
||||
|
||||
const fetchMembers = useCallback(async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
const res = await fetch('/api/team', {
|
||||
credentials: 'include',
|
||||
cache: 'no-store',
|
||||
});
|
||||
if (res.status === 401) {
|
||||
window.location.assign('/vcon/login');
|
||||
return;
|
||||
}
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Failed to load team members.');
|
||||
}
|
||||
setMembers(Array.isArray(data.members) ? data.members : []);
|
||||
if (data.currentRole) {
|
||||
setCurrentRole(data.currentRole);
|
||||
}
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to load team members.');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, [error]);
|
||||
|
||||
useEffect(() => {
|
||||
void fetchMembers();
|
||||
}, [fetchMembers]);
|
||||
|
||||
const isOwner = currentRole === 'owner';
|
||||
|
||||
const handleCreateMember = async (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
if (!newEmail.trim() || !newPassword) {
|
||||
error('Email and password are required.');
|
||||
return;
|
||||
}
|
||||
setCreating(true);
|
||||
try {
|
||||
const res = await fetch('/api/team', {
|
||||
method: 'POST',
|
||||
credentials: 'include',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
email: newEmail.trim(),
|
||||
password: newPassword,
|
||||
role: newRole,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Failed to create team member.');
|
||||
}
|
||||
success(`Added ${data.member?.email || newEmail} (${newRole}).`);
|
||||
setNewEmail('');
|
||||
setNewPassword('');
|
||||
setNewRole('viewer');
|
||||
setCreateOpen(false);
|
||||
await fetchMembers();
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to create team member.');
|
||||
} finally {
|
||||
setCreating(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleRoleChange = async (member: TeamMemberItem, nextRole: string) => {
|
||||
if (member.isPrimaryOwner) {
|
||||
error('Primary Owner role cannot be modified.');
|
||||
return;
|
||||
}
|
||||
setBusyId(member.id);
|
||||
try {
|
||||
const res = await fetch(`/api/team/${encodeURIComponent(member.id)}`, {
|
||||
method: 'PATCH',
|
||||
credentials: 'include',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ role: nextRole }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Role update failed.');
|
||||
}
|
||||
success(`Updated ${member.email} role to ${nextRole}.`);
|
||||
await fetchMembers();
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to update role.');
|
||||
} finally {
|
||||
setBusyId(null);
|
||||
}
|
||||
};
|
||||
|
||||
const handleToggleStatus = async (member: TeamMemberItem) => {
|
||||
if (member.isPrimaryOwner) {
|
||||
error('Primary Owner cannot be disabled.');
|
||||
return;
|
||||
}
|
||||
const nextStatus = member.status === 'active' ? 'disabled' : 'active';
|
||||
setBusyId(member.id);
|
||||
try {
|
||||
const res = await fetch(`/api/team/${encodeURIComponent(member.id)}`, {
|
||||
method: 'PATCH',
|
||||
credentials: 'include',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ status: nextStatus, revokeSessions: true }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Status update failed.');
|
||||
}
|
||||
success(
|
||||
nextStatus === 'disabled'
|
||||
? `Revoked access and active sessions for ${member.email}.`
|
||||
: `Re-enabled access for ${member.email}.`
|
||||
);
|
||||
await fetchMembers();
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to update member status.');
|
||||
} finally {
|
||||
setBusyId(null);
|
||||
}
|
||||
};
|
||||
|
||||
const handleDeleteMember = async (member: TeamMemberItem) => {
|
||||
if (member.isPrimaryOwner) {
|
||||
error('Primary Owner cannot be deleted.');
|
||||
return;
|
||||
}
|
||||
setBusyId(member.id);
|
||||
try {
|
||||
const res = await fetch(`/api/team/${encodeURIComponent(member.id)}`, {
|
||||
method: 'DELETE',
|
||||
credentials: 'include',
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!res.ok) {
|
||||
throw new Error(data?.error || 'Delete failed.');
|
||||
}
|
||||
success(`Removed ${member.email} from team.`);
|
||||
await fetchMembers();
|
||||
} catch (err: unknown) {
|
||||
error(err instanceof Error ? err.message : 'Unable to delete team member.');
|
||||
} finally {
|
||||
setBusyId(null);
|
||||
}
|
||||
};
|
||||
|
||||
const activeCount = members.filter((m) => m.status === 'active').length;
|
||||
const ownerCount = members.filter((m) => m.role === 'owner').length;
|
||||
const operatorCount = members.filter(
|
||||
(m) => m.role === 'admin' || m.role === 'operator'
|
||||
).length;
|
||||
const viewerCount = members.filter((m) => m.role === 'viewer').length;
|
||||
|
||||
return (
|
||||
<PageShell
|
||||
title="Team & RBAC"
|
||||
actions={
|
||||
<div className="flex items-center gap-1.5">
|
||||
<Link href="/vcon/audit">
|
||||
<Button variant="secondary" size="sm" icon="FileText">
|
||||
Audit Log
|
||||
</Button>
|
||||
</Link>
|
||||
<Button
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
icon="RefreshCw"
|
||||
onClick={() => void fetchMembers()}
|
||||
loading={loading}
|
||||
>
|
||||
Refresh
|
||||
</Button>
|
||||
{isOwner && (
|
||||
<Button
|
||||
variant="primary"
|
||||
size="sm"
|
||||
icon="UserPlus"
|
||||
onClick={() => setCreateOpen(true)}
|
||||
>
|
||||
Add Team Member
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-3">
|
||||
<StatStrip
|
||||
items={[
|
||||
{
|
||||
label: 'Total Members',
|
||||
value: `${members.length} (${activeCount} active)`,
|
||||
dot: 'ok',
|
||||
},
|
||||
{
|
||||
label: 'Owners',
|
||||
value: String(ownerCount),
|
||||
},
|
||||
{
|
||||
label: 'Admins / Operators',
|
||||
value: String(operatorCount),
|
||||
},
|
||||
{
|
||||
label: 'Viewers',
|
||||
value: String(viewerCount),
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<Panel
|
||||
title="Administrator Directory"
|
||||
meta={`ROLE: ${currentRole.toUpperCase()}`}
|
||||
>
|
||||
<TableWrap>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Email</TableHead>
|
||||
<TableHead>Role</TableHead>
|
||||
<TableHead>Status</TableHead>
|
||||
<TableHead>Token Version</TableHead>
|
||||
<TableHead>Created</TableHead>
|
||||
<TableHead>Actions</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{members.length === 0 ? (
|
||||
<TableEmpty
|
||||
colSpan={6}
|
||||
message={
|
||||
loading
|
||||
? 'Loading team members...'
|
||||
: 'No team members registered.'
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
members.map((row) => (
|
||||
<TableRow key={row.id}>
|
||||
<TableCell>
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="font-mono text-[11.5px] font-medium text-[var(--text)]">
|
||||
{row.email}
|
||||
</span>
|
||||
{row.isPrimaryOwner && (
|
||||
<span className="border border-[var(--accent)] bg-[var(--surface-2)] px-1.5 py-0.5 text-[9.5px] uppercase tracking-wider text-[var(--accent)]">
|
||||
Primary Owner
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
{row.isPrimaryOwner || !isOwner ? (
|
||||
<span className="border border-[var(--border)] bg-[var(--surface-2)] px-1.5 py-0.5 font-mono text-[10.5px] uppercase text-[var(--text)]">
|
||||
{row.role}
|
||||
</span>
|
||||
) : (
|
||||
<Select
|
||||
compact
|
||||
aria-label={`Role for ${row.email}`}
|
||||
value={row.role === 'operator' ? 'admin' : row.role}
|
||||
disabled={busyId === row.id}
|
||||
onChange={(e) =>
|
||||
void handleRoleChange(row, e.target.value)
|
||||
}
|
||||
options={[
|
||||
{ value: 'viewer', label: 'Viewer (Read-Only)' },
|
||||
{ value: 'admin', label: 'Admin / Operator' },
|
||||
{ value: 'owner', label: 'Owner (Full Control)' },
|
||||
]}
|
||||
/>
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span
|
||||
className={
|
||||
row.status === 'active'
|
||||
? 'border border-[var(--success)]/40 bg-[var(--success)]/10 px-1.5 py-0.5 text-[10px] uppercase text-[var(--success)]'
|
||||
: 'border border-[var(--danger)]/40 bg-[var(--danger)]/10 px-1.5 py-0.5 text-[10px] uppercase text-[var(--danger)]'
|
||||
}
|
||||
>
|
||||
{row.status}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="font-mono text-[11px] text-[var(--text-muted)]">
|
||||
v{row.tokenVersion}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<span className="font-mono text-[11px] text-[var(--text-muted)]">
|
||||
{row.createdAt ? row.createdAt.slice(0, 10) : '—'}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
{row.isPrimaryOwner ? (
|
||||
<span className="text-[11px] text-[var(--text-faint)]">
|
||||
Immutable (id=1)
|
||||
</span>
|
||||
) : isOwner ? (
|
||||
<div className="flex items-center gap-1.5">
|
||||
<Button
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
disabled={busyId === row.id}
|
||||
onClick={() => void handleToggleStatus(row)}
|
||||
>
|
||||
{row.status === 'active' ? 'Revoke' : 'Activate'}
|
||||
</Button>
|
||||
<Button
|
||||
variant="danger"
|
||||
size="sm"
|
||||
disabled={busyId === row.id}
|
||||
onClick={() => void handleDeleteMember(row)}
|
||||
>
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
<span className="text-[11px] text-[var(--text-faint)]">
|
||||
Owner required
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
))
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</TableWrap>
|
||||
</Panel>
|
||||
</div>
|
||||
|
||||
<Modal
|
||||
isOpen={createOpen}
|
||||
onClose={() => setCreateOpen(false)}
|
||||
title="Add Administrator / Team Member"
|
||||
>
|
||||
<form onSubmit={handleCreateMember} className="flex flex-col gap-3">
|
||||
<Input
|
||||
label="Email Address"
|
||||
type="email"
|
||||
placeholder="operator@datastack.dev"
|
||||
value={newEmail}
|
||||
onChange={(e) => setNewEmail(e.target.value)}
|
||||
required
|
||||
/>
|
||||
<Input
|
||||
label="Initial Password (min 8 chars)"
|
||||
type="password"
|
||||
placeholder="StrongPassword123!"
|
||||
value={newPassword}
|
||||
onChange={(e) => setNewPassword(e.target.value)}
|
||||
required
|
||||
/>
|
||||
<Select
|
||||
label="Assigned RBAC Role"
|
||||
value={newRole}
|
||||
onChange={(e) => setNewRole(e.target.value)}
|
||||
options={[
|
||||
{
|
||||
value: 'viewer',
|
||||
label: 'Viewer — Read-only dashboards, events, clients & exports',
|
||||
},
|
||||
{
|
||||
value: 'admin',
|
||||
label: 'Admin / Operator — Manage SDK clients, tokens & flushes',
|
||||
},
|
||||
{
|
||||
value: 'owner',
|
||||
label: 'Owner — Full platform settings, secrets & team control',
|
||||
},
|
||||
]}
|
||||
/>
|
||||
<div className="flex justify-end gap-2 pt-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
onClick={() => setCreateOpen(false)}
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="submit"
|
||||
variant="primary"
|
||||
size="sm"
|
||||
loading={creating}
|
||||
>
|
||||
Create Member
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
</Modal>
|
||||
</PageShell>
|
||||
);
|
||||
}
|
||||
|
||||
export default TeamWorkspace;
|
||||
@@ -83,6 +83,32 @@ export const CORE_SCHEMA_TABLE_DDLS = [
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS admin_team_members (
|
||||
id TEXT PRIMARY KEY,
|
||||
email TEXT NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'viewer',
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
password_salt BYTEA NOT NULL,
|
||||
password_hash BYTEA NOT NULL,
|
||||
token_version INTEGER NOT NULL DEFAULT 1,
|
||||
invited_by TEXT NOT NULL DEFAULT 'owner',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
)
|
||||
`,
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS admin_audit_logs (
|
||||
id TEXT PRIMARY KEY,
|
||||
actor_email TEXT NOT NULL,
|
||||
actor_role TEXT NOT NULL,
|
||||
action TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
ip_address TEXT NOT NULL DEFAULT '',
|
||||
metadata_json TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
)
|
||||
`,
|
||||
] as const;
|
||||
|
||||
export const CORE_SCHEMA_ALTER_DDLS = [
|
||||
|
||||
@@ -3,10 +3,42 @@ const encoder = new TextEncoder();
|
||||
const DEV_DEFAULT_SESSION_SECRET =
|
||||
"datastack-admin-session-default-secret-32-chars";
|
||||
|
||||
export type AdminRole = "owner" | "admin" | "operator" | "viewer";
|
||||
|
||||
export interface AdminSessionPayload {
|
||||
email: string;
|
||||
expiresAt: number;
|
||||
tokenVersion: number;
|
||||
role?: AdminRole;
|
||||
memberId?: string;
|
||||
}
|
||||
|
||||
export function normalizeAdminRole(
|
||||
raw: unknown,
|
||||
fallback: AdminRole = "viewer",
|
||||
): AdminRole {
|
||||
if (typeof raw !== "string") return fallback;
|
||||
const clean = raw.trim().toLowerCase();
|
||||
if (clean === "owner") return "owner";
|
||||
if (clean === "admin") return "admin";
|
||||
if (clean === "operator") return "operator";
|
||||
if (clean === "viewer") return "viewer";
|
||||
return fallback;
|
||||
}
|
||||
|
||||
export function getAdminRoleRank(role?: string): number {
|
||||
const normalized = normalizeAdminRole(role, "owner");
|
||||
if (normalized === "owner") return 3;
|
||||
if (normalized === "admin" || normalized === "operator") return 2;
|
||||
return 1;
|
||||
}
|
||||
|
||||
export function formatAdminRoleLabel(role?: string): string {
|
||||
const normalized = normalizeAdminRole(role, "owner");
|
||||
if (normalized === "owner") return "Owner";
|
||||
if (normalized === "admin") return "Admin";
|
||||
if (normalized === "operator") return "Operator";
|
||||
return "Viewer";
|
||||
}
|
||||
|
||||
export function getAdminSessionSecret(): string | null {
|
||||
@@ -50,14 +82,19 @@ export async function createAdminSessionToken(
|
||||
secret: string,
|
||||
expiresAt: number,
|
||||
tokenVersion = 1,
|
||||
role?: AdminRole,
|
||||
memberId?: string,
|
||||
) {
|
||||
const safeVersion =
|
||||
Number.isInteger(tokenVersion) && tokenVersion >= 1 ? tokenVersion : 1;
|
||||
const payload = encodeBase64Url(
|
||||
encoder.encode(
|
||||
JSON.stringify({ email, expiresAt, tokenVersion: safeVersion }),
|
||||
),
|
||||
);
|
||||
const payloadObj: AdminSessionPayload = {
|
||||
email,
|
||||
expiresAt,
|
||||
tokenVersion: safeVersion,
|
||||
...(role ? { role: normalizeAdminRole(role, "owner") } : {}),
|
||||
...(memberId ? { memberId } : {}),
|
||||
};
|
||||
const payload = encodeBase64Url(encoder.encode(JSON.stringify(payloadObj)));
|
||||
const signature = await crypto.subtle.sign(
|
||||
"HMAC",
|
||||
await importKey(secret),
|
||||
@@ -94,7 +131,12 @@ export async function getAdminSessionPayload(
|
||||
Number.isInteger(session.tokenVersion) &&
|
||||
session.tokenVersion >= 1
|
||||
) {
|
||||
return session;
|
||||
return {
|
||||
...session,
|
||||
...(session.role
|
||||
? { role: normalizeAdminRole(session.role, "owner") }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
return null;
|
||||
} catch {
|
||||
|
||||
+901
-39
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,281 @@
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { DatabaseEngine } from "@/core/Database/database";
|
||||
|
||||
export type AuditActionType =
|
||||
| "ADMIN_LOGIN"
|
||||
| "ADMIN_PROFILE_UPDATED"
|
||||
| "ADMIN_PASSWORD_UPDATED"
|
||||
| "TEAM_MEMBER_CREATED"
|
||||
| "TEAM_MEMBER_UPDATED"
|
||||
| "TEAM_MEMBER_REVOKED"
|
||||
| "TEAM_MEMBER_DELETED"
|
||||
| "CLIENT_CREATED"
|
||||
| "CLIENT_UPDATED"
|
||||
| "CLIENT_TOKEN_ROTATED"
|
||||
| "CLIENT_DELETED"
|
||||
| "SETTINGS_UPDATED"
|
||||
| "S3_ARCHIVE_TRIGGERED"
|
||||
| "WEBHOOK_TEST_TRIGGERED"
|
||||
| string;
|
||||
|
||||
export interface AuditLogEntry {
|
||||
id: string;
|
||||
actorEmail: string;
|
||||
actorRole: string;
|
||||
action: AuditActionType;
|
||||
target: string;
|
||||
ipAddress: string;
|
||||
metadata: Record<string, unknown>;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
export interface RecordAuditLogInput {
|
||||
actorEmail: string;
|
||||
actorRole: string;
|
||||
action: AuditActionType;
|
||||
target: string;
|
||||
ipAddress?: string;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface ListAuditLogsQuery {
|
||||
limit?: number;
|
||||
action?: string;
|
||||
actorEmail?: string;
|
||||
search?: string;
|
||||
}
|
||||
|
||||
const MAX_IN_MEMORY_AUDIT_LOGS = 1000;
|
||||
|
||||
type AuditLogGlobals = typeof globalThis & {
|
||||
__dataStackAuditLogs?: AuditLogEntry[];
|
||||
};
|
||||
|
||||
const globals = globalThis as AuditLogGlobals;
|
||||
|
||||
function getInMemoryAuditLogs(): AuditLogEntry[] {
|
||||
if (!Array.isArray(globals.__dataStackAuditLogs)) {
|
||||
globals.__dataStackAuditLogs = [];
|
||||
}
|
||||
return globals.__dataStackAuditLogs;
|
||||
}
|
||||
|
||||
function generateAuditLogId(): string {
|
||||
return `aud_${Date.now()}_${randomBytes(6).toString("hex")}`;
|
||||
}
|
||||
|
||||
function sanitizeMetadata(
|
||||
metadata?: Record<string, unknown>,
|
||||
): Record<string, unknown> {
|
||||
if (!metadata || typeof metadata !== "object" || Array.isArray(metadata)) {
|
||||
return {};
|
||||
}
|
||||
const REDACTED_KEYS = new Set([
|
||||
"password",
|
||||
"newPassword",
|
||||
"currentPassword",
|
||||
"token",
|
||||
"secret",
|
||||
"jwtSecret",
|
||||
"s3SecretAccessKey",
|
||||
"s3SecretAccessKeyEnc",
|
||||
"webhookSecret",
|
||||
"webhookSecretEnc",
|
||||
"prometheusMetricsToken",
|
||||
"prometheusMetricsTokenEnc",
|
||||
]);
|
||||
const clean: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(metadata)) {
|
||||
if (REDACTED_KEYS.has(key)) {
|
||||
clean[key] = "[REDACTED]";
|
||||
} else {
|
||||
clean[key] = value;
|
||||
}
|
||||
}
|
||||
return clean;
|
||||
}
|
||||
|
||||
export async function recordAdminAuditLogAsync(
|
||||
input: RecordAuditLogInput,
|
||||
): Promise<AuditLogEntry> {
|
||||
const entry: AuditLogEntry = {
|
||||
id: generateAuditLogId(),
|
||||
actorEmail: (input.actorEmail || "system").trim().toLowerCase(),
|
||||
actorRole: (input.actorRole || "owner").trim().toLowerCase(),
|
||||
action: (input.action || "UNKNOWN_ACTION").trim(),
|
||||
target: (input.target || "system").trim(),
|
||||
ipAddress: (input.ipAddress || "").trim(),
|
||||
metadata: sanitizeMetadata(input.metadata),
|
||||
createdAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
// Always maintain bounded in-memory copy for fast lookup & fallback
|
||||
const mem = getInMemoryAuditLogs();
|
||||
mem.unshift(entry);
|
||||
if (mem.length > MAX_IN_MEMORY_AUDIT_LOGS) {
|
||||
mem.length = MAX_IN_MEMORY_AUDIT_LOGS;
|
||||
}
|
||||
|
||||
const pool = DatabaseEngine.getPool();
|
||||
if (pool) {
|
||||
try {
|
||||
await DatabaseEngine.ensureSchema();
|
||||
await pool.query(
|
||||
`
|
||||
INSERT INTO admin_audit_logs (
|
||||
id,
|
||||
actor_email,
|
||||
actor_role,
|
||||
action,
|
||||
target,
|
||||
ip_address,
|
||||
metadata_json,
|
||||
created_at
|
||||
)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8::timestamptz)
|
||||
`,
|
||||
[
|
||||
entry.id,
|
||||
entry.actorEmail,
|
||||
entry.actorRole,
|
||||
entry.action,
|
||||
entry.target,
|
||||
entry.ipAddress,
|
||||
JSON.stringify(entry.metadata),
|
||||
entry.createdAt,
|
||||
],
|
||||
);
|
||||
} catch {
|
||||
// Never block or crash administrative actions if audit table write experiences transient DB failure
|
||||
}
|
||||
}
|
||||
|
||||
return entry;
|
||||
}
|
||||
|
||||
export function recordAdminAuditLog(input: RecordAuditLogInput): void {
|
||||
void recordAdminAuditLogAsync(input);
|
||||
}
|
||||
|
||||
export async function listAdminAuditLogsAsync(
|
||||
query: ListAuditLogsQuery = {},
|
||||
): Promise<AuditLogEntry[]> {
|
||||
const rawLimit =
|
||||
typeof query.limit === "number" && Number.isFinite(query.limit)
|
||||
? Math.floor(query.limit)
|
||||
: 100;
|
||||
const limit = Math.min(Math.max(rawLimit, 1), 500);
|
||||
const actionFilter = query.action?.trim().toUpperCase() || "";
|
||||
const actorFilter = query.actorEmail?.trim().toLowerCase() || "";
|
||||
const searchFilter = query.search?.trim().toLowerCase() || "";
|
||||
|
||||
const pool = DatabaseEngine.getPool();
|
||||
if (pool) {
|
||||
try {
|
||||
await DatabaseEngine.ensureSchema();
|
||||
const conditions: string[] = [];
|
||||
const params: unknown[] = [];
|
||||
|
||||
if (actionFilter && actionFilter !== "ALL") {
|
||||
params.push(actionFilter);
|
||||
conditions.push(`UPPER(action) = $${params.length}`);
|
||||
}
|
||||
if (actorFilter) {
|
||||
params.push(actorFilter);
|
||||
conditions.push(`LOWER(actor_email) = $${params.length}`);
|
||||
}
|
||||
if (searchFilter) {
|
||||
params.push(`%${searchFilter}%`);
|
||||
const idx = params.length;
|
||||
conditions.push(
|
||||
`(LOWER(actor_email) LIKE $${idx} OR LOWER(action) LIKE $${idx} OR LOWER(target) LIKE $${idx} OR LOWER(ip_address) LIKE $${idx})`,
|
||||
);
|
||||
}
|
||||
|
||||
params.push(limit);
|
||||
const whereClause =
|
||||
conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : "";
|
||||
const result = await pool.query<{
|
||||
id: string;
|
||||
actor_email: string;
|
||||
actor_role: string;
|
||||
action: string;
|
||||
target: string;
|
||||
ip_address: string;
|
||||
metadata_json: string;
|
||||
created_at: Date | string;
|
||||
}>(
|
||||
`
|
||||
SELECT
|
||||
id,
|
||||
actor_email,
|
||||
actor_role,
|
||||
action,
|
||||
target,
|
||||
ip_address,
|
||||
metadata_json,
|
||||
created_at
|
||||
FROM admin_audit_logs
|
||||
${whereClause}
|
||||
ORDER BY created_at DESC
|
||||
LIMIT $${params.length}
|
||||
`,
|
||||
params,
|
||||
);
|
||||
|
||||
return result.rows.map((row) => {
|
||||
let parsedMeta: Record<string, unknown> = {};
|
||||
try {
|
||||
const parsed = JSON.parse(row.metadata_json || "{}");
|
||||
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
|
||||
parsedMeta = parsed as Record<string, unknown>;
|
||||
}
|
||||
} catch {
|
||||
parsedMeta = {};
|
||||
}
|
||||
return {
|
||||
id: row.id,
|
||||
actorEmail: row.actor_email,
|
||||
actorRole: row.actor_role,
|
||||
action: row.action,
|
||||
target: row.target,
|
||||
ipAddress: row.ip_address || "",
|
||||
metadata: parsedMeta,
|
||||
createdAt:
|
||||
row.created_at instanceof Date
|
||||
? row.created_at.toISOString()
|
||||
: new Date(row.created_at).toISOString(),
|
||||
};
|
||||
});
|
||||
} catch {
|
||||
// Fallback to in-memory log buffer if PostgreSQL query fails
|
||||
}
|
||||
}
|
||||
|
||||
return getInMemoryAuditLogs()
|
||||
.filter((entry) => {
|
||||
if (
|
||||
actionFilter &&
|
||||
actionFilter !== "ALL" &&
|
||||
entry.action.toUpperCase() !== actionFilter
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (actorFilter && entry.actorEmail.toLowerCase() !== actorFilter) {
|
||||
return false;
|
||||
}
|
||||
if (searchFilter) {
|
||||
const haystack =
|
||||
`${entry.actorEmail} ${entry.actorRole} ${entry.action} ${entry.target} ${entry.ipAddress}`.toLowerCase();
|
||||
if (!haystack.includes(searchFilter)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
})
|
||||
.slice(0, limit);
|
||||
}
|
||||
|
||||
export function clearAuditLogsForTests(): void {
|
||||
globals.__dataStackAuditLogs = [];
|
||||
}
|
||||
+67
-15
@@ -1,27 +1,79 @@
|
||||
import {
|
||||
type AdminRole,
|
||||
getAdminRoleRank,
|
||||
normalizeAdminRole,
|
||||
} from "@/core/auth/adminSession";
|
||||
import { verifyRequestAdminSession } from "@/core/auth/adminUsers";
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
|
||||
export const ADMIN_SESSION_COOKIE = "datastack_admin_session";
|
||||
|
||||
export async function requireAdminApiSession(req: NextRequest) {
|
||||
export interface RequireAdminSessionOptions {
|
||||
minRole?: AdminRole;
|
||||
allowedRoles?: AdminRole[];
|
||||
}
|
||||
|
||||
export async function requireAdminApiSession(
|
||||
req: NextRequest,
|
||||
options?: RequireAdminSessionOptions,
|
||||
) {
|
||||
const token = req.cookies.get(ADMIN_SESSION_COOKIE)?.value;
|
||||
const auth = await verifyRequestAdminSession(token);
|
||||
if (auth.valid) {
|
||||
return { valid: true as const, auth };
|
||||
if (!auth.valid) {
|
||||
return {
|
||||
valid: false as const,
|
||||
response: NextResponse.json(
|
||||
{
|
||||
error:
|
||||
auth.reason === "UNAUTHORIZED"
|
||||
? "Authentication required."
|
||||
: "Admin authentication is temporarily unavailable.",
|
||||
},
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
valid: false as const,
|
||||
response: NextResponse.json(
|
||||
{
|
||||
error:
|
||||
auth.reason === "UNAUTHORIZED"
|
||||
? "Authentication required."
|
||||
: "Admin authentication is temporarily unavailable.",
|
||||
},
|
||||
{ status: auth.reason === "UNAUTHORIZED" ? 401 : 503 },
|
||||
),
|
||||
};
|
||||
const currentRole = normalizeAdminRole(auth.session.role, "owner");
|
||||
|
||||
if (options?.minRole) {
|
||||
const currentRank = getAdminRoleRank(currentRole);
|
||||
const requiredRank = getAdminRoleRank(options.minRole);
|
||||
if (currentRank < requiredRank) {
|
||||
return {
|
||||
valid: false as const,
|
||||
response: NextResponse.json(
|
||||
{
|
||||
error: `INSUFFICIENT_ROLE_PERMISSION: Role '${currentRole}' does not have permission for this action (requires '${options.minRole}').`,
|
||||
code: "INSUFFICIENT_ROLE_PERMISSION",
|
||||
currentRole,
|
||||
requiredRole: options.minRole,
|
||||
},
|
||||
{ status: 403 },
|
||||
),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
if (options?.allowedRoles && options.allowedRoles.length > 0) {
|
||||
const allowedRanks = options.allowedRoles.map((r) => normalizeAdminRole(r));
|
||||
if (!allowedRanks.includes(currentRole)) {
|
||||
return {
|
||||
valid: false as const,
|
||||
response: NextResponse.json(
|
||||
{
|
||||
error: `INSUFFICIENT_ROLE_PERMISSION: Role '${currentRole}' is not permitted for this action.`,
|
||||
code: "INSUFFICIENT_ROLE_PERMISSION",
|
||||
currentRole,
|
||||
allowedRoles: allowedRanks,
|
||||
},
|
||||
{ status: 403 },
|
||||
),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { valid: true as const, auth };
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -13,6 +13,7 @@ function isProtectedApi(pathname: string, method: string) {
|
||||
if (pathname === "/api/auth/setup" && method === "POST") return false;
|
||||
if (pathname === "/api/events" && method === "POST") return false;
|
||||
if (pathname === "/api/health" && method === "GET") return false;
|
||||
if (pathname === "/api/prometheus" && method === "GET") return false;
|
||||
return pathname.startsWith("/api/");
|
||||
}
|
||||
|
||||
|
||||
+71
-25
@@ -1,7 +1,7 @@
|
||||
// REFERENCE SCHEMA ONLY — NOT EXECUTED BY PRISMA CLI AT RUNTIME.
|
||||
// Canonical PostgreSQL runtime DDL is owned by `core/Database/database.ts` (Port 3000 via `pg`)
|
||||
// and mirrored 1:1 in `server/index.js` (Port 4000 standalone container).
|
||||
// Keep all 5 models below strictly synchronized with `core/Database/database.ts` (verified by tests).
|
||||
// Keep all 7 models below strictly synchronized with `core/Database/database.ts` (verified by tests).
|
||||
|
||||
generator client {
|
||||
provider = "prisma-client-js"
|
||||
@@ -12,20 +12,23 @@ datasource db {
|
||||
url = env("DATABASE_URL")
|
||||
}
|
||||
|
||||
// CRITICAL: PostgreSQL is ONLY for Admin / Config / SDK Registration — NOT for collected data.
|
||||
// All collected data is strictly written to CSV files in /storage/data/{clientId}/{YYYY-MM-DD}.csv
|
||||
// CRITICAL: PostgreSQL is ONLY for Admin / Config / SDK Registration / Audit Governance — NOT for collected telemetry data.
|
||||
// All collected telemetry data is strictly written to CSV / .csv.gz files in /storage/data/{clientId}/{YYYY-MM-DD}.csv
|
||||
|
||||
model ClientSDK {
|
||||
clientId String @id @map("client_id")
|
||||
name String
|
||||
secret String
|
||||
token String
|
||||
rateLimit Int @default(100) @map("rate_limit") // Max events per second
|
||||
status String @default("active") // active | revoked
|
||||
totalEvents BigInt @default(0) @map("total_events")
|
||||
lastActive DateTime? @map("last_active") @db.Timestamptz(6)
|
||||
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(6)
|
||||
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(6)
|
||||
clientId String @id @map("client_id")
|
||||
name String
|
||||
secret String
|
||||
token String
|
||||
rateLimit Int @default(100) @map("rate_limit") // Max events per second
|
||||
storageQuotaMb Int @default(0) @map("storage_quota_mb") // 0 = unlimited
|
||||
allowedIps String @default("") @map("allowed_ips") // Comma-separated IPv4/IPv6/CIDR allowlist
|
||||
expiresAt DateTime? @map("expires_at") @db.Timestamptz(6)
|
||||
status String @default("active") // active | revoked
|
||||
totalEvents BigInt @default(0) @map("total_events")
|
||||
lastActive DateTime? @map("last_active") @db.Timestamptz(6)
|
||||
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(6)
|
||||
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(6)
|
||||
|
||||
@@map("client_sdks")
|
||||
}
|
||||
@@ -58,19 +61,62 @@ model AdminLoginAttempt {
|
||||
@@map("admin_login_attempts")
|
||||
}
|
||||
|
||||
model AdminTeamMember {
|
||||
id String @id
|
||||
email String @unique
|
||||
role String @default("viewer") // owner | admin | operator | viewer
|
||||
status String @default("active") // active | disabled
|
||||
passwordSalt Bytes @map("password_salt")
|
||||
passwordHash Bytes @map("password_hash")
|
||||
tokenVersion Int @default(1) @map("token_version")
|
||||
invitedBy String @default("owner") @map("invited_by")
|
||||
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(6)
|
||||
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(6)
|
||||
|
||||
@@map("admin_team_members")
|
||||
}
|
||||
|
||||
model AdminAuditLog {
|
||||
id String @id
|
||||
actorEmail String @map("actor_email")
|
||||
actorRole String @map("actor_role")
|
||||
action String
|
||||
target String
|
||||
ipAddress String @default("") @map("ip_address")
|
||||
metadataJson String @default("{}") @map("metadata_json")
|
||||
createdAt DateTime @default(now()) @map("created_at") @db.Timestamptz(6)
|
||||
|
||||
@@map("admin_audit_logs")
|
||||
}
|
||||
|
||||
model AdminSettings {
|
||||
id String @id @default("default")
|
||||
serverPort Int @default(4000) @map("server_port")
|
||||
webPort Int @default(3000) @map("web_port")
|
||||
maxEventsPerSec Int @default(100) @map("max_events_per_sec")
|
||||
maxPayloadSizeKb Int @default(100) @map("max_payload_size_kb")
|
||||
batchFlushThreshold Int @default(500) @map("batch_flush_threshold")
|
||||
batchFlushIntervalMs Int @default(2000) @map("batch_flush_interval_ms")
|
||||
fileRotationMb Int @default(100) @map("file_rotation_mb")
|
||||
jwtSecret String @map("jwt_secret")
|
||||
enableIdempotency Boolean @default(true) @map("enable_idempotency")
|
||||
autoPruneDays Int @default(30) @map("auto_prune_days")
|
||||
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(6)
|
||||
id String @id @default("default")
|
||||
serverPort Int @default(4000) @map("server_port")
|
||||
webPort Int @default(3000) @map("web_port")
|
||||
maxEventsPerSec Int @default(100) @map("max_events_per_sec")
|
||||
maxPayloadSizeKb Int @default(100) @map("max_payload_size_kb")
|
||||
batchFlushThreshold Int @default(500) @map("batch_flush_threshold")
|
||||
batchFlushIntervalMs Int @default(2000) @map("batch_flush_interval_ms")
|
||||
fileRotationMb Int @default(100) @map("file_rotation_mb")
|
||||
jwtSecret String @map("jwt_secret")
|
||||
enableIdempotency Boolean @default(true) @map("enable_idempotency")
|
||||
autoPruneDays Int @default(30) @map("auto_prune_days")
|
||||
compressAfterDays Int @default(0) @map("compress_after_days")
|
||||
enableS3Backup Boolean @default(false) @map("enable_s3_backup")
|
||||
s3Endpoint String @default("") @map("s3_endpoint")
|
||||
s3Region String @default("auto") @map("s3_region")
|
||||
s3Bucket String @default("") @map("s3_bucket")
|
||||
s3Prefix String @default("datastack/") @map("s3_prefix")
|
||||
s3AccessKeyId String @default("") @map("s3_access_key_id")
|
||||
s3SecretAccessKeyEnc String @default("") @map("s3_secret_access_key_enc")
|
||||
diskPauseThresholdPct Int @default(95) @map("disk_pause_threshold_pct")
|
||||
enableWebhookAlerts Boolean @default(false) @map("enable_webhook_alerts")
|
||||
webhookUrl String @default("") @map("webhook_url")
|
||||
webhookSecretEnc String @default("") @map("webhook_secret_enc")
|
||||
webhookCooldownSec Int @default(300) @map("webhook_cooldown_sec")
|
||||
enablePrometheusMetrics Boolean @default(false) @map("enable_prometheus_metrics")
|
||||
prometheusMetricsTokenEnc String @default("") @map("prometheus_metrics_token_enc")
|
||||
updatedAt DateTime @default(now()) @updatedAt @map("updated_at") @db.Timestamptz(6)
|
||||
|
||||
@@map("admin_settings")
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
# DataStack Node.js & TypeScript SDK (`@datastack/sdk` v2.2.0)
|
||||
|
||||
Official server-side Node.js and TypeScript SDK for **DataStack**. Provides 1:1 transport, fallback, deduplication, and offline spooling parity with the official Python SDK (`sdk/python/datastack`).
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Primary Real-Time WebSocket Transport (`socket.io-client`)**: Streams single events (`sync_data`) and batches (`batch_sync`) to Port `4000` with explicit server persistence acknowledgments.
|
||||
- **Automatic HTTP Fallback (`POST /api/events`)**: Seamlessly falls back to Port `3000` REST ingestion whenever the WebSocket transport is disconnected or unreachable.
|
||||
- **Offline Disk Spooling (`.datastack_spool/spool.jsonl`)**: Persists undelivered events to local disk when both WebSocket and HTTP endpoints are offline, and automatically drains the spool via `flushSpool()` when connectivity recovers.
|
||||
- **Direct File Ingestion (`sendFile()`)**: Streams `.csv`, `.json`, `.jsonl`, and `.ndjson` files with deterministic SHA-256 `eventId` generation (`file_<stem>_<row>_<hash>`) so retries never duplicate rows.
|
||||
- **Typed Error Hierarchy (`V2-E12`)**: Distinguishes terminal errors (`DataStackAuthError` for `401 CLIENT_TOKEN_EXPIRED` / `403 CLIENT_IP_FORBIDDEN`, `DataStackQuotaExceededError` for `507 STORAGE_QUOTA_EXCEEDED` / `DISK_SPACE_LOW`, `DataStackConfigError`) from retryable `429 RATE_LIMIT_EXCEEDED` (`DataStackRateLimitError`) and auto-splittable `413 PAYLOAD_TOO_LARGE` (`DataStackPayloadTooLargeError`).
|
||||
- **Graceful Shutdown Drain (`await client.shutdown(timeoutMs)`)**: Waits for in-flight requests and drains pending spooled records before closing the socket.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Install from the self-hosted repository path:
|
||||
|
||||
```bash
|
||||
npm install /path/to/datastack/sdk/node/datastack
|
||||
```
|
||||
|
||||
Or import directly inside a monorepo / TypeScript workspace:
|
||||
|
||||
```ts
|
||||
import { DataStack } from './sdk/node/datastack';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration (`datastack.config.json` or Constructor Options)
|
||||
|
||||
Create `datastack.config.json` in your Node.js project root (or pass options directly to `new DataStack({...})`):
|
||||
|
||||
```json
|
||||
{
|
||||
"serverUrl": "http://localhost:4000",
|
||||
"apiUrl": "http://localhost:3000",
|
||||
"clientToken": "YOUR_SIGNED_CLIENT_JWT_TOKEN",
|
||||
"clientId": "my_service_prod",
|
||||
"autoConnect": true,
|
||||
"enableHttpFallback": true,
|
||||
"batchChunkSize": 250,
|
||||
"offlineSpoolEnabled": true,
|
||||
"offlineSpoolDir": ".datastack_spool"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quickstart
|
||||
|
||||
```ts
|
||||
import { DataStack } from '@datastack/sdk';
|
||||
|
||||
const client = new DataStack({
|
||||
serverUrl: process.env.DATASTACK_SERVER_URL || 'http://localhost:4000',
|
||||
apiUrl: process.env.DATASTACK_API_URL || 'http://localhost:3000',
|
||||
token: process.env.DATASTACK_CLIENT_TOKEN,
|
||||
clientId: 'checkout_api',
|
||||
});
|
||||
|
||||
// 1. Single Event
|
||||
const res = await client.send({
|
||||
event_type: 'order_completed',
|
||||
order_id: 'ord_9481',
|
||||
amount_usd: 129.99,
|
||||
});
|
||||
|
||||
// 2. Batch Events
|
||||
const batchRes = await client.sendBatch([
|
||||
{ metric: 'cpu_load', value: 0.42, host: 'worker-01' },
|
||||
{ metric: 'mem_used_mb', value: 1840, host: 'worker-01' },
|
||||
]);
|
||||
|
||||
// 3. Direct File Ingestion (.csv, .json, .jsonl, .ndjson)
|
||||
const fileRes = await client.sendFile('./backfill_events.csv', { chunkSize: 250 });
|
||||
|
||||
// 4. Graceful Shutdown Drain
|
||||
await client.shutdown(5000);
|
||||
```
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "@datastack/sdk",
|
||||
"version": "2.2.0",
|
||||
"description": "Official DataStack Node.js & TypeScript Client SDK — Real-time Socket.IO telemetry streaming with automatic HTTP fallback, deterministic file ingestion, and offline disk spooling.",
|
||||
"main": "./index.ts",
|
||||
"types": "./index.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./index.ts",
|
||||
"default": "./index.ts"
|
||||
}
|
||||
},
|
||||
"files": [
|
||||
"index.ts",
|
||||
"types.ts",
|
||||
"spool.ts",
|
||||
"README.md"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"csv-parse": "^7.0.3",
|
||||
"socket.io-client": "^4.8.4"
|
||||
},
|
||||
"keywords": [
|
||||
"datastack",
|
||||
"telemetry",
|
||||
"socketio",
|
||||
"csv-partitioning",
|
||||
"nodejs-sdk",
|
||||
"typescript-sdk"
|
||||
],
|
||||
"license": "MIT"
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import type { DataStackEnvelope } from './types';
|
||||
|
||||
export interface DiskSpoolerOptions {
|
||||
spoolDir?: string;
|
||||
maxSizeMb?: number;
|
||||
enabled?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Durable newline-delimited JSON (`.datastack_spool/spool.jsonl`) local disk spooler
|
||||
* for offline telemetry persistence when both Port 4000 WebSocket and Port 3000 HTTP
|
||||
* endpoints are temporarily unreachable.
|
||||
*/
|
||||
export class DiskSpooler {
|
||||
public readonly spoolDir: string;
|
||||
public readonly spoolFile: string;
|
||||
public readonly maxBytes: number;
|
||||
public readonly enabled: boolean;
|
||||
|
||||
constructor(options: DiskSpoolerOptions = {}) {
|
||||
this.spoolDir = path.resolve(options.spoolDir || '.datastack_spool');
|
||||
this.spoolFile = path.join(this.spoolDir, 'spool.jsonl');
|
||||
const maxSizeMb = Number(options.maxSizeMb ?? 50);
|
||||
this.maxBytes = Math.max(1, Number.isFinite(maxSizeMb) ? maxSizeMb : 50) * 1024 * 1024;
|
||||
this.enabled = options.enabled !== false;
|
||||
}
|
||||
|
||||
private ensureDir(): void {
|
||||
if (!this.enabled) return;
|
||||
if (!fs.existsSync(this.spoolDir)) {
|
||||
fs.mkdirSync(this.spoolDir, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Appends one or more event envelopes to `spool.jsonl`.
|
||||
* Returns the number of events persisted to disk.
|
||||
*/
|
||||
public enqueue(items: DataStackEnvelope | DataStackEnvelope[]): number {
|
||||
if (!this.enabled) return 0;
|
||||
const list = Array.isArray(items) ? items : [items];
|
||||
if (list.length === 0) return 0;
|
||||
|
||||
this.ensureDir();
|
||||
const currentSize = this.sizeBytes();
|
||||
const lines = list.map((item) => JSON.stringify(item)).join('\n') + '\n';
|
||||
const byteLength = Buffer.byteLength(lines, 'utf8');
|
||||
|
||||
if (currentSize + byteLength > this.maxBytes) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
const fd = fs.openSync(this.spoolFile, 'a');
|
||||
try {
|
||||
fs.writeFileSync(fd, lines, 'utf8');
|
||||
fs.fsyncSync(fd);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
return list.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads up to `limit` spooled event envelopes from the head of `spool.jsonl`
|
||||
* without removing them until `removeFirst(count)` is called after server ACK.
|
||||
*/
|
||||
public peekBatch(limit = 250): DataStackEnvelope[] {
|
||||
if (!this.enabled || !fs.existsSync(this.spoolFile)) {
|
||||
return [];
|
||||
}
|
||||
const raw = fs.readFileSync(this.spoolFile, 'utf8');
|
||||
const lines = raw
|
||||
.split('\n')
|
||||
.map((l) => l.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
const results: DataStackEnvelope[] = [];
|
||||
for (const line of lines.slice(0, Math.max(1, limit))) {
|
||||
try {
|
||||
const parsed = JSON.parse(line) as DataStackEnvelope;
|
||||
if (parsed && typeof parsed === 'object' && parsed.eventId && parsed.data) {
|
||||
results.push(parsed);
|
||||
}
|
||||
} catch {
|
||||
// Skip malformed line
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomically removes the first `removeCount` lines from `spool.jsonl`
|
||||
* via a temporary file + `fsync` + `renameSync`.
|
||||
*/
|
||||
public removeFirst(removeCount: number): void {
|
||||
if (!this.enabled || removeCount <= 0 || !fs.existsSync(this.spoolFile)) {
|
||||
return;
|
||||
}
|
||||
const raw = fs.readFileSync(this.spoolFile, 'utf8');
|
||||
const lines = raw
|
||||
.split('\n')
|
||||
.map((l) => l.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
const remaining = lines.slice(removeCount);
|
||||
if (remaining.length === 0) {
|
||||
try {
|
||||
fs.unlinkSync(this.spoolFile);
|
||||
} catch {
|
||||
// Ignore if already removed
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
this.ensureDir();
|
||||
const tmpFile = `${this.spoolFile}.${process.pid}.${Date.now()}.tmp`;
|
||||
const payload = remaining.join('\n') + '\n';
|
||||
try {
|
||||
const fd = fs.openSync(tmpFile, 'w');
|
||||
try {
|
||||
fs.writeFileSync(fd, payload, 'utf8');
|
||||
fs.fsyncSync(fd);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
fs.renameSync(tmpFile, this.spoolFile);
|
||||
} catch (err) {
|
||||
try {
|
||||
if (fs.existsSync(tmpFile)) {
|
||||
fs.unlinkSync(tmpFile);
|
||||
}
|
||||
} catch {
|
||||
// Ignore cleanup failure
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
public count(): number {
|
||||
if (!this.enabled || !fs.existsSync(this.spoolFile)) {
|
||||
return 0;
|
||||
}
|
||||
try {
|
||||
const raw = fs.readFileSync(this.spoolFile, 'utf8');
|
||||
return raw
|
||||
.split('\n')
|
||||
.map((l) => l.trim())
|
||||
.filter(Boolean).length;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
public sizeBytes(): number {
|
||||
if (!this.enabled || !fs.existsSync(this.spoolFile)) {
|
||||
return 0;
|
||||
}
|
||||
try {
|
||||
return fs.statSync(this.spoolFile).size;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
public clear(): void {
|
||||
if (fs.existsSync(this.spoolFile)) {
|
||||
try {
|
||||
fs.unlinkSync(this.spoolFile);
|
||||
} catch {
|
||||
// Ignore
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
export type DataStackTransportMode = 'websocket' | 'http' | 'spool';
|
||||
|
||||
export type DataStackConnectionState =
|
||||
| 'IDLE_WS'
|
||||
| 'HTTP_FALLBACK'
|
||||
| 'OFFLINE_SPOOL'
|
||||
| 'DISCONNECTED';
|
||||
|
||||
export type DataStackEventPayload = Record<string, unknown>;
|
||||
|
||||
export interface DataStackConfigOptions {
|
||||
/** Port 4000 Socket.IO ingestion server URL (e.g. http://localhost:4000) */
|
||||
serverUrl?: string;
|
||||
/** Port 3000 Next.js HTTP API URL (e.g. http://localhost:3000) */
|
||||
apiUrl?: string;
|
||||
/** Signed client JWT token issued from /vcon/clients */
|
||||
token?: string;
|
||||
/** Alias for token */
|
||||
clientToken?: string;
|
||||
/** Expected normalized clientId (validated against token subject if both present) */
|
||||
clientId?: string;
|
||||
/** Optional path to datastack.config.json */
|
||||
configPath?: string;
|
||||
/** Automatically connect WebSocket on instantiation (default: false in Node tests, true when configured) */
|
||||
autoConnect?: boolean;
|
||||
/** Fall back to HTTP POST /api/events when WebSocket is disconnected (default: true) */
|
||||
enableHttpFallback?: boolean;
|
||||
/** Background heartbeat recovery interval in milliseconds (default: 5000) */
|
||||
heartbeatIntervalMs?: number;
|
||||
/** Maximum retry attempts for transient or 429 rate-limited chunks (default: 3) */
|
||||
maxRetries?: number;
|
||||
/** Maximum records per batch chunk (default: 250, max: 1000) */
|
||||
batchChunkSize?: number;
|
||||
/** Enable local disk spooling when both WebSocket and HTTP are unreachable (default: true) */
|
||||
offlineSpoolEnabled?: boolean;
|
||||
/** Directory path for offline spool JSONL file (default: ".datastack_spool") */
|
||||
offlineSpoolDir?: string;
|
||||
/** Maximum offline spool file size in megabytes (default: 50) */
|
||||
maxSpoolSizeMb?: number;
|
||||
/** Per-request timeout in milliseconds for WebSocket ACKs and HTTP requests (default: 5000) */
|
||||
requestTimeoutMs?: number;
|
||||
/** Custom fetch implementation for testing or custom agents */
|
||||
fetchImpl?: typeof fetch;
|
||||
}
|
||||
|
||||
export interface ResolvedDataStackConfig {
|
||||
serverUrl: string;
|
||||
apiUrl: string;
|
||||
clientToken: string;
|
||||
clientId: string;
|
||||
autoConnect: boolean;
|
||||
enableHttpFallback: boolean;
|
||||
heartbeatIntervalMs: number;
|
||||
maxRetries: number;
|
||||
batchChunkSize: number;
|
||||
offlineSpoolEnabled: boolean;
|
||||
offlineSpoolDir: string;
|
||||
maxSpoolSizeMb: number;
|
||||
requestTimeoutMs: number;
|
||||
}
|
||||
|
||||
export interface DataStackEnvelope {
|
||||
clientId: string;
|
||||
eventId: string;
|
||||
timestamp: string;
|
||||
data: DataStackEventPayload;
|
||||
}
|
||||
|
||||
export interface DataStackSendResult {
|
||||
success: boolean;
|
||||
transport: DataStackTransportMode;
|
||||
eventId?: string;
|
||||
accepted?: number;
|
||||
spooled?: number;
|
||||
duplicates?: number;
|
||||
code?: string;
|
||||
error?: string;
|
||||
rateLimitedEventIds?: string[];
|
||||
unprocessedItems?: DataStackEventPayload[];
|
||||
}
|
||||
|
||||
export interface DataStackFileSendResult extends DataStackSendResult {
|
||||
filePath: string;
|
||||
totalRows: number;
|
||||
acceptedRows: number;
|
||||
spooledRows: number;
|
||||
chunksSent: number;
|
||||
}
|
||||
|
||||
export interface DataStackStatusSnapshot {
|
||||
connected: boolean;
|
||||
transportState: DataStackConnectionState;
|
||||
clientId: string;
|
||||
serverUrl: string;
|
||||
apiUrl: string;
|
||||
spooledCount: number;
|
||||
spooledBytes: number;
|
||||
inFlightRequests: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Base error class for all DataStack Node.js / TypeScript SDK errors (V2-E12).
|
||||
*/
|
||||
export class DataStackError extends Error {
|
||||
public readonly code: string;
|
||||
public readonly statusCode?: number;
|
||||
public readonly details?: Record<string, unknown>;
|
||||
|
||||
constructor(
|
||||
message: string,
|
||||
code = 'DATASTACK_ERROR',
|
||||
statusCode?: number,
|
||||
details?: Record<string, unknown>
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'DataStackError';
|
||||
this.code = code;
|
||||
this.statusCode = statusCode;
|
||||
this.details = details;
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackConfigError extends DataStackError {
|
||||
constructor(message: string, details?: Record<string, unknown>) {
|
||||
super(message, 'INVALID_SDK_CONFIG', 400, details);
|
||||
this.name = 'DataStackConfigError';
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackAuthError extends DataStackError {
|
||||
constructor(
|
||||
message: string,
|
||||
code:
|
||||
| 'AUTH_FAILED'
|
||||
| 'CLIENT_TOKEN_EXPIRED'
|
||||
| 'CLIENT_IP_FORBIDDEN'
|
||||
| 'CLIENT_REVOKED' = 'AUTH_FAILED',
|
||||
statusCode = 401,
|
||||
details?: Record<string, unknown>
|
||||
) {
|
||||
super(message, code, statusCode, details);
|
||||
this.name = 'DataStackAuthError';
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackRateLimitError extends DataStackError {
|
||||
public readonly rateLimitedEventIds: string[];
|
||||
|
||||
constructor(
|
||||
message: string,
|
||||
rateLimitedEventIds: string[] = [],
|
||||
details?: Record<string, unknown>
|
||||
) {
|
||||
super(message, 'RATE_LIMIT_EXCEEDED', 429, details);
|
||||
this.name = 'DataStackRateLimitError';
|
||||
this.rateLimitedEventIds = rateLimitedEventIds;
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackQuotaExceededError extends DataStackError {
|
||||
constructor(
|
||||
message: string,
|
||||
code: 'STORAGE_QUOTA_EXCEEDED' | 'DISK_SPACE_LOW' = 'STORAGE_QUOTA_EXCEEDED',
|
||||
details?: Record<string, unknown>
|
||||
) {
|
||||
super(message, code, 507, details);
|
||||
this.name = 'DataStackQuotaExceededError';
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackPayloadTooLargeError extends DataStackError {
|
||||
constructor(message: string, details?: Record<string, unknown>) {
|
||||
super(message, 'PAYLOAD_TOO_LARGE', 413, details);
|
||||
this.name = 'DataStackPayloadTooLargeError';
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackFileError extends DataStackError {
|
||||
constructor(message: string, details?: Record<string, unknown>) {
|
||||
super(message, 'FILE_INGESTION_ERROR', 400, details);
|
||||
this.name = 'DataStackFileError';
|
||||
}
|
||||
}
|
||||
|
||||
export class DataStackShutdownTimeoutError extends DataStackError {
|
||||
constructor(message: string, details?: Record<string, unknown>) {
|
||||
super(message, 'SHUTDOWN_FLUSH_TIMEOUT', 408, details);
|
||||
this.name = 'DataStackShutdownTimeoutError';
|
||||
}
|
||||
}
|
||||
+21
-3
@@ -617,11 +617,17 @@ let activeActivityFlushPromise = null;
|
||||
|
||||
let schemaEnsurePromise = null;
|
||||
let schemaEnsurePool = null;
|
||||
let hasHydratedSettingsFromDb = false;
|
||||
|
||||
function isDbReadyForMaintenance() {
|
||||
return !dbPool || hasHydratedSettingsFromDb === true;
|
||||
}
|
||||
|
||||
function setDbPoolForTests(pool) {
|
||||
dbPool = pool;
|
||||
schemaEnsurePromise = null;
|
||||
schemaEnsurePool = null;
|
||||
hasHydratedSettingsFromDb = false;
|
||||
}
|
||||
|
||||
async function flushPendingClientActivity() {
|
||||
@@ -1089,7 +1095,7 @@ function applyRuntimeSettingsObject(settingsObj) {
|
||||
const nextCompressDays = Math.floor(parsedCompress);
|
||||
const changed = nextCompressDays !== runtimeConfig.compressAfterDays;
|
||||
runtimeConfig.compressAfterDays = nextCompressDays;
|
||||
if (changed && nextCompressDays > 0) {
|
||||
if (changed && nextCompressDays > 0 && isDbReadyForMaintenance()) {
|
||||
void writer.compressOldPartitionsAsync(nextCompressDays).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
@@ -1101,7 +1107,12 @@ function applyRuntimeSettingsObject(settingsObj) {
|
||||
const nextPruneDays = Math.floor(parsedPrune);
|
||||
const changed = nextPruneDays !== runtimeConfig.autoPruneDays;
|
||||
runtimeConfig.autoPruneDays = nextPruneDays;
|
||||
if (changed && nextPruneDays > 0 && !runtimeConfig.enableS3Backup) {
|
||||
if (
|
||||
changed &&
|
||||
nextPruneDays > 0 &&
|
||||
!runtimeConfig.enableS3Backup &&
|
||||
isDbReadyForMaintenance()
|
||||
) {
|
||||
void writer.pruneExpiredPartitionsAsync(nextPruneDays).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
@@ -1290,7 +1301,11 @@ async function syncSettingsFromDatabase() {
|
||||
FROM admin_settings
|
||||
WHERE id = 'default'`,
|
||||
);
|
||||
if (!res.rowCount || res.rowCount === 0) return true;
|
||||
if (!res.rowCount || res.rowCount === 0) {
|
||||
hasHydratedSettingsFromDb = true;
|
||||
return true;
|
||||
}
|
||||
hasHydratedSettingsFromDb = true;
|
||||
const row = res.rows[0];
|
||||
const parsedPrune =
|
||||
row.auto_prune_days !== undefined && row.auto_prune_days !== null
|
||||
@@ -1459,6 +1474,7 @@ app.post('/internal/security-check', (req, res) => {
|
||||
|
||||
// Internal instant settings synchronization triggered by Next.js POST /api/settings
|
||||
app.post('/internal/settings-sync', (req, res) => {
|
||||
hasHydratedSettingsFromDb = true;
|
||||
const result = applyRuntimeSettingsObject(req.body || {});
|
||||
res.json({
|
||||
success: true,
|
||||
@@ -1811,6 +1827,7 @@ async function startIngestionServer() {
|
||||
}
|
||||
|
||||
pruneTimer = setInterval(() => {
|
||||
if (!isDbReadyForMaintenance()) return;
|
||||
if (runtimeConfig.compressAfterDays > 0) {
|
||||
void writer
|
||||
.compressOldPartitionsAsync(runtimeConfig.compressAfterDays)
|
||||
@@ -1860,6 +1877,7 @@ module.exports = {
|
||||
applyRuntimeSettingsObject,
|
||||
ensureDatabaseSchema,
|
||||
syncSettingsFromDatabase,
|
||||
isDbReadyForMaintenance,
|
||||
gracefulShutdown,
|
||||
startIngestionServer,
|
||||
setRedisClientForTests,
|
||||
|
||||
@@ -476,3 +476,133 @@ test("DEPLOY-BUG-02, 03 & 04: Port 4000 memoizes schema DDLs, hydrates rotated J
|
||||
ingestionServer.stopTimers();
|
||||
}
|
||||
});
|
||||
|
||||
test("V2-Deploy-Fix-01 (DEPLOY-V2-BUG-01, 02 & 03): middleware permits GET /api/prometheus Bearer scrapes, Port 4000 blocks unhydrated DB pruning, and prisma/schema.prisma mirrors all 7 V2 tables", async () => {
|
||||
// 1. DEPLOY-V2-BUG-01: middleware.ts allows GET /api/prometheus without a browser cookie so route handler can verify Bearer token
|
||||
const { middleware } = await import("../middleware");
|
||||
const { NextRequest } = await import("next/server");
|
||||
|
||||
const promReq = new NextRequest("http://localhost:3000/api/prometheus", {
|
||||
method: "GET",
|
||||
headers: {
|
||||
authorization: "Bearer prom_scrape_token_123",
|
||||
},
|
||||
});
|
||||
const promMwRes = await middleware(promReq);
|
||||
assert.notEqual(
|
||||
promMwRes.status,
|
||||
401,
|
||||
"middleware.ts must not block GET /api/prometheus with 401 before route handler verifies Bearer token",
|
||||
);
|
||||
assert.equal(promMwRes.headers.get("x-middleware-next"), "1");
|
||||
|
||||
// Meanwhile, protected admin APIs like GET /api/metrics still require admin session cookie in middleware
|
||||
const prevSecret = process.env.DATASTACK_ADMIN_SESSION_SECRET;
|
||||
process.env.DATASTACK_ADMIN_SESSION_SECRET =
|
||||
"test-session-secret-32-characters-minimum-length";
|
||||
try {
|
||||
const metricsReq = new NextRequest("http://localhost:3000/api/metrics", {
|
||||
method: "GET",
|
||||
});
|
||||
const metricsMwRes = await middleware(metricsReq);
|
||||
assert.equal(metricsMwRes.status, 401);
|
||||
} finally {
|
||||
if (prevSecret === undefined) {
|
||||
delete process.env.DATASTACK_ADMIN_SESSION_SECRET;
|
||||
} else {
|
||||
process.env.DATASTACK_ADMIN_SESSION_SECRET = prevSecret;
|
||||
}
|
||||
}
|
||||
|
||||
// 2. DEPLOY-V2-BUG-02: Port 4000 isDbReadyForMaintenance() is false when dbPool is configured but unhydrated
|
||||
const prevDisableListen = process.env.DATASTACK_DISABLE_LISTEN;
|
||||
process.env.DATASTACK_DISABLE_LISTEN = "1";
|
||||
const ingestionServer = require("../server/index.js") as {
|
||||
setDbPoolForTests: (pool: unknown) => void;
|
||||
isDbReadyForMaintenance: () => boolean;
|
||||
syncSettingsFromDatabase: () => Promise<boolean>;
|
||||
stopTimers: () => void;
|
||||
};
|
||||
if (prevDisableListen === undefined) {
|
||||
delete process.env.DATASTACK_DISABLE_LISTEN;
|
||||
} else {
|
||||
process.env.DATASTACK_DISABLE_LISTEN = prevDisableListen;
|
||||
}
|
||||
|
||||
try {
|
||||
// Without dbPool (in-memory mode), maintenance is ready immediately
|
||||
ingestionServer.setDbPoolForTests(null);
|
||||
assert.equal(ingestionServer.isDbReadyForMaintenance(), true);
|
||||
|
||||
// With dbPool attached but before hydration succeeds, maintenance is blocked
|
||||
let shouldFailDb = true;
|
||||
const fakePool = {
|
||||
async query(sql: string) {
|
||||
if (shouldFailDb) {
|
||||
throw new Error("DB starting up");
|
||||
}
|
||||
if (/SELECT\s+max_events_per_sec/i.test(sql)) {
|
||||
return { rowCount: 0, rows: [] };
|
||||
}
|
||||
return { rowCount: 0, rows: [] };
|
||||
},
|
||||
};
|
||||
ingestionServer.setDbPoolForTests(fakePool);
|
||||
assert.equal(
|
||||
ingestionServer.isDbReadyForMaintenance(),
|
||||
false,
|
||||
"Port 4000 must block background pruning while DATABASE_URL pool is unhydrated",
|
||||
);
|
||||
assert.equal(await ingestionServer.syncSettingsFromDatabase(), false);
|
||||
assert.equal(ingestionServer.isDbReadyForMaintenance(), false);
|
||||
|
||||
// Once DB hydrates, maintenance becomes ready
|
||||
shouldFailDb = false;
|
||||
assert.equal(await ingestionServer.syncSettingsFromDatabase(), true);
|
||||
assert.equal(ingestionServer.isDbReadyForMaintenance(), true);
|
||||
} finally {
|
||||
ingestionServer.setDbPoolForTests(null);
|
||||
ingestionServer.stopTimers();
|
||||
}
|
||||
|
||||
// 3. DEPLOY-V2-BUG-03: prisma/schema.prisma mirrors all 7 V2 tables and V2 columns
|
||||
const prismaSrc = readFileSync(
|
||||
resolve(process.cwd(), "prisma/schema.prisma"),
|
||||
"utf8",
|
||||
);
|
||||
for (const table of [
|
||||
"client_sdks",
|
||||
"admin_settings",
|
||||
"deleted_clients",
|
||||
"admin_users",
|
||||
"admin_login_attempts",
|
||||
"admin_team_members",
|
||||
"admin_audit_logs",
|
||||
]) {
|
||||
assert.match(
|
||||
prismaSrc,
|
||||
new RegExp(`@@map\\("${table}"\\)`),
|
||||
`prisma/schema.prisma must map ${table}`,
|
||||
);
|
||||
}
|
||||
for (const col of [
|
||||
"storage_quota_mb",
|
||||
"allowed_ips",
|
||||
"expires_at",
|
||||
"compress_after_days",
|
||||
"enable_s3_backup",
|
||||
"s3_secret_access_key_enc",
|
||||
"disk_pause_threshold_pct",
|
||||
"enable_webhook_alerts",
|
||||
"webhook_secret_enc",
|
||||
"enable_prometheus_metrics",
|
||||
"prometheus_metrics_token_enc",
|
||||
]) {
|
||||
assert.match(
|
||||
prismaSrc,
|
||||
new RegExp(`@map\\("${col}"\\)`),
|
||||
`prisma/schema.prisma must map V2 column ${col}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -9,11 +9,11 @@ import test from "node:test";
|
||||
import { NextRequest } from "next/server";
|
||||
import { io as ioClient, type Socket } from "socket.io-client";
|
||||
import { PATCH as patchClientRoute } from "../app/api/clients/[id]/route";
|
||||
import { GET as clientsGet, POST as clientsPost } from "../app/api/clients/route";
|
||||
import { POST as clientsPost } from "../app/api/clients/route";
|
||||
import { POST as eventsPost } from "../app/api/events/route";
|
||||
import { GET as exportGet } from "../app/api/export/route";
|
||||
import { GET as prometheusGet } from "../app/api/prometheus/route";
|
||||
import { GET as settingsGet, POST as settingsPost } from "../app/api/settings/route";
|
||||
import { POST as settingsPost } from "../app/api/settings/route";
|
||||
import { POST as webhookTestPost } from "../app/api/settings/webhook-test/route";
|
||||
import { SettingsManager } from "../core/adminSettings/settingsManager";
|
||||
import {
|
||||
@@ -113,16 +113,16 @@ test("V2-Phase-02 [IP Allowlist & Token Expiry]: validateAllowedIpsInput, isClie
|
||||
// Expiration validation & evaluation
|
||||
assert.deepEqual(validateClientExpiresAtInput(null), {
|
||||
valid: true,
|
||||
normalized: null,
|
||||
iso: null,
|
||||
});
|
||||
assert.deepEqual(validateClientExpiresAtInput(""), {
|
||||
valid: true,
|
||||
normalized: null,
|
||||
iso: null,
|
||||
});
|
||||
const validExp = validateClientExpiresAtInput("2030-06-15T12:00:00.000Z");
|
||||
assert.equal(validExp.valid, true);
|
||||
if (validExp.valid) {
|
||||
assert.equal(validExp.normalized, "2030-06-15T12:00:00.000Z");
|
||||
assert.equal(validExp.iso, "2030-06-15T12:00:00.000Z");
|
||||
}
|
||||
const badExp = validateClientExpiresAtInput("not-a-timestamp");
|
||||
assert.equal(badExp.valid, false);
|
||||
@@ -170,7 +170,7 @@ test("V2-Phase-02 [Webhook Notifier & /api/settings/webhook-test]: HMAC-SHA256 s
|
||||
|
||||
try {
|
||||
// 1. Format & signature verification
|
||||
const formatted = formatOutboundWebhookBody({
|
||||
const formattedJsonStr = formatOutboundWebhookBody({
|
||||
event: "DISK_SPACE_LOW",
|
||||
severity: "critical",
|
||||
title: "Disk Threshold Reached",
|
||||
@@ -179,34 +179,37 @@ test("V2-Phase-02 [Webhook Notifier & /api/settings/webhook-test]: HMAC-SHA256 s
|
||||
timestamp: "2026-11-19T12:00:00.000Z",
|
||||
details: { usedPct: 95.4 },
|
||||
});
|
||||
const formatted = JSON.parse(formattedJsonStr) as {
|
||||
text: string;
|
||||
content: string;
|
||||
};
|
||||
assert.ok(formatted.text.includes("DISK_SPACE_LOW"));
|
||||
assert.equal(formatted.content, formatted.text);
|
||||
|
||||
const bodyJson = JSON.stringify(formatted);
|
||||
const sig = buildWebhookSignature(bodyJson, "webhook_secret_xyz");
|
||||
const sig = buildWebhookSignature(formattedJsonStr, "webhook_secret_xyz");
|
||||
assert.ok(sig.startsWith("sha256="));
|
||||
|
||||
// 2. Cooldown deduplication test
|
||||
const first = await dispatchWebhookAlertAsync(
|
||||
{
|
||||
enabled: true,
|
||||
url: webhookUrl,
|
||||
secret: "webhook_secret_xyz",
|
||||
cooldownSec: 120,
|
||||
},
|
||||
{
|
||||
event: "CLIENT_QUOTA_EXCEEDED",
|
||||
event: "STORAGE_QUOTA_EXCEEDED",
|
||||
severity: "warning",
|
||||
title: "Client Quota Exceeded",
|
||||
message: "Quota reached for svc_alpha",
|
||||
clientId: "svc_alpha",
|
||||
},
|
||||
{
|
||||
enabled: true,
|
||||
webhookUrl,
|
||||
webhookSecret: "webhook_secret_xyz",
|
||||
cooldownSec: 120,
|
||||
},
|
||||
);
|
||||
assert.equal(first.ok, true);
|
||||
assert.equal(receivedRequests.length, 1);
|
||||
assert.equal(
|
||||
receivedRequests[0].headers["x-datastack-event"],
|
||||
"CLIENT_QUOTA_EXCEEDED",
|
||||
"STORAGE_QUOTA_EXCEEDED",
|
||||
);
|
||||
assert.equal(
|
||||
receivedRequests[0].headers["x-datastack-signature"],
|
||||
@@ -216,21 +219,22 @@ test("V2-Phase-02 [Webhook Notifier & /api/settings/webhook-test]: HMAC-SHA256 s
|
||||
// Duplicate event for same client within cooldown window is suppressed
|
||||
const second = await dispatchWebhookAlertAsync(
|
||||
{
|
||||
enabled: true,
|
||||
url: webhookUrl,
|
||||
secret: "webhook_secret_xyz",
|
||||
cooldownSec: 120,
|
||||
},
|
||||
{
|
||||
event: "CLIENT_QUOTA_EXCEEDED",
|
||||
event: "STORAGE_QUOTA_EXCEEDED",
|
||||
severity: "warning",
|
||||
title: "Client Quota Exceeded",
|
||||
message: "Quota reached again for svc_alpha",
|
||||
clientId: "svc_alpha",
|
||||
},
|
||||
{
|
||||
enabled: true,
|
||||
webhookUrl,
|
||||
webhookSecret: "webhook_secret_xyz",
|
||||
cooldownSec: 120,
|
||||
},
|
||||
);
|
||||
assert.equal(second.ok, true);
|
||||
assert.equal(second.suppressedByCooldown, true);
|
||||
assert.equal(second.skipped, true);
|
||||
assert.equal(second.reason, "COOLDOWN_ACTIVE");
|
||||
assert.equal(receivedRequests.length, 1);
|
||||
|
||||
// 3. Test POST /api/settings/webhook-test route (200 & 502 WEBHOOK_DELIVERY_FAILED V2-E06)
|
||||
@@ -271,17 +275,44 @@ test("V2-Phase-02 [Webhook Notifier & /api/settings/webhook-test]: HMAC-SHA256 s
|
||||
assert.equal(failTestData.code, "WEBHOOK_DELIVERY_FAILED");
|
||||
} finally {
|
||||
await new Promise<void>((resolve) => mockWebhookServer.close(() => resolve()));
|
||||
await DataBufferEngine.shutdown();
|
||||
resetAdminStateForTests();
|
||||
}
|
||||
});
|
||||
|
||||
test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (403 V2-E07) & Token expiration (401 V2-E08)", async () => {
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), "datastack-v2-p2-sec-"));
|
||||
const prevStorageDir = process.env.DATASTACK_STORAGE_DIR;
|
||||
const prevDisableListen = process.env.DATASTACK_DISABLE_LISTEN;
|
||||
const prevDbUrl = process.env.DATABASE_URL;
|
||||
process.env.DATASTACK_STORAGE_DIR = tmpDir;
|
||||
CSVStorageManager.resetForTests(tmpDir);
|
||||
await CSVStorageManager.initializeAsync();
|
||||
process.env.DATASTACK_DISABLE_LISTEN = "1";
|
||||
delete process.env.DATABASE_URL;
|
||||
|
||||
const origBaseStorageDir = CSVStorageManager.baseStorageDir;
|
||||
CSVStorageManager.baseStorageDir = tmpDir;
|
||||
CSVStorageManager.clearQuotaCaches();
|
||||
clearWebhookCooldownCacheForTests();
|
||||
|
||||
const serverMod = require("../server/index.js") as {
|
||||
server: http.Server;
|
||||
io: { close: (cb: () => void) => void };
|
||||
writer: { baseDir: string; clearQuotaCaches: () => void };
|
||||
verifyClientInDb: (
|
||||
clientId: string,
|
||||
token: string,
|
||||
bypassCache?: boolean,
|
||||
clientIp?: string,
|
||||
) => Promise<{ ok: boolean; reason?: string }>;
|
||||
setDbPoolForTests: (pool: unknown) => void;
|
||||
stopTimers: () => void;
|
||||
};
|
||||
const origWriterBaseDir = serverMod.writer.baseDir;
|
||||
serverMod.writer.baseDir = tmpDir;
|
||||
serverMod.writer.clearQuotaCaches();
|
||||
|
||||
const openSockets: Socket[] = [];
|
||||
|
||||
try {
|
||||
const adminCookie = await makeAdminCookie();
|
||||
|
||||
@@ -295,6 +326,7 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
"content-type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
clientId: "ip_restricted_svc",
|
||||
name: "IP Restricted Client",
|
||||
rateLimit: 100,
|
||||
storageQuotaMb: 0,
|
||||
@@ -303,7 +335,7 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(createRes.status, 200);
|
||||
assert.equal(createRes.status, 201);
|
||||
const createData = await createRes.json();
|
||||
const client = createData.client;
|
||||
assert.equal(client.allowedIps, "10.10.0.0/16");
|
||||
@@ -320,7 +352,7 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
},
|
||||
body: JSON.stringify({
|
||||
eventId: "evt_ip_denied",
|
||||
payload: { step: 1 },
|
||||
data: { step: 1 },
|
||||
}),
|
||||
}),
|
||||
);
|
||||
@@ -339,7 +371,7 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
},
|
||||
body: JSON.stringify({
|
||||
eventId: "evt_ip_allowed",
|
||||
payload: { step: 2 },
|
||||
data: { step: 2 },
|
||||
}),
|
||||
}),
|
||||
);
|
||||
@@ -377,7 +409,7 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
},
|
||||
body: JSON.stringify({
|
||||
eventId: "evt_expired",
|
||||
payload: { step: 3 },
|
||||
data: { step: 3 },
|
||||
}),
|
||||
}),
|
||||
);
|
||||
@@ -386,93 +418,120 @@ test("V2-Phase-02 [HTTP & Socket Ingestion Security]: Per-client IP allowlist (4
|
||||
assert.equal(expiredJson.code, "CLIENT_TOKEN_EXPIRED");
|
||||
|
||||
// 5. Verify Port 4000 Socket.IO server enforces IP allowlist & token expiry
|
||||
process.env.DATASTACK_DISABLE_LISTEN = "1";
|
||||
const ingestionApp = require("../server/index.js");
|
||||
let mockExpiresAt: string | null = pastIso;
|
||||
const mockAllowedIps = "10.10.0.0/16";
|
||||
serverMod.setDbPoolForTests({
|
||||
query: async (sql: string) => {
|
||||
if (sql.includes("FROM client_sdks")) {
|
||||
return {
|
||||
rowCount: 1,
|
||||
rows: [
|
||||
{
|
||||
client_id: client.clientId,
|
||||
token: client.token,
|
||||
status: "active",
|
||||
rate_limit: 100,
|
||||
storage_quota_mb: 0,
|
||||
allowed_ips: mockAllowedIps,
|
||||
expires_at: mockExpiresAt,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
return { rowCount: 0, rows: [] };
|
||||
},
|
||||
});
|
||||
|
||||
// Sync expired client to Port 4000 in-memory cache and verify verifyClientInDb returns CLIENT_TOKEN_EXPIRED
|
||||
await new Promise<void>((resolve) =>
|
||||
ingestionApp.server.listen(0, "127.0.0.1", () => resolve()),
|
||||
const expiredCheck = await serverMod.verifyClientInDb(
|
||||
client.clientId,
|
||||
client.token,
|
||||
true,
|
||||
"10.10.42.9",
|
||||
);
|
||||
const port4000 = (ingestionApp.server.address() as AddressInfo).port;
|
||||
assert.equal(expiredCheck.allowed, false);
|
||||
assert.equal(expiredCheck.code, "CLIENT_TOKEN_EXPIRED");
|
||||
|
||||
try {
|
||||
const expiredCheck = await ingestionApp.verifyClientInDb(
|
||||
client.clientId,
|
||||
client.token,
|
||||
true,
|
||||
"10.10.42.9",
|
||||
);
|
||||
assert.equal(expiredCheck.ok, false);
|
||||
assert.equal(expiredCheck.reason, "CLIENT_TOKEN_EXPIRED");
|
||||
// Clear expiration, keep IP allowlist 10.10.0.0/16, and check 127.0.0.1
|
||||
mockExpiresAt = null;
|
||||
const ipForbiddenCheck = await serverMod.verifyClientInDb(
|
||||
client.clientId,
|
||||
client.token,
|
||||
true,
|
||||
"127.0.0.1",
|
||||
);
|
||||
assert.equal(ipForbiddenCheck.allowed, false);
|
||||
assert.equal(ipForbiddenCheck.code, "CLIENT_IP_FORBIDDEN");
|
||||
|
||||
// Clear expiration via PATCH, keep IP allowlist 10.10.0.0/16, and check Socket.IO handshake from 127.0.0.1
|
||||
await patchClientRoute(
|
||||
new NextRequest(
|
||||
`http://localhost:3000/api/clients/${encodeURIComponent(client.clientId)}`,
|
||||
{
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
cookie: adminCookie,
|
||||
"content-type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
expiresAt: null,
|
||||
}),
|
||||
},
|
||||
),
|
||||
{ params: Promise.resolve({ id: client.clientId }) },
|
||||
);
|
||||
// Start Port 4000 server on ephemeral port and verify Socket.IO handshake from 127.0.0.1 is rejected with CLIENT_IP_FORBIDDEN
|
||||
await new Promise<void>((resolve) =>
|
||||
serverMod.server.listen(0, "127.0.0.1", () => resolve()),
|
||||
);
|
||||
const port4000 = (serverMod.server.address() as AddressInfo).port;
|
||||
|
||||
const ipForbiddenCheck = await ingestionApp.verifyClientInDb(
|
||||
client.clientId,
|
||||
client.token,
|
||||
true,
|
||||
"127.0.0.1",
|
||||
);
|
||||
assert.equal(ipForbiddenCheck.ok, false);
|
||||
assert.equal(ipForbiddenCheck.reason, "CLIENT_IP_FORBIDDEN");
|
||||
|
||||
// Socket.IO connection from 127.0.0.1 should fail handshake with CLIENT_IP_FORBIDDEN
|
||||
const socketError = await new Promise<string>((resolve) => {
|
||||
const sock: Socket = ioClient(`http://127.0.0.1:${port4000}`, {
|
||||
auth: { clientId: client.clientId, token: client.token },
|
||||
transports: ["websocket"],
|
||||
reconnection: false,
|
||||
});
|
||||
sock.on("connect_error", (err) => {
|
||||
sock.close();
|
||||
resolve(err.message);
|
||||
});
|
||||
sock.on("connect", () => {
|
||||
sock.close();
|
||||
resolve("CONNECTED_UNEXPECTEDLY");
|
||||
});
|
||||
const socketError = await new Promise<string>((resolve) => {
|
||||
const sock: Socket = ioClient(`http://127.0.0.1:${port4000}`, {
|
||||
auth: { clientId: client.clientId, token: client.token },
|
||||
transports: ["websocket"],
|
||||
reconnection: false,
|
||||
timeout: 3000,
|
||||
});
|
||||
assert.ok(socketError.includes("CLIENT_IP_FORBIDDEN"));
|
||||
} finally {
|
||||
await new Promise<void>((resolve) =>
|
||||
ingestionApp.server.close(() => resolve()),
|
||||
);
|
||||
}
|
||||
openSockets.push(sock);
|
||||
sock.on("connect_error", (err) => {
|
||||
sock.disconnect();
|
||||
resolve(err.message);
|
||||
});
|
||||
sock.on("connect", () => {
|
||||
sock.disconnect();
|
||||
resolve("CONNECTED_UNEXPECTEDLY");
|
||||
});
|
||||
});
|
||||
assert.ok(socketError.includes("CLIENT_IP_FORBIDDEN"));
|
||||
} finally {
|
||||
await DataBufferEngine.flushAllAsync();
|
||||
SDKManager.stopActivityFlushTimerForTests();
|
||||
if (prevStorageDir === undefined) {
|
||||
delete process.env.DATASTACK_STORAGE_DIR;
|
||||
} else {
|
||||
process.env.DATASTACK_STORAGE_DIR = prevStorageDir;
|
||||
for (const s of openSockets) {
|
||||
try {
|
||||
s.disconnect();
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
CSVStorageManager.resetForTests();
|
||||
serverMod.writer.baseDir = origWriterBaseDir;
|
||||
serverMod.writer.clearQuotaCaches();
|
||||
serverMod.setDbPoolForTests(null);
|
||||
serverMod.stopTimers();
|
||||
await new Promise<void>((resolve) => {
|
||||
serverMod.io.close(() => {
|
||||
if (serverMod.server.listening) {
|
||||
serverMod.server.close(() => resolve());
|
||||
} else {
|
||||
resolve();
|
||||
}
|
||||
});
|
||||
});
|
||||
SDKManager.stopActivityFlushTimerForTests();
|
||||
CSVStorageManager.clearQuotaCaches();
|
||||
await DataBufferEngine.shutdown();
|
||||
CSVStorageManager.baseStorageDir = origBaseStorageDir;
|
||||
if (prevStorageDir === undefined) delete process.env.DATASTACK_STORAGE_DIR;
|
||||
else process.env.DATASTACK_STORAGE_DIR = prevStorageDir;
|
||||
if (prevDisableListen === undefined) delete process.env.DATASTACK_DISABLE_LISTEN;
|
||||
else process.env.DATASTACK_DISABLE_LISTEN = prevDisableListen;
|
||||
if (prevDbUrl === undefined) delete process.env.DATABASE_URL;
|
||||
else process.env.DATABASE_URL = prevDbUrl;
|
||||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||||
resetAdminStateForTests();
|
||||
}
|
||||
});
|
||||
|
||||
test("V2-Phase-02 [Prometheus Metrics & NDJSON Streaming Export]: GET /api/prometheus (V2-FEAT-04 / V2-E09) & GET /api/export?format=ndjson (V2-FEAT-06)", async () => {
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), "datastack-v2-p2-prom-"));
|
||||
const prevStorageDir = process.env.DATASTACK_STORAGE_DIR;
|
||||
const prevDbUrl = process.env.DATABASE_URL;
|
||||
process.env.DATASTACK_STORAGE_DIR = tmpDir;
|
||||
CSVStorageManager.resetForTests(tmpDir);
|
||||
await CSVStorageManager.initializeAsync();
|
||||
delete process.env.DATABASE_URL;
|
||||
|
||||
const origBaseStorageDir = CSVStorageManager.baseStorageDir;
|
||||
CSVStorageManager.baseStorageDir = tmpDir;
|
||||
CSVStorageManager.clearQuotaCaches();
|
||||
|
||||
try {
|
||||
const adminCookie = await makeAdminCookie();
|
||||
@@ -546,7 +605,7 @@ test("V2-Phase-02 [Prometheus Metrics & NDJSON Streaming Export]: GET /api/prome
|
||||
assert.ok(metricsBody.includes("datastack_disk_used_percent"));
|
||||
|
||||
// 5. Seed events and export as streaming NDJSON (GET /api/export?format=ndjson) (V2-FEAT-06)
|
||||
await CSVStorageManager.appendBatchAsync("client_ndjson", [
|
||||
await CSVStorageManager.appendBatch([
|
||||
{
|
||||
clientId: "client_ndjson",
|
||||
eventId: "evt_nd_1",
|
||||
@@ -575,10 +634,10 @@ test("V2-Phase-02 [Prometheus Metrics & NDJSON Streaming Export]: GET /api/prome
|
||||
.get("content-type")
|
||||
?.includes("application/x-ndjson"),
|
||||
);
|
||||
const disposition = ndjsonRes.headers.get("content-disposition") || "";
|
||||
assert.ok(
|
||||
ndjsonRes.headers
|
||||
.get("content-disposition")
|
||||
?.includes("datastack-client_ndjson.ndjson"),
|
||||
disposition.includes("client_ndjson_export_") &&
|
||||
disposition.endsWith('.ndjson"'),
|
||||
);
|
||||
const ndjsonText = await ndjsonRes.text();
|
||||
const lines = ndjsonText
|
||||
@@ -610,8 +669,6 @@ test("V2-Phase-02 [Prometheus Metrics & NDJSON Streaming Export]: GET /api/prome
|
||||
DEFAULT_ALERTING_SETTINGS,
|
||||
);
|
||||
} finally {
|
||||
await DataBufferEngine.flushAllAsync();
|
||||
SDKManager.stopActivityFlushTimerForTests();
|
||||
await SettingsManager.updateSettingsAsync({
|
||||
enableWebhookAlerts: false,
|
||||
webhookUrl: "",
|
||||
@@ -620,12 +677,15 @@ test("V2-Phase-02 [Prometheus Metrics & NDJSON Streaming Export]: GET /api/prome
|
||||
enablePrometheusMetrics: false,
|
||||
clearPrometheusToken: true,
|
||||
});
|
||||
if (prevStorageDir === undefined) {
|
||||
delete process.env.DATASTACK_STORAGE_DIR;
|
||||
} else {
|
||||
process.env.DATASTACK_STORAGE_DIR = prevStorageDir;
|
||||
}
|
||||
CSVStorageManager.resetForTests();
|
||||
SDKManager.stopActivityFlushTimerForTests();
|
||||
await DataBufferEngine.shutdown();
|
||||
CSVStorageManager.clearQuotaCaches();
|
||||
CSVStorageManager.baseStorageDir = origBaseStorageDir;
|
||||
if (prevStorageDir === undefined) delete process.env.DATASTACK_STORAGE_DIR;
|
||||
else process.env.DATASTACK_STORAGE_DIR = prevStorageDir;
|
||||
if (prevDbUrl === undefined) delete process.env.DATABASE_URL;
|
||||
else process.env.DATABASE_URL = prevDbUrl;
|
||||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||||
resetAdminStateForTests();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,621 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { after, beforeEach, describe, it } from "node:test";
|
||||
import { NextRequest } from "next/server";
|
||||
import { GET as auditGet } from "@/app/api/audit/route";
|
||||
import { POST as loginPost } from "@/app/api/auth/login/route";
|
||||
import { GET as profileGet, PUT as profilePut } from "@/app/api/auth/profile/route";
|
||||
import {
|
||||
DELETE as clientDelete,
|
||||
GET as clientGet,
|
||||
PATCH as clientPatch,
|
||||
} from "@/app/api/clients/[id]/route";
|
||||
import { GET as clientsGet, POST as clientsPost } from "@/app/api/clients/route";
|
||||
import { GET as settingsGet, POST as settingsPost } from "@/app/api/settings/route";
|
||||
import { POST as webhookTestPost } from "@/app/api/settings/webhook-test/route";
|
||||
import { GET as storageGet, POST as storagePost } from "@/app/api/storage/route";
|
||||
import {
|
||||
DELETE as teamMemberDelete,
|
||||
PATCH as teamMemberPatch,
|
||||
} from "@/app/api/team/[id]/route";
|
||||
import { GET as teamGet, POST as teamPost } from "@/app/api/team/route";
|
||||
import {
|
||||
createAdminSessionToken,
|
||||
getAdminSessionSecret,
|
||||
} from "@/core/auth/adminSession";
|
||||
import {
|
||||
createInitialAdmin,
|
||||
resetAdminStateForTests,
|
||||
} from "@/core/auth/adminUsers";
|
||||
import { DataBufferEngine } from "@/core/data/buffer";
|
||||
|
||||
const ORIGINAL_ENV = { ...process.env };
|
||||
|
||||
function restoreEnv() {
|
||||
for (const key of Object.keys(process.env)) {
|
||||
if (!(key in ORIGINAL_ENV)) {
|
||||
delete process.env[key];
|
||||
}
|
||||
}
|
||||
Object.assign(process.env, ORIGINAL_ENV);
|
||||
}
|
||||
|
||||
describe("V2-Phase-03: Multi-Admin RBAC & Immutable Admin Audit Log", () => {
|
||||
beforeEach(async () => {
|
||||
restoreEnv();
|
||||
delete process.env.DATABASE_URL;
|
||||
process.env.DATASTACK_ALLOW_DEMO_ADMIN = "false";
|
||||
process.env.INGESTION_INTERNAL_URL = "http://127.0.0.1:1";
|
||||
resetAdminStateForTests();
|
||||
const g = globalThis as {
|
||||
__dataStackClientsMap?: Map<string, unknown>;
|
||||
__dataStackDeletedClients?: Set<string>;
|
||||
};
|
||||
g.__dataStackClientsMap?.clear();
|
||||
g.__dataStackDeletedClients?.clear();
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
restoreEnv();
|
||||
resetAdminStateForTests();
|
||||
await DataBufferEngine.shutdown();
|
||||
});
|
||||
|
||||
it("1. Enforces Owner, Admin/Operator, and Viewer RBAC matrix (V2-E10 INSUFFICIENT_ROLE_PERMISSION)", async () => {
|
||||
const createdOwner = await createInitialAdmin(
|
||||
"owner@datastack.dev",
|
||||
"OwnerPass123!",
|
||||
);
|
||||
assert.equal(createdOwner, true);
|
||||
const secret = getAdminSessionSecret()!;
|
||||
|
||||
// Owner session cookie (backward-compatible 4-arg call defaults to owner)
|
||||
const ownerCookie = await createAdminSessionToken(
|
||||
"owner@datastack.dev",
|
||||
secret,
|
||||
Date.now() + 3600_000,
|
||||
1,
|
||||
);
|
||||
|
||||
// Owner creates an Admin/Operator team member and a Viewer team member
|
||||
const createAdminRes = await teamPost(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "operator@datastack.dev",
|
||||
password: "OperatorPass123!",
|
||||
role: "admin",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(createAdminRes.status, 201);
|
||||
const createdAdminMember = (await createAdminRes.json()).member;
|
||||
assert.equal(createdAdminMember.role, "admin");
|
||||
|
||||
const createViewerRes = await teamPost(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "viewer@datastack.dev",
|
||||
password: "ViewerPass123!",
|
||||
role: "viewer",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(createViewerRes.status, 201);
|
||||
const createdViewerMember = (await createViewerRes.json()).member;
|
||||
assert.equal(createdViewerMember.role, "viewer");
|
||||
|
||||
// Sign in via POST /api/auth/login as Operator and Viewer
|
||||
const operatorLoginRes = await loginPost(
|
||||
new NextRequest("http://localhost:3000/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
email: "operator@datastack.dev",
|
||||
password: "OperatorPass123!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(operatorLoginRes.status, 200);
|
||||
const operatorCookie =
|
||||
operatorLoginRes.cookies.get("datastack_admin_session")?.value;
|
||||
assert.ok(operatorCookie);
|
||||
|
||||
const viewerLoginRes = await loginPost(
|
||||
new NextRequest("http://localhost:3000/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
email: "viewer@datastack.dev",
|
||||
password: "ViewerPass123!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerLoginRes.status, 200);
|
||||
const viewerCookie =
|
||||
viewerLoginRes.cookies.get("datastack_admin_session")?.value;
|
||||
assert.ok(viewerCookie);
|
||||
|
||||
// Check /api/auth/profile reflects each member's role
|
||||
const viewerProfileRes = await profileGet(
|
||||
new NextRequest("http://localhost:3000/api/auth/profile", {
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerProfileRes.status, 200);
|
||||
const viewerProfile = (await viewerProfileRes.json()).profile;
|
||||
assert.equal(viewerProfile.role, "Viewer");
|
||||
assert.equal(viewerProfile.roleKey, "viewer");
|
||||
|
||||
// Viewer can read GET /api/clients, GET /api/settings, GET /api/storage, GET /api/team
|
||||
const viewerGetClients = await clientsGet(
|
||||
new NextRequest("http://localhost:3000/api/clients", {
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerGetClients.status, 200);
|
||||
|
||||
const viewerGetSettings = await settingsGet(
|
||||
new NextRequest("http://localhost:3000/api/settings", {
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerGetSettings.status, 200);
|
||||
|
||||
const viewerGetStorage = await storageGet(
|
||||
new NextRequest("http://localhost:3000/api/storage", {
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerGetStorage.status, 200);
|
||||
|
||||
// Viewer CANNOT mutate settings, clients, storage, or team -> 403 INSUFFICIENT_ROLE_PERMISSION
|
||||
const viewerPostSettings = await settingsPost(
|
||||
new NextRequest("http://localhost:3000/api/settings", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${viewerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ maxEventsPerSec: 250 }),
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerPostSettings.status, 403);
|
||||
assert.equal(
|
||||
(await viewerPostSettings.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
const viewerPostClients = await clientsPost(
|
||||
new NextRequest("http://localhost:3000/api/clients", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${viewerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
clientId: "viewer_blocked_client",
|
||||
name: "Blocked",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerPostClients.status, 403);
|
||||
assert.equal(
|
||||
(await viewerPostClients.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
const viewerPostStorage = await storagePost(
|
||||
new NextRequest("http://localhost:3000/api/storage", {
|
||||
method: "POST",
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(viewerPostStorage.status, 403);
|
||||
assert.equal(
|
||||
(await viewerPostStorage.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
// Operator CAN create, update, rotate token, and delete SDK clients
|
||||
const operatorCreateClient = await clientsPost(
|
||||
new NextRequest("http://localhost:3000/api/clients", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${operatorCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
clientId: "op_managed_client",
|
||||
name: "Operator Client",
|
||||
rateLimit: 300,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(operatorCreateClient.status, 201);
|
||||
|
||||
// Viewer CANNOT reveal full client JWT via ?includeToken=true
|
||||
const viewerRevealToken = await clientGet(
|
||||
new NextRequest(
|
||||
"http://localhost:3000/api/clients/op_managed_client?includeToken=true",
|
||||
{
|
||||
headers: { cookie: `datastack_admin_session=${viewerCookie}` },
|
||||
},
|
||||
),
|
||||
{ params: Promise.resolve({ id: "op_managed_client" }) },
|
||||
);
|
||||
assert.equal(viewerRevealToken.status, 403);
|
||||
assert.equal(
|
||||
(await viewerRevealToken.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
// Operator CAN reveal full client JWT and rotate it
|
||||
const operatorRotateToken = await clientPatch(
|
||||
new NextRequest("http://localhost:3000/api/clients/op_managed_client", {
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${operatorCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ action: "regenerate_token" }),
|
||||
}),
|
||||
{ params: Promise.resolve({ id: "op_managed_client" }) },
|
||||
);
|
||||
assert.equal(operatorRotateToken.status, 200);
|
||||
|
||||
// Operator CANNOT modify platform settings or manage team members (Owner-only)
|
||||
const operatorPostSettings = await settingsPost(
|
||||
new NextRequest("http://localhost:3000/api/settings", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${operatorCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ maxEventsPerSec: 500 }),
|
||||
}),
|
||||
);
|
||||
assert.equal(operatorPostSettings.status, 403);
|
||||
assert.equal(
|
||||
(await operatorPostSettings.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
const operatorWebhookTest = await webhookTestPost(
|
||||
new NextRequest("http://localhost:3000/api/settings/webhook-test", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${operatorCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ webhookUrl: "https://hooks.example.com/test" }),
|
||||
}),
|
||||
);
|
||||
assert.equal(operatorWebhookTest.status, 403);
|
||||
assert.equal(
|
||||
(await operatorWebhookTest.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
|
||||
const operatorPostTeam = await teamPost(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${operatorCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "unauthorized@datastack.dev",
|
||||
password: "Password123!",
|
||||
role: "viewer",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(operatorPostTeam.status, 403);
|
||||
assert.equal(
|
||||
(await operatorPostTeam.json()).code,
|
||||
"INSUFFICIENT_ROLE_PERMISSION",
|
||||
);
|
||||
});
|
||||
|
||||
it("2. Protects Primary Owner (id=1 / owner-1) from deletion or demotion (V2-E11 CANNOT_REVOKE_PRIMARY_OWNER) and revokes team sessions on role/status change", async () => {
|
||||
await createInitialAdmin("primary@datastack.dev", "PrimaryPass123!");
|
||||
const secret = getAdminSessionSecret()!;
|
||||
const ownerCookie = await createAdminSessionToken(
|
||||
"primary@datastack.dev",
|
||||
secret,
|
||||
Date.now() + 3600_000,
|
||||
1,
|
||||
);
|
||||
|
||||
// Attempt to delete primary owner -> 400 CANNOT_REVOKE_PRIMARY_OWNER
|
||||
const delOwnerRes = await teamMemberDelete(
|
||||
new NextRequest("http://localhost:3000/api/team/owner-1", {
|
||||
method: "DELETE",
|
||||
headers: { cookie: `datastack_admin_session=${ownerCookie}` },
|
||||
}),
|
||||
{ params: Promise.resolve({ id: "owner-1" }) },
|
||||
);
|
||||
assert.equal(delOwnerRes.status, 400);
|
||||
assert.equal(
|
||||
(await delOwnerRes.json()).code,
|
||||
"CANNOT_REVOKE_PRIMARY_OWNER",
|
||||
);
|
||||
|
||||
// Attempt to demote or disable primary owner -> 400 CANNOT_REVOKE_PRIMARY_OWNER
|
||||
const patchOwnerRes = await teamMemberPatch(
|
||||
new NextRequest("http://localhost:3000/api/team/owner-1", {
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ role: "viewer", status: "disabled" }),
|
||||
}),
|
||||
{ params: Promise.resolve({ id: "owner-1" }) },
|
||||
);
|
||||
assert.equal(patchOwnerRes.status, 400);
|
||||
assert.equal(
|
||||
(await patchOwnerRes.json()).code,
|
||||
"CANNOT_REVOKE_PRIMARY_OWNER",
|
||||
);
|
||||
|
||||
// Create a secondary member, log in, then disable them and verify session cookie is immediately invalidated
|
||||
const createdRes = await teamPost(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "temp@datastack.dev",
|
||||
password: "TempPassword123!",
|
||||
role: "viewer",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(createdRes.status, 201);
|
||||
const memberId = (await createdRes.json()).member.id as string;
|
||||
|
||||
const loginRes = await loginPost(
|
||||
new NextRequest("http://localhost:3000/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
email: "temp@datastack.dev",
|
||||
password: "TempPassword123!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(loginRes.status, 200);
|
||||
const tempCookie = loginRes.cookies.get("datastack_admin_session")?.value;
|
||||
assert.ok(tempCookie);
|
||||
|
||||
// Disable team member -> bumps token_version and sets status='disabled'
|
||||
const disableRes = await teamMemberPatch(
|
||||
new NextRequest(`http://localhost:3000/api/team/${memberId}`, {
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ status: "disabled", revokeSessions: true }),
|
||||
}),
|
||||
{ params: Promise.resolve({ id: memberId }) },
|
||||
);
|
||||
assert.equal(disableRes.status, 200);
|
||||
|
||||
// Old cookie must now be rejected 401
|
||||
const afterDisableCheck = await clientsGet(
|
||||
new NextRequest("http://localhost:3000/api/clients", {
|
||||
headers: { cookie: `datastack_admin_session=${tempCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(afterDisableCheck.status, 401);
|
||||
});
|
||||
|
||||
it("3. Records immutable admin audit logs across login, team, client, and settings mutations and supports filtered GET /api/audit queries", async () => {
|
||||
await createInitialAdmin("audit-owner@datastack.dev", "AuditPass123!");
|
||||
const loginRes = await loginPost(
|
||||
new NextRequest("http://localhost:3000/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
"x-real-ip": "203.0.113.77",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "audit-owner@datastack.dev",
|
||||
password: "AuditPass123!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(loginRes.status, 200);
|
||||
const ownerCookie = loginRes.cookies.get("datastack_admin_session")?.value!;
|
||||
|
||||
// Create a client, update settings, and delete the client
|
||||
const createClientRes = await clientsPost(
|
||||
new NextRequest("http://localhost:3000/api/clients", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
"x-real-ip": "203.0.113.77",
|
||||
},
|
||||
body: JSON.stringify({
|
||||
clientId: "audited_sdk_client",
|
||||
name: "Audited Client",
|
||||
rateLimit: 120,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(createClientRes.status, 201);
|
||||
|
||||
const updateSettingsRes = await settingsPost(
|
||||
new NextRequest("http://localhost:3000/api/settings", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
"x-real-ip": "203.0.113.77",
|
||||
},
|
||||
body: JSON.stringify({ maxEventsPerSec: 400 }),
|
||||
}),
|
||||
);
|
||||
assert.equal(updateSettingsRes.status, 200);
|
||||
|
||||
const deleteClientRes = await clientDelete(
|
||||
new NextRequest(
|
||||
"http://localhost:3000/api/clients/audited_sdk_client?purgeFiles=false",
|
||||
{
|
||||
method: "DELETE",
|
||||
headers: {
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
"x-real-ip": "203.0.113.77",
|
||||
},
|
||||
},
|
||||
),
|
||||
{ params: Promise.resolve({ id: "audited_sdk_client" }) },
|
||||
);
|
||||
assert.equal(deleteClientRes.status, 200);
|
||||
|
||||
// Query GET /api/audit
|
||||
const auditRes = await auditGet(
|
||||
new NextRequest("http://localhost:3000/api/audit?limit=50", {
|
||||
headers: { cookie: `datastack_admin_session=${ownerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(auditRes.status, 200);
|
||||
const auditJson = await auditRes.json();
|
||||
assert.ok(Array.isArray(auditJson.logs));
|
||||
const actions = auditJson.logs.map((l: { action: string }) => l.action);
|
||||
assert.ok(actions.includes("ADMIN_LOGIN"));
|
||||
assert.ok(actions.includes("CLIENT_CREATED"));
|
||||
assert.ok(actions.includes("SETTINGS_UPDATED"));
|
||||
assert.ok(actions.includes("CLIENT_DELETED"));
|
||||
|
||||
// Filter by action=CLIENT_CREATED
|
||||
const filteredRes = await auditGet(
|
||||
new NextRequest(
|
||||
"http://localhost:3000/api/audit?action=CLIENT_CREATED&q=audited_sdk_client",
|
||||
{
|
||||
headers: { cookie: `datastack_admin_session=${ownerCookie}` },
|
||||
},
|
||||
),
|
||||
);
|
||||
assert.equal(filteredRes.status, 200);
|
||||
const filteredJson = await filteredRes.json();
|
||||
assert.equal(filteredJson.logs.length, 1);
|
||||
assert.equal(filteredJson.logs[0].target, "audited_sdk_client");
|
||||
assert.equal(filteredJson.logs[0].ipAddress, "203.0.113.77");
|
||||
assert.equal(filteredJson.logs[0].actorEmail, "audit-owner@datastack.dev");
|
||||
|
||||
// Team listing includes primary owner + team members
|
||||
const teamListRes = await teamGet(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
headers: { cookie: `datastack_admin_session=${ownerCookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(teamListRes.status, 200);
|
||||
const teamListJson = await teamListRes.json();
|
||||
assert.equal(teamListJson.members[0].id, "owner-1");
|
||||
assert.equal(teamListJson.members[0].isPrimaryOwner, true);
|
||||
|
||||
// Restore default settings before shutdown
|
||||
await settingsPost(
|
||||
new NextRequest("http://localhost:3000/api/settings", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({ maxEventsPerSec: 100 }),
|
||||
}),
|
||||
);
|
||||
await DataBufferEngine.shutdown();
|
||||
});
|
||||
|
||||
it("4. Allows secondary team member to rotate their own password via PUT /api/auth/profile and invalidates prior cookie", async () => {
|
||||
await createInitialAdmin("root@datastack.dev", "RootPassword123!");
|
||||
const secret = getAdminSessionSecret()!;
|
||||
const ownerCookie = await createAdminSessionToken(
|
||||
"root@datastack.dev",
|
||||
secret,
|
||||
Date.now() + 3600_000,
|
||||
1,
|
||||
);
|
||||
|
||||
await teamPost(
|
||||
new NextRequest("http://localhost:3000/api/team", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${ownerCookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "selfupdate@datastack.dev",
|
||||
password: "InitialPassword123!",
|
||||
role: "operator",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
|
||||
const loginRes = await loginPost(
|
||||
new NextRequest("http://localhost:3000/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
email: "selfupdate@datastack.dev",
|
||||
password: "InitialPassword123!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(loginRes.status, 200);
|
||||
const v1Cookie = loginRes.cookies.get("datastack_admin_session")?.value!;
|
||||
|
||||
const rotateRes = await profilePut(
|
||||
new NextRequest("http://localhost:3000/api/auth/profile", {
|
||||
method: "PUT",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
cookie: `datastack_admin_session=${v1Cookie}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: "selfupdate@datastack.dev",
|
||||
currentPassword: "InitialPassword123!",
|
||||
newPassword: "RotatedPassword456!",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
assert.equal(rotateRes.status, 200);
|
||||
const v2Cookie = rotateRes.cookies.get("datastack_admin_session")?.value!;
|
||||
assert.ok(v2Cookie);
|
||||
|
||||
// Old v1 cookie is now revoked
|
||||
const staleRes = await profileGet(
|
||||
new NextRequest("http://localhost:3000/api/auth/profile", {
|
||||
headers: { cookie: `datastack_admin_session=${v1Cookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(staleRes.status, 401);
|
||||
|
||||
// New v2 cookie retains Operator role
|
||||
const freshRes = await profileGet(
|
||||
new NextRequest("http://localhost:3000/api/auth/profile", {
|
||||
headers: { cookie: `datastack_admin_session=${v2Cookie}` },
|
||||
}),
|
||||
);
|
||||
assert.equal(freshRes.status, 200);
|
||||
const freshJson = await freshRes.json();
|
||||
assert.equal(freshJson.profile.role, "Operator");
|
||||
assert.equal(freshJson.profile.roleKey, "operator");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,398 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import {
|
||||
DataStack,
|
||||
DataStackAuthError,
|
||||
DataStackConfigError,
|
||||
DataStackFileError,
|
||||
DataStackQuotaExceededError,
|
||||
SDK_VERSION,
|
||||
makeDeterministicFileEventId,
|
||||
} from '../sdk/node/datastack';
|
||||
|
||||
function makeFakeJwt(clientId: string): string {
|
||||
const header = Buffer.from(JSON.stringify({ alg: 'HS256', typ: 'JWT' })).toString('base64url');
|
||||
const payload = Buffer.from(JSON.stringify({ clientId, iat: 1700000000 })).toString('base64url');
|
||||
return `${header}.${payload}.fakesignature`;
|
||||
}
|
||||
|
||||
test('V2-Phase-04 (1): Node.js / TypeScript SDK packaging metadata, JWT clientId verification, and V2-E12 config validation', async () => {
|
||||
const pkgRaw = await fs.readFile(
|
||||
path.join(process.cwd(), 'sdk', 'node', 'datastack', 'package.json'),
|
||||
'utf8'
|
||||
);
|
||||
const pkg = JSON.parse(pkgRaw) as {
|
||||
name: string;
|
||||
version: string;
|
||||
engines?: { node?: string };
|
||||
};
|
||||
assert.equal(pkg.name, '@datastack/sdk');
|
||||
assert.equal(pkg.version, '2.2.0');
|
||||
assert.equal(SDK_VERSION, '2.2.0');
|
||||
assert.equal(pkg.engines?.node, '>=20.0.0');
|
||||
|
||||
const token = makeFakeJwt('billing_service');
|
||||
|
||||
// Mismatched clientId vs JWT token claim must throw DataStackConfigError (V2-E12)
|
||||
assert.throws(
|
||||
() =>
|
||||
new DataStack({
|
||||
serverUrl: 'http://localhost:4000',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
clientId: 'different_service',
|
||||
autoConnect: false,
|
||||
}),
|
||||
(err: unknown) =>
|
||||
err instanceof DataStackConfigError &&
|
||||
err.code === 'INVALID_SDK_CONFIG' &&
|
||||
/does not match JWT token subject/i.test(err.message)
|
||||
);
|
||||
|
||||
// Invalid serverUrl scheme must throw DataStackConfigError
|
||||
assert.throws(
|
||||
() =>
|
||||
new DataStack({
|
||||
serverUrl: 'ftp://invalid-endpoint',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
autoConnect: false,
|
||||
}),
|
||||
(err: unknown) => err instanceof DataStackConfigError
|
||||
);
|
||||
|
||||
// Matching normalized clientId succeeds
|
||||
const client = new DataStack({
|
||||
serverUrl: 'http://localhost:4000',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
clientId: ' BILLING_SERVICE ',
|
||||
autoConnect: false,
|
||||
offlineSpoolEnabled: false,
|
||||
});
|
||||
assert.equal(client.clientId, 'billing_service');
|
||||
|
||||
// Empty payload after metadata stripping throws DataStackConfigError
|
||||
await assert.rejects(
|
||||
() => client.send({ eventId: 'evt_only_meta', timestamp: '2026-11-20T00:00:00Z' }),
|
||||
(err: unknown) => err instanceof DataStackConfigError
|
||||
);
|
||||
});
|
||||
|
||||
test('V2-Phase-04 (2): HTTP fallback single/batch send, recursive 413 batch splitting, 429 partial retry, and terminal V2-E12 error mapping', async () => {
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ds-node-sdk-http-'));
|
||||
try {
|
||||
const token = makeFakeJwt('checkout_api');
|
||||
const calls: Array<{ body: Record<string, unknown> }> = [];
|
||||
let simulateMode:
|
||||
| 'ok'
|
||||
| 'split_413'
|
||||
| 'partial_429'
|
||||
| 'expired_401'
|
||||
| 'ip_403'
|
||||
| 'quota_507' = 'ok';
|
||||
let partial429Count = 0;
|
||||
|
||||
const mockFetch: typeof fetch = async (_input, init) => {
|
||||
const body = JSON.parse(String(init?.body || '{}')) as Record<string, unknown>;
|
||||
calls.push({ body });
|
||||
|
||||
if (simulateMode === 'expired_401') {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
code: 'CLIENT_TOKEN_EXPIRED',
|
||||
error: 'Client credential has expired',
|
||||
}),
|
||||
{ status: 401, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
|
||||
if (simulateMode === 'ip_403') {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
code: 'CLIENT_IP_FORBIDDEN',
|
||||
error: 'Client IP not in allowlist',
|
||||
}),
|
||||
{ status: 403, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
|
||||
if (simulateMode === 'quota_507') {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
code: 'STORAGE_QUOTA_EXCEEDED',
|
||||
error: 'Client storage quota exceeded',
|
||||
}),
|
||||
{ status: 507, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
|
||||
const batch = Array.isArray(body.batch)
|
||||
? (body.batch as Array<{ eventId: string; data: Record<string, unknown> }>)
|
||||
: null;
|
||||
|
||||
if (simulateMode === 'split_413' && batch && batch.length > 2) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
code: 'PAYLOAD_TOO_LARGE',
|
||||
error: 'Batch exceeds maxPayloadSizeKb',
|
||||
}),
|
||||
{ status: 413, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
|
||||
if (simulateMode === 'partial_429' && batch && partial429Count === 0) {
|
||||
partial429Count += 1;
|
||||
const rateLimitedIds = [batch[batch.length - 1].eventId];
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
accepted: batch.length - 1,
|
||||
rateLimitedEventIds: rateLimitedIds,
|
||||
}),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
eventId: body.eventId || 'evt_ok',
|
||||
accepted: batch ? batch.length : 1,
|
||||
}),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
};
|
||||
|
||||
const client = new DataStack({
|
||||
serverUrl: 'http://localhost:4000',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
autoConnect: false,
|
||||
offlineSpoolDir: tmpDir,
|
||||
fetchImpl: mockFetch,
|
||||
});
|
||||
|
||||
// 1. Single event via HTTP fallback
|
||||
const singleRes = await client.send({
|
||||
event_type: 'payment_authorized',
|
||||
amount: 99.5,
|
||||
});
|
||||
assert.equal(singleRes.success, true);
|
||||
assert.equal(singleRes.transport, 'http');
|
||||
assert.equal(singleRes.accepted, 1);
|
||||
|
||||
// 2. Recursive 413 PAYLOAD_TOO_LARGE splitting (4 items -> split into 2 + 2)
|
||||
simulateMode = 'split_413';
|
||||
const splitRes = await client.sendBatch([
|
||||
{ seq: 1 },
|
||||
{ seq: 2 },
|
||||
{ seq: 3 },
|
||||
{ seq: 4 },
|
||||
]);
|
||||
assert.equal(splitRes.success, true);
|
||||
assert.equal(splitRes.accepted, 4);
|
||||
|
||||
// 3. Partial 429 RATE_LIMIT_EXCEEDED retry
|
||||
simulateMode = 'partial_429';
|
||||
const retryRes = await client.sendBatch([
|
||||
{ item: 'a' },
|
||||
{ item: 'b' },
|
||||
{ item: 'c' },
|
||||
]);
|
||||
assert.equal(retryRes.success, true);
|
||||
assert.equal(retryRes.accepted, 3);
|
||||
|
||||
// 4. Terminal error mapping (401 CLIENT_TOKEN_EXPIRED, 403 CLIENT_IP_FORBIDDEN, 507 STORAGE_QUOTA_EXCEEDED)
|
||||
simulateMode = 'expired_401';
|
||||
await assert.rejects(
|
||||
() => client.send({ test: 'expired' }),
|
||||
(err: unknown) =>
|
||||
err instanceof DataStackAuthError &&
|
||||
err.code === 'CLIENT_TOKEN_EXPIRED' &&
|
||||
err.statusCode === 401
|
||||
);
|
||||
|
||||
simulateMode = 'ip_403';
|
||||
await assert.rejects(
|
||||
() => client.send({ test: 'ip_blocked' }),
|
||||
(err: unknown) =>
|
||||
err instanceof DataStackAuthError &&
|
||||
err.code === 'CLIENT_IP_FORBIDDEN' &&
|
||||
err.statusCode === 403
|
||||
);
|
||||
|
||||
simulateMode = 'quota_507';
|
||||
await assert.rejects(
|
||||
() => client.send({ test: 'quota_full' }),
|
||||
(err: unknown) =>
|
||||
err instanceof DataStackQuotaExceededError &&
|
||||
err.code === 'STORAGE_QUOTA_EXCEEDED' &&
|
||||
err.statusCode === 507
|
||||
);
|
||||
} finally {
|
||||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('V2-Phase-04 (3): Offline disk spooling (.datastack_spool/spool.jsonl), flushSpool replay, and graceful shutdown drain', async () => {
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ds-node-sdk-spool-'));
|
||||
try {
|
||||
const token = makeFakeJwt('iot_gateway');
|
||||
let networkOnline = false;
|
||||
const deliveredEvents: Array<Record<string, unknown>> = [];
|
||||
|
||||
const mockFetch: typeof fetch = async (_input, init) => {
|
||||
if (!networkOnline) {
|
||||
throw new Error('ECONNREFUSED: Network unreachable');
|
||||
}
|
||||
const body = JSON.parse(String(init?.body || '{}')) as Record<string, unknown>;
|
||||
if (Array.isArray(body.batch)) {
|
||||
for (const item of body.batch as Array<Record<string, unknown>>) {
|
||||
deliveredEvents.push(item);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
accepted: body.batch.length,
|
||||
}),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
}
|
||||
deliveredEvents.push(body);
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
accepted: 1,
|
||||
}),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
};
|
||||
|
||||
const client = new DataStack({
|
||||
serverUrl: 'http://localhost:4000',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
autoConnect: false,
|
||||
offlineSpoolEnabled: true,
|
||||
offlineSpoolDir: tmpDir,
|
||||
fetchImpl: mockFetch,
|
||||
});
|
||||
|
||||
// While offline, events spool to disk
|
||||
const s1 = await client.send({ sensor: 'pressure', kpa: 101.3 });
|
||||
assert.equal(s1.success, true);
|
||||
assert.equal(s1.transport, 'spool');
|
||||
assert.equal(s1.spooled, 1);
|
||||
|
||||
const s2 = await client.sendBatch([
|
||||
{ sensor: 'temp', c: 22.1 },
|
||||
{ sensor: 'temp', c: 22.4 },
|
||||
]);
|
||||
assert.equal(s2.success, true);
|
||||
assert.equal(s2.transport, 'spool');
|
||||
assert.equal(s2.spooled, 2);
|
||||
assert.equal(client.status().spooledCount, 3);
|
||||
|
||||
// Network comes back online -> shutdown(5000) automatically drains the spool
|
||||
networkOnline = true;
|
||||
const shutdownRes = await client.shutdown(5000);
|
||||
assert.equal(shutdownRes.drained, true);
|
||||
assert.equal(shutdownRes.remainingInFlight, 0);
|
||||
assert.equal(shutdownRes.remainingSpooled, 0);
|
||||
assert.equal(deliveredEvents.length, 3);
|
||||
assert.equal(client.status().spooledCount, 0);
|
||||
} finally {
|
||||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('V2-Phase-04 (4): Direct file ingestion (sendFile for .csv, .json, .jsonl) with deterministic SHA-256 eventId and /vcon/sdk multi-language tabs', async () => {
|
||||
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ds-node-sdk-files-'));
|
||||
try {
|
||||
const token = makeFakeJwt('batch_importer');
|
||||
const receivedBatches: Array<Array<{ eventId: string; data: Record<string, unknown> }>> = [];
|
||||
|
||||
const mockFetch: typeof fetch = async (_input, init) => {
|
||||
const body = JSON.parse(String(init?.body || '{}')) as {
|
||||
batch?: Array<{ eventId: string; data: Record<string, unknown> }>;
|
||||
};
|
||||
if (Array.isArray(body.batch)) {
|
||||
receivedBatches.push(body.batch);
|
||||
}
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: true,
|
||||
accepted: body.batch?.length ?? 1,
|
||||
}),
|
||||
{ status: 200, headers: { 'Content-Type': 'application/json' } }
|
||||
);
|
||||
};
|
||||
|
||||
const client = new DataStack({
|
||||
serverUrl: 'http://localhost:4000',
|
||||
apiUrl: 'http://localhost:3000',
|
||||
token,
|
||||
autoConnect: false,
|
||||
offlineSpoolEnabled: false,
|
||||
fetchImpl: mockFetch,
|
||||
});
|
||||
|
||||
// 1. CSV file ingestion
|
||||
const csvPath = path.join(tmpDir, 'sensors.csv');
|
||||
await fs.writeFile(
|
||||
csvPath,
|
||||
'device,reading,status\ndev_01,42.5,OK\ndev_02,19.8,WARN\n',
|
||||
'utf8'
|
||||
);
|
||||
const csvRes = await client.sendFile(csvPath, { chunkSize: 10 });
|
||||
assert.equal(csvRes.success, true);
|
||||
assert.equal(csvRes.totalRows, 2);
|
||||
assert.equal(csvRes.acceptedRows, 2);
|
||||
assert.equal(receivedBatches[0][0].eventId, makeDeterministicFileEventId('sensors', 0, {
|
||||
device: 'dev_01',
|
||||
reading: '42.5',
|
||||
status: 'OK',
|
||||
}));
|
||||
|
||||
// 2. JSONL file ingestion
|
||||
const jsonlPath = path.join(tmpDir, 'events.jsonl');
|
||||
await fs.writeFile(
|
||||
jsonlPath,
|
||||
'{"event":"login","user":"u1"}\n{"event":"logout","user":"u1"}\n',
|
||||
'utf8'
|
||||
);
|
||||
const jsonlRes = await client.sendFile(jsonlPath);
|
||||
assert.equal(jsonlRes.success, true);
|
||||
assert.equal(jsonlRes.totalRows, 2);
|
||||
assert.equal(jsonlRes.acceptedRows, 2);
|
||||
|
||||
// 3. Unsupported extension throws DataStackFileError
|
||||
const badPath = path.join(tmpDir, 'notes.txt');
|
||||
await fs.writeFile(badPath, 'hello', 'utf8');
|
||||
await assert.rejects(
|
||||
() => client.sendFile(badPath),
|
||||
(err: unknown) => err instanceof DataStackFileError
|
||||
);
|
||||
|
||||
// 4. Verify /vcon/sdk page includes Python SDK, Node.js / TypeScript SDK, and HTTP / cURL tabs
|
||||
const sdkPageSrc = await fs.readFile(
|
||||
path.join(process.cwd(), 'app', 'vcon', 'sdk', 'page.tsx'),
|
||||
'utf8'
|
||||
);
|
||||
assert.match(sdkPageSrc, /Python SDK \(v2\.2\.0\)/);
|
||||
assert.match(sdkPageSrc, /Node\.js \/ TypeScript SDK \(v2\.2\.0\)/);
|
||||
assert.match(sdkPageSrc, /@datastack\/sdk/);
|
||||
assert.match(sdkPageSrc, /datastack\.config\.json/);
|
||||
assert.match(sdkPageSrc, /await client\.shutdown\(5000\)/);
|
||||
} finally {
|
||||
await fs.rm(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
Reference in new issue
Block a user