- Standardize Python SDK (`datastack`) documentation with a new multi-file suite. - Update `datastack-sdk` to serve as a backward-compatible shim for the canonical `datastack` package. - Update project manifest and memory map to reflect current SDK structure and v2.2.0 features.
4.7 KiB
4.7 KiB
Operational Workflows & User Journeys
This document provides step-by-step procedures for common operational workflows, from first-time administrator onboarding to credential rotation and disaster recovery.
Workflow 1: Initial Administrator Onboarding
When setting up a newly deployed instance:
Deploy Stack ──► Open /vcon/setup ──► Enter Setup Key & Admin Email ──► Create Password ──► Redirect to /vcon/dashboard
- Visit your instance URL at
/vcon/setup(e.g.,https://your-domain.example/vcon/setup). - If an administrator already exists, DataStack redirects to
/vcon/login. - Provide the required credentials:
- Setup Key: Enter the
DATASTACK_ADMIN_SETUP_KEYconfigured in your.env. - Admin Email: Enter the primary operational contact email.
- Password: Enter a strong password (minimum 8 characters, checked against common passwords).
- Setup Key: Enter the
- Click Complete Setup.
- DataStack hashes the password using
crypto.scrypt, creates the first row inadmin_users, establishes a signed session cookie, and redirects to/vcon/dashboard.
Workflow 2: Onboarding an External Client Application
Connecting a new producer (Python app, IoT gateway, mobile app, microservice):
/vcon/clients ──► Register Client ──► Copy JWT Token ──► Configure Client App ──► Stream Events
- Navigate to Clients (
/vcon/clients). - Click + Register Client.
- Specify:
Client Identifier: e.g.,checkout_service_prodClient Name: e.g.,Checkout Service ProductionRate Limit: e.g.,250events/second
- Click Register Client.
- A modal appears displaying the newly signed HMAC-SHA256 JWT token.
- Copy the token and configure your application environment:
DATASTACK_CLIENT_ID=checkout_service_prod DATASTACK_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... DATASTACK_SERVER_URL=https://your-domain.example - Start your producer. Events will immediately stream into the system and appear in the live dashboard activity feed.
Workflow 3: Rotating a Client Token
If a client token has leaked or requires routine security rotation:
/vcon/clients ──► Rotate Token ──► Copy New Token ──► Update Client Config ──► Reconnect
- Open
/vcon/clients. - Locate the client row and click Rotate Token.
- Confirm the action in the confirmation modal.
- DataStack generates a new client secret, signs a fresh JWT, updates PostgreSQL, and notifies Port 4000 via
/internal/client-sync. - Active WebSocket connections using the prior token are disconnected immediately.
- Copy the new token and update your producer application's environment. Once restarted, the client reconnects with the new token.
Workflow 4: Rotating the Platform JWT Secret
In the event of an infrastructure-wide security review or platform secret compromise:
- Open
/vcon/settings. - In the Security & Secrets card, click Rotate Platform Secret.
- Confirm the security alert.
- DataStack:
- Generates a new cryptographically secure 48-character platform JWT secret.
- Saves the secret to
admin_settings. - Iterates through all registered clients in PostgreSQL and re-signs their tokens using the new master secret (
SDKManager.resignAllClientTokensAsync()). - Pushes the new secret and re-signed tokens to Port 4000 via
/internal/settings-sync.
- Navigate to
/vcon/clientsto export or view the refreshed client tokens to distribute to client application teams.
Workflow 5: Backup & Disaster Recovery
DataStack stores state in two distinct layers: PostgreSQL (metadata) and filesystem volumes (telemetry data).
Backing Up the PostgreSQL Control Plane
docker exec -t datastack-postgres pg_dump -U datastack datastack_db > datastack_meta_$(date +%F).sql
Backing Up Telemetry Partitions
Telemetry files reside in the named Docker volume datastack-data (or mounted host path ./storage/data).
# Archive all client partition directories
docker run --rm \
-v datastack-data:/data:ro \
-v $(pwd):/backup \
alpine tar czf /backup/datastack_partitions_$(date +%F).tar.gz -C /data .
Restoring from Backup
- Stop the ingestion and web containers:
docker compose down - Restore the database:
docker compose up -d postgres cat datastack_meta_2026-11-17.sql | docker exec -i datastack-postgres psql -U datastack datastack_db - Restore the partition volume:
docker run --rm \ -v datastack-data:/data \ -v $(pwd):/backup \ alpine tar xzf /backup/datastack_partitions_2026-11-17.tar.gz -C /data - Restart the full stack:
docker compose up -d