Md Najmul d39d9f92f9 docs: update AI Studio design token audits
Refresh design token inventory files to reflect the latest UI sizing and layout configurations from the sandbox prototype.
2026-10-23 11:43:49 -05:00

DataStack — Self-Hosted Universal Realtime Data Collection Platform

DataStack is a high-throughput, self-hosted data collection platform designed for high-frequency event ingestion over WebSockets with buffered batch persistence to CSV.

Architecture

Process Port Role
Next.js App Router 3000 Frontend UI, /vcon/* admin console, streaming CSV export /api/export
Express + Socket.IO 4000 WebSocket handshake, JWT client auth, event intake
CSV Writer Worker — In-memory buffer flush to CSV (500 rows / 2s)

Data Persistence

  • CSV Storage Path: /storage/data/{clientId}/{YYYY-MM-DD}.csv
  • Schema: clientId, eventId, timestamp(ISO), payload(JSON)
  • Rotation Rule: Per-day OR 100MB threshold (whichever triggers first).
  • PostgreSQL: Used strictly for SDK client registration and admin settings. Zero collected data is inserted into the database.

Python SDK

Install the client:

pip install ./sdk-python

Usage:

from datastack_sdk import DataStackClient

client = DataStackClient(server_url="http://localhost:4000", token="JWT_TOKEN")
if client.connect():
    client.sync({"metric": "load", "value": 42})
    client.disconnect()

Running the Platform

# Terminal 1: Web Interface & Admin Console
npm run dev

# Terminal 2: Express Socket.IO Ingestion Engine
npm run server
S
Description
DataStack is a free, open-source, self-hosted real-time data collection and telemetry logging platform developed by Vib Tools.
Readme MIT
4.3 MiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 74.1%
JavaScript 11.6%
HTML 7.9%
Python 6.2%
CSS 0.2%