Files
DataStack/docs/dev/README.md
T
Team Dark Devil 4279da45bb chore: establish cross-IDE synchronization protocol
Implement `.ai-rules`, formalize `MEMORY.md` as the source of truth, and initialize `docs/` structure to ensure consistent architectural context across development environments.
2026-11-18 07:11:11 +06:00

3.0 KiB

Developer & Contributor Guide

Welcome to the DataStack Developer Documentation. This section contains in-depth architectural specifications, development workflows, compilation instructions, and contribution conventions for engineers working on the DataStack codebase.


Developer Documentation Index

Explore the core technical guides:

1. Architecture Blueprint

Detailed breakdown of DataStack's decoupled dual-runtime model:

  • Port 3000 (web): Next.js 15 App Router, React 19, serverless REST API, streaming CSV/JSON export.
  • Port 4000 (ingestion): Dedicated Express 5 + Socket.IO real-time ingestion server.
  • Internal Service Bridge: Authenticated /internal/* communication via DATASTACK_INTERNAL_SECRET.
  • Storage Subsystem: In-memory and Redis-backed buffering, cooperative partition file locking, and streaming row counters.
  • PostgreSQL Control-Plane: Schema provisioning and metadata storage rules (pg.Pool + prisma/schema.prisma).

2. Local Development Setup

Instructions for configuring your local machine:

  • Prerequisites (Node.js 22, npm, PostgreSQL 16, Redis 7).
  • Environment variable configuration (.env.example).
  • Dual-terminal local workflow (npm run dev + npm run server).
  • Database schema initialization.

3. Build, Compilation & Testing

Standard build and testing commands:

  • Compiling Next.js standalone output (npm run build).
  • Code quality and linting (npm run lint).
  • Running the comprehensive TypeScript test suite with node:test and tsx.
  • Executing Python SDK unit tests.
  • Docker multi-stage container builds.

4. Contribution Rules & Conventions

Guidelines for submitting pull requests and modifying code:

  • Universal Multi-IDE Memory Rule (MEMORY.md & .ai-rules).
  • Single-source-of-truth UI primitives in components/ui/ (0rem radius, zero shadows).
  • Architectural guardrails (zero telemetry rows in PostgreSQL, no Socket.IO in Next.js).
  • Pull Request and code review checklist.

High-Level Repository Layout

datastack/
├── app/                  # Next.js 15 App Router (Frontend, /vcon/*, and /api/*)
├── components/           # UI primitives (ui/), layout shells (layout/), and sections (sections/)
├── core/                 # Backend TypeScript modules (auth, Database, data, Storage, Security, sdk)
├── server/               # Dedicated Port 4000 Express + Socket.IO service & CSV batch worker
├── sdk/python/datastack/ # Official pip-installable Python SDK
├── sdk-python/           # Backward-compatible Python SDK wrapper
├── storage/data/         # Partitioned CSV telemetry storage root
├── docker/               # Production Dockerfiles, Nginx proxy configs, and docker-compose.yml
├── nginx/                # Local Nginx development configuration
├── tests/                # Automated test suites (phase01 - phase07)
└── docs/                 # User and developer documentation