Implement `.ai-rules`, formalize `MEMORY.md` as the source of truth, and initialize `docs/` structure to ensure consistent architectural context across development environments.
59 lines
3.0 KiB
Markdown
59 lines
3.0 KiB
Markdown
# Developer & Contributor Guide
|
|
|
|
Welcome to the **DataStack Developer Documentation**. This section contains in-depth architectural specifications, development workflows, compilation instructions, and contribution conventions for engineers working on the DataStack codebase.
|
|
|
|
---
|
|
|
|
## Developer Documentation Index
|
|
|
|
Explore the core technical guides:
|
|
|
|
### 1. [Architecture Blueprint](./architecture.md)
|
|
Detailed breakdown of DataStack's decoupled dual-runtime model:
|
|
- **Port 3000 (`web`)**: Next.js 15 App Router, React 19, serverless REST API, streaming CSV/JSON export.
|
|
- **Port 4000 (`ingestion`)**: Dedicated Express 5 + Socket.IO real-time ingestion server.
|
|
- **Internal Service Bridge**: Authenticated `/internal/*` communication via `DATASTACK_INTERNAL_SECRET`.
|
|
- **Storage Subsystem**: In-memory and Redis-backed buffering, cooperative partition file locking, and streaming row counters.
|
|
- **PostgreSQL Control-Plane**: Schema provisioning and metadata storage rules (`pg.Pool` + `prisma/schema.prisma`).
|
|
|
|
### 2. [Local Development Setup](./setup.md)
|
|
Instructions for configuring your local machine:
|
|
- Prerequisites (Node.js 22, npm, PostgreSQL 16, Redis 7).
|
|
- Environment variable configuration (`.env.example`).
|
|
- Dual-terminal local workflow (`npm run dev` + `npm run server`).
|
|
- Database schema initialization.
|
|
|
|
### 3. [Build, Compilation & Testing](./build.md)
|
|
Standard build and testing commands:
|
|
- Compiling Next.js standalone output (`npm run build`).
|
|
- Code quality and linting (`npm run lint`).
|
|
- Running the comprehensive TypeScript test suite with `node:test` and `tsx`.
|
|
- Executing Python SDK unit tests.
|
|
- Docker multi-stage container builds.
|
|
|
|
### 4. [Contribution Rules & Conventions](./contributing.md)
|
|
Guidelines for submitting pull requests and modifying code:
|
|
- **Universal Multi-IDE Memory Rule** (`MEMORY.md` & `.ai-rules`).
|
|
- Single-source-of-truth UI primitives in `components/ui/` (`0rem` radius, zero shadows).
|
|
- Architectural guardrails (zero telemetry rows in PostgreSQL, no Socket.IO in Next.js).
|
|
- Pull Request and code review checklist.
|
|
|
|
---
|
|
|
|
## High-Level Repository Layout
|
|
|
|
```text
|
|
datastack/
|
|
├── app/ # Next.js 15 App Router (Frontend, /vcon/*, and /api/*)
|
|
├── components/ # UI primitives (ui/), layout shells (layout/), and sections (sections/)
|
|
├── core/ # Backend TypeScript modules (auth, Database, data, Storage, Security, sdk)
|
|
├── server/ # Dedicated Port 4000 Express + Socket.IO service & CSV batch worker
|
|
├── sdk/python/datastack/ # Official pip-installable Python SDK
|
|
├── sdk-python/ # Backward-compatible Python SDK wrapper
|
|
├── storage/data/ # Partitioned CSV telemetry storage root
|
|
├── docker/ # Production Dockerfiles, Nginx proxy configs, and docker-compose.yml
|
|
├── nginx/ # Local Nginx development configuration
|
|
├── tests/ # Automated test suites (phase01 - phase07)
|
|
└── docs/ # User and developer documentation
|
|
```
|