Files
DataStack/sdk/python/datastack/README.md
Team Dark Devil 40a6855724 feat(sdk): refactor Python SDK documentation
- Standardize Python SDK (`datastack`) documentation with a new multi-file suite.
- Update `datastack-sdk` to serve as a backward-compatible shim for the canonical `datastack` package.
- Update project manifest and memory map to reflect current SDK structure and v2.2.0 features.
2026-11-18 10:03:40 +06:00

3.5 KiB

DataStack Official Python SDK (datastack)

PyPI version Python: 3.8+ License: MIT Maintained by Vib Tools

The official production-grade Python SDK for DataStack — high-throughput, self-hosted telemetry ingestion and partitioned CSV persistence platform.


Key Capabilities

  • Zero-Boilerplate Auto-Discovery: Automatically finds and loads config.datastack.py in your working directory.
  • Dual-Transport Resilience: Real-time Socket.IO streaming on Port 4000 with automatic fallback to HTTP REST (POST /api/events) on Port 3000.
  • Zero-CPU Heartbeat Monitor: Background recovery thread sleeps in 0-CPU IDLE mode when healthy; wakes only on network disruption.
  • Offline Disk Spooling: Automatically appends unsent events to .datastack_spool/spool.jsonl when offline and replays them atomically upon reconnection.
  • Direct File Streaming: Stream .csv, .json, .jsonl, and .ndjson files with deterministic SHA-256 deduplication and auto-halving on payload limits.
  • Backoff & Rate-Limit Pacing: Lossless partial-batch retries on 429 RATE_LIMIT_EXCEEDED with jitter.

Installation

# From PyPI
pip install datastack

# Or from local repository
pip install ./sdk/python/datastack

Quick Example

Create config.datastack.py in your project root:

# config.datastack.py
DATASTACK_CONFIG = {
    "SERVER_URL": "http://localhost:4000",
    "API_URL": "http://localhost:3000",
    "CLIENT_TOKEN": "YOUR_SIGNED_CLIENT_JWT",
}

Then stream telemetry in Python:

from datastack import DataStack

with DataStack() as client:
    # 1. Single Event
    client.send({"service": "order_worker", "status": "completed", "amount": 99.50})

    # 2. Batch Events
    client.send_batch([
        {"metric": "cpu", "value": 45.2},
        {"metric": "ram_mb", "value": 2048},
    ])

    # 3. Direct File Streaming
    client.send_file("device_logs.csv", chunk_size=250)

Documentation Suite

Exhaustive technical documentation is available in the docs/ directory:


Author & License

  • Organization: Vib Tools
  • Author: Md Anik (ANIK.D.ART) — info@vibtools.top
  • License: MIT License