Files
DataStack/sdk/python/datastack/docs/LIFECYCLE.md
T
Team Dark Devil 40a6855724 feat(sdk): refactor Python SDK documentation
- Standardize Python SDK (`datastack`) documentation with a new multi-file suite.
- Update `datastack-sdk` to serve as a backward-compatible shim for the canonical `datastack` package.
- Update project manifest and memory map to reflect current SDK structure and v2.2.0 features.
2026-11-18 10:03:40 +06:00

5.4 KiB

Lifecycle, Resilience & Offline Spooling

This document explains the internal connection state machine, the zero-CPU Heartbeat Monitor, and the thread-safe offline disk spooler powering the DataStack Python SDK.


1. Connection State Machine (ConnectionState)

The client transitions automatically through five lifecycle states:

 ┌────────────────┐         WebSocket Drop          ┌────────────────┐
 │ CONNECTED_IDLE ├────────────────────────────────►│ FALLBACK_HTTP  │
 │ (Heartbeat     │◄────────────────────────────────┤ (Heartbeat     │
 │  IDLE Mode)    │       Reconnected & Verified    │  ACTIVE_REC)   │
 └───────┬────────┘        (via ping_verify)        └───────┬────────┘
         │                                                  │ Both WS & HTTP
         │                                                  │ Unreachable
         │                                                  ▼
         │                                          ┌────────────────┐
         │                                          │ OFFLINE_SPOOL  │
         │                                          │ (Local Disk    │
         │                                          │  spool.jsonl)  │
         │                                          └───────┬────────┘
         │                                                  │
         │            client.disconnect() or                │ Network Returns
         │            Fatal Auth Revocation                 │
         ▼                                                  ▼
 ┌────────────────┐                                 ┌────────────────┐
 │    STOPPED     │◄────────────────────────────────┤  RECONNECTING_ │
 │                │                                 │     VERIFY     │
 └────────────────┘                                 └────────────────┘

State Details

State Transport Used Heartbeat Mode Description
CONNECTED_IDLE WebSocket (Port 4000) IDLE (0% CPU) Primary WebSocket connection is active and verified. The Heartbeat thread sleeps, consuming zero network or CPU resources.
FALLBACK_HTTP HTTP REST (Port 3000) ACTIVE_RECOVERY WebSocket connection dropped. Outgoing events automatically route via HTTP POST /api/events. The Heartbeat thread wakes up and probes WebSocket reconnection in the background.
OFFLINE_SPOOL Local Disk Queue ACTIVE_RECOVERY The client device has lost internet connectivity completely. Telemetry events are appended to .datastack_spool/spool.jsonl.
RECONNECTING_VERIFY WebSocket ACTIVE_RECOVERY Reconnection handshake in progress. The client issues a ping_verify emission and replays spooled disk batches.
STOPPED None STOPPED Client was closed via disconnect() or credentials were permanently revoked (401/403). All background threads terminate.

2. Zero-CPU Heartbeat Monitor (HeartbeatMonitor)

Traditional SDKs constantly emit polling pings even when connections are healthy, wasting bandwidth and battery. DataStack implements an Event-Driven Sleep Architecture:

  1. When Healthy: Once connect() succeeds and passes ping_verify, the monitor clears its internal wake event (_wake_event.clear()). The background thread remains completely idle.
  2. On Disconnect: The Socket.IO disconnect event or an HTTP delivery trigger wakes the thread (_wake_event.set()).
  3. Background Reconnection: Every HEARTBEAT_INTERVAL_SEC (default: 5.0s), the monitor attempts to reconnect and verify the WebSocket.
  4. Return to Idle: Once verified, the thread drains any offline spooled events and immediately returns to sleep.

3. Offline Disk Spooling (DiskSpooler)

When edge devices, field laptops, or factory floor terminals lose internet connectivity:

Storage Mechanics

  • Events are appended to .datastack_spool/spool.jsonl using UTF-8 JSON lines.
  • Each event preserves its deterministic eventId, timestamp, and clientId.
  • Disk capacity is protected by MAX_SPOOL_SIZE_MB (default: 50 MB).

Atomic Drain & Crash Recovery

  • When connectivity returns, the Heartbeat Monitor reads spooled records in batches of BATCH_CHUNK_SIZE (default: 250).
  • Batches are emitted to the server.
  • The spool file is trimmed atomically using ack_and_requeue_front() and os.replace(), guaranteeing that if the process crashes mid-stream, unacknowledged records remain at the head of the file without duplication or data loss.

4. Resource Disposal & Cleanup

Always disconnect the client when your application shuts down to avoid dangling threads:

client = DataStack()
try:
    client.send({"metric": "active"})
finally:
    client.disconnect()

Or use the context manager:

with DataStack() as client:
    client.send({"metric": "active"})