- Standardize Python SDK (`datastack`) documentation with a new multi-file suite. - Update `datastack-sdk` to serve as a backward-compatible shim for the canonical `datastack` package. - Update project manifest and memory map to reflect current SDK structure and v2.2.0 features.
91 lines
5.4 KiB
Markdown
91 lines
5.4 KiB
Markdown
# Lifecycle, Resilience & Offline Spooling
|
|
|
|
This document explains the internal connection state machine, the zero-CPU Heartbeat Monitor, and the thread-safe offline disk spooler powering the DataStack Python SDK.
|
|
|
|
---
|
|
|
|
## 1. Connection State Machine (`ConnectionState`)
|
|
|
|
The client transitions automatically through five lifecycle states:
|
|
|
|
```text
|
|
┌────────────────┐ WebSocket Drop ┌────────────────┐
|
|
│ CONNECTED_IDLE ├────────────────────────────────►│ FALLBACK_HTTP │
|
|
│ (Heartbeat │◄────────────────────────────────┤ (Heartbeat │
|
|
│ IDLE Mode) │ Reconnected & Verified │ ACTIVE_REC) │
|
|
└───────┬────────┘ (via ping_verify) └───────┬────────┘
|
|
│ │ Both WS & HTTP
|
|
│ │ Unreachable
|
|
│ ▼
|
|
│ ┌────────────────┐
|
|
│ │ OFFLINE_SPOOL │
|
|
│ │ (Local Disk │
|
|
│ │ spool.jsonl) │
|
|
│ └───────┬────────┘
|
|
│ │
|
|
│ client.disconnect() or │ Network Returns
|
|
│ Fatal Auth Revocation │
|
|
▼ ▼
|
|
┌────────────────┐ ┌────────────────┐
|
|
│ STOPPED │◄────────────────────────────────┤ RECONNECTING_ │
|
|
│ │ │ VERIFY │
|
|
└────────────────┘ └────────────────┘
|
|
```
|
|
|
|
### State Details
|
|
|
|
| State | Transport Used | Heartbeat Mode | Description |
|
|
| :--- | :--- | :--- | :--- |
|
|
| **`CONNECTED_IDLE`** | WebSocket (Port 4000) | `IDLE` (0% CPU) | Primary WebSocket connection is active and verified. The Heartbeat thread sleeps, consuming zero network or CPU resources. |
|
|
| **`FALLBACK_HTTP`** | HTTP REST (Port 3000) | `ACTIVE_RECOVERY` | WebSocket connection dropped. Outgoing events automatically route via HTTP `POST /api/events`. The Heartbeat thread wakes up and probes WebSocket reconnection in the background. |
|
|
| **`OFFLINE_SPOOL`** | Local Disk Queue | `ACTIVE_RECOVERY` | The client device has lost internet connectivity completely. Telemetry events are appended to `.datastack_spool/spool.jsonl`. |
|
|
| **`RECONNECTING_VERIFY`** | WebSocket | `ACTIVE_RECOVERY` | Reconnection handshake in progress. The client issues a `ping_verify` emission and replays spooled disk batches. |
|
|
| **`STOPPED`** | None | `STOPPED` | Client was closed via `disconnect()` or credentials were permanently revoked (`401`/`403`). All background threads terminate. |
|
|
|
|
---
|
|
|
|
## 2. Zero-CPU Heartbeat Monitor (`HeartbeatMonitor`)
|
|
|
|
Traditional SDKs constantly emit polling pings even when connections are healthy, wasting bandwidth and battery. DataStack implements an **Event-Driven Sleep Architecture**:
|
|
|
|
1. **When Healthy**: Once `connect()` succeeds and passes `ping_verify`, the monitor clears its internal wake event (`_wake_event.clear()`). The background thread remains completely idle.
|
|
2. **On Disconnect**: The Socket.IO `disconnect` event or an HTTP delivery trigger wakes the thread (`_wake_event.set()`).
|
|
3. **Background Reconnection**: Every `HEARTBEAT_INTERVAL_SEC` (default: 5.0s), the monitor attempts to reconnect and verify the WebSocket.
|
|
4. **Return to Idle**: Once verified, the thread drains any offline spooled events and immediately returns to sleep.
|
|
|
|
---
|
|
|
|
## 3. Offline Disk Spooling (`DiskSpooler`)
|
|
|
|
When edge devices, field laptops, or factory floor terminals lose internet connectivity:
|
|
|
|
### Storage Mechanics
|
|
- Events are appended to `.datastack_spool/spool.jsonl` using UTF-8 JSON lines.
|
|
- Each event preserves its deterministic `eventId`, `timestamp`, and `clientId`.
|
|
- Disk capacity is protected by `MAX_SPOOL_SIZE_MB` (default: 50 MB).
|
|
|
|
### Atomic Drain & Crash Recovery
|
|
- When connectivity returns, the Heartbeat Monitor reads spooled records in batches of `BATCH_CHUNK_SIZE` (default: 250).
|
|
- Batches are emitted to the server.
|
|
- The spool file is trimmed atomically using `ack_and_requeue_front()` and `os.replace()`, guaranteeing that if the process crashes mid-stream, unacknowledged records remain at the head of the file without duplication or data loss.
|
|
|
|
---
|
|
|
|
## 4. Resource Disposal & Cleanup
|
|
|
|
Always disconnect the client when your application shuts down to avoid dangling threads:
|
|
|
|
```python
|
|
client = DataStack()
|
|
try:
|
|
client.send({"metric": "active"})
|
|
finally:
|
|
client.disconnect()
|
|
```
|
|
|
|
Or use the context manager:
|
|
```python
|
|
with DataStack() as client:
|
|
client.send({"metric": "active"})
|
|
```
|