- Standardize Python SDK (`datastack`) documentation with a new multi-file suite. - Update `datastack-sdk` to serve as a backward-compatible shim for the canonical `datastack` package. - Update project manifest and memory map to reflect current SDK structure and v2.2.0 features.
7.4 KiB
API Reference Specification
This document provides the exhaustive specification for all public classes, methods, parameters, and return structures in the DataStack Python SDK.
1. Primary Client: DataStack / DataStackClient
Import path:
from datastack import DataStack, DataStackClient
# Both identifiers point to the same underlying class.
Constructor: __init__(...)
DataStack(
server_url: Optional[str] = None,
token: Optional[str] = None,
api_url: Optional[str] = None,
config_path: Optional[str] = None,
auto_connect: Optional[bool] = None,
reconnection_attempts: Optional[int] = None,
reconnection_delay: Optional[float] = None,
client_id: Optional[str] = None,
**kwargs: Any,
)
| Parameter | Type | Default | Description |
|---|---|---|---|
server_url |
str |
None (loads from config or http://localhost:4000) |
URL for Port 4000 Socket.IO ingestion service. |
token |
str |
None (loads from config or env) |
Signed HMAC-SHA256 client JWT token. |
api_url |
str |
None (inferred from server_url or http://localhost:3000) |
URL for Port 3000 REST API and automatic HTTP fallback. |
config_path |
str |
None |
Path to a specific Python configuration file. |
auto_connect |
bool |
None (defaults to True if config loaded) |
Whether to connect immediately during initialization. |
reconnection_attempts |
int |
None (alias for MAX_RETRIES, default: 3) |
Maximum number of retry attempts for transient errors. |
reconnection_delay |
float |
None (alias for HEARTBEAT_INTERVAL_SEC) |
Delay in seconds between reconnection probes. |
client_id |
str |
None |
Explicit client identifier slug (must match JWT sub/clientId). |
**kwargs |
Any |
— | Any configuration overrides passed to load_config(). |
Core Public Methods
connect(wait_timeout: Optional[float] = None) -> bool
Establishes the primary WebSocket connection to Port 4000 and verifies the handshake via ping_verify.
- Behavior:
- If WebSocket succeeds: Drains any offline spooled events and transitions the Heartbeat Monitor to
CONNECTED_IDLE(zero-CPU IDLE mode). - If WebSocket fails and
enable_http_fallbackisTrue: Checks HTTP API health. If online, entersFALLBACK_HTTPand starts background recovery. - If both are unreachable and
offline_spool_enabledisTrue: EntersOFFLINE_SPOOL.
- If WebSocket succeeds: Drains any offline spooled events and transitions the Heartbeat Monitor to
- Returns:
Trueif connected to WebSocket or HTTP fallback is operational;Falseif offline. - Raises:
DataStackAuthErrorif credentials are rejected (401/403).
disconnect() -> None
Stops the background Heartbeat Monitor thread and gracefully closes any active Socket.IO connection.
send(data: Dict[str, Any], event_id: Optional[str] = None, timestamp: Optional[str] = None, timeout: Optional[float] = None) -> Dict[str, Any]
Sends a single telemetry event.
- Parameters:
data(Dict[str, Any]): Arbitrary JSON-serializable dictionary containing the event payload.event_id(str, optional): Unique event identifier for idempotency. Auto-generated (evt_<timestamp>_<uuid>) if omitted.timestamp(str, optional): ISO 8601 UTC timestamp. Auto-generated (_utc_now_iso()) if omitted.timeout(float, optional): Timeout in seconds for waiting for server acknowledgment.
- Returns: Dictionary with delivery details:
{ "status": "ok", "eventId": "evt_1700000000000_a1b2c3d4", "transport": "WEBSOCKET" # or "HTTP_FALLBACK" or "OFFLINE_SPOOL" }
sync(...) -> Dict[str, Any]
Backward-compatible alias for send().
client.sync(data={"status": "active"}, timeout=5.0)
send_batch(events: Optional[List[Dict[str, Any]]] = None, timeout: Optional[float] = None, file_name: Optional[str] = None, *, records: Optional[List[Dict[str, Any]]] = None) -> Dict[str, Any]
Transmits a list of event dictionaries in a single network round-trip.
- Parameters:
events(List[Dict[str, Any]]): List of event dictionaries (accepts either raw data dicts or wrapped envelopes{eventId, timestamp, data}).records: Keyword-only alias forevents.timeout: Network ACK timeout.file_name(optional): Source filename if sending structured file records.
- Built-in Auto-Recovery:
- Auto-Splitting on
413 PAYLOAD_TOO_LARGE: Recursively halves the batch (chunk // 2) and sends each half. - Lossless Partial Rate-Limit Retry: If the server returns
rateLimitedEventIds, the SDK automatically waits 1.05s for the 1-second sliding window to reset and retries only the throttled items.
- Auto-Splitting on
- Returns:
{ "success": True, "processed": 500, "rejected": 0, "duplicates": 0, "spooledCount": 0, "transport": "WEBSOCKET" }
batch_sync(...) -> Dict[str, Any]
Backward-compatible alias for send_batch(). Accepts either events or records.
send_file(file_path: Union[str, Path], chunk_size: Optional[int] = None, extra_metadata: Optional[Dict[str, Any]] = None) -> Dict[str, Any]
Streams a local file directly into DataStack in deterministic chunks.
- Supported File Types:
.csv,.json,.jsonl,.ndjson. - Parameters:
file_path: Path to local data file.chunk_size: Rows per batch chunk (defaults toBATCH_CHUNK_SIZE, default: 250).extra_metadata: Optional key-value dictionary merged into every record.
- Features:
- Deterministic SHA-256 event ID generation (
file_{stem}_{row}_{hash}) preventing duplicates on re-runs. - UTF-8 with Byte Order Mark (
utf-8-sig) support.
- Deterministic SHA-256 event ID generation (
- Returns:
{ "success": True, "fileName": "transactions.csv", "fileType": "csv", "chunksSent": 4, "processed": 1000, "rejected": 0, "duplicates": 0, "spooledCount": 0, "transport": "WEBSOCKET" }
status() -> Dict[str, Any]
Returns a diagnostic snapshot of the client instance:
{
"connectedWebSocket": True,
"clientId": "pos_terminal_01",
"serverUrl": "http://localhost:4000",
"apiUrl": "http://localhost:3000",
"configSource": "/path/to/config.datastack.py",
"connectionState": "CONNECTED_IDLE",
"heartbeatMode": "IDLE", # "IDLE" or "ACTIVE_RECOVERY"
"recoveryAttempts": 0,
"lastError": None,
"offlineSpoolPending": 0,
"offlineSpoolBytes": 0
}
2. Configuration Class: DataStackConfig
Dataclass storing resolved SDK settings:
from datastack import DataStackConfig
@dataclass
class DataStackConfig:
server_url: str = "http://localhost:4000"
api_url: str = "http://localhost:3000"
client_token: str = ""
client_id: str = ""
auto_connect: bool = True
enable_http_fallback: bool = True
heartbeat_interval_sec: float = 5.0
request_timeout_sec: float = 8.0
max_retries: int = 3
retry_backoff_base_sec: float = 0.5
batch_chunk_size: int = 250
offline_spool_enabled: bool = True
offline_spool_dir: str = ".datastack_spool"
max_spool_size_mb: int = 50
config_source_path: Optional[str] = None
3. Configuration Loader: load_config
from datastack import load_config
config: DataStackConfig = load_config(
config_path=None,
require_token=False,
**overrides
)
Discovers config.datastack.py, layers environment variables (DATASTACK_*), and merges keyword overrides.