docs: update architecture documentation and manifests
Refine technical documentation to reflect current architecture, component structure, and routing conventions for the vcon console.
This commit is contained in:
1 parent
bb938c33c0
commit
367dd819f8
64 files changed
+910
-2661
No files matched your search
+74
-20
@@ -1,48 +1,102 @@
|
||||
# DataStack Python SDK
|
||||
# DataStack Python SDK (`datastack-sdk`)
|
||||
|
||||
Official Python client for the **DataStack Universal Realtime Data Collection Platform**.
|
||||
[](../LICENSE)
|
||||
[](https://www.python.org/)
|
||||
[](https://vib.tools/)
|
||||
|
||||
Official Python client library for **DataStack — Self-Hosted Real-Time Telemetry & Event Logging Platform**, developed by **[Vib Tools](https://vib.tools/)**.
|
||||
|
||||
`datastack-sdk` enables Python services, IoT sensor nodes, microservice workers, and research pipelines to stream structured JSON telemetry events to a DataStack server over Socket.IO with deterministic `threading.Event` acknowledgments.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Install directly via pip:
|
||||
Install directly from the `sdk-python` directory:
|
||||
|
||||
```bash
|
||||
pip install ./sdk-python
|
||||
```
|
||||
Or in development:
|
||||
|
||||
For editable development mode:
|
||||
|
||||
```bash
|
||||
cd sdk-python && pip install -e .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from datastack_sdk import DataStackClient
|
||||
|
||||
# 1. Initialize client with JWT from /vcon/clients console
|
||||
# 1. Initialize the client with a signed JWT token from /vcon/clients or /vcon/sdk
|
||||
client = DataStackClient(
|
||||
server_url="http://localhost:4000",
|
||||
token="YOUR_CLIENT_JWT_TOKEN"
|
||||
token="YOUR_CLIENT_JWT_TOKEN",
|
||||
)
|
||||
|
||||
# 2. Connect to Ingestion Engine
|
||||
# 2. Connect to the DataStack Socket.IO server
|
||||
if client.connect():
|
||||
# 3. Stream single realtime event
|
||||
res = client.sync({
|
||||
"action": "user_checkout",
|
||||
"order_id": "ORD-9482",
|
||||
"amount": 249.50,
|
||||
"currency": "USD"
|
||||
})
|
||||
# 3. Stream a single structured event and wait for server acknowledgment
|
||||
res = client.sync(
|
||||
data={
|
||||
"event_type": "service_heartbeat",
|
||||
"service": "billing-worker",
|
||||
"latency_ms": 18.4,
|
||||
"status": "healthy",
|
||||
},
|
||||
timeout=5.0,
|
||||
)
|
||||
print("Sync result:", res)
|
||||
|
||||
# 4. Stream batch of records
|
||||
batch_res = client.batch_sync([
|
||||
{"metric": "cpu_load", "value": 42.1},
|
||||
{"metric": "mem_usage", "value": 78.5},
|
||||
{"metric": "disk_io", "value": 1120}
|
||||
])
|
||||
# 4. Stream a batch of events in a single round-trip
|
||||
batch_res = client.batch_sync(
|
||||
records=[
|
||||
{"metric": "cpu_load", "value": 42.1, "unit": "%"},
|
||||
{"metric": "mem_usage", "value": 68.5, "unit": "%"},
|
||||
{"metric": "disk_free_gb", "value": 124.0, "unit": "GB"},
|
||||
],
|
||||
timeout=10.0,
|
||||
)
|
||||
print("Batch result:", batch_res)
|
||||
|
||||
# 5. Disconnect cleanly
|
||||
client.disconnect()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### `DataStackClient(server_url="http://localhost:4000", token="", reconnection_attempts=5, reconnection_delay=1)`
|
||||
Creates a new Socket.IO client instance configured for DataStack handshake authentication.
|
||||
|
||||
### `client.connect(wait_timeout=5) -> bool`
|
||||
Establishes a WebSocket/Socket.IO connection to the DataStack Port 4000 ingestion server using `auth={"token": self.token}`. Returns `True` on success or `False` on connection failure.
|
||||
|
||||
### `client.sync(data, event_id=None, timeout=5.0) -> dict`
|
||||
Emits a single event (`sync_data`) and blocks via `threading.Event.wait(timeout)` until the server responds.
|
||||
- **Returns**: `{"success": True, "eventId": "evt_..."}` or `{"success": False, "eventId": "evt_...", "error": "..."}`.
|
||||
|
||||
### `client.batch_sync(records, timeout=10.0) -> dict`
|
||||
Emits a list of event dictionaries (`batch_sync`) and blocks via `threading.Event.wait(timeout)` until the server acknowledges the batch.
|
||||
- **Returns**: `{"success": True, "processed": N, "rejected": M}`.
|
||||
|
||||
### `client.disconnect()`
|
||||
Cleanly closes the active Socket.IO connection.
|
||||
|
||||
---
|
||||
|
||||
## Author & Support
|
||||
|
||||
Developed and maintained by **[Vib Tools](https://vib.tools/)**:
|
||||
- **Website**: [https://vib.tools/](https://vib.tools/)
|
||||
- **GitHub**: [https://github.com/vibtoolshq](https://github.com/vibtoolshq)
|
||||
- **General Email**: [hello@vibtools.io](mailto:hello@vibtools.io)
|
||||
- **Technical Support**: [support@vibtools.io](mailto:support@vibtools.io)
|
||||
|
||||
## License
|
||||
|
||||
Released under the [MIT License](../LICENSE).
|
||||
@@ -1,13 +1,10 @@
|
||||
# Python SDK Manifest
|
||||
|
||||
## Purpose
|
||||
Pip-installable Python client SDK (`datastack-sdk`) enabling Python applications, scrapers, and telemetry daemons to stream real-time events to DataStack.
|
||||
Pip-installable Python client SDK (`datastack-sdk`) enabling Python services, IoT telemetry agents, and analytics applications to stream structured real-time events to DataStack.
|
||||
|
||||
## Contents
|
||||
- `setup.py`: Standard setuptools configuration
|
||||
- `datastack_sdk/client.py`: `DataStackClient` class with automatic reconnection, JWT handshake, and sync/batch_sync methods
|
||||
- `README.md`: Quickstart guide and API reference.
|
||||
|
||||
## Dependencies
|
||||
- `python-socketio[client]`
|
||||
- `requests`
|
||||
- `setup.py`: Package metadata and dependencies (`python-socketio[client]`, `requests`).
|
||||
- `datastack_sdk/__init__.py`: Exports `DataStackClient`.
|
||||
- `datastack_sdk/client.py`: `DataStackClient` implementation featuring JWT handshake authentication, automatic reconnection, and thread-safe acknowledgment waiting (`threading.Event`) for `sync()` and `batch_sync()`.
|
||||
- `README.md`: Installation instructions, quickstart example, and API reference.
|
||||
+24
-4
@@ -3,18 +3,38 @@ from setuptools import setup, find_packages
|
||||
setup(
|
||||
name="datastack-sdk",
|
||||
version="1.0.0",
|
||||
description="Python client SDK for DataStack Universal Realtime Data Collection Platform",
|
||||
long_description=open("README.md").read(),
|
||||
description="Official Python client SDK for DataStack — Self-Hosted Real-Time Telemetry & Event Logging Platform",
|
||||
long_description=open("README.md", encoding="utf-8").read(),
|
||||
long_description_content_type="text/markdown",
|
||||
author="DataStack Engine Team",
|
||||
author="Vib Tools",
|
||||
author_email="hello@vibtools.io",
|
||||
url="https://github.com/vibtoolshq",
|
||||
project_urls={
|
||||
"Homepage": "https://vib.tools/",
|
||||
"Source": "https://github.com/vibtoolshq",
|
||||
"Support": "mailto:support@vibtools.io",
|
||||
},
|
||||
license="MIT",
|
||||
packages=find_packages(),
|
||||
install_requires=[
|
||||
"python-socketio[client]>=5.11.0",
|
||||
"requests>=2.31.0",
|
||||
"websocket-client>=1.7.0"
|
||||
"websocket-client>=1.7.0",
|
||||
],
|
||||
classifiers=[
|
||||
"Development Status :: 5 - Production/Stable",
|
||||
"Intended Audience :: Developers",
|
||||
"Intended Audience :: System Administrators",
|
||||
"Intended Audience :: Science/Research",
|
||||
"Topic :: System :: Logging",
|
||||
"Topic :: System :: Monitoring",
|
||||
"Topic :: Software Development :: Libraries :: Python Modules",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Operating System :: OS Independent",
|
||||
],
|
||||
|
||||
Reference in new issue
Block a user