docs: update architecture documentation and manifests

Refine technical documentation to reflect current architecture, component structure, and routing conventions for the vcon console.
This commit is contained in:
Team Dark Devil committed 2026-11-17 21:22:55 +06:00
1 parent bb938c33c0
commit 367dd819f8
64 files changed
+910 -2661

No files matched your search

+74 -20
View File
@@ -1,48 +1,102 @@
# DataStack Python SDK
# DataStack Python SDK (`datastack-sdk`)
Official Python client for the **DataStack Universal Realtime Data Collection Platform**.
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](../LICENSE)
[![Python 3.8+](https://img.shields.io/badge/Python-3.8%2B-3776AB)](https://www.python.org/)
[![Maintained by Vib Tools](https://img.shields.io/badge/Maintained%20by-Vib%20Tools-0f172a)](https://vib.tools/)
Official Python client library for **DataStack — Self-Hosted Real-Time Telemetry & Event Logging Platform**, developed by **[Vib Tools](https://vib.tools/)**.
`datastack-sdk` enables Python services, IoT sensor nodes, microservice workers, and research pipelines to stream structured JSON telemetry events to a DataStack server over Socket.IO with deterministic `threading.Event` acknowledgments.
---
## Installation
Install directly via pip:
Install directly from the `sdk-python` directory:
```bash
pip install ./sdk-python
```
Or in development:
For editable development mode:
```bash
cd sdk-python && pip install -e .
```
---
## Quick Start
```python
from datastack_sdk import DataStackClient
# 1. Initialize client with JWT from /vcon/clients console
# 1. Initialize the client with a signed JWT token from /vcon/clients or /vcon/sdk
client = DataStackClient(
server_url="http://localhost:4000",
token="YOUR_CLIENT_JWT_TOKEN"
token="YOUR_CLIENT_JWT_TOKEN",
)
# 2. Connect to Ingestion Engine
# 2. Connect to the DataStack Socket.IO server
if client.connect():
# 3. Stream single realtime event
res = client.sync({
"action": "user_checkout",
"order_id": "ORD-9482",
"amount": 249.50,
"currency": "USD"
})
# 3. Stream a single structured event and wait for server acknowledgment
res = client.sync(
data={
"event_type": "service_heartbeat",
"service": "billing-worker",
"latency_ms": 18.4,
"status": "healthy",
},
timeout=5.0,
)
print("Sync result:", res)
# 4. Stream batch of records
batch_res = client.batch_sync([
{"metric": "cpu_load", "value": 42.1},
{"metric": "mem_usage", "value": 78.5},
{"metric": "disk_io", "value": 1120}
])
# 4. Stream a batch of events in a single round-trip
batch_res = client.batch_sync(
records=[
{"metric": "cpu_load", "value": 42.1, "unit": "%"},
{"metric": "mem_usage", "value": 68.5, "unit": "%"},
{"metric": "disk_free_gb", "value": 124.0, "unit": "GB"},
],
timeout=10.0,
)
print("Batch result:", batch_res)
# 5. Disconnect cleanly
client.disconnect()
```
---
## API Reference
### `DataStackClient(server_url="http://localhost:4000", token="", reconnection_attempts=5, reconnection_delay=1)`
Creates a new Socket.IO client instance configured for DataStack handshake authentication.
### `client.connect(wait_timeout=5) -> bool`
Establishes a WebSocket/Socket.IO connection to the DataStack Port 4000 ingestion server using `auth={"token": self.token}`. Returns `True` on success or `False` on connection failure.
### `client.sync(data, event_id=None, timeout=5.0) -> dict`
Emits a single event (`sync_data`) and blocks via `threading.Event.wait(timeout)` until the server responds.
- **Returns**: `{"success": True, "eventId": "evt_..."}` or `{"success": False, "eventId": "evt_...", "error": "..."}`.
### `client.batch_sync(records, timeout=10.0) -> dict`
Emits a list of event dictionaries (`batch_sync`) and blocks via `threading.Event.wait(timeout)` until the server acknowledges the batch.
- **Returns**: `{"success": True, "processed": N, "rejected": M}`.
### `client.disconnect()`
Cleanly closes the active Socket.IO connection.
---
## Author & Support
Developed and maintained by **[Vib Tools](https://vib.tools/)**:
- **Website**: [https://vib.tools/](https://vib.tools/)
- **GitHub**: [https://github.com/vibtoolshq](https://github.com/vibtoolshq)
- **General Email**: [hello@vibtools.io](mailto:hello@vibtools.io)
- **Technical Support**: [support@vibtools.io](mailto:support@vibtools.io)
## License
Released under the [MIT License](../LICENSE).
+5 -8
View File
@@ -1,13 +1,10 @@
# Python SDK Manifest
## Purpose
Pip-installable Python client SDK (`datastack-sdk`) enabling Python applications, scrapers, and telemetry daemons to stream real-time events to DataStack.
Pip-installable Python client SDK (`datastack-sdk`) enabling Python services, IoT telemetry agents, and analytics applications to stream structured real-time events to DataStack.
## Contents
- `setup.py`: Standard setuptools configuration
- `datastack_sdk/client.py`: `DataStackClient` class with automatic reconnection, JWT handshake, and sync/batch_sync methods
- `README.md`: Quickstart guide and API reference.
## Dependencies
- `python-socketio[client]`
- `requests`
- `setup.py`: Package metadata and dependencies (`python-socketio[client]`, `requests`).
- `datastack_sdk/__init__.py`: Exports `DataStackClient`.
- `datastack_sdk/client.py`: `DataStackClient` implementation featuring JWT handshake authentication, automatic reconnection, and thread-safe acknowledgment waiting (`threading.Event`) for `sync()` and `batch_sync()`.
- `README.md`: Installation instructions, quickstart example, and API reference.
+24 -4
View File
@@ -3,18 +3,38 @@ from setuptools import setup, find_packages
setup(
name="datastack-sdk",
version="1.0.0",
description="Python client SDK for DataStack Universal Realtime Data Collection Platform",
long_description=open("README.md").read(),
description="Official Python client SDK for DataStack — Self-Hosted Real-Time Telemetry & Event Logging Platform",
long_description=open("README.md", encoding="utf-8").read(),
long_description_content_type="text/markdown",
author="DataStack Engine Team",
author="Vib Tools",
author_email="hello@vibtools.io",
url="https://github.com/vibtoolshq",
project_urls={
"Homepage": "https://vib.tools/",
"Source": "https://github.com/vibtoolshq",
"Support": "mailto:support@vibtools.io",
},
license="MIT",
packages=find_packages(),
install_requires=[
"python-socketio[client]>=5.11.0",
"requests>=2.31.0",
"websocket-client>=1.7.0"
"websocket-client>=1.7.0",
],
classifiers=[
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"Intended Audience :: Science/Research",
"Topic :: System :: Logging",
"Topic :: System :: Monitoring",
"Topic :: Software Development :: Libraries :: Python Modules",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],