Files
DataStack/docs/users/tutorials.md
Team Dark Devil 4279da45bb chore: establish cross-IDE synchronization protocol
Implement `.ai-rules`, formalize `MEMORY.md` as the source of truth, and initialize `docs/` structure to ensure consistent architectural context across development environments.
2026-11-18 07:11:11 +06:00

167 lines
4.5 KiB
Markdown

# Usage Tutorials
This document provides hands-on tutorials for sending data to DataStack, integrating client SDKs, and exporting datasets.
---
## Tutorial 1: Sending Events via HTTP REST (`curl`)
The fastest way to send data to DataStack without writing application code is via the HTTP REST API.
### 1. Retrieve Your Client Token
In `/vcon/clients`, register a client (e.g., `bash_script_01`) and copy the generated Bearer JWT token.
### 2. Send a Single Event
```bash
curl -X POST "https://your-domain.example/api/events" \
-H "Authorization: Bearer YOUR_CLIENT_JWT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"eventId": "order_1001",
"timestamp": "2026-11-17T12:00:00Z",
"data": {
"item": "Mechanical Keyboard",
"price": 129.99,
"status": "completed"
}
}'
```
**Expected Response**:
```json
{
"success": true,
"eventId": "order_1001",
"event": {
"clientId": "bash_script_01",
"eventId": "order_1001",
"timestamp": "2026-11-17T12:00:00Z",
"payload": "{\"item\":\"Mechanical Keyboard\",\"price\":129.99,\"status\":\"completed\"}"
}
}
```
### 3. Send a Batch of Events
```bash
curl -X POST "https://your-domain.example/api/events" \
-H "Authorization: Bearer YOUR_CLIENT_JWT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"batch": [
{ "eventId": "evt_01", "data": { "metric": "cpu", "value": 45.2 } },
{ "eventId": "evt_02", "data": { "metric": "ram", "value": 72.8 } }
]
}'
```
---
## Tutorial 2: Integrating the Python Client SDK
DataStack provides an official, pip-installable Python SDK featuring real-time Socket.IO streaming, automatic HTTP fallback, and offline disk spooling.
### 1. Install the SDK
```bash
pip install datastack
# Or install backward-compatible package:
pip install datastack-sdk
```
### 2. Create Configuration (`config.datastack.py`)
In the root of your Python project, create a `config.datastack.py` file:
```python
DATASTACK_CONFIG = {
"SERVER_URL": "https://your-domain.example", # WebSocket & proxy endpoint
"CLIENT_TOKEN": "YOUR_CLIENT_JWT_TOKEN", # Token from /vcon/clients
"AUTO_CONNECT": True,
"ENABLE_HTTP_FALLBACK": True,
"OFFLINE_SPOOL_ENABLED": True,
}
```
### 3. Stream Telemetry from Python
```python
from datastack import DataStack
import time
# Client automatically discovers and loads config.datastack.py
client = DataStack()
print(f"Connected to DataStack as client: {client.client_id}")
for i in range(5):
response = client.send(
data={
"sensor_id": "temp_probe_04",
"temperature_celsius": 24.5 + i * 0.2,
"humidity_percent": 55,
},
event_id=f"reading_{i}_{int(time.time())}"
)
print(f"Event sent: {response}")
time.sleep(1)
client.disconnect()
```
---
## Tutorial 3: Streaming Local Files (`send_file`)
If you have offline log files or CSV datasets that need to be ingested into DataStack, use the SDK's built-in file streaming:
```python
from datastack import DataStack
client = DataStack()
# Supports .csv, .json, .jsonl, and .ndjson
result = client.send_file(
file_path="./daily_orders.csv",
chunk_size=250, # Stream in batches of 250 records
extra_metadata={"imported_by": "nightly_cron", "env": "production"}
)
print(f"File Ingestion Complete:")
print(f" Processed rows: {result['processed']}")
print(f" Duplicates skipped: {result['duplicates']}")
print(f" Chunks sent: {result['chunksSent']}")
```
- **Lossless Handling**: If your server rate limits the file, the SDK automatically pauses and retries.
- **Deduplication**: Rows without an explicit `eventId` receive deterministic SHA-256 IDs, preventing duplicate rows if the script runs twice.
---
## Tutorial 4: Exporting Data for Analysis
You can stream historical partitions for analysis in pandas, Python, or command-line scripts.
### 1. Stream CSV Export via `curl`
```bash
curl -X GET "https://your-domain.example/api/export?format=csv&clientId=bash_script_01" \
-H "Cookie: datastack_admin_session=YOUR_ADMIN_SESSION_TOKEN" \
-o export_bash_script.csv
```
### 2. Load Directly into pandas
```python
import pandas as pd
import requests
import io
session_cookie = "YOUR_ADMIN_SESSION_COOKIE"
export_url = "https://your-domain.example/api/export?format=csv&clientId=all"
response = requests.get(
export_url,
cookies={"datastack_admin_session": session_cookie},
stream=True
)
df = pd.read_csv(io.StringIO(response.text))
print("Exported DataFrame:")
print(df.head())
```