Initial commit: tischlerctrl home automation project
This commit is contained in:
sebseb7
335
implementation_plan.md
Normal file
335
implementation_plan.md
Normal file
@@ -0,0 +1,335 @@
|
||||
# Sensor Data Collection System
|
||||
|
||||
A Node.js server that collects sensor data from multiple agents via WebSocket, stores it in SQLite with automatic data summarization and retention policies.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Central Server (Node.js)"
|
||||
WS[WebSocket Server :8080]
|
||||
DB[(SQLite Database)]
|
||||
AGG[Aggregation Job]
|
||||
WS --> DB
|
||||
AGG --> DB
|
||||
end
|
||||
|
||||
subgraph "AC Infinity Agent (Node.js)"
|
||||
AC[AC Infinity Client]
|
||||
AC -->|polls every 60s| ACAPI[AC Infinity Cloud API]
|
||||
AC -->|WebSocket| WS
|
||||
end
|
||||
|
||||
subgraph "Tapo Agent (Rust)"
|
||||
TAPO[Tapo Client]
|
||||
TAPO -->|polls every 60s| PLUG[Tapo P100/P110]
|
||||
TAPO -->|WebSocket| WS
|
||||
end
|
||||
|
||||
subgraph "Custom CLI Agent"
|
||||
CLI[Shell Script]
|
||||
CLI -->|WebSocket| WS
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Review Required
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **Tapo Agent Language Choice**: I recommend **Rust** for the Tapo agent because:
|
||||
> - Compiles to a single ~2MB static binary
|
||||
> - Uses ~5-10MB RAM at runtime
|
||||
> - Excellent [tapo crate](https://crates.io/crates/tapo) already exists
|
||||
> - Easy cross-compilation for Raspberry Pi
|
||||
>
|
||||
> Alternatively, I could write it in **Go** (would need to implement protocol from scratch) or as a **Node.js** agent (but you mentioned wanting it lightweight).
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **AC Infinity Credentials**: The AC Infinity API requires email/password authentication to their cloud service. These will need to be stored in configuration.
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
tischlerctrl/
|
||||
├── server/
|
||||
│ ├── package.json
|
||||
│ ├── src/
|
||||
│ │ ├── index.js # Entry point
|
||||
│ │ ├── config.js # Configuration loader
|
||||
│ │ ├── db/
|
||||
│ │ │ ├── schema.js # SQLite schema + migrations
|
||||
│ │ │ └── queries.js # Database operations
|
||||
│ │ ├── websocket/
|
||||
│ │ │ ├── server.js # WebSocket server
|
||||
│ │ │ └── handlers.js # Message handlers
|
||||
│ │ └── jobs/
|
||||
│ │ ├── aggregator.js # Data summarization job
|
||||
│ │ └── cleanup.js # Data retention cleanup
|
||||
│ └── data/
|
||||
│ └── sensors.db # SQLite database file
|
||||
│
|
||||
├── agents/
|
||||
│ ├── ac-infinity/
|
||||
│ │ ├── package.json
|
||||
│ │ └── src/
|
||||
│ │ ├── index.js # Entry point
|
||||
│ │ ├── config.js # Configuration
|
||||
│ │ ├── ac-client.js # AC Infinity API client
|
||||
│ │ └── ws-client.js # WebSocket client with reconnect
|
||||
│ │
|
||||
│ ├── tapo/
|
||||
│ │ ├── Cargo.toml
|
||||
│ │ └── src/
|
||||
│ │ └── main.rs # Rust Tapo agent
|
||||
│ │
|
||||
│ └── cli/
|
||||
│ └── sensor-send # Shell script CLI tool
|
||||
│
|
||||
├── .env.example # Example environment variables
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### Server - Database Schema
|
||||
|
||||
#### [NEW] [schema.js](file:///home/seb/src/tischlerctrl/server/src/db/schema.js)
|
||||
|
||||
SQLite tables:
|
||||
|
||||
```sql
|
||||
-- API keys for agent authentication
|
||||
CREATE TABLE api_keys (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
key TEXT UNIQUE NOT NULL,
|
||||
name TEXT NOT NULL, -- e.g., "ac-infinity-agent"
|
||||
device_prefix TEXT NOT NULL, -- e.g., "ac:" or "tapo:"
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
last_used_at DATETIME
|
||||
);
|
||||
|
||||
-- Raw sensor data (1-minute resolution, kept for 1 week)
|
||||
CREATE TABLE sensor_data (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp DATETIME NOT NULL,
|
||||
device TEXT NOT NULL, -- e.g., "ac:controller-69-grow"
|
||||
channel TEXT NOT NULL, -- e.g., "temperature", "humidity", "power"
|
||||
value REAL NOT NULL,
|
||||
INDEX idx_sensor_data_time (timestamp),
|
||||
INDEX idx_sensor_data_device (device, channel)
|
||||
);
|
||||
|
||||
-- 10-minute aggregated data (kept for 1 month)
|
||||
CREATE TABLE sensor_data_10m (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp DATETIME NOT NULL, -- Rounded to 10-min boundary
|
||||
device TEXT NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
value REAL NOT NULL, -- Averaged value
|
||||
sample_count INTEGER NOT NULL,
|
||||
UNIQUE(timestamp, device, channel)
|
||||
);
|
||||
|
||||
-- 1-hour aggregated data (kept forever)
|
||||
CREATE TABLE sensor_data_1h (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp DATETIME NOT NULL, -- Rounded to 1-hour boundary
|
||||
device TEXT NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
value REAL NOT NULL, -- Averaged value
|
||||
sample_count INTEGER NOT NULL,
|
||||
UNIQUE(timestamp, device, channel)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Server - WebSocket Protocol
|
||||
|
||||
#### [NEW] [server.js](file:///home/seb/src/tischlerctrl/server/src/websocket/server.js)
|
||||
|
||||
**Authentication Flow:**
|
||||
1. Client connects to `ws://server:8080`
|
||||
2. Client sends: `{ "type": "auth", "apiKey": "xxx" }`
|
||||
3. Server validates API key, responds: `{ "type": "auth", "success": true, "devicePrefix": "ac:" }`
|
||||
4. Client is now authenticated and can send data
|
||||
|
||||
**Data Ingestion Message:**
|
||||
```json
|
||||
{
|
||||
"type": "data",
|
||||
"readings": [
|
||||
{ "device": "controller-69-grow", "channel": "temperature", "value": 24.5 },
|
||||
{ "device": "controller-69-grow", "channel": "humidity", "value": 65.2 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Server prepends `devicePrefix` to device names and adds timestamp.
|
||||
|
||||
**Keepalive:**
|
||||
- Server sends `ping` every 30 seconds
|
||||
- Client responds with `pong`
|
||||
- Connection closed after 90 seconds of no response
|
||||
|
||||
---
|
||||
|
||||
### Server - Aggregation Jobs
|
||||
|
||||
#### [NEW] [aggregator.js](file:///home/seb/src/tischlerctrl/server/src/jobs/aggregator.js)
|
||||
|
||||
Runs every 10 minutes:
|
||||
|
||||
1. **10-minute aggregation**:
|
||||
- Select data from `sensor_data` older than 10 minutes
|
||||
- Group by device, channel, and 10-minute bucket
|
||||
- Calculate average, insert into `sensor_data_10m`
|
||||
|
||||
2. **1-hour aggregation**:
|
||||
- Select data from `sensor_data_10m` older than 1 hour
|
||||
- Group by device, channel, and 1-hour bucket
|
||||
- Calculate weighted average, insert into `sensor_data_1h`
|
||||
|
||||
#### [NEW] [cleanup.js](file:///home/seb/src/tischlerctrl/server/src/jobs/cleanup.js)
|
||||
|
||||
Runs every hour:
|
||||
- Delete from `sensor_data` where timestamp < NOW - 7 days
|
||||
- Delete from `sensor_data_10m` where timestamp < NOW - 30 days
|
||||
|
||||
---
|
||||
|
||||
### AC Infinity Agent
|
||||
|
||||
#### [NEW] [ac-client.js](file:///home/seb/src/tischlerctrl/agents/ac-infinity/src/ac-client.js)
|
||||
|
||||
Port of the TypeScript AC Infinity client to JavaScript ES modules:
|
||||
|
||||
- `login(email, password)` → Returns userId token
|
||||
- `getDevicesListAll()` → Returns all controllers with sensor readings
|
||||
- Polling interval: 60 seconds
|
||||
- Extracts: temperature, humidity, VPD (if available) per controller
|
||||
|
||||
**Data extraction from API response:**
|
||||
```javascript
|
||||
// Each device in response has:
|
||||
// - devId, devName
|
||||
// - devSettings.temperature (°C * 100)
|
||||
// - devSettings.humidity (% * 100)
|
||||
// We normalize and send to server
|
||||
```
|
||||
|
||||
#### [NEW] [ws-client.js](file:///home/seb/src/tischlerctrl/agents/ac-infinity/src/ws-client.js)
|
||||
|
||||
WebSocket client with:
|
||||
- Auto-reconnect with exponential backoff (1s → 2s → 4s → ... → 60s max)
|
||||
- Authentication on connect
|
||||
- Heartbeat response
|
||||
- Message queue during disconnection
|
||||
|
||||
---
|
||||
|
||||
### Tapo Agent (Rust)
|
||||
|
||||
#### [NEW] [main.rs](file:///home/seb/src/tischlerctrl/agents/tapo/src/main.rs)
|
||||
|
||||
Uses [tapo crate](https://crates.io/crates/tapo) for P100/P110 communication.
|
||||
|
||||
**Features:**
|
||||
- Configuration via environment variables or TOML file
|
||||
- WebSocket client with tungstenite crate
|
||||
- Auto-reconnect with backoff
|
||||
- Polls devices every 60 seconds
|
||||
|
||||
**Data collected:**
|
||||
| Device | Channel | Description |
|
||||
|--------|---------|-------------|
|
||||
| P100 | `state` | 0 = off, 1 = on |
|
||||
| P110 | `state` | 0 = off, 1 = on |
|
||||
| P110 | `power` | Current power in watts |
|
||||
| P110 | `energy_today` | Energy used today in Wh |
|
||||
|
||||
**Build for Raspberry Pi:**
|
||||
```bash
|
||||
# Cross-compile for ARM
|
||||
cross build --release --target armv7-unknown-linux-gnueabihf
|
||||
# Binary: ~2MB, runs with ~8MB RAM
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Custom CLI Agent
|
||||
|
||||
#### [NEW] [sensor-send](file:///home/seb/src/tischlerctrl/agents/cli/sensor-send)
|
||||
|
||||
A shell script using `websocat` (lightweight WebSocket CLI tool):
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Usage: sensor-send --device=mydevice --channel=temp --value=23.5
|
||||
|
||||
API_KEY="${SENSOR_API_KEY:-}"
|
||||
SERVER="${SENSOR_SERVER:-ws://localhost:8080}"
|
||||
|
||||
sensor-send mydevice temperature 23.5
|
||||
```
|
||||
|
||||
Requires: `websocat` (single binary, ~3MB, available via cargo or apt)
|
||||
|
||||
---
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
### Server `.env`
|
||||
```bash
|
||||
PORT=8080
|
||||
DB_PATH=./data/sensors.db
|
||||
# Generate API keys via CLI: node src/cli/generate-key.js "ac-infinity" "ac:"
|
||||
```
|
||||
|
||||
### AC Infinity Agent `.env`
|
||||
```bash
|
||||
SERVER_URL=ws://192.168.1.100:8080
|
||||
API_KEY=your-api-key-here
|
||||
AC_EMAIL=your@email.com
|
||||
AC_PASSWORD=your-password
|
||||
POLL_INTERVAL_MS=60000
|
||||
```
|
||||
|
||||
### Tapo Agent `config.toml`
|
||||
```toml
|
||||
server_url = "ws://192.168.1.100:8080"
|
||||
api_key = "your-api-key-here"
|
||||
poll_interval_secs = 60
|
||||
|
||||
[[devices]]
|
||||
ip = "192.168.1.50"
|
||||
name = "grow-light-plug"
|
||||
type = "P110" # or "P100"
|
||||
tapo_email = "your@email.com"
|
||||
tapo_password = "your-tapo-password"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Plan
|
||||
|
||||
### Automated Tests
|
||||
1. **Server unit tests**: Database operations, aggregation logic
|
||||
2. **Integration test**: Start server, connect mock agent, verify data flow
|
||||
3. **Run commands**:
|
||||
```bash
|
||||
cd server && npm test
|
||||
cd agents/ac-infinity && npm test
|
||||
```
|
||||
|
||||
### Manual Verification
|
||||
1. Start server, verify WebSocket accepts connections
|
||||
2. Send test data via CLI agent, verify it appears in database
|
||||
3. Wait 10+ minutes, verify aggregation runs and data appears in `sensor_data_10m`
|
||||
4. Connect AC Infinity agent with real credentials, verify sensor readings
|
||||
5. Deploy Tapo agent to Raspberry Pi, verify plug data collection
|
||||
Reference in New Issue
Block a user