Add initial MQTT scrubber service scaffold
This commit is contained in:
@@ -1,3 +1,117 @@
|
||||
# mqqt-scrubber
|
||||
|
||||
Mqtt scrubber in golang.
|
||||
Small Go daemon for collecting MQTT topics, normalizing mostly Tasmota payloads, and writing them into InfluxDB v3.
|
||||
|
||||
## Current status
|
||||
|
||||
This repository now contains:
|
||||
|
||||
- a milestone work plan in `docs/WORKPLAN.md`
|
||||
- a technical design in `docs/ARCHITECTURE.md`
|
||||
- an initial runnable scaffold for the background service
|
||||
|
||||
## Scope for v1
|
||||
|
||||
- subscribe to MQTT topics, primarily `tele/+/SENSOR` and `tele/+/STATE`
|
||||
- parse Tasmota JSON payloads into normalized records
|
||||
- batch writes to InfluxDB v3 using the HTTP `write_lp` endpoint
|
||||
- run as a long-lived background process with reconnect and graceful shutdown
|
||||
|
||||
## Project layout
|
||||
|
||||
```text
|
||||
cmd/mqqt-scrubber/ application entrypoint
|
||||
internal/config/ config loading and validation
|
||||
internal/influx/ InfluxDB v3 HTTP writer
|
||||
internal/model/ internal message and record types
|
||||
internal/mqtt/ MQTT subscriber wrapper
|
||||
internal/parser/ Tasmota payload parsing and flattening
|
||||
internal/pipeline/ batching, flushing, and service orchestration
|
||||
docs/ work plan and architecture notes
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The service loads configuration from a JSON file passed with `-config` and then applies environment variable overrides.
|
||||
|
||||
Start with:
|
||||
|
||||
```bash
|
||||
cp config.example.json config.json
|
||||
```
|
||||
|
||||
Supported environment variables:
|
||||
|
||||
- `MQTT_SCRUBBER_MQTT_BROKER`
|
||||
- `MQTT_SCRUBBER_MQTT_USERNAME`
|
||||
- `MQTT_SCRUBBER_MQTT_PASSWORD`
|
||||
- `MQTT_SCRUBBER_MQTT_CLIENT_ID`
|
||||
- `MQTT_SCRUBBER_MQTT_TOPICS`
|
||||
- `MQTT_SCRUBBER_MQTT_QOS`
|
||||
- `MQTT_SCRUBBER_INFLUX_URL`
|
||||
- `MQTT_SCRUBBER_INFLUX_DATABASE`
|
||||
- `MQTT_SCRUBBER_INFLUX_TOKEN`
|
||||
- `MQTT_SCRUBBER_INFLUX_PRECISION`
|
||||
- `MQTT_SCRUBBER_APP_BATCH_SIZE`
|
||||
- `MQTT_SCRUBBER_APP_BUFFER_SIZE`
|
||||
- `MQTT_SCRUBBER_APP_FLUSH_INTERVAL`
|
||||
- `MQTT_SCRUBBER_APP_FLUSH_TIMEOUT`
|
||||
- `MQTT_SCRUBBER_APP_LOG_LEVEL`
|
||||
- `MQTT_SCRUBBER_APP_HEALTH_ADDRESS`
|
||||
|
||||
`MQTT_SCRUBBER_MQTT_TOPICS` expects a comma-separated list.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
go run ./cmd/mqqt-scrubber -config config.json
|
||||
```
|
||||
|
||||
The process also exposes simple health endpoints when `health_address` is set:
|
||||
|
||||
- `/healthz`
|
||||
- `/readyz`
|
||||
- `/metrics`
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
go build ./cmd/mqqt-scrubber
|
||||
```
|
||||
|
||||
## Docker
|
||||
|
||||
Build the runtime image:
|
||||
|
||||
```bash
|
||||
docker build -t mqqt-scrubber .
|
||||
```
|
||||
|
||||
Run it with your config mounted in:
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-p 8080:8080 \
|
||||
-v "$PWD/config.json:/app/config.json:ro" \
|
||||
mqqt-scrubber
|
||||
```
|
||||
|
||||
The container health check uses `http://127.0.0.1:8080/healthz`.
|
||||
|
||||
## Docker Compose
|
||||
|
||||
Bring the service up with the included compose file:
|
||||
|
||||
```bash
|
||||
docker-compose up --build -d
|
||||
```
|
||||
|
||||
It expects a local `config.json` next to the compose file and exposes port `8080` for health and metrics.
|
||||
|
||||
## Notes
|
||||
|
||||
- The repo name is kept as `mqqt-scrubber` to match the existing folder.
|
||||
- The current parser is intentionally narrow and optimized for Tasmota telemetry first.
|
||||
- Writes use line protocol over HTTP against InfluxDB v3 `/api/v3/write_lp`.
|
||||
- The Docker image includes a non-root runtime user and a simple HTTP health endpoint.
|
||||
- Runtime counters are exposed in Prometheus-style text format at `/metrics`.
|
||||
Reference in New Issue
Block a user