feat: per-sink delivery, journal forwarding, and operational API
CI / lint (push) Canceled after 0s
CI / test (push) Canceled after 0s
CI / release (push) Canceled after 0s

Persistence:
- one delivery state per sink, with claim, crash recovery, and index replacement
- retention for delivered and dead records
- LMDB writes moved off the event loop

Pipeline:
- one attempt per dispatch, with the next try scheduled by next_retry_at
- backpressure through max_queue_size
- reload re-reads YAML; SIGTERM drains and SIGHUP reloads

Plugins:
- journald source, enricher and filter processors, Loki sink
- built-ins register when entry points are missing

API and CLI:
- health, metrics, per-sink queue, and dead-letter replay
- startup prints the pipeline on one line

Tests:
- dispatch, plugins, metrics, CLI, and persistence coverage
This commit is contained in:
ahp
2026-10-14 01:58:07 +03:30
parent 5e081144d5
commit 06d233054b
38 changed files with 3282 additions and 557 deletions
+41
View File
@@ -8,18 +8,59 @@ pid_file: /run/daena/daena.pid
logging:
level: info
format: json
verbose: false
api:
grpc:
host: 127.0.0.1
port: 8642
max_workers: 10
metrics:
enabled: true
host: 127.0.0.1
port: 8643
pipeline:
batch_size: 100
flush_interval: 5.0
max_queue_size: 10000
retention:
delivered_ttl_seconds: 86400
dead_ttl_seconds: 604800
interval_seconds: 60
health:
max_pending_age_seconds: 300
# journald source, enricher, filter, and loki sink are registered plugins.
# Example:
# sources:
# - name: system
# type: journald
# options:
# units: ["ssh.service"]
# cursor_path: /var/lib/daena/journal.cursor
# processors:
# - name: host
# type: enricher
# options:
# hostname: true
# static:
# env: prod
# - name: drop_debug
# type: filter
# options:
# drop:
# body.priority: "7"
# sinks:
# - name: loki
# type: loki
# options:
# url: http://127.0.0.1:3100/loki/api/v1/push
# labels:
# job: daena
persistence:
backend: lmdb
lmdb: