feat: per-sink delivery, journal forwarding, and operational API
Persistence: - one delivery state per sink, with claim, crash recovery, and index replacement - retention for delivered and dead records - LMDB writes moved off the event loop Pipeline: - one attempt per dispatch, with the next try scheduled by next_retry_at - backpressure through max_queue_size - reload re-reads YAML; SIGTERM drains and SIGHUP reloads Plugins: - journald source, enricher and filter processors, Loki sink - built-ins register when entry points are missing API and CLI: - health, metrics, per-sink queue, and dead-letter replay - startup prints the pipeline on one line Tests: - dispatch, plugins, metrics, CLI, and persistence coverage
This commit is contained in:
@@ -8,18 +8,59 @@ pid_file: /run/daena/daena.pid
|
||||
logging:
|
||||
level: info
|
||||
format: json
|
||||
verbose: false
|
||||
|
||||
api:
|
||||
grpc:
|
||||
host: 127.0.0.1
|
||||
port: 8642
|
||||
max_workers: 10
|
||||
metrics:
|
||||
enabled: true
|
||||
host: 127.0.0.1
|
||||
port: 8643
|
||||
|
||||
pipeline:
|
||||
batch_size: 100
|
||||
flush_interval: 5.0
|
||||
max_queue_size: 10000
|
||||
|
||||
retention:
|
||||
delivered_ttl_seconds: 86400
|
||||
dead_ttl_seconds: 604800
|
||||
interval_seconds: 60
|
||||
|
||||
health:
|
||||
max_pending_age_seconds: 300
|
||||
|
||||
# journald source, enricher, filter, and loki sink are registered plugins.
|
||||
# Example:
|
||||
# sources:
|
||||
# - name: system
|
||||
# type: journald
|
||||
# options:
|
||||
# units: ["ssh.service"]
|
||||
# cursor_path: /var/lib/daena/journal.cursor
|
||||
# processors:
|
||||
# - name: host
|
||||
# type: enricher
|
||||
# options:
|
||||
# hostname: true
|
||||
# static:
|
||||
# env: prod
|
||||
# - name: drop_debug
|
||||
# type: filter
|
||||
# options:
|
||||
# drop:
|
||||
# body.priority: "7"
|
||||
# sinks:
|
||||
# - name: loki
|
||||
# type: loki
|
||||
# options:
|
||||
# url: http://127.0.0.1:3100/loki/api/v1/push
|
||||
# labels:
|
||||
# job: daena
|
||||
|
||||
persistence:
|
||||
backend: lmdb
|
||||
lmdb:
|
||||
|
||||
Reference in New Issue
Block a user