feat: implement daena MVP core service
Complete implementation of the daena log/event collection service:
Core runtime:
- DaenaRuntime service lifecycle (start/stop/reload)
- Pipeline orchestrator with source/persist/dispatch flow
- LMDB persistence backend with key-prefix partitioning
- Plugin discovery via Python entry points
Pipeline:
- Source/Processor/Sink ABC interfaces with DeliveryResult
- DummySource for testing and development
- DummySink and HTTPSink implementations
- PassthroughProcessor
- Record dataclass for in-memory records
- StoredRecord model with RecordState tracking
REST API (FastAPI, v1):
- GET /system/{status,health,config}
- GET /plugins
- GET /queue, /queue/{pending,dead}
- GET /sinks
- Versioned under /api/v1 prefix
CLI (Typer):
- status, health, queue, sinks, plugins, config, version
Persistence (LMDB):
- Single-database key-prefix design (no named DBs)
- Index keys: idx:{state}:{ts}\x00{id}
- Record keys: rec:{id}
- Full CRUD: put/get/ack/nack/dead/update_state
- Pending/retry/dead-letter queries with state filtering
- delete_old for TTL-based retention
Deployment:
- systemd unit file (deploy/daena.service)
- Example YAML config (config/daena.yaml)
- Entry points: daena (CLI), daena-serve (uvicorn server)
Quality:
- 23 unit tests covering config, persistence, pipeline
- ruff linting and formatting pass clean
- Py3.13 compatible, uv-managed, pyenv-pinned
- GPG-signed commit on dev branch
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from daena.config import DaenaConfig
|
||||
from daena.persistence.lmdb import LMDBBackend
|
||||
from daena.persistence.models import StoredRecord
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_db_path(tmp_path: Path) -> Path:
|
||||
return tmp_path / "lmdb"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def lmdb_backend(tmp_db_path: Path) -> Iterator[LMDBBackend]:
|
||||
backend = LMDBBackend(
|
||||
path=str(tmp_db_path),
|
||||
map_size=10_485_760,
|
||||
)
|
||||
backend.open()
|
||||
yield backend
|
||||
backend.close()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sample_record() -> StoredRecord:
|
||||
return StoredRecord(
|
||||
source="test_source",
|
||||
record_type="test",
|
||||
body={"message": "hello"},
|
||||
metadata={"env": "test"},
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def minimal_config() -> DaenaConfig:
|
||||
return DaenaConfig()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config_file(tmp_path: Path) -> Path:
|
||||
path = tmp_path / "daena.yaml"
|
||||
data = {
|
||||
"data_dir": "/tmp/daena_data",
|
||||
"logging": {"level": "debug", "format": "json"},
|
||||
"sources": [
|
||||
{
|
||||
"name": "test",
|
||||
"type": "dummy",
|
||||
"enabled": True,
|
||||
"options": {"interval": 1.0},
|
||||
}
|
||||
],
|
||||
"sinks": [
|
||||
{
|
||||
"name": "output",
|
||||
"type": "dummy",
|
||||
"enabled": True,
|
||||
"options": {},
|
||||
}
|
||||
],
|
||||
}
|
||||
with open(path, "w") as f:
|
||||
yaml.dump(data, f)
|
||||
return path
|
||||
Reference in New Issue
Block a user