pargar/docs/operations.md

Operations

Health

curl -k "$PUBLIC_URL/api/health"    # {"status":"ok"}
kubectl -n pargar get pods,ingressroute,svc
kubectl -n pargar logs deploy/pargar-backend --tail=100

Backup

Compose

./scripts/backup.sh [outdir]

Output: postgres.sql.gz + RustFS data archive (when possible).

Helm / k3s

With backup.enabled=true:

• CronJob pargar-pg-backup on backup.schedule (default daily 03:00 UTC)
• Files on PVC pargar-backups
• Auto-delete older than backup.retentionDays

Manual restore (summary):

# Example: copy dump from pod/PVC then
gunzip -c pargar-YYYYMMDD.sql.gz | psql "$DATABASE_URL"

Back up object storage separately (bucket versioning / RustFS PVC snapshot).

Metrics and tracing

• GET /metrics — without METRICS_TOKEN and outside DevMode → 404
• Header X-Metrics-Token or Authorization: Bearer …
• OTLP: OTEL_EXPORTER_OTLP_ENDPOINT + OTEL_SERVICE_NAME

Updating images on k3s

./scripts/k3s-load-images.sh
./scripts/helm-k3s.sh
kubectl -n pargar rollout restart deploy/pargar-backend deploy/pargar-frontend
kubectl -n pargar rollout status deploy/pargar-backend

The local tag uses IfNotPresent; always restart after import.

Common troubleshooting

Symptom	Likely cause	Action
Seed video will not play	sample.mp4 missing in storage	Check sample media logs; ensure file is in the image; restart backend
Sudden 401 after upgrade	JWT secret changed	Do not rotate the secret unless intentional; users must log in again
Backend CrashLoop after changing DB password in Secret	Old PVC password ≠ new Secret	Restore previous Secret password or re-seed DB
Repeated 429	Rate limit	Wait; honor Retry-After; DevMode behaves differently
Empty CAPTCHA in tests	CAPTCHA_EXPOSE_ANSWER only with DevMode	Tests set both true
Helm lint/template differs from cluster	lookup empty without API	Install/upgrade against a real cluster
Frontend does not show changes	Stale image	Rebuild + import + rollout restart

Logging

• Requests with request_id, user_id, status, duration
• Security events: login, recovery, password reset, …
• Critical-path internals: writeInternalErr (log cause, generic user message)

Useful scripts

Script	Purpose
scripts/compose-dev.sh	Development Compose
scripts/compose-prod.sh	Production Compose
scripts/k3s-load-images.sh	Build + import into k3s containerd
scripts/helm-k3s.sh	helm upgrade --install
scripts/backup.sh	Compose backup
scripts/regenerate-sample-mp4.sh	Rebuild color bars with ffmpeg