feat: native systemd journal socket handler (structured per-field export) #1

Open
opened 2026-08-01 13:07:49 +02:00 by heiko · 0 comments
Owner

Context

Wiring hslog into xr-invoiced surfaced a gap: our current journal format (FormatJournal/FormatColorJournal) is the common <priority>-prefix-on-stdout trick, relying on systemd's own stdout-capture to parse the <N> prefix (per sd-daemon convention). It writes into whatever io.Writer the caller supplies, consistent with every other hslog handler.

xr-invoiced previously shipped (v0.0.15, "native systemd journal handler with per-field structured logging") a different, richer integration: it dials /run/systemd/journal/socket (unixgram) directly and exports each slog attr as its own uppercase journal field (e.g. MAILBOX=INBOX, TAGGED=true), on top of a human-readable MESSAGE= line. That makes attrs queryable via journalctl MAILBOX=INBOX — something the stdout-prefix trick cannot do, since everything besides the priority ends up folded into one opaque MESSAGE field.

Why this doesn't fit today's Format model

  1. Different contract. Every hslog handler today is "format text into the io.Writer you gave me" (NewHandler(w, format, opts); detectKind/colorable etc. operate on w). A native journal-socket handler ignores w — it dials a fixed system path and only falls back to w (e.g. os.Stderr) if the dial fails.
  2. Different detection signal. FormatJournal's auto-detection (detectKind) triggers on JOURNAL_STREAM (is stdout specifically captured by the journal?). The native-socket approach is meaningful whenever INVOCATION_ID is set (are we a systemd unit at all?), independent of where stdout goes.

So this isn't a new kind value in Format — it needs its own constructor, e.g.:

func NewJournaldHandler(fallback io.Writer, opts *Options) slog.Handler

(dials the socket; on failure, falls back to a plain/text handler over fallback).

Reference implementation

We have a working, tested implementation to port from at go.schlittermann.de/ius/xr-invoiced (as of writing, in handler_journal.go on master, ~166 lines):

  • Connects via net.Dial("unixgram", "/run/systemd/journal/socket"); falls back to slog.NewTextHandler(fallback, ...) if the dial fails.
  • Native journal wire protocol: KEY=value\n for single-line values; for values containing \n, the length-prefixed binary form (KEY\n + uint64LE(len) + value + \n).
  • PRIORITY field mapped from slog.Level (0-tier mapping: >=Error -> 3, >=Warn -> 4, >=Info -> 6, else 7).
  • MESSAGE field: human-readable text with attrs inlined as key=value (mirrors what a text handler would show).
  • Every other attr also gets its own uppercase, sanitized journal field ([A-Z0-9_] only, leading underscore avoided since it's reserved for trusted journal fields).
  • WithAttrs/WithGroup supported (attrs accumulate; groups currently flatten, matching hslog's own group handling elsewhere would need reconciling).

Ask

Port/adapt this into hslog as a dedicated handler + constructor (not a Format value), with tests covering: the wire framing (both branches), field-name sanitization, the priority mapping, and the fallback-when-socket-unavailable path (probably via a net.Listen("unixgram", ...) test double rather than the real system path).

Until this lands, xr-invoiced is keeping its own handler_journal.go for journal-mode logging (hybrid approach: hslog for plain/color-plain/jsonl, our own handler for journal) rather than regressing the structured-export feature.

## Context Wiring hslog into `xr-invoiced` surfaced a gap: our current `journal` format (`FormatJournal`/`FormatColorJournal`) is the common `<priority>`-prefix-on-stdout trick, relying on systemd's own stdout-capture to parse the `<N>` prefix (per `sd-daemon` convention). It writes into whatever `io.Writer` the caller supplies, consistent with every other hslog handler. `xr-invoiced` previously shipped (v0.0.15, "native systemd journal handler with per-field structured logging") a different, richer integration: it dials `/run/systemd/journal/socket` (`unixgram`) directly and exports **each slog attr as its own uppercase journal field** (e.g. `MAILBOX=INBOX`, `TAGGED=true`), on top of a human-readable `MESSAGE=` line. That makes attrs queryable via `journalctl MAILBOX=INBOX` — something the stdout-prefix trick cannot do, since everything besides the priority ends up folded into one opaque `MESSAGE` field. ## Why this doesn't fit today's Format model 1. **Different contract.** Every hslog handler today is "format text into the `io.Writer` you gave me" (`NewHandler(w, format, opts)`; `detectKind`/`colorable` etc. operate on `w`). A native journal-socket handler ignores `w` — it dials a fixed system path and only falls back to `w` (e.g. `os.Stderr`) if the dial fails. 2. **Different detection signal.** `FormatJournal`'s auto-detection (`detectKind`) triggers on `JOURNAL_STREAM` (is *stdout specifically* captured by the journal?). The native-socket approach is meaningful whenever `INVOCATION_ID` is set (are we a systemd unit at all?), independent of where stdout goes. So this isn't a new `kind` value in `Format` — it needs its own constructor, e.g.: ```go func NewJournaldHandler(fallback io.Writer, opts *Options) slog.Handler ``` (dials the socket; on failure, falls back to a plain/text handler over `fallback`). ## Reference implementation We have a working, tested implementation to port from at `go.schlittermann.de/ius/xr-invoiced` (as of writing, in `handler_journal.go` on `master`, ~166 lines): - Connects via `net.Dial("unixgram", "/run/systemd/journal/socket")`; falls back to `slog.NewTextHandler(fallback, ...)` if the dial fails. - Native journal wire protocol: `KEY=value\n` for single-line values; for values containing `\n`, the length-prefixed binary form (`KEY\n` + `uint64LE(len)` + value + `\n`). - `PRIORITY` field mapped from `slog.Level` (0-tier mapping: >=Error -> 3, >=Warn -> 4, >=Info -> 6, else 7). - `MESSAGE` field: human-readable text with attrs inlined as `key=value` (mirrors what a text handler would show). - Every other attr also gets its own uppercase, sanitized journal field (`[A-Z0-9_]` only, leading underscore avoided since it's reserved for trusted journal fields). - `WithAttrs`/`WithGroup` supported (attrs accumulate; groups currently flatten, matching hslog's own group handling elsewhere would need reconciling). ## Ask Port/adapt this into hslog as a dedicated handler + constructor (not a `Format` value), with tests covering: the wire framing (both branches), field-name sanitization, the priority mapping, and the fallback-when-socket-unavailable path (probably via a `net.Listen("unixgram", ...)` test double rather than the real system path). Until this lands, `xr-invoiced` is keeping its own `handler_journal.go` for journal-mode logging (hybrid approach: hslog for plain/color-plain/jsonl, our own handler for journal) rather than regressing the structured-export feature.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
heiko/hslog#1
No description provided.