XRechnung IMAP Scanner

A Go service that connects to an IMAP server, monitors mailboxes in real-time via IMAP IDLE, detects XRechnung invoices (UBL / UN-CEFACT CII XML and Factur-X hybrid PDFs), and applies a custom IMAP flag to matching messages for easy filtering in mail clients.

User Guide — Thunderbird setup: see and filter tagged invoices

Administration Guide — installation, configuration, deployment

Features

• IMAP IDLE listener: Real-time monitoring of incoming mail; auto-reconnect on connection drop
• Multiple mailboxes: Monitor several mailboxes in parallel, each with its own IMAP connection
• Mailbox heartbeat logs: Every 60 minutes per mailbox, logs a summary window with processed/tagged/errors/reconnects and initial/startup/idle source counters
• XRechnung detection:
  • UBL Invoice (namespace: urn:oasis:names:specification:ubl:schema:xsd:Invoice-2)
  • UN-CEFACT CII (namespace: urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100)
  • CustomizationID validation against XRechnung profiles
  • Factur-X/ZUGFeRD hybrid PDFs (embedded XML via PDF object model, pdfcpu)
• Persistent state: Tracks UIDVALIDITY + last-scanned UID per mailbox; no re-scan on restart
• Custom IMAP keyword: Configurable keyword name (default: ius-xrechnung)
• Secure deployment: systemd unit with hardening, env var config, graceful shutdown

Quick Start

go build -o xr-invoiced ./cmd/xr-invoiced/
./xr-invoiced --doctor                    # verify IMAP credentials
./xr-invoiced --since "last month"        # preview what would be tagged
./xr-invoiced --apply                     # tag messages for real, enter IDLE

How It Works

1. Startup: Loads config, connects to IMAP (one connection per mailbox)
2. Initial scan: On first run or after UIDVALIDITY change, scans all existing messages; logs are marked scan=initial
3. State/Resume scan: On normal startup, resumes from saved state and scans only messages newer than the last-seen UID; logs are marked scan=startup
4. State persistence: Saves per-mailbox UIDVALIDITY + LastUID to JSON file
5. IDLE loop: Enters IMAP IDLE and listens for new messages; after wake-up, every processed message is logged with scan=idle
6. Message processing: For each new message:
   • Fetches full body + headers
   • Walks MIME attachments
   • Checks each attachment for XRechnung
   • Applies custom flag if match found
7. Reconnection: On IDLE timeout (≈28 min) or connection drop, auto-reconnects and resumes
8. Heartbeat: Every 60 minutes per mailbox, emits heartbeat with counters for the previous 60-minute window:
   • processed, tagged, errors, reconnects
   • initial, startup, idle

Logging

All log output goes to stdout only by default. --log-format=syslog/color-syslog is available if you want syslog-style <priority> framing on stdout, but auto-detection never selects it on its own.

The output format is autodetected based on the runtime environment:

│ Condition │ Format │ Description │ │──────────────────────────────────│─────────────────│──────────────────────────────────────│ │ systemd unit (auto) │ journal │ native journal export, structured fields, journalctl KEY=value filterable │ │ stdout is a terminal (auto) │ color-plain │ human-readable, ANSI colors │ │ otherwise, e.g. redirected (auto)│ jsonl │ structured JSON, one object per line │

Override with --log-format <format>: auto (default), plain/color-plain, jsonl/color-jsonl, syslog/color-syslog, journal/color-journal.

Architecture

cmd/xr-invoiced/main.go           Entry point; flag parsing; logging setup; signal handling
internal/app/app.go               Orchestration: doctor/list/remove-keyword, per-mailbox scan+IDLE loop
internal/config/config.go         Viper-based YAML + env override
internal/heartbeat/heartbeat.go   Per-mailbox activity counters + periodic heartbeat log
internal/infomessage/             One-time "tagging is active" info message
internal/journalhandler/          Native systemd journal socket handler (structured fields)
internal/traceio/                 Truncating writer for --debug=imap wire traces
internal/scanner/scanner.go       Orchestrator: fetch → walk → detect → flag
internal/scanner/attachment.go    MIME part walker
internal/state/state.go           Per-mailbox UIDVALIDITY + LastUID persistence
internal/xrechnung/detector.go    XML namespace + CustomizationID validation
internal/xrechnung/pdf.go         Factur-X/ZUGFeRD PDF embedded-file extraction (pdfcpu)

IMAP client (TLS/STARTTLS, IDLE, fetch, search, store) is go.schlittermann.de/heiko/imapclient, an external dependency rather than an in-repo package.

Known Limitations & Future Work

• SASL mechanisms: Uses plain LOGIN; no OAuth2/SASL XOAUTH2 support
• Email notifications: No alerting on scan errors; structured log output only
• Parallel processing: Messages are processed sequentially; parallel fetch/detect is a planned enhancement

License

MIT (or your chosen license)

References

• IMAP RFC 3501 & RFC 2177 (IDLE)
• XRechnung Specification
• Factur-X / ZUGFeRD Standard
• go-imap Documentation

Glossary

Term	Meaning
Flag	Generic IMAP per-message attribute (RFC 3501). Covers system flags and keywords.
System flag	Predefined by the RFC, backslash-prefixed: \Seen, \Answered, \Flagged, \Deleted, \Draft.
Keyword	User-defined flag without backslash, e.g. ius-xrechnung. This is what xr-invoiced sets.
Tag	Thunderbird's UI name for an IMAP keyword. Same data, different vocabulary.