Avoid full unflagged mailbox rescan on every IDLE wakeup #12

New issue

Closed

opened 2026-05-11 18:52:27 +02:00 by heiko · 4 comments

heiko commented

2026-05-11 18:52:27 +02:00

Owner

Problem

internal/imap/idle.go searches all messages without the custom keyword after each IDLE cycle.

Reference: internal/imap/idle.go:102.

On a mailbox with many historical non-invoice messages, every new message can trigger a full rescan of all unflagged mail. In dry-run mode this is worse because no messages are flagged, so the same historical set is repeatedly scanned.

Expected

IDLE wakeups should primarily process new UIDs. Broad recovery scans should be bounded, scheduled, or made explicit.

Suggested fix

Limit the unflagged search to a recent UID/date range, run it only on explicit recovery/force mode, or make it configurable with clear defaults.

## Problem `internal/imap/idle.go` searches all messages without the custom keyword after each IDLE cycle. Reference: `internal/imap/idle.go:102`. On a mailbox with many historical non-invoice messages, every new message can trigger a full rescan of all unflagged mail. In dry-run mode this is worse because no messages are flagged, so the same historical set is repeatedly scanned. ## Expected IDLE wakeups should primarily process new UIDs. Broad recovery scans should be bounded, scheduled, or made explicit. ## Suggested fix Limit the unflagged search to a recent UID/date range, run it only on explicit recovery/force mode, or make it configurable with clear defaults.

heiko commented

2026-05-11 19:13:30 +02:00

Author

Owner

Fixed by 77735bb (fix: IDLE wakeup no longer rescans entire mailbox).

That commit removes the SearchUnflaggedUIDs(customFlag) call from the IDLE hot path and changes the loop to search only for new UIDs (> lastUID), repeating until no more new messages arrive.

The regression branch regression/issue-12-idle-unflagged-scan now passes on current master.

Fixed by 77735bb (fix: IDLE wakeup no longer rescans entire mailbox). That commit removes the SearchUnflaggedUIDs(customFlag) call from the IDLE hot path and changes the loop to search only for new UIDs (> lastUID), repeating until no more new messages arrive. The regression branch regression/issue-12-idle-unflagged-scan now passes on current master.

heiko closed this issue

2026-05-11 19:13:30 +02:00

heiko commented

2026-05-11 20:38:16 +02:00

Author

Owner

Branch renamed per convention:

Old: regression/issue-12-idle-unflagged-scan
New: fix/12-idle-unflagged-scan

The old remote branch has been deleted. This issue remains closed because the regression guard already passes on current master.

Branch renamed per convention: Old: regression/issue-12-idle-unflagged-scan New: fix/12-idle-unflagged-scan The old remote branch has been deleted. This issue remains closed because the regression guard already passes on current master.

heiko commented

2026-05-11 20:55:56 +02:00

Author

Owner

Implementation branch: fix/12-idle-unflagged-scan-codex

Implementation branch: [fix/12-idle-unflagged-scan-codex](https://forgejo.schlittermann.de/IUS/xr-invoiced/src/branch/fix/12-idle-unflagged-scan-codex)

heiko commented

2026-05-11 21:08:07 +02:00

Author

Owner

Closing as fixed.

The issue was addressed by commit 77735bb, fix: IDLE wakeup no longer rescans entire mailbox.

That change removed the unbounded SearchUnflaggedUIDs(customFlag) call from the IDLE hot path. After each IDLE wakeup the scanner now searches only for new UIDs greater than lastUID, repeating until no more new messages arrive.

Regression coverage exists on branch fix/12-idle-unflagged-scan and passes on current master.

Closing as fixed. The issue was addressed by commit 77735bb, fix: IDLE wakeup no longer rescans entire mailbox. That change removed the unbounded SearchUnflaggedUIDs(customFlag) call from the IDLE hot path. After each IDLE wakeup the scanner now searches only for new UIDs greater than lastUID, repeating until no more new messages arrive. Regression coverage exists on branch fix/12-idle-unflagged-scan and passes on current master.