Grain 0.8.0-rc5

Original release notes

GRAIN v0.8.0-rc5

Fixes from rc4's first production run Release candidate 10.1.2026 Full Changelog: https://github.com/0ceanSlim/grain/compare/v0.8.0-rc4...v0.8.0-rc5

---

rc4 ran on a live relay for ten days, and that run turned up a long-standing bug: events containing &, < or > were acknowledged and then silently thrown away. rc5 fixes it, along with the problems the same run exposed. Purge now counts an event's age from when the relay received it. Request filters are strict. The public stats endpoint no longer scans the whole database per call. Logs land where --data-dir says.

---

Events with `&`, `<` or `>` are stored

The bug: any event whose content or tags contained &, < or > got OK true, was logged as stored, and was never written. That included URLs with query strings in content or in r/imeta tags. The cause: grain encoded events for nostrdb with Go's default JSON encoder, which writes those characters as \u0026-style escapes, and nostrdb's parser can't read escapes. Every release since grain moved to nostrdb did this. On the relay where it was found, not one stored note contained any of the three.
Profiles were erased, not just dropped. For replaceable events (profiles, contact lists, relay lists), the old version was deleted before the new one was handed to the database. A profile update containing & left the author with no profile at all.
The fix: grain now builds the JSON it hands nostrdb, and computes event ids, with exactly the escaping NIP-01 specifies. The id check also handles U+2028/U+2029, control characters and a literal \u0026 typed in a note, and matches nostr-tools on all of them.
No more silent drops. Before anything is queued, and before a replaceable's old version or a NIP-09 target is deleted, grain runs nostrdb's own parser and id check on the event. If nostrdb would refuse it, the client gets OK false and the operator gets an ERROR line with the event id, kind and size.
Clients that use nostrdb can read these events too. Outgoing messages used the same escaping, and Damus and Notedeck ingest relay messages with nostrdb. Events are now sent with &, < and > as-is.
Events dropped before this fix were never written, so there is nothing to recover. Each author's next publish will land.

Purge counts age from arrival

The bug: purge measured keep_interval_hours from each event's created_at. Clients often publish events long after writing them: republished profiles, contact lists, relay lists, backfills. Such an event arrived already past the window and was deleted at the next purge run, a few minutes later.
New default: event_purge.retention_clock: received. An event that arrives more than late_arrival_minutes (default 10) after its created_at has its arrival time recorded, and purge ages it from then. Events that arrive sooner are aged from created_at, so they can be purged at most that many minutes early. Arrivals are kept in arrivals.log next to the database. The first arrival counts, so re-sending a stored event doesn't restart its clock.
retention_clock: created_at restores the old behavior. Both settings are in the dashboard's Event Purge form.
Purge reaches every kind. In category mode, purge found which kinds to walk by sampling the newest 50,000 events. On a larger database, any kind whose events were all older than that was never purged. It now checks every kind number for events past the cutoff, about 0.6 s per run.

Stricter request filters

Malformed filters are rejected, not widened. A filter field with the wrong type used to be dropped, so {"kinds":["1"]}, a 65-character author, a non-hex id or "limit": -1 each returned events from the whole database. REQ and COUNT now parse filters strictly and answer bad input with CLOSED "invalid: filter N: ...".
An empty list matches nothing. An empty ids, authors, kinds or #tag list now matches no events, as in nostr-tools and go-nostr. It used to match everything.
Prefix lookups still work. ids and authors prefixes of 1-64 hex characters stay supported. The first prefix query on a connection gets an advisory NOTICE, because a prefix can't be verified against a full id.

Subscriptions and messages

Evictions are announced. At max_subscriptions_per_client, a new subscription replaces the oldest and the relay sends CLOSED for it. The relay used to pick a random victim without telling the client, and also evicted when a REQ only replaced an existing subscription.
No reply to CLOSE. NIP-01 reserves CLOSED for subscriptions the relay ends, so a client's own CLOSE now gets no answer. A malformed CLOSE gets a NOTICE.
NOTICE matches NIP-01. Notices are now ["NOTICE", <message>]. Some used to carry an extra empty element.
NIP-77 is refused. grain doesn't support negentropy yet. NEG-OPEN now gets an immediate NEG-ERR, so a client falls back to REQ instead of waiting out a timeout.

Relay stats

GET /api/v1/relay/stats is cached. This public endpoint used to scan the whole database twice per request, about 340 ms at 300k events, so any caller could burn the relay's CPU. It now serves from a 30-second cache that refreshes in the background, and no longer returns an empty body while startup work is running.
Kind buckets are fixed. The "list" bucket reports kind 30000, so by_kind no longer has two rows for kind 1. The dashboard donut shows a single "other" slice, and on phones it stacks above its legend.
total_messages_sent is a real total. A 5-minute stats log reset the same counter, so the dashboard number went down as well as up. It now only grows.

Startup and logging

--data-dir covers every log line. Lines written before the data directory was resolved, including the whole nostrdb open, used to land in debug.log in the working directory. They now go to the data directory from the first line.
Slow startup steps are visible. The nostrdb open, the startup purge and the NIP-40 expiration bootstrap each log how long they took.
Config restarts say so. A config change restarts the server instance, which used to print "Server is running" again. The new instance now says it restarted after a config change.

Web

Unknown pages return 404. Any path that isn't a real page used to return 200 with the app shell, which misled uptime monitors and crawlers. The shell still renders, so a mistyped URL lands on the home page as before.
/favicon.ico serves the icon. It used to return HTML.

Housekeeping

docs/design/ is gone. The Beyond-1.0 plans are tracked as issues instead: #107 (search across the relay pool) and #108 (rich event rendering).

---

Upgrading

Purge behavior changes by default. With retention_clock unset, purge counts age from arrival. Arrival tracking starts with this version, and only while purging is enabled. Events stored before then keep the created_at clock. Set retention_clock: created_at to keep the old behavior.
The first purge may delete more than usual. Kinds that the old sampling never reached are now purged. Each run is still capped at 100,000 deletes.
From rc3 or earlier: nostrdb upgrades the database from v5 to v6 on first start and rebuilds its profile search index. rc4 did this too, although its notes said nothing was migrated. Expect the first start on a large database to take longer. rc5's startup timings in debug.log show where the time goes.
Malformed filters now fail. Clients that sent malformed filters, and relied on getting results back anyway, now get CLOSED "invalid: ...".
New config keys: event_purge.retention_clock and event_purge.late_arrival_minutes. Both are optional and default as described above.
Still a release candidate. Please file anything you hit before the final v0.8.0.