Answer every record, count what was not stored, and let a second writer wait #14

Merged

LNK merged 1 commit from claude/honest-when-busy into main

2026-09-04 21:08:37 +00:00

LNK commented

2026-09-04 21:08:36 +00:00

Contributor

The first of the review's cheap items, taken together because they are one fault: the client lied about itself when busy.

What changes. Every record is answered stored, rejected with the reason, or dropped with the error. Collectors deliver through one helper that sends, reads the answer, reconnects once if needed, and returns whether the gate stored the record; eighteen hand-rolled send blocks became that call. Only stored counts as emitted; the rest is events_dropped on the heartbeat, and the store carries a collector_dropped counter moved by the difference. A submit failure is a counter and an answer, not a line on stderr. A heartbeat answered with the wrong shape is treated as a lost connection. The store opens with a five-second busy timeout so the uploader's writes wait for the gate's transaction instead of failing with "database is locked".

Proof. Delivery counts only what the gate stored; a wrong-shaped answer is not a delivery; a record is answered with what became of it; dropped counts reach the store once; a second writer waits instead of failing, and that test fails with the timeout removed. Linux verifier green on the pinned toolchain; the clock and Windows crates cross-check clean. Not run here: the end-to-end suite over real pipes, which now reads the acknowledgement after each record, and the Windows verifier.

The first of the review's cheap items, taken together because they are one fault: the client lied about itself when busy. **What changes.** Every record is answered `stored`, `rejected` with the reason, or `dropped` with the error. Collectors deliver through one helper that sends, reads the answer, reconnects once if needed, and returns whether the gate stored the record; eighteen hand-rolled send blocks became that call. Only `stored` counts as emitted; the rest is `events_dropped` on the heartbeat, and the store carries a `collector_dropped` counter moved by the difference. A submit failure is a counter and an answer, not a line on stderr. A heartbeat answered with the wrong shape is treated as a lost connection. The store opens with a five-second busy timeout so the uploader's writes wait for the gate's transaction instead of failing with "database is locked". **Proof.** Delivery counts only what the gate stored; a wrong-shaped answer is not a delivery; a record is answered with what became of it; dropped counts reach the store once; a second writer waits instead of failing, and that test fails with the timeout removed. Linux verifier green on the pinned toolchain; the clock and Windows crates cross-check clean. Not run here: the end-to-end suite over real pipes, which now reads the acknowledgement after each record, and the Windows verifier.

LNK added 1 commit

2026-09-04 21:08:37 +00:00

Answer every record, count what was not stored, and let a second writer wait 2ac7435a18

A collector counted every send as a record. A send the gate rejected at admission, a send
that failed on the pipe, the resend after a reconnect — each moved `events_emitted` by one,
so the heartbeat's figure was the number of attempts and said nothing about the store. It
lied exactly when the gate was busy, which is when the number is wanted. The gate, for its
part, answered heartbeats and nothing else: a record vanished into the channel and the
collector never learned whether it existed.

Every record is answered now. The gate says `stored`, or `rejected` with the reason it
refused the record, or `dropped` with the error that stopped the write — and a write that
fails is no longer a line on stderr but a count the store carries and an answer the
collector hears, so neither side believes the record is somewhere. Collectors deliver through
one helper in `here-ipc` that sends, reads the answer, remakes the connection once if it has
gone, and returns whether the gate stored the record; eighteen hand-rolled send blocks across
the collectors became that one call, and the local `send` each of them carried is gone. Only
`stored` counts as emitted. Everything else is `events_dropped`, which travels on the
heartbeat beside the emitted count, so the two together are what the collector tried to
deliver and the store's `collector_dropped` counter is what never arrived. A heartbeat
answered with the wrong shape — an acknowledgement where a grant was due — is the pipe out
of step, and every collector treats it as a lost connection and gets a fresh one.

One record is in flight per connection, which is the cost and is also the backpressure made
explicit: a collector cannot hand over the next record until the gate has applied this one.

The store opens with a five-second busy timeout. Two processes write the file — the gate,
and the uploader marking what it sent — and without the timeout the second got "database is
locked" the instant the first held a transaction, an unhandled write failure and not a
recorded drop. Now it waits its turn.

Tests: a record the gate stored is delivered and a rejected one is not; an answer of the
wrong shape is not a delivery; a record is answered with what became of it and why; a
collector's dropped count reaches the store once, by the difference between heartbeats; a
second writer waits for the first instead of failing — which fails with the timeout removed,
checked by removing it.

The end-to-end suite that drives the real gate reads the answer after each record it sends,
so its next heartbeat's reply is its own; it is Windows-only and was not run here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

LNK merged commit a151ad34c2 into main

2026-09-04 21:08:37 +00:00

LNK referenced this pull request from a commit

2026-09-04 21:08:37 +00:00

Merge pull request #14 from claude/honest-when-busy

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

mikebrandon/here-telemetry-rnd!14

No description provided.

Rows
Columns