5 — Billboard: Prototype → Production Hardening

Rank 6 of 28 · Tier A · app/workers/billboard_users, app/BillboardGateway, docs/billboard/ Stack: TypeScript (Hono), Cloudflare Workers, D1, R2, KV, Durable Objects, Hyperdrive → Postgres, WASM (kissfft), GitHub CI Status: ~60 commits over the hardening campaign. Upload live in production. CI gates enforced.

Why this ranks here. This is the production engineering story, and it is the one a bank cares about most. It starts from a written audit that names everything broken, then closes the gaps one numbered commit at a time, with the CI gates that stop them reopening. Google calls this "reduce toil and make the fix stick." Banks call it "evidence."

It starts with an audit, not code

docs/billboard/ is a 17-chapter engineering manual written before the remediation began: executive summary, system architecture, iOS client, audio engine, feature surfaces, recognition, edge, workers, data layer, an API contract diff, upload current-state, security & auth, a gap register, a Spotify parity matrix (14 of 52 features), an upload v2 design, a remediation roadmap, and a task list.

The audit's findings were blunt:

• Five distinct API hosts serve one feature mode. No single gateway; the iOS client fans out directly to each.
• billboard-feed-api was a mock data generator — writes were discarded.
• BillboardGateway Express origin was a scaffold with one health route.
• Auth was forgeable.
• No audio upload path existed at all.

What was then built

Auth made real — [F-2]

RS256 JWT verification replacing forgeable auth, split into four privilege tiers, with identity bound to the token rather than trusted from the body. Applied to global-media-worker as well.

The data layer made real — [F-3], [F-8], [D-4]

Nine missing D1 tables defined; column drift between code and schema bridged; billboard-feed-db wired and its schema applied; ProfileService moved to Postgres after Profile was 404ing against an empty D1 table.

Resumable multipart upload — [F-1], [W-3], [A-9]

R2 multipart ingest with 10 MiB equal parts, an audio probe, and an IngestDO Durable Object coordinating the session. TUS uploads made actually resumable. publishDraft made atomic.

Contract discipline in CI — [M-7], [F-10], [CI], [M-3]

openapi.yaml promoted to the single source of truth with generation and contract tests; an OpenAPI contract for billboard-users enforced in CI; contract-diff and shadow-type gates; and the first CI Billboard ever had — typecheck, tests, and a real end-to-end pass on every push.

Moderation, auto-moderation and DMCA — [S-3], [auto-mod], [dmca]

Review-before-playback, report intake, queue, decision, takedown. Then a three-way automated classifier in front of the human queue, with the thresholds, the cost, and what the classifier cannot catch recorded. Then DMCA: expeditious removal, notice, counter-notice, repeat-infringer policy.

Audio fingerprinting — [F-6]

Built kissfft.wasm — the module JamzFingerprintDO had always loaded and never actually had. A Shazam-style FFT constellation matcher running in a Durable Object at the edge.

Search and discovery — [F-9], [F-11], [F-4]

Typo-tolerant search; a home surface that leads with what you were doing; the following feed (empty for every user) fixed along with two unbound bindings; /v1 contract aliases plus four endpoints that had never existed.

The habit worth showing

Read the commit titles. They are written as claims with evidence:

[F-11] the following feed was empty for every user; fix it and the two unbound bindings [F-5] Remove the fake R2 types, and fix the three bugs they were hiding [F-6] Build kissfft.wasm — the module JamzFingerprintDO loads and never had [seed] fix truncated Stream uids — the seeded catalog would not have played docs: record the scope leak and adopt pathspec-scoped commits

That last one is a process failure, caught and corrected in public. Interviewers notice.

Scope

Interview surface this opens

• Edge compute constraints: no TCP sockets, and how that reshapes architecture
• Durable Objects as single-writer coordinators, and where they bottleneck
• Multipart/resumable upload protocol design and failure recovery
• Contract-first development and how you enforce it mechanically
• Building a gap register and burning it down without breaking live users