26 — Media & Upload Infrastructure
Additional project 2 of 4 · app/GlobalCloudflareAdapter, app/workers/global_upload_service, app/BillboardGateway/worker
Stack: Go (adapter + Kafka/Redis adapters + KV proxy), TypeScript Workers (Hono), R2 multipart, Cloudflare Stream, Durable Objects, TUS
Where this sits. Large-file ingest is a genuinely hard distributed problem — resumability, partial failure, orphaned parts, and a coordinator that must be a single writer — and this is the system that solves it across five product modes.
Two halves
The Go adapter — app/GlobalCloudflareAdapter (7,855 lines)
bootstrap/, cloud_store.go, event_router.go, kafka_adapter/,
redis_adapter.go, kv_proxy.go, upload_service/, global_upload_service/,
chat_service/, db/.
It is the bridge between the edge and the host: a Worker that cannot open a
TCP socket still needs Kafka, Redis, and Cassandra, so the adapter terminates
those protocols host-side and exposes them over HTTP. The kv_proxy is the
same idea for key-value access.
This is the general form of the constraint that shaped the Activity Graph: the edge cannot speak your protocols, so something host-resident has to.
The edge worker — app/workers/global_upload_service
One ingest surface serving five modes:
BillboardUploadService.ts music
BoxOfficeUploadService.ts video
StorefrontUploadService.ts commerce
BeaconUploadService.ts beacon
BeaconAgroService.ts
DatingDistroService.ts DatingStreamBridge.ts
auth.ts jwt.ts routes.ts service.ts RedisAdapter.ts
Upload v2 — the design that shipped
| Decision | Why |
|---|---|
| R2 multipart, no presigned URLs | Presigning hands the client a credential you cannot revoke mid-upload; multipart through the Worker keeps authorization on every part |
| 10 MiB equal parts | Uniform parts make part numbering, retry, and progress arithmetic trivial |
| Audio probe on ingest | Reject an unplayable file at upload time, not at first play |
IngestDO Durable Object | The upload session needs exactly one writer; a DO is that primitive at the edge |
TUS made genuinely resumable ([W-3]) | It advertised resumability it did not have |
publishDraft made atomic ([A-9]) | A half-published draft is worse than a failed one |
Auth on the media path
[F-2] — global-media-worker gained real RS256 verification with identity
bound to the token, rather than trusting an identity supplied in the request
body. On an upload path, body-trusted identity means anyone can write to
anyone's library.
Delivery
billboard-gateway-edge serves audio from R2 with HTTP range requests,
which is what makes seeking work without downloading the whole object. Video
goes through Cloudflare Stream with HLS/DASH manifests and a playback token
that refreshes ([H-4]/[H-6]).
One shipped defect worth naming: [seed] fix truncated Stream uids — the seeded catalog would not have played. A truncated id fails at playback, far from
where it was introduced.
Interview surface this opens
- Multipart upload: part sizing, retry, orphaned-part cleanup, completion races
- Presigned URLs vs proxied uploads — the revocation trade-off
- Durable Objects as a single-writer session coordinator
- HTTP range requests and byte-range streaming
- Terminating TCP protocols host-side for a socket-less edge runtime