# 27 — Push & Notification Delivery Pipeline

**Additional project 3 of 4** · `app/GlobalChatService` (`pushconsumer.go`, `pushsender.go`, `devicetokens.go`), `app/BillboardActivityService/internal/notifier`, `docs/APNS-VOIP-PUSH-SETUP.txt`
**Stack:** Go, APNs (token + certificate auth), PushKit VoIP, Kafka consumer, Cassandra device-token store

> **Where this sits.** Push is the last mile of every realtime system, and it is
> the one that fails silently. This covers both kinds — ordinary notifications
> and **VoIP pushes**, which have completely different semantics and a
> completely different failure mode.

---

## The pipeline

```
message written / activity event
        │
        ▼
 Kafka  ──► pushconsumer.go       consumes; decides who needs waking
        │
        ▼
 devicetokens.go                  token store, per user, per device, per env
        │
        ▼
 pushsender.go ──► APNs  ──► device
```

Plus the [Activity Graph's](01-billboard-activity-graph.md)
`internal/notifier` — **consumer group 2**, which turns activity into
notification rows in Postgres, deduped by a unique index on `activity_id` with
`ON CONFLICT DO NOTHING`.

That dedupe is the push-specific consequence of at-least-once delivery: a
duplicated Kafka event must not become **two buzzes on someone's phone.**
Idempotency here is not an abstraction — it is a user-visible artifact.

## The two certificates, and why they are not interchangeable

`docs/APNS-VOIP-PUSH-SETUP.txt` explains this in plain language before any
configuration:

**The APNs certificate (regular push)** is for messages. Someone sends a chat
message while the app is closed; Apple wakes the phone and shows a banner.

**The VoIP certificate** is different in kind. When Apple delivers a VoIP push,
**iOS launches the app in the background even if the user force-quit it**, and
gives it a moment to present the full-screen ringing UI. A regular push cannot
do that — it can only show a banner.

> The consequence, stated bluntly in the doc: *without the VoIP certificate
> working, a call to a phone whose app is not currently on screen does nothing
> at all. The phone never rings.*

That is the correct way to document a dependency: not "configure the VoIP cert"
but "here is the feature that is entirely absent until you do."

## The operational honesty

The guide is addressed to *"the operator holding the APNs and VoIP certificates
on a local Mac"* and opens with **"Read Part 0 before doing anything. It
explains what the certificates actually do and why you cannot finish this
today."**

Writing a runbook that begins by telling the reader the task is not completable
in one sitting — and why — is the same instinct as the
[BLOCKED/INFERRED separation](12-technical-documentation.md) in the audit
documents.

## Device token management

`devicetokens.go` handles the parts that make push quietly break in production:
multiple devices per user, sandbox vs production environments, token rotation
on reinstall, and unregistering tokens APNs reports as invalid. A push system
that never prunes dead tokens degrades into mostly-failing sends.

## Interview surface this opens

- Push as the fallback leg of a realtime delivery system: when do you send one?
- Idempotency where a duplicate is a user-visible defect, not a data one
- APNs token-based vs certificate auth, and PushKit's distinct guarantees
- Device-token lifecycle: rotation, multi-device fan-out, invalid-token pruning
- Writing a runbook that names what the reader cannot finish today