14 — Billboard Player: Audio UI & Engine

UI project 1 of 5 · IOS/TheOne/Billboard Stack: SwiftUI, Swift Concurrency (actors), AVFoundation / AVPlayer, Accelerate (vDSP), MediaPlayer Scope: 128 files · 57,669 lines of Swift

Where this sits. It is the flagship consumer surface in the tree and the most directly Spotify-relevant product artifact: a full music player with a real audio engine underneath it, not a list of songs with a play button.

The surface

Views 2/
  BillboardMiniPlayer.swift      persistent now-playing bar
  BillboardFullPlayer.swift      full-screen transport, scrubber, queue
  Player/                        player subviews
  Explore/ExploreSubViews/BillboardPlayerView.swift
ViewModels 2/
  BillboardPlayerViewModel.swift observable transport state
Engine 2/
  BillboardAudioEngine.swift     AVPlayer wrapped in AudioEngineActor
Utils 2/
  AudioSession.swift             AVAudioSession category & interruption handling
  AudioTapManager.swift          MTAudioProcessingTap install/teardown
  AudioSpectrumAnalyzer.swift    live spectrum for the visualizer
Shazaam/
  JamzAudioEngine.swift          vDSP FFT constellation fingerprinting

The engineering worth talking about

The actor boundary

BillboardAudioEngine puts AVPlayer behind an AudioEngineActor. Audio state is mutated from a media thread, an interruption handler, a remote-command handler, and the UI simultaneously — a classic data race. A Swift actor makes the serialization a compile-time property instead of a discipline, and the UI reads a @MainActor-isolated projection of it.

This is the same instinct as the fatal map race in the Go call path: identify the shared mutable state, then give it exactly one owner.

The audio tap

AudioTapManager installs an MTAudioProcessingTap on the player's audio mix to pull raw PCM while playback continues. That is what feeds AudioSpectrumAnalyzer for the visualizer — and it is a genuinely awkward API (C callbacks, manual retain semantics, teardown that must happen before the item is released or you leak or crash).

On-device fingerprinting

JamzAudioEngine performs FFT via Accelerate/vDSP and builds a constellation of spectral peaks — the Shazam approach — then hands the fingerprint to JamzRecognitionService for matching. The server-side counterpart is the WASM kissfft module running in a Durable Object, described in the Edge Platform. Same algorithm, two runtimes, both written here.

Session and interruption correctness

AudioSession.swift handles the category/mode configuration and the interruption lifecycle — a phone call arriving mid-track, a route change when headphones are unplugged, and resumption afterward. Getting this wrong is the single most common way a music app feels broken.

Known state, stated honestly

The Billboard audit found the player's API surface fanned out across five distinct hosts with no gateway, and that BillboardGateway.swift targeted a Worker whose source is not in this repo. The player UI is complete; parts of what it talks to were not.

Interview surface this opens

• Actor isolation for real-time media, and why @MainActor alone is insufficient
• MTAudioProcessingTap lifecycle and C-interop memory rules
• FFT-based audio fingerprinting: windowing, peak picking, constellation hashing
• AVAudioSession interruption/route-change handling
• Gapless playback, prebuffering, and queue management