avfoundation-media-pipeline-workflow

AVFoundation Media Pipeline Workflow

Purpose

Guide AVFoundation capture, playback, asset loading, reader, writer, export, and sample-buffer pipeline design and repair. This skill owns media-pipeline shape, typed AVFoundation surface choice, and back-pressure policy, while leaving app audio-session policy, engine graphs, Core Media timing internals, and Xcode execution to their owning skills.

When To Use

• Use this skill for AVCaptureSession, capture inputs and outputs, capture queues, AVPlayer, AVPlayerItem, AVAsset, AVURLAsset, AVAsyncProperty, async asset loading, AVAssetReader, AVAssetWriter, export, transcode, and sample-buffer append loops.
• Use this skill when existing code blocks the main thread, uses deprecated AVAsset synchronous properties, misuses loadValuesAsynchronously(forKeys:), appends sample buffers without back-pressure, confuses capture authorization with microphone permission, or mixes capture, UI, writer, and player ownership in one object.
• Recommend avfaudio-session-workflow when the primary problem is app audio intent, routes, interruptions, or microphone permission.
• Recommend coremedia-timing-samplebuffer-workflow when the primary issue is sample-buffer timestamps, format descriptions, timebases, clocks, or synchronization.
• Recommend camera-capture-depth-workflow when the primary issue is camera discovery, device controls, formats, rotation, photo capture, depth, calibration, synchronized camera outputs, or computational capture.

Single-Path Workflow

1. Classify the media pipeline:
   • capture setup
   • player or playback state
   • asset inspection or async loading
   • reader or writer loop
   • export or transcode
   • sample-buffer append or back-pressure
   • repair of an existing implementation
2. Apply the Apple docs gate:
   • read current AVFoundation documentation for the pipeline surface
   • state the documented behavior relied on before recommending changes
   • apply ../../shared/references/apple-media-type-ownership.md before introducing custom media wrappers, raw numeric timing, strings, or dictionaries
3. Choose the pipeline owner and queues:
   • capture-session configuration owner
   • serial capture session queue
   • player item and UI update boundary
   • async asset loading boundary
   • reader and writer queue
   • cancellation and teardown path
4. Repair common failure modes:
   • AVCaptureSession.startRunning() on the main queue
   • deprecated synchronous AVAsset property reads in Swift clients
   • callback-based asset loading left in otherwise async Swift code
   • writer loops that ignore isReadyForMoreMediaData
   • raw strings, dictionaries, or custom structs replacing AVAsset, AVAssetTrack, AVPlayerItem, AVCaptureConnection, AVAssetReaderOutput, AVAssetWriterInput, CMSampleBuffer, or writer receiver types without a documented boundary reason
   • capture authorization, recording permission, and Info.plist gates collapsed together
   • sample-buffer timing issues that need Core Media handoff
5. Return one recommendation with:
   • pipeline class
   • documented Apple behavior relied on
   • owner, queue, and async-loading plan
   • back-pressure and cancellation plan
   • repair findings
   • validation and handoff expectation