web-audio-api

Web Audio API Skill

1. Overview

This skill provides Web Audio API expertise for creating audio feedback, voice processing, and sound effects in the JARVIS AI Assistant.

Risk Level: LOW - Audio processing with minimal security surface

Primary Use Cases:

• HUD audio feedback (beeps, alerts)
• Voice input processing
• Spatial audio for 3D HUD elements
• Real-time audio visualization
• Text-to-speech integration

2. Core Responsibilities

2.1 Fundamental Principles

1. TDD First: Write tests before implementation for all audio components
2. Performance Aware: Optimize for 60fps with minimal audio latency
3. User Gesture Required: Audio context must be started after user interaction
4. Resource Cleanup: Close audio contexts and disconnect nodes on unmount
5. AudioWorklet for Processing: Use AudioWorklet for heavy DSP operations
6. Accessibility: Provide visual alternatives to audio feedback
7. Volume Control: Respect system and user volume preferences
8. Error Handling: Gracefully handle audio permission denials

3. Technology Stack & Versions

3.1 Browser Support

Browser	AudioContext	AudioWorklet
Chrome	35+	66+
Firefox	25+	76+
Safari	14.1+	14.1+

3.2 TypeScript Types

// types/audio.ts
interface AudioFeedbackOptions {
  frequency: number
  duration: number
  type: OscillatorType
  volume: number
}

interface SpatialAudioPosition {
  x: number
  y: number
  z: number
}

4. Implementation Patterns

4.1 Audio Context Management

// composables/useAudioContext.ts
export function useAudioContext() {
  const audioContext = ref<AudioContext | null>(null)
  const isInitialized = ref(false)

  async function initialize() {
    if (audioContext.value) return
    audioContext.value = new AudioContext()
    if (audioContext.value.state === 'suspended') await audioContext.value.resume()
    isInitialized.value = true
  }

  onUnmounted(() => {
    audioContext.value?.close()
    audioContext.value = null
  })

  return { audioContext: readonly(audioContext), isInitialized: readonly(isInitialized), initialize }
}