web-interface

Web Interface Skill for Voice Assistant

Build a production-ready Next.js web interface that connects to the Voice Assistant backend.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                      BROWSER (Next.js)                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │
│  │ Voice Record │  │ Chat UI      │  │ Push-to-Talk/        │  │
│  │ (MediaRecorder)│ │ (React)     │  │ Wake Word (optional) │  │
│  └──────────────┘  └──────────────┘  └──────────────────────┘  │
│           │                │                    │               │
│           └────────────────┴────────────────────┘               │
│                            │ WebSocket                          │
└────────────────────────────┼────────────────────────────────────┘
                             │
┌────────────────────────────┼────────────────────────────────────┐
│                      BACKEND (FastAPI)                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │
│  │ WebSocket    │  │ REST API     │  │ Audio Streaming      │  │
│  │ Handler      │  │ Endpoints    │  │ Processor            │  │
│  └──────────────┘  └──────────────┘  └──────────────────────┘  │
│           │                │                    │               │
│           └────────────────┴────────────────────┘               │
│                            │                                    │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │              Voice Assistant Core                         │  │
│  │  STT → Intent → Memory → Planner → Tools → LLM → TTS     │  │
│  └──────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Quick Start

1. Backend Setup (FastAPI with WebSocket)

Create src/api/websocket_server.py:

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
import asyncio
import json
import base64

app = FastAPI(title="Voice Assistant API")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # Configure for production
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

class ConnectionManager:
    def __init__(self):
        self.active_connections: list[WebSocket] = []

    async def connect(self, websocket: WebSocket):
        await websocket.accept()
        self.active_connections.append(websocket)

    def disconnect(self, websocket: WebSocket):
        self.active_connections.remove(websocket)

    async def send_json(self, websocket: WebSocket, data: dict):
        await websocket.send_json(data)

manager = ConnectionManager()

@app.websocket("/ws/voice")
async def websocket_endpoint(websocket: WebSocket):
    await manager.connect(websocket)
    try:
        while True:
            data = await websocket.receive_json()
            # Process audio/text and return response
            response = await process_message(data)
            await manager.send_json(websocket, response)
    except WebSocketDisconnect:
        manager.disconnect(websocket)