80-livekit-agents-panaversity-agentfactory-1f8ecf89

Build Your LiveKit Agents Skill

Before learning LiveKit Agents—the framework powering ChatGPT's Advanced Voice Mode—you'll own a LiveKit Agents skill.

This is skill-first learning. You build the skill, then the chapter teaches you what it knows and how to make it better. By the end, you have a production-ready voice agent AND a reusable skill for building more.

---

Why LiveKit Agents?

In September 2023, OpenAI unveiled ChatGPT Voice Mode. The technology behind it? LiveKit. When OpenAI launched the feature, they also released LiveKit Agents—an open source framework that made it easy for developers to build their own voice AI agents.

LiveKit Agents was used in every demo during the GPT-4o unveil. The framework now powers voice-driven AI products across the industry—from startups to enterprises building Digital FTEs that can hear, speak, and reason in realtime.

What you're learning: Production voice agent architecture from the framework that runs at scale.

---

Step 1: Clone Skills-Lab Fresh

Every chapter starts fresh. No state assumptions.

1. Go to github.com/panaversity/claude-code-skills-lab
2. Click the green Code button
3. Select Download ZIP
4. Extract the ZIP file
5. Open the extracted folder in your terminal

cd claude-code-skills-lab
claude

Why fresh? Skills accumulate across chapters. A fresh start ensures your LiveKit skill builds on clean foundations, not inherited state.

---

Step 2: Write Your LEARNING-SPEC.md

Before asking Claude to build anything, define what you want to learn. This is specification-first learning—you specify intent, then the system executes.

Create a new file:

touch LEARNING-SPEC.md

Write your specification:

# LiveKit Agents Skill

## What I Want to Learn
Build voice agents using LiveKit's production framework—the same technology
powering ChatGPT's Advanced Voice Mode.

## Why This Matters
- LiveKit Agents handles the hard parts: WebRTC, turn detection, interruptions
- Understanding the framework means understanding what works at scale
- Every voice-enabled Digital FTE I build will use these patterns

## Success Criteria
- [ ] Create voice agent that responds to speech
- [ ] Implement function calling (tool use via voice)
- [ ] Handle interruptions gracefully (barge-in)
- [ ] Understand deployment to Kubernetes

## Key Questions I Have
1. How do Agents, AgentSessions, and Workers relate to each other?
2. How does semantic turn detection work? Why is it better than silence-based?
3. How do I integrate MCP tools into a voice agent?
4. What's the difference between VoicePipelineAgent and MultimodalAgent?
5. How do I handle phone calls (SIP integration)?

## What I Already Know
- Part 10: Chat interfaces, streaming, tool calling UI
- Part 7: Kubernetes deployment, containerization
- Part 6: Agent SDKs (OpenAI, Claude, Google ADK)

## What I'm Not Trying to Learn Yet
- Pipecat (that's Chapter 81)
- Raw OpenAI Realtime API (that's Chapter 82)
- Phone number provisioning details (that's Chapter 84)