search

Search Videos Using Natural Language

Search indexed videos using natural language descriptions. TwelveLabs interprets your query to find matching content based on visual elements, actions, sounds, and on-screen text.

When to Use This Skill

Use this skill when the user:

• Asks to find something in a video ("find the part where someone laughs")
• Wants to search for content ("search for a red car")
• Asks about when/where something happens ("when does the presenter mention AI?")
• Describes a scene they want to locate ("look for the sunset scene")

Instructions

Step 1: Extract the Search Query

Identify what the user wants to find from their natural language request. Convert their question into a search query.

Examples:

• "Find the part where someone is walking" → query: "a person walking"
• "When does the presenter talk about AI?" → query: "presenter talking about AI"
• "Look for any red cars in my videos" → query: "red car"

Step 2: Call the Search MCP Tool

Use the mcp__twelvelabs-mcp__search tool:

Tool: mcp__twelvelabs-mcp__search
Parameters:
  query: "<extracted search query>"

Note: The search is performed across all videos in the default index. To search a specific index, provide the optional indexId parameter.

Step 3: Display Search Results

Format the search results clearly for the user. The search returns matching segments with timestamps.

For each video with matching segments, display:

• Filename of the video (if available)
• URL of the video stream (if available)
• Bulleted list of start and end times for matching segments

Example output with results:

Found matches for: "a person walking"

**video_filename.mp4**
Stream: https://stream.url/video.m3u8

Matching segments:
- 00:12 - 00:28 (16 seconds)
- 01:45 - 02:03 (18 seconds)
- 03:30 - 03:45 (15 seconds)

**another_video.mp4**
Stream: https://stream.url/another.m3u8

Matching segments:
- 00:05 - 00:15 (10 seconds)

Found 4 matching segments across 2 videos.