gpt-multimodal

OpenAI Vision Analysis Skill

Purpose

This skill enables image analysis, scene understanding, text extraction, and multi-frame comparison using OpenAI's vision-capable GPT models (e.g., gpt-4o, gpt-5). It supports single and multiple images analysis and sequential frames for temporal analysis.

When to Use

• Analyzing image content (objects, scenes, colors, spatial relationships)
• Extracting and reading text from images (OCR via vision models)
• Comparing multiple images to detect differences or changes
• Processing video frames to understand temporal progression
• Generating detailed image descriptions or captions
• Answering questions about visual content

Required Libraries

The following Python libraries are required:

from openai import OpenAI
import base64
import json
import os
from pathlib import Path

Input Requirements

• File formats: JPG, JPEG, PNG, WEBP, non-animated GIF
• Image quality: Clear and legible; minimum 512×512px recommended
• File size: Under 20MB per image recommended
• Maximum per request: Up to 500 images, 50MB total payload
• URL or Base64: Images can be provided as URLs or base64-encoded data

Output Schema

All analysis results should be returned as valid JSON conforming to this schema:

{
  "success": true,
  "model": "gpt-5",
  "analysis": "Detailed description or analysis of the image content...",
  "metadata": {
    "image_count": 1,
    "detail_level": "high",
    "tokens_used": 850,
    "processing_time_ms": 1234
  },
  "extracted_data": {
    "objects": ["car", "person", "building"],
    "text_found": "Sample text from image",
    "colors": ["blue", "white", "gray"],
    "scene_type": "urban street"
  },
  "warnings": []
}

Field Descriptions

• success: Boolean indicating whether the API call succeeded
• model: The GPT model used for analysis (e.g., "gpt-4o", "gpt-5")
• analysis: Complete textual analysis or description from the model
• metadata.image_count: Number of images analyzed in this request
• metadata.detail_level: Detail parameter used ("low", "high", or "auto")
• metadata.tokens_used: Approximate token count for the request
• metadata.processing_time_ms: Time taken to process the request
• extracted_data: Structured information extracted from the image(s)
• warnings: Array of issues or limitations encountered