openai-vision

OpenAI Vision Analysis Skill

Purpose

This skill enables image analysis, scene understanding, text extraction, and multi-frame comparison using OpenAI's vision-capable GPT models (e.g., gpt-4o, gpt-4o-mini). It supports single images, multiple images for comparison, and sequential frames for temporal analysis.

When to Use

• Analyzing image content (objects, scenes, colors, spatial relationships)
• Extracting and reading text from images (OCR via vision models)
• Comparing multiple images to detect differences or changes
• Processing video frames to understand temporal progression
• Generating detailed image descriptions or captions
• Answering questions about visual content

Required Libraries

The following Python libraries are required:

from openai import OpenAI
import base64
import json
import os
from pathlib import Path

Input Requirements

• File formats: JPG, JPEG, PNG, WEBP, non-animated GIF
• Image sources: URL, Base64-encoded data, or local file paths
• Size limits: Up to 20MB per image; total request payload under 50MB
• Maximum images: Up to 500 images per request
• Image quality: Clear, legible content; avoid watermarks or heavy distortions

Output Schema

Analysis results should be returned as valid JSON conforming to this schema:

{
  "success": true,
  "images_analyzed": 1,
  "analysis": {
    "description": "A detailed scene description...",
    "objects": [
      {"name": "car", "color": "red", "position": "foreground center"},
      {"name": "tree", "count": 3, "position": "background"}
    ],
    "text_content": "Any text visible in the image...",
    "colors": ["blue", "green", "white"],
    "scene_type": "outdoor/urban"
  },
  "comparison": {
    "differences": ["Object X appeared", "Color changed from A to B"],
    "similarities": ["Background unchanged", "Layout consistent"]
  },
  "metadata": {
    "model_used": "gpt-4o",
    "detail_level": "high",
    "token_usage": {"prompt": 1500, "completion": 200}
  },
  "warnings": []
}