visual-intelligence

Visual Intelligence

Integrate your app with iOS Visual Intelligence to let users find app content by pointing their camera at objects.

When This Skill Activates

• User wants camera-based search in their app
• User asks about visual search integration
• User wants to surface app content in system searches
• User needs to handle visual intelligence queries

Overview

Visual Intelligence lets users:

1. Point camera at objects or use screenshots
2. System identifies what they're looking at
3. Your app provides matching content
4. Results appear in system UI

Your app implements:

• IntentValueQuery to receive search requests
• AppEntity types for searchable content
• Display representations for results

Platform Availability (WWDC26 297)

• Visual Intelligence runs on iOS, iPadOS, and macOS — the same entities, query, and OpenIntent code works unchanged on all three. Handle both camera captures of physical objects (iOS) and screenshots of digital media (iPad/Mac) as input.
• On Mac, the input pixel buffer can be much larger than what you'd encounter on iPhone — consider whether resizing is necessary before matching.

Quick Start

1. Import Frameworks

import VisualIntelligence
import AppIntents

2. Create App Entity

struct ProductEntity: AppEntity {
    var id: String
    var name: String
    var price: String
    var imageName: String

    static var typeDisplayRepresentation: TypeDisplayRepresentation {
        TypeDisplayRepresentation(
            name: LocalizedStringResource("Product"),
            numericFormat: "\(placeholder: .int) products"
        )
    }

    var displayRepresentation: DisplayRepresentation {
        DisplayRepresentation(
            title: "\(name)",
            subtitle: "\(price)",
            image: .init(named: imageName)
        )
    }

    // Deep link URL
    var appLinkURL: URL? {
        URL(string: "myapp://product/\(id)")
    }
}

3. Create Intent Value Query