coreml-optimizer

CoreML Optimizer

Expert guidance for optimizing machine learning models for Apple's CoreML framework on iOS and macOS devices.

When to Use This Skill

Use this skill when:

• Converting PyTorch/TensorFlow models to CoreML format
• Optimizing CoreML model size and inference latency
• Targeting the Neural Engine for maximum performance
• Debugging slow model inference or compute unit issues
• Applying quantization, palettization, or pruning
• Profiling model performance with Instruments
• Troubleshooting accuracy degradation after compression

Speed Optimization Checklist

The critical path to fast CoreML inference:

1. Verify Compute Unit Configuration

Many "slow" models are accidentally CPU-bound. Configure via MLModelConfiguration.computeUnits:

Option	Description
.all	Uses all available compute units including Neural Engine (default, recommended)
.cpuAndNeuralEngine	CPU + Neural Engine, excludes GPU
.cpuAndGPU	CPU + GPU, excludes Neural Engine
.cpuOnly	Forces CPU-only execution (for debugging/consistency)

In Swift:

let config = MLModelConfiguration()
config.computeUnits = .all  // .cpuAndNeuralEngine, .cpuAndGPU, .cpuOnly
let model = try MLModel(contentsOf: modelURL, configuration: config)

Benchmark each configuration - if .all isn't faster than .cpuAndGPU, your model may not be hitting the Neural Engine.

2. Apply Weight Compression with coremltools

CoreML execution commonly uses float16 where possible. Use coremltools.optimize for further compression:

import coremltools as ct
import coremltools.optimize as cto

# 8-bit quantization (2-4x speedup for memory-bound models)
config = cto.coreml.OptimizationConfig(
    global_config=cto.coreml.OpLinearQuantizerConfig(
        mode="linear_symmetric",
        dtype="int8",
        granularity="per_channel"
    )
)

model = ct.models.MLModel("Model.mlpackage")
quantized = cto.coreml.linear_quantize_weights(model, config=config)
quantized.save("Model_int8.mlpackage")