computer-vision

Computer Vision

Computer vision is the engineering discipline of making machines interpret and act on visual data — images, video, point clouds, and multi-modal streams. This skill spans the full pipeline from raw pixel ingestion through model training, evaluation, export, and production deployment, with emphasis on the practical trade-offs between accuracy, latency, and hardware constraints. Canonical frameworks are PyTorch/torchvision, Ultralytics, OpenCV, and the ONNX ecosystem.

Core Concepts

Object Detection

YOLOv8 / YOLOv11 (Ultralytics) is the workhorse for real-time detection. Training: model = YOLO('yolov8n.pt'); model.train(data='dataset.yaml', epochs=100, imgsz=640, batch=16, device='cuda'). Dataset YAML format: path, train, val, nc (num classes), names. Key hyperparameters: mosaic augmentation (default on, disable last 10 epochs via close_mosaic=10), mixup, degrees, hsv_h/s/v. For small objects, use a higher imgsz (1280) and anchor-free head benefits directly. Export: model.export(format='onnx', opset=17, dynamic=True) or format='engine' for TensorRT.

RT-DETR (Real-Time Detection Transformer) from Baidu: eliminates NMS via set-based prediction, so latency is more deterministic. Use RTDETRv2 via Ultralytics or the original PaddleDetection implementation. Backbone is typically ResNet-50/101 or HGNetv2. Slower to train than YOLO but competitive accuracy at similar FPS on GPU. Good choice when NMS jitter causes downstream tracking instability.

NMS tuning: conf threshold (default 0.25 for inference, raise to 0.5+ for precision-sensitive tasks), iou threshold (default 0.7 for NMS suppression). agnostic_nms=True suppresses across classes — useful when class overlap is likely (e.g., person vs. cyclist partially occluded).

COCO metrics: report [email protected] and [email protected]:0.95. The latter is the standard benchmark metric; @0.5 is more forgiving. Per-class AP breakdown often reveals dataset imbalance issues before overall mAP does.