Name: Computer Vision Engineering: From Image Classification to Object Detection and Segmentation
Author: step-3.5-flash

Computer Vision Task Taxonomy and Development Environment Setup

Computer vision (CV) as a discipline emerged from the intersection of computer science, neuroscience, and signal processing, with its foundational roots tracing back to the 1959 discovery by Hubel and Wiesel that mammalian visual cortex neurons respond to oriented edges in visual stimuli. Over the past two decades, the field has undergone a paradigm shift from handcrafted feature engineering (e.g., SIFT, HOG) to data-driven deep learning approaches, enabling unprecedented performance on perceptual tasks that were previously considered intractable for machines. At the core of modern CV engineering lies a well-defined task taxonomy that categorizes problems by the granularity of output they require, with image classification, object detection, and image segmentation forming the foundational triad of most industrial and research applications. This chapter first formalizes this task taxonomy, clarifies the technical evolution and industrial use cases of each task, then walks through the end-to-end setup of a production-grade CV development environment, and concludes with a hands-on image preprocessing practice to validate the environment, laying the groundwork for the practical content of subsequent chapters.

Core CV Task Taxonomy: Definitions, Evolution, and Industrial Scenarios

The CV task taxonomy is defined by the level of granularity required for output interpretation, with each task building on the capabilities of lower-granularity tasks while introducing new technical challenges. We formalize the three core tasks below, alongside their technical evolution milestones and real-world industrial deployment cases.

1.1 Image Classification: Single-Label Semantic Mapping

Image classification is the most fundamental CV task, where the goal is to assign a single semantic category label to an entire input image. Mathematically, it is defined as a mapping function $f: \mathbb{R}^{H \times W \times C} \rightarrow \mathcal{C}$, where $H, W, C$ are the height, width, and number of channels of the input image, and $\mathcal{C}$ is a predefined set of class labels. The output is a probability distribution over all classes, with the highest-probability class assigned as the prediction.

The modern era of deep learning-based image classification was catalyzed by the 2012 AlexNet architecture[^1], which achieved a top-5 error rate of 15.3% on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), outperforming the second-best entry by over 10 percentage points. This was followed by a series of architectural innovations: VGG (2014) introduced stacked 3x3 convolutional layers as a standard building block; GoogLeNet (2014) proposed the Inception module to improve computational efficiency; ResNet (2015) solved the degradation problem of deep networks via residual skip connections, enabling training of networks with hundreds or even thousands of layers; and Vision Transformers (ViT, 2020) introduced the transformer architecture from NLP to CV, achieving state-of-the-art performance on large-scale datasets.

Industrial application scenarios for image classification are extremely widespread, particularly in scenarios where coarse-grained semantic judgment is required. A representative case is industrial printed circuit board (PCB) defect detection: traditional manual inspection has a defect漏检率 of 15-20% and an efficiency of less than 100 units per person per hour. By fine-tuning a ResNet50 model pre-trained on ImageNet on a dataset of 50,000 labeled PCB images (covering 12 common defect types such as short circuits, open circuits, and virtual soldering), the classification accuracy can reach 99.2%, with an inspection efficiency of over 1000 units per hour, reducing the漏检率 to less than 2% and saving 80% of labor costs. Another common scenario is content moderation for social media platforms, where image classification models are used to identify prohibited content such as violence, pornography, and political sensitivity, with an accuracy of over 98% for mainstream platforms.

1.2 Object Detection: Multi-Instance Localization and Classification

Object detection extends image classification by requiring not only the classification of all objects in an image, but also the localization of each object via bounding boxes. Mathematically, it is defined as a mapping function $f: \mathbb{R}^{H \times W \times C} \rightarrow \{(b_i, c_i, p_i)\}_{i=1}^N$, where $b_i = (x_i, y_i, w_i, h_i)$ is the coordinate and size of the $i$-th bounding box, $c_i$ is the class label, $p_i$ is the confidence score, and $N$ is the number of detected objects. Unlike classification, detection needs to handle the challenges of variable number of objects, overlapping objects, and small objects.

The technical evolution of object detection can be divided into two stages: the two-stage detector represented by R-CNN (2014)[^2], Faster R-CNN (2015), and Mask R-CNN (2017), and the one-stage detector represented by YOLO (2016), SSD (2016), and RetinaNet (2017). Two-stage detectors first generate region proposals via a region proposal network (RPN), then classify and refine the proposals, with higher accuracy but slower inference speed; one-stage detectors directly predict bounding boxes and classes on the entire image, with faster inference speed but slightly lower accuracy. In recent years, the rise of DETR (2020) and its variants (Deformable DETR, DINO, etc.) has introduced the transformer architecture to detection, eliminating the need for hand-designed components such as anchor boxes and non-maximum suppression (NMS), further improving the performance of small object detection and crowded scene detection.

Object detection is the most widely deployed CV task in industrial scenarios. A typical case is intelligent transportation: traffic violation capture systems use YOLOv8-based detection models to identify vehicles, pedestrians, traffic signs, and traffic lights in real-time, with an inference speed of 30 FPS on 1080P video, and an accuracy of 98.5% for identifying violations such as running red lights and driving in the wrong lane. Another case is retail shelf auditing: detection models are used to count the number and types of products on supermarket shelves, identify out-of-stock and misplaced items, with an audit efficiency 10 times higher than manual auditing, and an accuracy of over 95%. The core metric for evaluating detection performance is mAP (mean Average Precision), with the COCO dataset (the most widely used detection benchmark) using an IoU threshold of 0.5:0.95 (averaging mAP over IoU thresholds from 0.5 to 0.95 with a step of 0.05) as the main evaluation metric.

1.3 Image Segmentation: Pixel-Level Semantic Understanding

Image segmentation further refines the output granularity to the pixel level, assigning a semantic label to each pixel in the input image. It can be divided into three sub-tasks: 1. Semantic Segmentation: Assigns