Foundations of Computer Vision

Foundations of Computer Vision

> *"The important thing in science is not so much to obtain new facts as to discover new ways of thinking about them."* — Sir William Lawrence Bragg

This chapter addresses a fundamental question that every practitioner entering the field must confront: What exactly is computer vision, why has it become so critically important in our technological landscape, and what mathematical toolkit do I need to work effectively in this domain? Understanding these foundations is not merely academic—it determines how you approach problems, which tools you choose, and how you interpret results. The field of computer vision has undergone a dramatic transformation over the past decade, shifting from hand-engineered features and classical algorithms to learned representations and deep neural networks. Yet beneath this revolution lies a consistent mathematical framework that has guided researchers from the earliest days of the field.

We will explore the historical evolution of computer vision, tracing its intellectual lineage from the pattern recognition efforts of the 1960s through the deep learning breakthroughs of the 2010s. We will examine the real-world applications that have driven both research progress and industrial adoption. We will establish the essential mathematical prerequisites—linear algebra, calculus, and probability theory—that underpin virtually every algorithm in the field. We will also consider the relationship between biological visual systems and computational approaches, drawing insights from neuroscience that have shaped how we think about visual understanding. Finally, we will introduce the three core computer vision tasks—classification, detection, and segmentation—that form the backbone of this book and most practical applications today.

By the end of this chapter, you will have a comprehensive map of the computer vision landscape and the foundational knowledge necessary to proceed to the more advanced topics covered in subsequent chapters.

Historical Evolution: From Pattern Recognition to Deep Learning

The story of computer vision begins not with computers, but with a fundamental question about intelligence itself. In 1966, Seymour Papert and Marvin Minsky at the Massachusetts Institute of Technology launched the "Summer Vision Project," an ambitious attempt to construct a complete visual system within a single summer. While this optimistic timeline proved unrealistic, the project crystallized a vision for the field: that understanding images could be reduced to a series of algorithmic transformations, each step progressively abstracting raw pixel data into meaningful representations.

The Classical Era (1960s–1980s)

The early decades of computer vision research were dominated by a paradigm that now seems almost quaint: the belief that visual understanding could be achieved through explicit symbolic reasoning. Researchers like David Marr at MIT developed influential theories suggesting that vision was a process of constructing hierarchical descriptions from raw image data. Marr's framework, articulated in his posthumously published book *Vision* (1982), proposed that visual processing proceeded through stages: from primal sketches capturing edges and boundaries, through 2.5D sketches representing surface orientations, to 3D model representations in object-centered coordinates.

During this period, the field focused heavily on edge detection, feature extraction, and geometric modeling. Researchers developed algorithms to extract lines, curves, and basic shapes from images, often using techniques borrowed from signal processing and differential geometry. The assumption was that visual scenes could be parsed into discrete objects by first identifying boundaries and then fitting geometric primitives—circles, ellipses, polygons—to the extracted features.

```python

Classical edge detection using the Sobel operator

import numpy as np from scipy import ndimage

def sobel_edge_detection(image): """ Apply Sobel edge detection to a grayscale image. Returns gradient magnitude and direction. """

Sobel kernels for x and y gradients

sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]) sobel_y = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]])

Compute gradients

gx = ndimage.convolve(image.astype(float), sobel_x) gy = ndimage.convolve(image.astype(float), sobel_y)

Gradient magnitude

magnitude = np.sqrt(gx2 + gy2)

Gradient direction

direction = np.arctan2(gy, gx)

return magnitude, direction ```

This classical approach yielded important insights but struggled with the variability inherent in real-world images. Lighting conditions, object pose, occlusion, and background clutter created combinatorial explosion in the number of special cases that hand-crafted algorithms needed to handle. The recognition rate for even simple object categories remained disappointingly low throughout the 1970s and 1980s.

The Statistical Revolution (1990s–2000s)

The 1990s brought a shift toward statistical methods that would reshape the field's trajectory. Rather than attempting to explicitly model the physics of image formation or the geometry of objects, researchers began treating visual recognition as a problem of learning from examples. Support Vector Machines (SVMs), introduced by Vladimir Vapnik and colleagues, became the dominant classification method and achieved remarkable success on handwritten digit recognition tasks, famously surpassing human performance on the MNIST dataset.

This era also saw the emergence of robust feature descriptors that could handle some degree of viewpoint and illumination variation. David Lowe's Scale-Invariant Feature Transform (SIFT), published in 1999, represented a breakthrough in practical computer vision. SIFT features were designed to be invariant to scale and rotation, relatively robust to changes in illumination, and distinctive enough to enable reliable matching between images. The algorithm detected distinctive keypoints at multiple scales, computed orientation histograms for each keypoint, and constructed a feature descriptor based on gradient orientation histograms.

```python

Conceptual illustration of SIFT-like feature extraction

def extract_local_features(image, keypoints): """ Extract features at specified keypoint locations. Simplified illustration of the SIFT approach. """ features = []

for kp in keypoints: x, y, scale = kp['x'], kp['y'], kp['scale']

Define region of interest at this scale

window_size = int(3 * scale) x_start = max(0, x - window_size // 2) x_end = min(image.shape[1], x + window_size // 2) y_start = max(0, y - window_size // 2) y_end = min(image.shape[0], y + window_size // 2)

region = image[y_start:y_end, x_start:x_end]

Compute gradient orientations in the region

gx = np.gradient(region.astype(float), axis=1) gy = np.gradient(region.astype(float), axis=0) magnitudes = np.sqrt(gx2 + gy2) orientations = np.arctan2(gy, gx)

Create orientation histogram (simplified)

hist, _ = np.histogram(orientations, bins=8, weights=magnitudes) features.append(hist / (np.sum(hist) + 1e-8)) # Normalize

return np.array(features) ```

The introduction of PASCAL Visual Object Classes (VOC) challenge in 2005 provided a standardized benchmark that catalyzed progress in object detection. For the first time, researchers had a common metric for comparing algorithms and tracking improvements over time. The annual competitions revealed both the power and limitations of classical approaches: methods based on hand-crafted features like SIFT, HOG (Histogram of Oriented Gradient