Classification

Classification:

• Partition the feature space into two regions by finding the decision boundary that minimizes the error.

Sensors & Preprocessing
– Use a sensor (camera or microphone) for data capture.
– PR depends on bandwidth, resolution, sensitivity, distortion of the sensor.

• Pre-processing:

– Removal of noise in data.
– Segmentation (i.e., isolation of patterns of interest from background).

Feature Extraction
• How to choose a good set of features?
– Discriminative features
– Invariant features (e.g., translation, rotation and scale)
• Are there ways to automatically learn which features are best ?

How Many Features?

• Does adding more features always improve performance?
– It might be difficult and computationally expensive to extract certain features.
– Correlated features might not improve performance.
– “Curse” of dimensionality.

Curse of Dimensionality
• Adding too many features can, paradoxically, lead to a worsening of performance.
– Divide each of the input features into a number of intervals, so that the value of a feature can be specified approximately by
saying in which interval it lies.
– If each input feature is divided into M divisions, then the total number of cells is M d (d: # of features).
– Since each cell must contain at least one point, the number of training data grows exponentially with d.