Page 37

3.3

Classification

3.3.2

Training Methods

Classifiers can be trained using (Duda et al. [2001]):

Batch Training - all data from the training data set are presented to the clas-

sifier in the training process (this includes historic data in the re-training
case).

Stochastic Training - data used in the training process are randomly selected

from the training data set, i.e.

training data can be considered to be

stochastic variables. This training method is used when there are large
redundant data sets.

Incremental Training - data are only used once in the training process, i.e.

historic data is not used in the training process when the training data
grows. If the classifier using incremental training is kept up-to-date at all
times it is also called online training. This training method is used when
the growth and amount of data is so large that storing them is too expensive.
The classifiers proposed in paper F, G and H support incremental and online
training.

Decremental Training - In domains where classification data grows rapidly,

the classes or concepts might change over time; this is called con-
cept drift, Widmer and Kubat [1996]. The corresponding learning task
is called learning drifting concepts, Schwefel et al. [2003]. In order to
efficiently adapt to concept drift the classifier must unlearn the old con-
cepts, this is called forgetting or decremental training, Widmer and Kubat
[1996]. The classifier proposed in paper G support decremental training for
learning drifting concepts.

3.3.3

Linear and Nonlinear Classifiers

Classifiers that can discriminate between classes of non-rectangular shapes are
called nonlinear classifiers, Support Vector Machines and Artificial Neural
Networks using nonlinear kernel functions are examples of nonlinear classifiers.
See figure 3.3.3 for an example of a nonlinear classification problem. The XOR
problem is a well-known example of a non-rectangular classification problem; it
can be solved with a nonlinear classifier but not with a linear classifier. Examples
of linear classifiers are C4.5, Logistic Regression and Naive Bayes, Goldman
and Axtell [1995]. The classifiers proposed in paper F, G and H are linear.

<< - < - > - >>