them. By solving the equation for y and inserting the result into the expression
to be minimized, one gets the following unconstrained optimization problem:
min
(w,)R
n+1+m
f (w, ) =
2
D(Aw - e) - e
2
+
1
2
(w w +
2
)
(2)
Setting
f =
f
w
,
f
= 0 one gets:
w
X
=
A A +
I
-A e
-e A
1
+ m
-1
A De
-e De
=
I
+ E E
-1
A
-1
E De
B
(3)
E = [A - e], E R
m×(n+1)
Agarwal has showed that the Proximal SVM is directly transferable to a ridge
regression expression [1]. Fung and Mangasarian [10] later showed that (3) can
be rewritten to handle increments (E
i
, d
i
) and decrements (E
d
, d
d
), as shown
in (4). This decremental approach is based on time windows.
X =
w
=
I
+ E E + (E
i
) E
i
- (E
d
) E
d
-1
E d + (E
i
) d
i
- (E
d
) d
d
,
(4)
where d = De
.
3
PSVM Decremental Learning using Weight Decay
Coefficient
The basic idea is to reduce the effect of the existing (old) accumulated training
knowledge E E with an exponential weight decay coefficient .
w
=
I
+ · E E + E
i
E
i
-1
· E d + E
i
d
i
; 0, 1]
(5)
As opposed to the decremental approach in expression (4), the presented
weight decay approach does not require storage of increments (E
i
E
i
, E
i
d
i
)
later to be retrieved as decrements (E
d
E
d
, E
d
d
d
).
A hybrid approach is shown in expression (6), where one has both a soft
decremental effect using the weight decay coefficient as well as a hard decre-
mental effect using a fixed window of size W increments.
118
Incremental and Decremental PSVM Classifiers