In the previous activity, we classified objects based on their features using the Minimum Distance method.
In this activity, we will use Linear Discriminant Analysis or LDA. In LDA, the features are assumed to be linearly separable. The objects and features used are the same as in Activity 14.
Now for some math.
The discriminant function is given by:
The following discussion will be specific to the number of objects and features I used, so that it will also serve as an example. Below summarizes the number of test and training images:
In the discriminant function, p is the prior probability vector given by:
where n1 is the number of samples of group 1 (in this case 5), and so on, and N is the total number of samples (in this case 24).
C is the pooled covariance matrix given by:
where
Of course, \mu_i is the mean of the features of each group.
An object k is assigned to a group i where the f_i is maximum.
If the discussion is confusing, visit reference [1], which provides a more complete discussion. :)
Okay enough math. Here are the results:
The values in the table are the f_i values of an object k to each group i. The maximum f_i are highlighted for each sample. The number of classified objects for each group is summarized below.
We can see that the LDA is successful in classifying the objects. It was able to classify all objects correctly, except for 1 case, which gives the method a 96% accuracy. This is again much much higher, more than three times that of the chance proportion criterion of 26.08%.
Basically, LDA is similar to PCA where it finds a linear combination of variables that best represents the data [2]. A transformation function that "maximizes the ratio of between-class variance to within-class variance" [3]. In other words, the axis is transformed such that the clusters are more separated if the values are projected.
I give myself 10 points for this activity.
References:
[1] Teknomo, Kardi. Linear Discriminant Analysis. http://people.revoledu.com/kardi/tutorial/LDA/LDA.html#LDA
[2] Linear Discriminant Analysis. http://en.wikipedia.org/wiki/Linear_discriminant_analysis
[3] Linear Discriminant Analysis. http://www.dtreg.com/lda.htm
In this activity, we will use Linear Discriminant Analysis or LDA. In LDA, the features are assumed to be linearly separable. The objects and features used are the same as in Activity 14.
Now for some math.
The discriminant function is given by:
The following discussion will be specific to the number of objects and features I used, so that it will also serve as an example. Below summarizes the number of test and training images:
In the discriminant function, p is the prior probability vector given by:
where n1 is the number of samples of group 1 (in this case 5), and so on, and N is the total number of samples (in this case 24).
C is the pooled covariance matrix given by:
where
Of course, \mu_i is the mean of the features of each group.
An object k is assigned to a group i where the f_i is maximum.
If the discussion is confusing, visit reference [1], which provides a more complete discussion. :)
Okay enough math. Here are the results:
The values in the table are the f_i values of an object k to each group i. The maximum f_i are highlighted for each sample. The number of classified objects for each group is summarized below.
We can see that the LDA is successful in classifying the objects. It was able to classify all objects correctly, except for 1 case, which gives the method a 96% accuracy. This is again much much higher, more than three times that of the chance proportion criterion of 26.08%.
Basically, LDA is similar to PCA where it finds a linear combination of variables that best represents the data [2]. A transformation function that "maximizes the ratio of between-class variance to within-class variance" [3]. In other words, the axis is transformed such that the clusters are more separated if the values are projected.
I give myself 10 points for this activity.
References:
[1] Teknomo, Kardi. Linear Discriminant Analysis. http://people.revoledu.com/kardi/tutorial/LDA/LDA.html#LDA
[2] Linear Discriminant Analysis. http://en.wikipedia.org/wiki/Linear_discriminant_analysis
[3] Linear Discriminant Analysis. http://www.dtreg.com/lda.htm
0 comments:
Post a Comment