Linear Discriminant Analysis

This cookbook page introduces the application of linear discriminant analysis to multi-class classifications.

Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and CMulticlassLabels as

features_train = RealFeatures(f_feats_train)
features_test = RealFeatures(f_feats_test)
labels_train = MulticlassLabels(f_labels_train)
labels_test = MulticlassLabels(f_labels_test)
features_train = RealFeatures(f_feats_train);
features_test = RealFeatures(f_feats_test);
labels_train = MulticlassLabels(f_labels_train);
labels_test = MulticlassLabels(f_labels_test);
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
features_train = Modshogun::RealFeatures.new f_feats_train
features_test = Modshogun::RealFeatures.new f_feats_test
labels_train = Modshogun::MulticlassLabels.new f_labels_train
labels_test = Modshogun::MulticlassLabels.new f_labels_test
features_train <- RealFeatures(f_feats_train)
features_test <- RealFeatures(f_feats_test)
labels_train <- MulticlassLabels(f_labels_train)
labels_test <- MulticlassLabels(f_labels_test)
features_train = modshogun.RealFeatures(f_feats_train)
features_test = modshogun.RealFeatures(f_feats_test)
labels_train = modshogun.MulticlassLabels(f_labels_train)
labels_test = modshogun.MulticlassLabels(f_labels_test)
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
auto features_train = some<CDenseFeatures<float64_t>>(f_feats_train);
auto features_test = some<CDenseFeatures<float64_t>>(f_feats_test);
auto labels_train = some<CMulticlassLabels>(f_labels_train);
auto labels_test = some<CMulticlassLabels>(f_labels_test);

We create an instance of the CMCLDA classifier with feature matrix and label list. CMCLDA also has two default parameters, to set tolerance used in training and mark whether to store the within class covariances.

mc_lda = MCLDA(features_train, labels_train, 0.0001, True)
mc_lda = MCLDA(features_train, labels_train, 0.0001, true);
MCLDA mc_lda = new MCLDA(features_train, labels_train, 0.0001, true);
mc_lda = Modshogun::MCLDA.new features_train, labels_train, 0.0001, 1
mc_lda <- MCLDA(features_train, labels_train, 0.0001, TRUE)
mc_lda = modshogun.MCLDA(features_train, labels_train, 0.0001, True)
MCLDA mc_lda = new MCLDA(features_train, labels_train, 0.0001, true);
auto mc_lda = some<CMCLDA>(features_train, labels_train, 0.0001, true);

Then we train and apply it to the test data, which here gives CMulticlassLabels.

mc_lda.train()
labels_predict = mc_lda.apply_multiclass(features_test)
mc_lda.train();
labels_predict = mc_lda.apply_multiclass(features_test);
mc_lda.train();
MulticlassLabels labels_predict = mc_lda.apply_multiclass(features_test);
mc_lda.train 
labels_predict = mc_lda.apply_multiclass features_test
mc_lda$train()
labels_predict <- mc_lda$apply_multiclass(features_test)
mc_lda:train()
labels_predict = mc_lda:apply_multiclass(features_test)
mc_lda.train();
MulticlassLabels labels_predict = mc_lda.apply_multiclass(features_test);
mc_lda->train();
auto labels_predict = mc_lda->apply_multiclass(features_test);

We can extract the mean vector of one class. If we enabled storing covariance when creating instances, we can also extract the covariance matrix:

classlabel = 1
m = mc_lda.get_mean(classlabel)
c = mc_lda.get_cov()
classlabel = 1;
m = mc_lda.get_mean(classlabel);
c = mc_lda.get_cov();
int classlabel = 1;
DoubleMatrix m = mc_lda.get_mean(classlabel);
DoubleMatrix c = mc_lda.get_cov();
classlabel = 1
m = mc_lda.get_mean classlabel
c = mc_lda.get_cov 
classlabel <- 1
m <- mc_lda$get_mean(classlabel)
c <- mc_lda$get_cov()
classlabel = 1
m = mc_lda:get_mean(classlabel)
c = mc_lda:get_cov()
int classlabel = 1;
double[] m = mc_lda.get_mean(classlabel);
double[,] c = mc_lda.get_cov();
auto classlabel = 1;
auto m = mc_lda->get_mean(classlabel);
auto c = mc_lda->get_cov();

We can evaluate test performance via e.g. CMulticlassAccuracy.

evals = MulticlassAccuracy()
accuracy = evals.evaluate(labels_predict, labels_test)
evals = MulticlassAccuracy();
accuracy = evals.evaluate(labels_predict, labels_test);
MulticlassAccuracy evals = new MulticlassAccuracy();
double accuracy = evals.evaluate(labels_predict, labels_test);
evals = Modshogun::MulticlassAccuracy.new 
accuracy = evals.evaluate labels_predict, labels_test
evals <- MulticlassAccuracy()
accuracy <- evals$evaluate(labels_predict, labels_test)
evals = modshogun.MulticlassAccuracy()
accuracy = evals:evaluate(labels_predict, labels_test)
MulticlassAccuracy evals = new MulticlassAccuracy();
double accuracy = evals.evaluate(labels_predict, labels_test);
auto evals = some<CMulticlassAccuracy>();
auto accuracy = evals->evaluate(labels_predict, labels_test);