Quadratic Discriminant Analysis

Quadratic discriminant analysis (QDA) is used to separate measurements of two or more classes of objects by a quadric surface. For QDA, the class label \(y\) is assumed to be quadratic in the measurements of observations \(X\), i.e.:

\[\mathbf{x^{T}Ax} + \mathbf{b^{T}x} + c\]

QDA is a generalization of linear discriminant analysis (LDA). See Chapter 16 in [Bar12] for a detailed introduction.

Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and CMulticlassLabels as

features_train = RealFeatures(f_feats_train)
features_test = RealFeatures(f_feats_test)
labels_train = MulticlassLabels(f_labels_train)
labels_test = MulticlassLabels(f_labels_test)
features_train = RealFeatures(f_feats_train);
features_test = RealFeatures(f_feats_test);
labels_train = MulticlassLabels(f_labels_train);
labels_test = MulticlassLabels(f_labels_test);
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
features_train = Modshogun::RealFeatures.new f_feats_train
features_test = Modshogun::RealFeatures.new f_feats_test
labels_train = Modshogun::MulticlassLabels.new f_labels_train
labels_test = Modshogun::MulticlassLabels.new f_labels_test
features_train <- RealFeatures(f_feats_train)
features_test <- RealFeatures(f_feats_test)
labels_train <- MulticlassLabels(f_labels_train)
labels_test <- MulticlassLabels(f_labels_test)
features_train = modshogun.RealFeatures(f_feats_train)
features_test = modshogun.RealFeatures(f_feats_test)
labels_train = modshogun.MulticlassLabels(f_labels_train)
labels_test = modshogun.MulticlassLabels(f_labels_test)
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
auto features_train = some<CDenseFeatures<float64_t>>(f_feats_train);
auto features_test = some<CDenseFeatures<float64_t>>(f_feats_test);
auto labels_train = some<CMulticlassLabels>(f_labels_train);
auto labels_test = some<CMulticlassLabels>(f_labels_test);

We create an instance of the CQDA classifier with feature matrix and label list. CQDA also has two default arguments, to set tolerance used in training and mark whether to store the within class covariances

qda = QDA(features_train, labels_train, 0.0001, True)
qda = QDA(features_train, labels_train, 0.0001, true);
QDA qda = new QDA(features_train, labels_train, 0.0001, true);
qda = Modshogun::QDA.new features_train, labels_train, 0.0001, 1
qda <- QDA(features_train, labels_train, 0.0001, TRUE)
qda = modshogun.QDA(features_train, labels_train, 0.0001, True)
QDA qda = new QDA(features_train, labels_train, 0.0001, true);
auto qda = some<CQDA>(features_train, labels_train, 0.0001, true);

We run the train QDA algorithm and apply it to test data, which here gives CMulticlassLabels.

qda.train()
labels_predict = qda.apply_multiclass(features_test)
qda.train();
labels_predict = qda.apply_multiclass(features_test);
qda.train();
MulticlassLabels labels_predict = qda.apply_multiclass(features_test);
qda.train 
labels_predict = qda.apply_multiclass features_test
qda$train()
labels_predict <- qda$apply_multiclass(features_test)
qda:train()
labels_predict = qda:apply_multiclass(features_test)
qda.train();
MulticlassLabels labels_predict = qda.apply_multiclass(features_test);
qda->train();
auto labels_predict = qda->apply_multiclass(features_test);

We can extract the mean vector of one class. If we enabled storing covariance when creating instances, we can also extract the covariance matrix of the class:

classlabel = 1
m = qda.get_mean(classlabel)
c = qda.get_cov(classlabel)
classlabel = 1;
m = qda.get_mean(classlabel);
c = qda.get_cov(classlabel);
int classlabel = 1;
DoubleMatrix m = qda.get_mean(classlabel);
DoubleMatrix c = qda.get_cov(classlabel);
classlabel = 1
m = qda.get_mean classlabel
c = qda.get_cov classlabel
classlabel <- 1
m <- qda$get_mean(classlabel)
c <- qda$get_cov(classlabel)
classlabel = 1
m = qda:get_mean(classlabel)
c = qda:get_cov(classlabel)
int classlabel = 1;
double[] m = qda.get_mean(classlabel);
double[,] c = qda.get_cov(classlabel);
auto classlabel = 1;
auto m = qda->get_mean(classlabel);
auto c = qda->get_cov(classlabel);

We can evaluate test performance via e.g. CMulticlassAccuracy.

eval = MulticlassAccuracy()
accuracy = eval.evaluate(labels_predict, labels_test)
eval = MulticlassAccuracy();
accuracy = eval.evaluate(labels_predict, labels_test);
MulticlassAccuracy eval = new MulticlassAccuracy();
double accuracy = eval.evaluate(labels_predict, labels_test);
eval = Modshogun::MulticlassAccuracy.new 
accuracy = eval.evaluate labels_predict, labels_test
eval <- MulticlassAccuracy()
accuracy <- eval$evaluate(labels_predict, labels_test)
eval = modshogun.MulticlassAccuracy()
accuracy = eval:evaluate(labels_predict, labels_test)
MulticlassAccuracy eval = new MulticlassAccuracy();
double accuracy = eval.evaluate(labels_predict, labels_test);
auto eval = some<CMulticlassAccuracy>();
auto accuracy = eval->evaluate(labels_predict, labels_test);