Multi-class Linear Machine

We extend the application of linear machines to multi-class datasets by constructing generic multiclass classifiers with ensembles of binary classifiers.

In this example, we show how to apply CLibLinear to multi-class cases with CLinearMulticlassMachine.

See the linear SVM cookbook for the infomration about CLibLinear binary classifier.

Example

Imagine we have files with training and test data. We create CDenseFeatures (here 64 bit floats aka RealFeatures) and CMulticlassLabels as

features_train = RealFeatures(f_feats_train)
features_test = RealFeatures(f_feats_test)
labels_train = MulticlassLabels(f_labels_train)
labels_test = MulticlassLabels(f_labels_test)
features_train = RealFeatures(f_feats_train);
features_test = RealFeatures(f_feats_test);
labels_train = MulticlassLabels(f_labels_train);
labels_test = MulticlassLabels(f_labels_test);
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
features_train = Modshogun::RealFeatures.new f_feats_train
features_test = Modshogun::RealFeatures.new f_feats_test
labels_train = Modshogun::MulticlassLabels.new f_labels_train
labels_test = Modshogun::MulticlassLabels.new f_labels_test
features_train <- RealFeatures(f_feats_train)
features_test <- RealFeatures(f_feats_test)
labels_train <- MulticlassLabels(f_labels_train)
labels_test <- MulticlassLabels(f_labels_test)
features_train = modshogun.RealFeatures(f_feats_train)
features_test = modshogun.RealFeatures(f_feats_test)
labels_train = modshogun.MulticlassLabels(f_labels_train)
labels_test = modshogun.MulticlassLabels(f_labels_test)
RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
MulticlassLabels labels_train = new MulticlassLabels(f_labels_train);
MulticlassLabels labels_test = new MulticlassLabels(f_labels_test);
auto features_train = some<CDenseFeatures<float64_t>>(f_feats_train);
auto features_test = some<CDenseFeatures<float64_t>>(f_feats_test);
auto labels_train = some<CMulticlassLabels>(f_labels_train);
auto labels_test = some<CMulticlassLabels>(f_labels_test);

We use CLibLinear as base classifier and create an instance of CLibLinear.

classifier = LibLinear()
classifier = LibLinear();
LibLinear classifier = new LibLinear();
classifier = Modshogun::LibLinear.new 
classifier <- LibLinear()
classifier = modshogun.LibLinear()
LibLinear classifier = new LibLinear();
auto classifier = some<CLibLinear>();

In order to run CLinearMulticlassMachine, we need to specify an multi-class strategy from CMulticlassOneVsRestStrategy and CMulticlassOneVsOneStrategy.

strategy = MulticlassOneVsOneStrategy()
strategy = MulticlassOneVsOneStrategy();
MulticlassOneVsOneStrategy strategy = new MulticlassOneVsOneStrategy();
strategy = Modshogun::MulticlassOneVsOneStrategy.new 
strategy <- MulticlassOneVsOneStrategy()
strategy = modshogun.MulticlassOneVsOneStrategy()
MulticlassOneVsOneStrategy strategy = new MulticlassOneVsOneStrategy();
auto strategy = some<CMulticlassOneVsOneStrategy>();

We create an instance of the CLinearMulticlassMachine classifier by passing it the strategy, dataset, binary classifer and the labels.

mc_classifier = LinearMulticlassMachine(strategy, features_train, classifier, labels_train)
mc_classifier = LinearMulticlassMachine(strategy, features_train, classifier, labels_train);
LinearMulticlassMachine mc_classifier = new LinearMulticlassMachine(strategy, features_train, classifier, labels_train);
mc_classifier = Modshogun::LinearMulticlassMachine.new strategy, features_train, classifier, labels_train
mc_classifier <- LinearMulticlassMachine(strategy, features_train, classifier, labels_train)
mc_classifier = modshogun.LinearMulticlassMachine(strategy, features_train, classifier, labels_train)
LinearMulticlassMachine mc_classifier = new LinearMulticlassMachine(strategy, features_train, classifier, labels_train);
auto mc_classifier = some<CLinearMulticlassMachine>(strategy, features_train, classifier, labels_train);

Then we train and apply it to test data, which here gives CMulticlassLabels.

mc_classifier.train()
labels_predict = mc_classifier.apply_multiclass(features_test)
mc_classifier.train();
labels_predict = mc_classifier.apply_multiclass(features_test);
mc_classifier.train();
MulticlassLabels labels_predict = mc_classifier.apply_multiclass(features_test);
mc_classifier.train 
labels_predict = mc_classifier.apply_multiclass features_test
mc_classifier$train()
labels_predict <- mc_classifier$apply_multiclass(features_test)
mc_classifier:train()
labels_predict = mc_classifier:apply_multiclass(features_test)
mc_classifier.train();
MulticlassLabels labels_predict = mc_classifier.apply_multiclass(features_test);
mc_classifier->train();
auto labels_predict = mc_classifier->apply_multiclass(features_test);

We can evaluate test performance via e.g. CMulticlassAccuracy.

eval = MulticlassAccuracy()
accuracy = eval.evaluate(labels_predict, labels_test)
eval = MulticlassAccuracy();
accuracy = eval.evaluate(labels_predict, labels_test);
MulticlassAccuracy eval = new MulticlassAccuracy();
double accuracy = eval.evaluate(labels_predict, labels_test);
eval = Modshogun::MulticlassAccuracy.new 
accuracy = eval.evaluate labels_predict, labels_test
eval <- MulticlassAccuracy()
accuracy <- eval$evaluate(labels_predict, labels_test)
eval = modshogun.MulticlassAccuracy()
accuracy = eval:evaluate(labels_predict, labels_test)
MulticlassAccuracy eval = new MulticlassAccuracy();
double accuracy = eval.evaluate(labels_predict, labels_test);
auto eval = some<CMulticlassAccuracy>();
auto accuracy = eval->evaluate(labels_predict, labels_test);