SHOGUN  4.0.0
Examples for Octave Modular Interface

This page lists ready to run shogun examples for the Octave Modular interface.

To run the examples issue

octave name_of_example.m

or start up octave and then type

name_of_example

# Classifier

• examples/documented/octave_modular/classifier_domainadaptationsvm_modular.m
% In this example we demonstrate how to use SVMs in a domain adaptation
% scenario. Here, we assume that we have two problem domains, one with
% an abundance of training data (source domain) and one with only a few
% training examples (target domain). These domains are assumed to be
% different but related enough to transfer information between them.
% Thus, we first train an SVM on the source domain and then subsequently
% pass this previously trained SVM object to the DASVM, that we train
% on the target domain. The DASVM internally computes a custom linear term
% (for the underlying quadratic program of the dual formulation of the SVM)
% based on the support vectors of the source SVM and the training examples
% of the target SVM. Finally, it can be used for prediction just as any other
% SVM object.
%

addpath('tools');
modshogun;

tmp=load_matrix('../data/label_train_dna.dat');
label_train_dna=tmp(1:50);
tmp=load_matrix('../data/fm_train_dna.dat');
fm_train_dna=tmp(:,1:50);
tmp=load_matrix('../data/label_train_dna.dat');
label_train_dna2=tmp(50:92);
tmp=load_matrix('../data/fm_train_dna.dat');
fm_train_dna2=tmp(:, 50:92);
fm_test_dna=load_matrix('../data/fm_test_dna.dat');
fm_test_dna2=tmp(:,50:92);

%if exist('SVMLight')
disp('Domain Adaptation SVM')

C = 1.0;
degree=3;

feats_train=StringCharFeatures(DNA);
feats_test=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test.set_features(fm_test_dna);
kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree);
labels=BinaryLabels(label_train_dna);
svm=SVMLight(C, kernel, labels);
svm.train();

%#####################################

disp('obtaining DA SVM from previously trained SVM')

feats_train2=StringCharFeatures(DNA);
feats_test2=StringCharFeatures(DNA);
feats_train2.set_features(fm_train_dna2);
feats_test2.set_features(fm_test_dna2);

kernel2=WeightedDegreeStringKernel(feats_train, feats_train, degree);

labels2=BinaryLabels(label_train_dna);

% we regularize versus the previously obtained solution
dasvm = DomainAdaptationSVM(C, kernel2, labels2, svm, 1.0);
dasvm.train();

out = dasvm.apply(feats_test2).get_labels();
%else
%disp('No support for SVMLight available.')
%end

• examples/documented/octave_modular/classifier_gmnpsvm_modular.m
% In this example a multi-class support vector machine is trained on a toy data
% set and the trained classifier is then used to predict labels of test
% examples. The training algorithm is based on BSVM formulation (L2-soft margin
% and the bias added to the objective function) which is solved by the Improved
% Mitchell-Demyanov-Malozemov algorithm. The training algorithm uses the Gaussian
% kernel of width 2.1 and the regularization constant C=1. The solver stops if the
% relative duality gap falls below 1e-5.
%
% For more details on the used SVM solver see
%  V.Franc: Optimization Algorithms for Kernel Methods. Research report.
%  CTU-CMP-2005-22. CTU FEL Prague. 2005.
%  ftp://cmp.felk.cvut.cz/pub/cmp/articles/franc/Franc-PhD.pdf .
%

modshogun

addpath('tools');
label_train_multiclass=load_matrix('../data/label_train_multiclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% gmnpsvm
disp('GMNPSVM')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=1;
labels=MulticlassLabels(label_train_multiclass);

svm=GMNPSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_gpbtsvm_modular.m
% In this example a two-class support vector machine classifier is trained on a
% toy data set and the trained classifier is then used to predict labels of test
% examples. As training algorithm Gradient Projection Decomposition Technique
% (GPDT) is used with SVM regularization parameter C=1 and a Gaussian
% kernel of width 2.1. The solver returns an epsilon-precise (epsilon=1e-5) solution.
%
% For more details on GPDT solver see http://dm.unife.it/gpdt .
%

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% gpbtsvm
disp('GPBTSVM')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=2;
labels=BinaryLabels(label_train_twoclass);

svm=GPBTSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_knn_modular.m
% This example shows usage of a k-nearest neighbor (KNN) classification rule on
% a toy data set. The number of the nearest neighbors is set to k=3 and the distances
% are measured by the Euclidean metric. Finally, the KNN rule is applied to predict
% labels of test examples.

modshogun

addpath('tools');
label_train_multiclass=load_matrix('../data/label_train_multiclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% knn
disp('KNN')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
distance=EuclideanDistance(feats_train, feats_train);

k=3;
num_threads=1;
labels=MulticlassLabels(label_train_multiclass);

knn=KNN(k, distance, labels);
knn.parallel.set_num_threads(num_threads);
knn.train();

output=knn.apply(feats_test).get_labels();

• examples/documented/octave_modular/classifier_lda_modular.m
% In this example a two-class linear classifier based on the Linear Discriminant
% Analysis (LDA) is trained on a toy data set and then the trained classifier is
% used to predict test examples. The regularization parameter, which corresponds
% to a weight of a unitary matrix added to the covariance matrix, is set to
% gamma=3.
%
% For more details on the LDA see e.g.
%     http://en.wikipedia.org/wiki/Linear_discriminant_analysis

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% lda
disp('LDA')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

gamma=3;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

lda=LDA(gamma, feats_train, labels);
lda.parallel.set_num_threads(num_threads);
lda.train();

lda.get_bias();
lda.get_w();
lda.set_features(feats_test);
lda.apply().get_labels();

• examples/documented/octave_modular/classifier_liblinear_modular.m
% In this example a two-class linear support vector machine classifier is trained
% on a toy data set and the trained classifier is then used to predict labels of
% test examples. As training algorithm the LIBLINEAR solver is used with the SVM
% regularization parameter C=0.9 and the bias in the classification rule switched
% on and the precision parameters epsilon=1e-5.
%
% For more details on LIBLINEAR see
%     http://www.csie.ntu.edu.tw/~cjlin/liblinear/

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% liblinear
disp('LibLinear')

realfeat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
realfeat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(realfeat);

C=1.2;
epsilon=1e-5;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

svm=LibLinear(C, feats_train, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.set_bias_enabled(true);
svm.train();

svm.set_features(feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_libsvm_minimal_modular.m
% In this example a two-class support vector machine classifier is trained on a
% 2-dimensional randomly generated data set and the trained classifier is used to
% predict labels of test examples. As training algorithm the LIBSVM solver is used
% with SVM regularization parameter C=1 and a Gaussian kernel of width 2.1.
%
% For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/

modshogun

num=1000;
dist=1;
width=2.1;
C=1;

traindata_real=[randn(2,num)-dist, randn(2,num)+dist];
testdata_real=[randn(2,num)-dist, randn(2,num)+dist];

trainlab=[-ones(1,num), ones(1,num)];
testlab=[-ones(1,num), ones(1,num)];

feats_train=RealFeatures(traindata_real);
feats_test=RealFeatures(testdata_real);
kernel=GaussianKernel(feats_train, feats_train, width);

labels=BinaryLabels(trainlab);
svm=LibSVM(C, kernel, labels);
svm.parallel.set_num_threads(8);
svm.train();
kernel.init(feats_train, feats_test);
out=svm.apply().get_labels();
testerr=mean(sign(out)~=testlab)

• examples/documented/octave_modular/classifier_libsvm_modular.m
% In this example a two-class support vector machine classifier is trained on a
% toy data set and the trained classifier is used to predict labels of test
% examples. As training algorithm the LIBSVM solver is used with SVM
% regularization parameter C=1 and a Gaussian kernel of width 2.1 and the
% precision parameter epsilon=1e-5. The example also shows how to retrieve the
% support vectors from the train SVM model.
%
% For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% libsvm
disp('LibSVM')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=2;
labels=BinaryLabels(label_train_twoclass);

svm=LibSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_libsvmoneclass_modular.m
% In this example a one-class support vector machine classifier is trained on a
% toy data set. The training algorithm finds a hyperplane in the RKHS which
% separates the training data from the origin. The one-class classifier is
% typically used to estimate the support of a high-dimesnional distribution.
% For more details see e.g.
%   B. Schoelkopf et al. Estimating the support of a high-dimensional
%   distribution. Neural Computation, 13, 2001, 1443-1471.
%
% In the example, the one-class SVM is trained by the LIBSVM solver with the
% regularization parameter C=1 and the Gaussian kernel of width 2.1 and the
% precision parameter epsilon=1e-5.
%
% For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% libsvm twoclass
disp('LibSVMOneClass')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=4;

svm=LibSVMOneClass(C, kernel);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_mpdsvm_modular.m
% In this example a two-class support vector machine classifier is trained on a
% toy data set and the trained classifier is used to predict labels of test
% examples. As training algorithm the Minimal Primal Dual SVM is used with SVM
% regularization parameter C=1 and a Gaussian kernel of width 1.2 and the
% precision parameter 1e-5.
%
% For more details on the MPD solver see
%  Kienzle, W. and B. Schölkopf: Training Support Vector Machines with Multiple
%  Equality Constraints. Machine Learning: ECML 2005, 182-193. (Eds.) Carbonell,
%  J. G., J. Siekmann, Springer, Berlin, Germany (11 2005)

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% mpdsvm
disp('MPDSVM')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

svm=MPDSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_multiclasslibsvm_modular.m
modshogun

addpath('tools');
label_train_multiclass=load_matrix('../data/label_train_multiclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

disp('MulticlassLibSVM')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
num_threads=8;
labels=MulticlassLabels(label_train_multiclass);

svm=MulticlassLibSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_newtonsvm_modular.m
modshogun

addpath('tools');
label=load_matrix('../data/label_train_dna.dat');
data=load_matrix('../data/fm_train_dna.dat');
fm_test_real=load_matrix('../data/fm_test_dna.dat');

% Newton SVM

disp('NewtonSVM')
data=double(data);
%fm_test_real=double(fm_test_real);
realfeat=RealFeatures(data);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
%realfeat=RealFeatures(fm_test_real);
%feats_test=SparseRealFeatures();
%feats_test.obtain_from_simple(realfeat);

C=1.0;
epsilon=1e-5;
num_threads=1;
label=double(label);
labels=BinaryLabels(label);

svm=NewtonSVM(C, feats_train, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.set_bias_enabled(true);
svm.train();
%svm.set_features(feats_test);
svm.get_bias();
svm.get_w();
%svm.apply().get_labels();

• examples/documented/octave_modular/classifier_perceptron_modular.m
% This example shows usage of the Perceptron algorithm for training a two-class
% linear classifier, i.e.  y = sign( <x,w>+b). The Perceptron algorithm works by
% iteratively passing though the training examples and applying the update rule on
% those examples which are misclassified by the current classifier. The Perceptron
% update rule reads
%
%   w(t+1) = w(t) + alpha * y_t * x_t
%   b(t+1) = b(t) + alpha * y_t
%
% where (x_t,y_t) is feature vector and label (must be +1/-1) of the misclassified example
%       (w(t),b(t)) are the current parameters of the linear classifier
%       (w(t+1),b(t+1)) are the new parameters of the linear classifier
%       alpha is the learning rate; in this examples alpha=1
%
% The Perceptron algorithm iterates until all training examples are correctly
% classified or the prescribed maximal number of iterations, in this example
% max_iter=1000, is reached.

modshogun

num=50;
label_train_twoclass=[-ones(1,num/2) ones(1,num/2)];
fm_train_real=[randn(5,num/2)-1, randn(5,num/2)+1];
fm_test_real=[randn(5,num)-1, randn(5,num)+1];

% perceptron
disp('Perceptron')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

learn_rate=1.;
max_iter=1000;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

perceptron=Perceptron(feats_train, labels);
perceptron.set_learn_rate(learn_rate);
perceptron.set_max_iter(max_iter);
perceptron.parallel.set_num_threads(num_threads);
perceptron.train();

perceptron.set_features(feats_test);
perceptron.apply().get_labels();

• examples/documented/octave_modular/classifier_svm_serialize_modular.m
% initialize modular shogun interface
modshogun

% add path to load matrix script
addpath('tools');

% some defines
C = 1.2;
width = 2.1;
epsilon = 1e-5;
num_threads = 2;

% get train features and labels
fm_train_real = load_matrix('../data/fm_train_real.dat');
fm_train_labels = load_matrix('../data/label_train_twoclass.dat');

% get test features and labels %fixme need example w/ test data/labels - using training data instead
fm_test_real = load_matrix('../data/fm_train_real.dat');
fm_test_labels = load_matrix('../data/label_train_twoclass.dat');

% create feature and label objects
feats_train = RealFeatures(fm_train_real);
feats_test = RealFeatures(fm_test_real);
labels = BinaryLabels(fm_train_labels);

% create kernel
kernel = GaussianKernel(feats_train, feats_train, width);

% create support vector machine
svm = LibSVM(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);

% train
svm.train();

% save to file
file = SerializableAsciiFile('test_svm.dat', 'w');
svm.save_serializable(file);
file.close();

% load classifier and verify with test features
file_new = SerializableAsciiFile('test_svm.dat', 'r');
svm_new = LibSVM();
svm_new.load_serializable(file_new);
file_new.close();
result = svm_new.apply(feats_test).get_labels();

result = sum(sign(result) == fm_test_labels) / columns(fm_test_labels);

• examples/documented/octave_modular/classifier_svmlight_modular.m
% In this example a two-class support vector machine classifier is trained on a
% DNA splice-site detection data set and the trained classifier is used to predict
% labels on test set. As training algorithm SVM^light is used with SVM
% regularization parameter C=1.2 and the Weighted Degree kernel of degree 20 and
% the precision parameter epsilon=1e-5.
%
% For more details on the SVM^light see
%  T. Joachims. Making large-scale SVM learning practical. In Advances in Kernel
%  Methods -- Support Vector Learning, pages 169-184. MIT Press, Cambridge, MA USA, 1999.
%
% For more details on the Weighted Degree kernel see
%  G. Raetsch, S.Sonnenburg, and B. Schoelkopf. RASE: recognition of alternatively
%  spliced exons in C. elegans. Bioinformatics, 21:369-377, June 2005.

modshogun

% Explicit examples on how to use the different classifiers

addpath('tools');
label_train_dna=load_matrix('../data/label_train_dna.dat');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% svm light
try
disp('SVMLight')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
degree=20;

kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree);

C=1.2;
epsilon=1e-5;
num_threads=3;
labels=BinaryLabels(label_train_dna);

svm=SVMLight(C, kernel, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.train();

kernel.init(feats_train, feats_test);
svm.apply().get_labels();
catch
disp('No support for SVMLight available.')
end

• examples/documented/octave_modular/classifier_svmlin_modular.m
% In this example a two-class linear support vector machine classifier (SVM) is
% trained on a toy data set and the trained classifier is used to predict labels
% of test examples. As training algorithm the SVMLIN solver is used with the SVM
% regularization parameter C=0.9 and the bias in the classification rule switched
% on and the precision parameter epsilon=1e-5. The example also shows how to
% retrieve parameters (vector w and bias b)) of the trained linear classifier.
%
% For more details on the SVMLIN solver see
%  V. Sindhwani, S.S. Keerthi. Newton Methods for Fast Solution of Semi-supervised
%  Linear SVMs. Large Scale Kernel Machines MIT Press (Book Chapter), 2007

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% svm lin
disp('SVMLin')

realfeat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
realfeat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(realfeat);

C=0.9;
epsilon=1e-5;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

svm=SVMLin(C, feats_train, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.set_bias_enabled(true);
svm.train();

svm.set_features(feats_test);
svm.get_bias();
svm.get_w();
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_svmocas_modular.m
% In this example a two-class linear support vector machine classifier is trained
% on a toy data set and the trained classifier is used to predict labels of test
% examples. As training algorithm the OCAS solver is used with the SVM
% regularization parameter C=0.9 and the bias term in the classification rule
% switched off and the precision parameter epsilon=1e-5 (duality gap).
%
% For more details on the OCAS solver see
%  V. Franc, S. Sonnenburg. Optimized Cutting Plane Algorithm for Large-Scale Risk
%  Minimization.The Journal of Machine Learning Research, vol. 10,
%  pp. 2157--2192. October 2009.
%

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% svm ocas
disp('SVMOcas')

realfeat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
realfeat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(realfeat);

C=0.9;
epsilon=1e-5;
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

svm=SVMOcas(C, feats_train, labels);
svm.set_epsilon(epsilon);
svm.parallel.set_num_threads(num_threads);
svm.set_bias_enabled(false);
svm.train();

svm.set_features(feats_test);
svm.apply().get_labels();

• examples/documented/octave_modular/classifier_svmsgd_modular.m
% In this example a two-class linear support vector machine classifier is trained
% on a toy data set and the trained classifier is used to predict labels of test
% examples. As training algorithm the Stochastic Gradient Descent (SGD) solver is
% used with the SVM regularization parameter C=0.9. The number of iterations, i.e.
% passes though all training examples, is set to num_iter=5 .
%
% For more details on the SGD solver see
%  L. Bottou, O. Bousquet. The tradeoff of large scale learning. In NIPS 20. MIT
%  Press. 2008.

modshogun

addpath('tools');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sgd
disp('SVMSGD')

realfeat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
realfeat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(realfeat);

C=0.9;
num_iter=5
num_threads=1;
labels=BinaryLabels(label_train_twoclass);

svm=SVMSGD(C, feats_train, labels);
svm.set_epochs(num_iter)
%svm.io.set_loglevel(0);
svm.train();

svm.set_features(feats_test);
svm.apply().get_labels();

# Clustering

• examples/documented/octave_modular/clustering_hierarchical_modular.m
% In this example an agglomerative hierarchical single linkage clustering method
% is used to cluster a given toy data set. Starting with each object being
% assigned to its own cluster clusters are iteratively merged. Here the clusters
% are merged that have the closest (minimum distance, here set via the Euclidean
% distance object) two elements.

modshogun

% Explicit examples on how to use clustering

addpath('tools');
fm_train=load_matrix('../data/fm_train_real.dat');

% Hierarchical
disp('Hierarchical')

merges=4;
feats_train=RealFeatures(fm_train);
feats_test=RealFeatures(fm_train);
distance=EuclideanDistance(feats_train, feats_train);

hierarchical=Hierarchical(merges, distance);
hierarchical.train();

distance.init(feats_train, feats_test);
mdist=hierarchical.get_merge_distances();
pairs=hierarchical.get_cluster_pairs();

• examples/documented/octave_modular/clustering_kmeans_modular.m
% In this example the k-means clustering method is used to cluster a given toy
% data set. In k-means clustering one tries to partition n observations into k
% clusters in which each observation belongs to the cluster with the nearest mean.
% The algorithm class constructor takes the number of clusters and a distance to
% be used as input. The distance used in this example is Euclidean distance.
% After training one can fetch the result of clustering by obtaining the cluster
% centers and their radiuses.

modshogun

% Explicit examples on how to use clustering

addpath('tools');
fm_train=load_matrix('../data/fm_train_real.dat');

% KMeans
disp('KMeans')

k=4;
feats_train=RealFeatures(fm_train);
distance=EuclideanDistance(feats_train, feats_train);

kmeans=KMeans(k, distance);
kmeans.train();

c=kmeans.get_cluster_centers();
r=kmeans.get_radiuses();

# Converter

• examples/documented/octave_modular/converter_jade_bss.m
% Blind Source Separation using the Jade Algorithm with Shogun
%
% Based on the example from scikit-learn
% http://scikit-learn.org/
%
% Kevin Hughes 2013

modshogun

% Generate sample data
n_samples = 2000;
time = linspace(0,10,n_samples);

% Source Signals
S = zeros(2, length(time));
S(1,:) = sin(2*time);
S(2,:) = sign(sin(3*time));
S += 0.2*rand(size(S));

% Standardize data
S = S ./ (std(S,0,2) * ones(1,n_samples));

% Mixing Matrix
A = [1 0.5; 0.5 1]

% Mix Signals
X = A*S;
mixed_signals = RealFeatures(X);

% Separating
jade = Jade();
signals = jade.apply(mixed_signals);
signals = RealFeatures.obtain_from_generic(signals);
S_ = signals.get_feature_matrix();
A_ = jade.get_mixing_matrix()

% Plot
figure();
subplot(311);
plot(time, S(1,:), 'b');
hold on;
plot(time, S(2,:), 'g');
set(gca, 'xtick', [])
title("True Sources");

subplot(312);
plot(time, X(1,:), 'b');
hold on;
plot(time, X(2,:), 'g');
set(gca, 'xtick', [])
title("Mixed Sources");

subplot(313);
plot(time, S_(1,:), 'b');
hold on;
plot(time, S_(2,:), 'g');
title("Estimated Sources");

# Distance

• examples/documented/octave_modular/distance_braycurtis_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CBrayCurtisDistance.html.
%
% Obviously, using the Bray Curtis distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% bray curtis distance
disp('BrayCurtisDistance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=BrayCurtisDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_canberra_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance (dissimilarity ratio) matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CCanberraMetric.html.
%
% Obviously, using the Canberra distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% canberra metric
disp('CanberaMetric')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=CanberraMetric(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_canberraword_modular.m
% This example shows how to compute the Canberra Word Distance.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% canberra word distance
disp('CanberraWordDistance')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

distance=CanberraWordDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_chebyshew_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance (maximum of absolute feature dimension differences) matrix is
% computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% (maximum of absolute feature dimension differences) matrix between these
% two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CChebyshewMetric.html.
%
% Obviously, using the Chebyshew distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% chebyshew metric
disp('ChebyshewMetric')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=ChebyshewMetric(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_chisquare_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CChiSquareDistance.html.
%
% Obviously, using the ChiSquare distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% chi square distance
disp('ChiSquareDistance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=ChiSquareDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_cosine_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CCosineDistance.html.
%
% Obviously, using the Cosine distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% cosine distance
disp('Cosine Distance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=CosineDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_euclidian_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CEuclidianDistance.html.
%
% Obviously, using the Euclidian distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% euclidean distance
disp('EuclideanDistance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=EuclideanDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_geodesic_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a
% pairwise distance (shortest path on a sphere) matrix is computed
% by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% (shortest path on a sphere) matrix between these two data sets is
% computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CGeodesicMetric.html.
%
% Obviously, using the Geodesic distance is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% geodesic metric
disp('GeodesicMetric')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=GeodesicMetric(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_hammingword_modular.m
% This example shows how to compute the Hamming Word Distance for string features.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% hamming word distance
disp('HammingWordDistance')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

use_sign=false;

distance=HammingWordDistance(feats_train, feats_train, use_sign);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_jensen_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance (divergence measure based on the Kullback-Leibler divergence) matrix
% is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% (divergence measure based on the Kullback-Leibler divergence) matrix between
% these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CJensenMetric.html.
%
% Obviously, using the Jensen-Shannon distance/divergence is not limited to
% this showcase example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% jensen metric
disp('JensenMetric')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=JensenMetric(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_manhatten_modular.m
% This example shows how to compute the Manhatten Distance.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% manhattan metric
disp('ManhattanMetric')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=ManhattanMetric(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_manhattenword_modular.m
% This example shows how to compute the Manahattan Distance for string features.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% manhattan word distance
disp('ManhattanWordDistance')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

distance=ManhattanWordDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_minkowski_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) and norm 'k' controls the processing of the given data points,
% where a pairwise distance matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% matrix between these two data sets is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CMinkowskiMetric.html.
%
% Obviously, using the Minkowski metric is not limited to this showcase
% example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% minkowski metric
disp('MinkowskiMetric')

k=3;

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=MinkowskiMetric(feats_train, feats_train, k);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_sparseeuclidian_modular.m
% This example shows how to compute the Eucledian Distance using a sparse encoding.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sparse euclidean distance
disp('SparseEuclideanDistance')

realfeat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(realfeat);
realfeat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(realfeat);

distance=SparseEuclideanDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

• examples/documented/octave_modular/distance_tanimoto_modular.m
% An approach as applied below, which shows the processing of input data
% from a file becomes a crucial factor for writing your own sample applications.
% This approach is just one example of what can be done using the distance
% functions provided by shogun.
%
% First, you need to determine what type your data will be, because this
% will determine the distance function you can use.
%
% This example loads two stored matrices of real values from different
% files and initializes the matrices to 'RealFeatures'.
% Each column of the matrices corresponds to one data point.
%
% The distance initialized by two data sets (the same data set as shown in the
% first call) controls the processing of the given data points, where a pairwise
% distance (extended Jaccard coefficient) matrix is computed by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% The method call 'init'* binds the given data sets, where a pairwise distance
% (extended Jaccard coefficient) matrix between these two data sets is computed
% by 'get_distance_matrix'.
%
% The resulting distance matrix can be reaccessed by 'get_distance_matrix'.
%
% *Note that the previous computed distance matrix can no longer be
% reaccessed by 'get_distance_matrix'.
%
% For more details see doc/classshogun_1_1CTanimotoDistance.html.
%
% Obviously, using the Tanimoto distance/coefficient is not limited to
% this showcase example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% tanimoto distance
disp('TanimotoDistance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

distance=TanimotoDistance(feats_train, feats_train);

dm_train=distance.get_distance_matrix();
distance.init(feats_train, feats_test);
dm_test=distance.get_distance_matrix();

# Distribution

• examples/documented/octave_modular/distribution_histogram_modular.m
% In this example the Histogram algorithm object computes a histogram over all
% 16bit unsigned integers in the features.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');

% Histogram
disp('Histogram')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats=StringWordFeatures(charfeat.get_alphabet());
feats.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats);
feats.add_preprocessor(preproc);
feats.apply_preprocessor();

histo=Histogram(feats);
histo.train();

histo.get_histogram();

num_examples=feats.get_num_vectors();
num_param=histo.get_num_model_parameters();
% for i=0:(num_examples-1),
%	for j=0:(num_param-1),
%		histo.get_log_derivative(j, i);
%	end
% end

histo.get_log_likelihood();
histo.get_log_likelihood_sample();

• examples/documented/octave_modular/distribution_hmm_modular.m
% In this example a hidden markov model with 3 states and 6 transitions is trained
% on a string data set. After calling the constructor of the HMM class specifying
% the number of states and transitions the model is trained. Via the Baum-Welch
% algorithm the optimal transition and emission probabilities are estimated. The
% best path, i.e. the path with highest probability given the model can then be
% calculated using get_best_path_state.

modshogun

addpath('tools');

leng=50;
rep=5;
weight=0.3;

% generate a sequence with characters 1-6 drawn from 3 loaded cubes
for i = 1:3,
a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ];
a{i}= a{i}(randperm(length(a{i})));
end

s=[];
for i = 1:size(a,2),
s= [ s i*ones(1,ceil(rep*rand)) ];
end
s=s(randperm(length(s)));
cubesequence={''};
for i = 1:length(s),
f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)}));
t=randperm(length(a{s(i)}));
r=a{s(i)}(t(1:f(i)));
cubesequence{1}=[cubesequence{1} char(r+'0')];
end

% HMM
disp('HMM')

N=3;
M=6;
pseudo=1e-1;
order=1;
gap=0;
reverse=false;
num_examples=2;
charfeat=StringCharFeatures(CUBE);
charfeat.set_features(cubesequence);
feats=StringWordFeatures(charfeat.get_alphabet());
feats.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats);
feats.add_preprocessor(preproc);
feats.apply_preprocessor();

hmm=HMM(feats, N, M, pseudo);
hmm.train();
% cheating, but enum BaumWelchViterbiType does not seem available
BW_NORMAL=0;
hmm.baum_welch_viterbi_train(BW_NORMAL);

num_examples=feats.get_num_vectors();
num_param=hmm.get_num_model_parameters();
for i=0:(num_examples-1),
for j=0:(num_param-1),
hmm.get_log_derivative(j, i);
end
end

best_path=0;
best_path_state=0;
for i=0:(num_examples-1),
best_path = best_path + hmm.best_path(i);
for j=0:(N-1),
best_path_state = best_path_state + hmm.get_best_path_state(i, j);
end
end

hmm.get_log_likelihood();
hmm.get_log_likelihood_sample();

• examples/documented/octave_modular/distribution_linearhmm_modular.m
% Trains an inhomogeneous Markov chain of order 3 on a DNA string data set. Due to
% the structure of the Markov chain it is very similar to a HMM with just one
% chain of connected hidden states - that is why we termed this linear HMM.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');

leng=50;
rep=5;
weight=0.3;

% generate a sequence with characters 1-6 drawn from 3 loaded cubes
for i = 1:3,
a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ];
a{i}= a{i}(randperm(length(a{i})));
end

s=[];
for i = 1:size(a,2),
s= [ s i*ones(1,ceil(rep*rand)) ];
end
s=s(randperm(length(s)));
cubesequence={''};
for i = 1:length(s),
f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)}));
t=randperm(length(a{s(i)}));
r=a{s(i)}(t(1:f(i)));
cubesequence{1}=[cubesequence{1} char(r+'0')];
end

% Linear HMM
disp('LinearHMM')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats=StringWordFeatures(charfeat.get_alphabet());
feats.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats);
feats.add_preprocessor(preproc);
feats.apply_preprocessor();

hmm=LinearHMM(feats);
hmm.train();

hmm.get_transition_probs();

num_examples=feats.get_num_vectors();
num_param=hmm.get_num_model_parameters();
for i=0:(num_examples-1),
for j=0:(num_param-1),
hmm.get_log_derivative(j, i);
end
end

hmm.get_log_likelihood();
hmm.get_log_likelihood_sample();

# Features

• examples/documented/octave_modular/features_sparse_modular.m
% This example demsonstrates how to encode sparse (most entries zero),
% real-valued features in shogun using SparseRealFeatures.

modshogun
rand('state',0);

disp('Sparse Features')

A=rand(3,5);
A(A<0.7)=0;
full(A)

% sparse representation X of dense matrix A
X = sparse(A)

% create sparse shogun features from dense matrix A
a=SparseRealFeatures(A)
a_out = a.get_full_feature_matrix()

% create sparse shogun features from sparse matrix X
a.set_sparse_feature_matrix(X)
a_out=a.get_full_feature_matrix()

% create sparse shogun features from sparse matrix X
a=SparseRealFeatures(X)
a_out=a.get_full_feature_matrix()

z=a.get_sparse_feature_matrix()

# Kernel

• examples/documented/octave_modular/kernel_auc_modular.m
% This example demonstrates the use of the AUC Kernel, which
% can be used to maximize AUC instead of margin in SVMs.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');

% auc
disp('AUC')

feats_train=RealFeatures(fm_train_real);
width=1.7;
subkernel=GaussianKernel(feats_train, feats_train, width);

kernel=AUCKernel(0, subkernel);
kernel.setup_auc_maximization( BinaryLabels(label_train_twoclass) );
km_train=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_chi2_modular.m
% This is an example for the initialization of the chi2-kernel on real data, where
% each column of the matrices corresponds to one training/test example.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% chi2
disp('Chi2')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=1.4;
size_cache=10;

kernel=Chi2Kernel(feats_train, feats_train, width, size_cache);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_combined_modular.m
% This is an example for the initialization of a combined kernel, which is a weighted sum of
% in this case three kernels on real valued data. The sub-kernel weights are all set to 1.
%

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% combined
disp('Combined')

kernel=CombinedKernel();
feats_train=CombinedFeatures();
feats_test=CombinedFeatures();

subkfeats_train=RealFeatures(fm_train_real);
subkfeats_test=RealFeatures(fm_test_real);
subkernel=GaussianKernel(10, 1.2);
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);

subkfeats_train=StringCharFeatures(DNA);
subkfeats_train.set_features(fm_train_dna);
subkfeats_test=StringCharFeatures(DNA);
subkfeats_test.set_features(fm_test_dna);
degree=3;
subkernel=FixedDegreeStringKernel(10, degree);
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);

subkfeats_train=StringCharFeatures(DNA);
subkfeats_train.set_features(fm_train_dna);
subkfeats_test=StringCharFeatures(DNA);
subkfeats_test.set_features(fm_test_dna);
subkernel=LocalAlignmentStringKernel(10);
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);

kernel.init(feats_train, feats_train);
km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_comm_ulong_string_modular.m
% This is an example for the initialization of the CommUlongString-kernel. This kernel
% sums over k-mere matches (k='order'). For efficient computing a preprocessor is used
% that extracts and sorts all k-mers. If 'use_sign' is set to one each k-mere is counted
% only once.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% comm_ulong_string
disp('CommUlongString')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringUlongFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortUlongString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringUlongFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

use_sign=false;

kernel=CommUlongStringKernel(feats_train, feats_train, use_sign);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_comm_word_string_modular.m
% This is an example for the initialization of the CommWordString-kernel (aka
% Spectrum or n-gram kernel; its name is derived from the unix command comm). This kernel
% sums over k-mere matches (k='order'). For efficient computing a preprocessor is used
% that extracts and sorts all k-mers. If 'use_sign' is set to one each k-mere is counted
% only once.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

order=4;
gap=0;
reverse=false;
use_sign=true;

% comm_word_string
disp('CommWordString')

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

use_sign=false;

kernel=CommWordStringKernel(feats_train, feats_train, use_sign);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_const_modular.m
% The constant kernel gives a trivial kernel matrix with all entries set to the same value
% defined by the argument 'c'.
%

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% const
disp('Const')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
c=23.;

kernel=ConstKernel(feats_train, feats_train, c);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_custom_modular.m
% A user defined custom kernel is assigned in this example, for which only the lower triangle
% may be given (set_triangle_kernel_matrix_from_triangle) or
% a full matrix (set_full_kernel_matrix_from_full), or a full matrix which is then internally stored as a
% triangle (set_triangle_kernel_matrix_from_full). Labels for the examples are given, a svm is trained and
% the svm is used to classify the examples.
%

modshogun;

C=1;
dim=7;

lab=sign(2*rand(1,dim) - 1);
data=rand(dim, dim);
symdata=data+data';

% custom
disp('Custom')

dim=7
data=rand(dim, dim);
symdata=data+data';
%lowertriangle=array([symdata[(x,y)] for x in xrange(symdata.shape[1]);
%	for y in xrange(symdata.shape[0]) if y<=x]);
%
kernel=CustomKernel();

%kernel.set_triangle_kernel_matrix_from_triangle(lowertriangle);
%km_triangletriangle=kernel.get_kernel_matrix();
%
kernel.set_triangle_kernel_matrix_from_full(symdata);
km_fulltriangle=kernel.get_kernel_matrix();
%
kernel.set_full_kernel_matrix_from_full(data);
km_fullfull=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_diag_modular.m
% This is an example for the initialization of the diag-kernel.
% The diag kernel has all kernel matrix entries but those on
% the main diagonal set to zero.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% diag
disp('Diag')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
diag=23.;

kernel=DiagKernel(feats_train, feats_train, diag);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_distance_modular.m
% With the distance kernel one can use any of the following distance metrics:
% BrayCurtisDistance()
% CanberraMetric()
% CanberraWordDistance()
% ChebyshewMetric()
% ChiSquareDistance()
% CosineDistance()
% Distance()
% EuclidianDistance()
% GeodesicMetric()
% HammingWordDistance()
% JensenMetric()
% ManhattanMetric()
% ManhattanWordDistance()
% MinkowskiMetric()
% RealDistance()
% SimpleDistance()
% SparseDistance()
% SparseEuclidianDistance()
% StringDistance()
% TanimotoDistance()
%

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');
label_train_twoclass=load_matrix('../data/label_train_twoclass.dat');

% distance
disp('Distance')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=1.7;
distance=EuclideanDistance();

kernel=DistanceKernel(feats_train, feats_test, width, distance);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_fixed_degree_string_modular.m
% The FixedDegree String kernel takes as input two strings of same size and counts the number of matches of length d.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% fixed_degree_string
disp('FixedDegreeString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
degree=3;

kernel=FixedDegreeStringKernel(feats_train, feats_train, degree);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_gaussian_modular.m
% The well known Gaussian kernel (swiss army knife for SVMs) on dense real valued features.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% gaussian
disp('Gaussian')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=1.9;

kernel=GaussianKernel(feats_train, feats_train, width);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_gaussian_shift_modular.m
% An experimental kernel inspired by the WeightedDegreePositionStringKernel and the Gaussian kernel.
% The idea is to shift the dimensions of the input vectors against eachother. 'shift_step' is the step
% size of the shifts and  max_shift is the maximal shift.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% gaussian_shift
disp('GaussianShift')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
width=1.8;
max_shift=2;
shift_step=1;

kernel=GaussianShiftKernel(
feats_train, feats_train, width, max_shift, shift_step);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_histogramword_modular.m
% The HistogramWordString computes the TOP kernel on inhomogeneous Markov Chains.

modshogun

addpath('tools');
label_train_dna=load_matrix('../data/label_train_dna.dat');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% plugin_estimate
disp('PluginEstimate w/ HistogramWord')

order=3;
gap=0;
reverse=false;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);

pie=PluginEstimate();
labels=BinaryLabels(label_train_dna);
pie.set_labels(labels);
pie.set_features(feats_train);
pie.train();

kernel=HistogramWordStringKernel(feats_train, feats_train, pie);
km_train=kernel.get_kernel_matrix();

kernel.init(feats_train, feats_test);
pie.set_features(feats_test);
pie.apply().get_labels();
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_linear_byte_modular.m
% This is an example for the initialization of a linear kernel on raw byte
% data.

modshogun

addpath('tools');
fm_train_byte=uint8(load_matrix('../data/fm_train_byte.dat'));
fm_test_byte=uint8(load_matrix('../data/fm_test_byte.dat'));

% linear byte
disp('LinearByte')

feats_train=ByteFeatures(RAWBYTE);
feats_train.set_feature_matrix(fm_train_byte);

feats_test=ByteFeatures(RAWBYTE);
feats_test.set_feature_matrix(fm_test_byte);

kernel=LinearKernel(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_linear_modular.m
% This is an example for the initialization of a linear kernel on real valued
% data using scaling factor 1.2.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% linear
disp('Linear')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
scale=1.2;

kernel=LinearKernel();
kernel.set_normalizer(AvgDiagKernelNormalizer(scale));
kernel.init(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_linear_string_modular.m
% This is an example for the initialization of a linear kernel on string data. The
% strings are all of the same length and consist of the characters 'ACGT' corresponding
% to the DNA-alphabet. Each column of the matrices of type char corresponds to
% one training/test example.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% linear_string
disp('LinearString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);

kernel=LinearStringKernel(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_linear_word_modular.m
% This is an example for the initialization of a linear kernel on word (2byte)
% data.

modshogun

addpath('tools');
fm_train_word=uint16(load_matrix('../data/fm_train_word.dat'));
fm_test_word=uint16(load_matrix('../data/fm_test_word.dat'));

% linear_word
disp('LinearWord')

feats_train=WordFeatures(fm_train_word);
feats_test=WordFeatures(fm_test_word);
do_rescale=true;
scale=1.4;

kernel=LinearKernel();
kernel.set_normalizer(AvgDiagKernelNormalizer(scale));
kernel.init(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_local_alignment_string_modular.m
% This is an example for the initialization of the local alignment kernel on
% DNA sequences, where each column of the matrices of type char corresponds to
% one training/test example.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% local_alignment_strin
disp('LocalAlignmentString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);

kernel=LocalAlignmentStringKernel(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_localityimprovedstring_modular.m
% This example initializes the locality improved string kernel. The locality improved string
% kernel is defined on sequences of the same length and inspects letters matching at
% corresponding positions in both sequences. The kernel sums over all matches in windows of
% length l and takes this sum to the power of 'inner_degree'. The sum over all these
% terms along the sequence is taken to the power of 'outer_degree'.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% locality_improved_string
disp('LocalityImprovedString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
l=5;
inner_degree=5;
outer_degree=7;

kernel=LocalityImprovedStringKernel(
feats_train, feats_train, l, inner_degree, outer_degree);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_matchwordstring_modular.m
% The class MatchWordStringKernel computes a variant of the polynomial
% kernel on strings of same length converted to a word alphabet.
%
% It is computed as
%
% k(x,x')= \sum_{i=0}^L I(x_i=x'_i)+c)^d
%
% where I is the indicator function which evaluates to 1 if its argument is
% true and to 0 otherwise.
%
% Note that additional normalisation is applied, i.e.
%     k'(x, x')=\frac{k(x, x')}{\sqrt{k(x, x)k( x', x')}}
%

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% match_word_string
disp('MatchWordString')

order=4;
gap=0;
reverse=false;
degree=3;
scale=1.4;
size_cache=10;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

kernel=MatchWordStringKernel(size_cache, degree);
kernel.set_normalizer(AvgDiagKernelNormalizer(scale));
kernel.init(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_oligo_string_modular.m
% This is an example initializing the oligo string kernel which takes distances
% between matching oligos (k-mers) into account via a gaussian. Variable 'k' defines the length
% of the oligo and variable 'w' the width of the gaussian. The oligo string kernel is
% implemented for the DNA-alphabet 'ACGT'.
%

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% oligo_string
disp('OligoString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
k=3;
width=1.2;
size_cache=10;

kernel=OligoStringKernel(size_cache, k, width);
kernel.init(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_poly_match_word_string.m
% This is an example for the initialization of the PolyMatchString kernel on string data.
% The PolyMatchString kernel sums over the matches of two stings of the same length and
% takes the sum to the power of 'degree'. The strings consist of the characters 'ACGT' corresponding
% to the DNA-alphabet. Each column of the matrices of type char corresponds to
% one training/test example.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

order=3;
gap=0;
reverse=false;

% poly_match_word_string
disp('PolyMatchWordString')

degree=2;
inhomogene=true;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

kernel=PolyMatchWordStringKernel(feats_train, feats_train, degree, inhomogene);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_poly_modular.m
% This example initializes the polynomial kernel with real data.
% If variable 'inhomogene' is 'True' +1 is added to the scalar product
% before taking it to the power of 'degree'. If 'use_normalization' is
% set to 'true' then kernel matrix will be normalized by the square roots
% of the diagonal entries.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% poly
disp('Poly')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
degree=4;
inhomogene=false;
use_normalization=true;

kernel=PolyKernel(
feats_train, feats_train, degree, inhomogene, use_normalization);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_polymatchstring_modular.m
% This is an example for the initialization of the PolyMatchString kernel on string data.
% The PolyMatchString kernel sums over the matches of two stings of the same length and
% takes the sum to the power of 'degree'. The strings consist of the characters 'ACGT' corresponding
% to the DNA-alphabet. Each column of the matrices of type char corresponds to
% one training/test example.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% poly_match_string
disp('PolyMatchString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
degree=3;
inhomogene=false;

kernel=PolyMatchStringKernel(feats_train, feats_train, degree, inhomogene);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_sigmoid_modular.m
% The standard Sigmoid kernel computed on dense real valued features.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sigmoid
disp('Sigmoid')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);
size_cache=10;
gamma=1.2;
coef0=1.3;

kernel=SigmoidKernel(feats_train, feats_train, size_cache, gamma, coef0);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_simple_locality_improved_string_modular.m
% SimpleLocalityImprovedString kernel, is a `simplified' and better performing version of the Locality improved kernel.

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% simple_locality_improved_string
disp('SimpleLocalityImprovedString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
l=5;
inner_degree=5;
outer_degree=7;

kernel=SimpleLocalityImprovedStringKernel(
feats_train, feats_train, l, inner_degree, outer_degree);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_sparsegaussian_modular.m
% The well known Gaussian kernel (swiss army knife for SVMs) on sparse real valued features.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sparse_gaussian - b0rked?
disp('SparseGaussian')

feat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(feat);
feat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(feat);
width=1.1;

kernel=GaussianKernel(feats_train, feats_train, width);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_sparselinear_modular.m
% Computes the standard linear kernel on sparse real valued features.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sparse_linear
disp('SparseLinear')

feat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(feat);
feat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(feat);
scale=1.1;

kernel=LinearKernel();
kernel.set_normalizer(AvgDiagKernelNormalizer(scale));
kernel.init(feats_train, feats_train);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_sparsepoly_modular.m
% Computes the standard polynomial kernel on sparse real valued features.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

% sparse_poly
disp('SparsePoly')

feat=RealFeatures(fm_train_real);
feats_train=SparseRealFeatures();
feats_train.obtain_from_simple(feat);
feat=RealFeatures(fm_test_real);
feats_test=SparseRealFeatures();
feats_test.obtain_from_simple(feat);
size_cache=10;
degree=3;
inhomogene=true;

kernel=PolyKernel(feats_train, feats_train, degree,
inhomogene, size_cache);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_top_modular.m
% The class TOPFeatures implements TOP kernel features obtained from
% two Hidden Markov models.
%
% It was used in
%
% K. Tsuda, M. Kawanabe, G. Raetsch, S. Sonnenburg, and K.R. Mueller. A new
% discriminative kernel from probabilistic models. Neural Computation,
% 14:2397-2414, 2002.
%
% which also has the details.
%
% Note that TOP-features are computed on the fly, so to be effective feature
% caching should be enabled.
%
% It inherits its functionality from CSimpleFeatures, which should be
% consulted for further reference.
%

modshogun

addpath('tools');

leng=28;
rep=5;
weight=0.3;

% generate a sequence with characters 1-6 drawn from 3 loaded cubes
for i = 1:3,
a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ];
a{i}= a{i}(randperm(length(a{i})));
end

s=[];
for i = 1:size(a,2),
s= [ s i*ones(1,ceil(rep*rand)) ];
end
s=s(randperm(length(s)));
cubesequence={''};
for i = 1:length(s),
f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)}));
t=randperm(length(a{s(i)}));
r=a{s(i)}(t(1:f(i)));
cubesequence{1}=[cubesequence{1} char(r+'0')];
end

% top_fisher
disp('TOP/Fisher on PolyKernel')

N=3;
M=6;
pseudo=1e-1;
order=1;
gap=0;
reverse=false;

charfeat=StringCharFeatures(CUBE);
charfeat.set_features(cubesequence);
wordfeats_train=StringWordFeatures(charfeat.get_alphabet());
wordfeats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(wordfeats_train);
wordfeats_train.add_preprocessor(preproc);
wordfeats_train.apply_preprocessor();

charfeat=StringCharFeatures(CUBE);
charfeat.set_features(cubesequence);
wordfeats_test=StringWordFeatures(charfeat.get_alphabet());
wordfeats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
wordfeats_test.add_preprocessor(preproc);
wordfeats_test.apply_preprocessor();

% cheating, BW_NORMAL is somehow not available
BW_NORMAL=0;
pos=HMM(wordfeats_train, N, M, pseudo);
pos.train();
pos.baum_welch_viterbi_train(BW_NORMAL);
neg=HMM(wordfeats_train, N, M, pseudo);
neg.train();
neg.baum_welch_viterbi_train(BW_NORMAL);
pos_clone=HMM(pos);
neg_clone=HMM(neg);
pos_clone.set_observations(wordfeats_test);
neg_clone.set_observations(wordfeats_test);

feats_train=TOPFeatures(10, pos, neg, false, false);
feats_test=TOPFeatures(10, pos_clone, neg_clone, false, false);
kernel=PolyKernel(feats_train, feats_train, 1, false, true);
km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

feats_train=FKFeatures(10, pos, neg);
feats_train.set_opt_a(-1); %estimate prior
feats_test=FKFeatures(10, pos_clone, neg_clone);
feats_test.set_a(feats_train.get_a()); %use prior from training data
kernel=PolyKernel(feats_train, feats_train, 1, false, true);
km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_weighted_comm_word_string_modular.m
% The WeightedCommWordString kernel may be used to compute the weighted
% spectrum kernel (i.e. a spectrum kernel for 1 to K-mers, where each k-mer
% length is weighted by some coefficient \f$\beta_k\f$) from strings that have
% been mapped into unsigned 16bit integers.
%
% These 16bit integers correspond to k-mers. To applicable in this kernel they
% need to be sorted (e.g. via the SortWordString pre-processor).
%
% It basically uses the algorithm in the unix "comm" command (hence the name)
% to compute:
%
% k({\bf x},({\bf x'})= \sum_{k=1}^K\beta_k\Phi_k({\bf x})\cdot \Phi_k({\bf x'})
%
% where \f$\Phi_k\f$ maps a sequence \f${\bf x}\f$ that consists of letters in
% \f$\Sigma\f$ to a feature vector of size \f$|\Sigma|^k\f$. In this feature
% vector each entry denotes how often the k-mer appears in that \f${\bf x}\f$.
%
% Note that this representation is especially tuned to small alphabets
% (like the 2-bit alphabet DNA), for which it enables spectrum kernels
% of order 8.
%
% For this kernel the linadd speedups are quite efficiently implemented using
% direct maps.
%

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% weighted_comm_word_string
disp('WeightedCommWordString')

order=3;
gap=0;
reverse=true;

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_train_dna);
feats_train=StringWordFeatures(charfeat.get_alphabet());
feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse);
preproc=SortWordString();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();

charfeat=StringCharFeatures(DNA);
charfeat.set_features(fm_test_dna);
feats_test=StringWordFeatures(charfeat.get_alphabet());
feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse);
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

use_sign=false;

kernel=WeightedCommWordStringKernel(feats_train, feats_train, use_sign);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_weighted_degree_position_string_modular.m
% The Weighted Degree Position String kernel (Weighted Degree kernel with shifts).
%
% The WD-shift kernel of order d compares two sequences X and
% Y of length L by summing all contributions of k-mer matches of
% lengths k in 1...d, weighted by coefficients beta_k
% allowing for a positional tolerance of up to shift s.
%

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% weighted_degree_position_string
disp('WeightedDegreePositionString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
degree=20;

kernel=WeightedDegreePositionStringKernel(feats_train, feats_train, degree);

%kernel.set_shifts(zeros(len(fm_train_dna[0]), dtype=int));

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/kernel_weighteddegreestring_modular.m
% The Weighted Degree String kernel.
%
% The WD kernel of order d compares two sequences X and
% Y of length L by summing all contributions of k-mer matches of
% lengths k in 1...d , weighted by coefficients beta_k. It
% is defined as
%
%     k(X, Y)=\sum_{k=1}^d\beta_k\sum_{l=1}^{L-k+1}I(u_{k,l}(X)=u_{k,l}(Y)).
%
% Here, $u_{k,l}(X)$ is the string of length k starting at position
% l of the sequence X and I(.) is the indicator function
% which evaluates to 1 when its argument is true and to 0
% otherwise.
%

modshogun

addpath('tools');
fm_train_dna=load_matrix('../data/fm_train_dna.dat');
fm_test_dna=load_matrix('../data/fm_test_dna.dat');

% weighted_degree_string
disp('WeightedDegreeString')

feats_train=StringCharFeatures(DNA);
feats_train.set_features(fm_train_dna);
feats_test=StringCharFeatures(DNA);
feats_test.set_features(fm_test_dna);
degree=20;

kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree);

%weights=arange(1,degree+1,dtype=double)[::-1]/ \
%	sum(arange(1,degree+1,dtype=double));
%kernel.set_wd_weights(weights);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

# Metric

• examples/documented/octave_modular/metric_lmnn_modular.m
modshogun;

label_train_fname='../data/label_train_multiclass.dat';
train_fname='../data/fm_train_real.dat';
test_fname='../data/fm_test_real.dat';

% wrap data into Shogun objects
feats_train=RealFeatures(CSVFile(train_fname));
feats_test=RealFeatures(CSVFile(test_fname));
labels_train=MulticlassLabels(CSVFile(label_train_fname));

% distance learning with LMNN
disp('LMNN')

% number of target neighbours per example
k=3;
lmnn=LMNN(feats_train,labels_train,k);
init_transform=eye(feats_train.get_num_features());
lmnn.train(init_transform);

% retrieve distance learnt by LMNN
lmnn_distance=lmnn.get_distance();

% perform multiclass classification using KNN with the distance learnt
knn=KNN(k,lmnn_distance,labels_train);
knn.train();
output=knn.apply(feats_test).get_labels();

# Mkl

• examples/documented/octave_modular/mkl_multiclass_modular.m
% In this example we show how to perform Multiple Kernel Learning (MKL)
% with the modular interface for multi-class classification.
% First, we create a number of base kernels and features.
% These kernels can capture different views of the same features, or actually
% consider entirely different features associated with the same example
% (e.g. DNA sequences = strings AND gene expression data = real values of the same tissue sample).
% The base kernels are then subsequently added to a CombinedKernel, which
% contains a weight for each kernel and encapsulates the base kernels
% from the training procedure. When the CombinedKernel between two examples is
% evaluated it computes the corresponding linear combination of kernels according to their weights.
% We then show how to create an MKLMultiClass classifier that trains an SVM and learns the optimal
% weighting of kernels (w.r.t. a given norm q) at the same time. The main difference to the binary
% classification version of MKL is that we can use more than two values as labels, when training
% the classifier.
% Finally, the example shows how to classify with a trained MKLMultiClass classifier.
%

modshogun

addpath('tools');
label_train_multiclass=load_matrix('../data/label_train_multiclass.dat');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

disp('Octave_modular')
% combined
disp('Combined')

kernel=CombinedKernel();
feats_train=CombinedFeatures();
feats_test=CombinedFeatures();

subkfeats_train=RealFeatures(fm_train_real);
subkfeats_test=RealFeatures(fm_test_real);
subkernel=GaussianKernel(10, 1.2);
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);

subkfeats_train=RealFeatures(fm_train_real);
subkfeats_test=RealFeatures(fm_test_real);
subkernel=LinearKernel();
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);

subkfeats_train=RealFeatures(fm_train_real);
subkfeats_test=RealFeatures(fm_test_real);
subkernel=PolyKernel(10,2);
feats_train.append_feature_obj(subkfeats_train);
feats_test.append_feature_obj(subkfeats_test);
kernel.append_kernel(subkernel);
kernel.init(feats_train, feats_train);

C=1.2;
epsilon=1e-5;
num_threads=1;
labels=MulticlassLabels(label_train_multiclass);

% MKL_MULTICLASS
disp('MKL_MULTICLASS')
mkl=MKLMulticlass(C, kernel, labels);
mkl.set_epsilon(epsilon);
mkl.parallel.set_num_threads(num_threads);
mkl.set_mkl_epsilon(0.001);
mkl.set_mkl_norm(1.5);
mkl.train();

kernel.init(feats_train, feats_test);
result=mkl.apply().get_labels();
result

# Preprocessor

• examples/documented/octave_modular/preprocessor_logplusone_modular.m
% In this example a kernel matrix is computed for a given real-valued data set.
% The kernel used is the Chi2 kernel which operates on real-valued vectors. It
% computes the chi-squared distance between sets of histograms. It is a very
% useful distance in image recognition (used to detect objects). The preprocessor
% LogPlusOne adds one to a dense real-valued vector and takes the logarithm of
% each component of it. It is most useful in situations where the inputs are
% counts: When one compares differences of small counts any difference may matter
% a lot, while small differences in large counts don't. This is what this log
% transformation controls for.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

%LogPlusOne
disp('LogPlusOne')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

preproc=LogPlusOne();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

width=1.4;
size_cache=10;

kernel=Chi2Kernel(feats_train, feats_train, width, size_cache);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/preprocessor_normone_modular.m
% In this example a kernel matrix is computed for a given real-valued data set.
% The kernel used is the Chi2 kernel which operates on real-valued vectors. It
% computes the chi-squared distance between sets of histograms. It is a very
% useful distance in image recognition (used to detect objects). The preprocessor
% NormOne, normalizes vectors to have norm 1.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

%NormOne
disp('NormOne')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

preproc=NormOne();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

width=1.4;
size_cache=10;

kernel=Chi2Kernel(feats_train, feats_train, width, size_cache);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

• examples/documented/octave_modular/preprocessor_prunevarsubmean_modular.m
% In this example a kernel matrix is computed for a given real-valued data set.
% The kernel used is the Chi2 kernel which operates on real-valued vectors. It
% computes the chi-squared distance between sets of histograms. It is a very
% useful distance in image recognition (used to detect objects). The preprocessor
% PruneVarSubMean substracts the mean from each feature and removes features that
% have zero variance.

modshogun

addpath('tools');
fm_train_real=load_matrix('../data/fm_train_real.dat');
fm_test_real=load_matrix('../data/fm_test_real.dat');

%PruneVarSubMean
disp('PruneVarSubMean')

feats_train=RealFeatures(fm_train_real);
feats_test=RealFeatures(fm_test_real);

preproc=PruneVarSubMean();
preproc.init(feats_train);
feats_train.add_preprocessor(preproc);
feats_train.apply_preprocessor();
feats_test.add_preprocessor(preproc);
feats_test.apply_preprocessor();

width=1.4;
size_cache=10;

kernel=Chi2Kernel(feats_train, feats_train, width, size_cache);

km_train=kernel.get_kernel_matrix();
kernel.init(feats_train, feats_test);
km_test=kernel.get_kernel_matrix();

# Regression

• examples/documented/octave_modular/regression_kernel_ridge_modular.m
modshogun

addpath('tools');
label_train=load_matrix('../data/label_train_twoclass.dat');
fm_train=load_matrix('../data/fm_train_real.dat');
fm_test=load_matrix('../data/fm_test_real.dat');

% kernel ridge regression
disp('KRR')

feats_train=RealFeatures(fm_train);
feats_test=RealFeatures(fm_test);
width=0.8;
kernel=GaussianKernel(feats_train, feats_train, width);

C=0.9;
tau=1e-6;
num_threads=1;
labels=RegressionLabels(label_train);

krr=KernelRidgeRegression(tau, kernel, labels);
krr.parallel.set_num_threads(num_threads);
krr.train();

kernel.init(feats_train, feats_test);
out=krr.apply().get_labels();

• examples/documented/octave_modular/regression_libsvr_modular.m
% In this example a support vector regression algorithm is trained on a
% real-valued toy data set. The underlying library used for the SVR training is
% LIBSVM. Both the epsilon-SVR and the nu-SVR formulation are used.
% Data is a noisy sine wave on which a Gaussian kernel is used.
%
% For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/ .

modshogun

addpath('tools');
label_train=load_matrix('../data/label_train_twoclass.dat');
fm_train=load_matrix('../data/fm_train_real.dat');
fm_test=load_matrix('../data/fm_test_real.dat');

%% libsvm based support vector regression
disp('LibSVR')

feats_train=RealFeatures(fm_train);
feats_test=RealFeatures(fm_test);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
tube_epsilon=1e-2;
num_threads=3;
labels=RegressionLabels(label_train);

svr=LibSVR(C, epsilon, kernel, labels);
svr.set_tube_epsilon(tube_epsilon);
svr.parallel.set_num_threads(num_threads);
svr.train();

kernel.init(feats_train, feats_test);
out=svr.apply().get_labels();

• examples/documented/octave_modular/regression_svrlight_modular.m
% In this example a support vector regression algorithm is trained on a
% real-valued toy data set. The underlying library used for the SVR training is
% SVM^light. The SVR is trained with regularization parameter C=1 and a gaussian
% kernel with width=2.1. The the label of both the train and the test data are
% fetched via svr.classify().get_labels().
%
% For more details on the SVM^light see
%  T. Joachims. Making large-scale SVM learning practical. In Advances in Kernel
%  Methods -- Support Vector Learning, pages 169-184. MIT Press, Cambridge, MA USA, 1999.

modshogun

addpath('tools');
label_train=load_matrix('../data/label_train_twoclass.dat');
fm_train=load_matrix('../data/fm_train_real.dat');
fm_test=load_matrix('../data/fm_test_real.dat');

% svmlight based support vector regression
try
disp('SVRLight')

feats_train=RealFeatures(fm_train);
feats_test=RealFeatures(fm_test);
width=2.1;
kernel=GaussianKernel(feats_train, feats_train, width);

C=1.2;
epsilon=1e-5;
tube_epsilon=1e-2;
num_threads=3;
labels=RegressioLabels(label_train);

svr=SVRLight(C, epsilon, kernel, labels);
svr.set_tube_epsilon(tube_epsilon);
svr.parallel.set_num_threads(num_threads);
svr.train();

kernel.init(feats_train, feats_test);
svr.apply().get_labels();
catch
disp('No support for SVRLight available.')
end

SHOGUN Machine Learning Toolbox - Documentation