Open in new window / Try shogun cloud
--- Log opened Tue Aug 21 00:00:17 2012
@sonney2kyoh_, alright. uploaded shogun 1.1.0-600:53
* sonney2k starts an upgrade to wheezy00:54
shogun-buildbot_build #330 of deb2 - static_interfaces is complete: Failure [failed test octave_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/330  blamelist: Soeren Sonnenburg <sonne@debian.org>00:59
yoh_sonney2k: haven't consciously used shogun for a while :-/01:33
yoh_btw -- remembering our discussion awhile ago -- have you come up with a built-in breakage of the ties for multi-class problems relying on voting of pair-wise classifiers (e.g. how it was in SVMs)01:34
yoh_?01:34
shogun-buildbot_build #420 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/420  blamelist: Soeren Sonnenburg <sonne@debian.org>01:34
shogun-buildbot_build #331 of deb2 - static_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/33101:40
@sonney2kyoh_, actually the whole multiclass code got rewritten this summer by pluskid (not online right now...) - we have tons of MC methods now and I simply don't know - ask on the mailinglist he will know01:46
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has joined #shogun02:18
shogun-buildbot_build #421 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/42102:18
shogun-buildbot_build #61 of nightly_none is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_none/builds/6103:01
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has quit [Ping timeout: 244 seconds]03:43
shogun-buildbot_build #72 of nightly_default is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/7203:44
-!- emrecelikten [~emre@trir-5d800761.pool.mediaWays.net] has quit [Ping timeout: 244 seconds]04:26
-!- emrecelikten [~emre@trir-4d0d941d.pool.mediaWays.net] has joined #shogun04:41
-!- zxtx [~zv@c-24-6-91-131.hsd1.ca.comcast.net] has quit [Ping timeout: 244 seconds]05:24
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun07:27
-!- uricamic [~uricamic@2001:718:2:1634:e560:dc03:82b4:748f] has joined #shogun07:38
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Read error: Operation timed out]09:09
-!- gsomix_ [~gsomix@80.234.27.140] has joined #shogun09:10
-!- gsomix [~gsomix@109.169.157.174] has quit [Read error: Connection reset by peer]09:10
-!- gsomix_ [~gsomix@80.234.27.140] has quit [Quit: Ex-Chat]09:38
CIA-52shogun: Sergey Lisitsyn master * r3a54e3f / src/shogun/classifier/FeatureBlockLogisticRegression.h : Updated doc of FeatureBlockLogisticRegression - http://git.io/KlGRsw09:57
-!- pluskid [72fa5039@gateway/web/freenode/ip.114.250.80.57] has joined #shogun09:58
-!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has joined #shogun10:19
yoohi alll10:43
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun10:48
-!- pluskid [72fa5039@gateway/web/freenode/ip.114.250.80.57] has quit [Quit: Page closed]11:18
CIA-52shogun: Sergey Lisitsyn master * r64f1532 / src/shogun/features/SubsetStack.h : Added get_last_subset doc - http://git.io/Zo3oEg11:54
_____________oh CIA-52 you are back11:56
_____________n4nd0: hey how is it going?11:59
CIA-52shogun: Sergey Lisitsyn master * rf9ca4a4 / src/shogun/transfer/multitask/MultitaskLeastSquaresRegression.h : Added doc for MultitaskLeastSquaresRegression - http://git.io/V3Yehw12:03
CIA-52shogun: Sergey Lisitsyn master * r2a89ec1 / src/shogun/structure/StateModel.h : Removed Math class reference in doc to avoid warnings - http://git.io/gI08Tw12:08
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 245 seconds]12:22
@sonney2k_____________, do you know if this is caused by the - sign in front of CMath? https://github.com/shogun-toolbox/shogun/commit/2a89ec12acb3c5946ec5c726ad3794bebba3e1d612:31
* wiking is talking on the phone german... after 4 years of not speaking at all :)))12:35
wikingpure fun that is :D12:35
_____________sonney2k: still warning??13:08
_____________wiking: ich spreche etwas deutsch! :D13:09
@sonney2k_____________, no but I was wondering if it is possible to keep CMath::INFTY there if one just moves the - sign13:12
_____________sonney2k: I had no idea so removed that13:13
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun13:13
-!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has quit [Quit: Page closed]13:20
CIA-52shogun: Sergey Lisitsyn master * r8d0d4e2 / src/shogun/lib/IndexBlockGroup.h : Updated doc of IndexBlockGroup - http://git.io/gmU9tg14:37
CIA-52shogun: Sergey Lisitsyn master * rd5c67f9 / src/shogun/latent/LatentSOSVM.h : Updated doc of LatentSOSVM - http://git.io/XxPDvA14:40
CIA-52shogun: Sergey Lisitsyn master * r4ec5ff5 / src/shogun/latent/LatentSOSVM.h : Updated LatentSOSVM apply doc causing warning - http://git.io/WSZk7g15:08
-!- pluskid [72f6b389@gateway/web/freenode/ip.114.246.179.137] has joined #shogun15:18
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun15:47
n4nd0hey _____________15:48
_____________nice nickname I have isn't it?15:49
_____________:)15:49
_____________n4nd0: what's up?15:58
n4nd0yeah really cool nickname15:59
n4nd0_____________: not much, I just came from university15:59
n4nd0_____________: you at job?15:59
_____________yeah15:59
_____________are your studies started already?16:00
n4nd0no, not yet16:00
n4nd0it was a meeting for first year students16:00
n4nd0I went there to give tips and stuff like that16:00
_____________did you give a tip about shoguning?16:02
_____________and vodka of course16:03
n4nd0haha16:04
n4nd0I just told my professor about my summer in SHOGUN16:04
_____________what does he think about it? :)16:04
n4nd0aah he just said, oh coding summer ;)16:05
-!- emrecelikten [~emre@trir-4d0d941d.pool.mediaWays.net] has quit [Ping timeout: 244 seconds]16:18
_____________n4nd0: do you expect you manage to put HM-SVM based on BMRM before release?16:38
n4nd0_____________: I don't think so since now I am focusing more on the ASR application16:39
_____________I see16:39
n4nd0_____________: do you think it is important to have it for now?16:39
_____________well I don't know16:39
n4nd0aham ok16:40
_____________I'd ask you to check mosek implementation for bugs16:40
_____________have you any test you used before?16:40
n4nd0I've tested and compared results with the hm-svm toolbox (the one written by Gunnar and Georg)16:41
_____________can there be any regression due to last changes?16:41
n4nd0what do you mean with regression?16:41
_____________in soft engineering mean :)16:42
n4nd0still :)16:42
n4nd0what do you mean? hehe16:42
_____________bug introduced by some change in other code16:42
n4nd0like to come back?16:42
n4nd0mmm I hope not, at least I believe it isnt'16:43
_____________okay16:43
n4nd0at least the hm-svm application returns exactly the same results as the toolbox16:43
_____________I think we will manage to release in healthy state at 1st of september16:43
n4nd0the only thing I think I am going to add before release is the PLiF support16:44
n4nd0this is already in a branch of my fork16:44
-!- alexlovesdata [~binder@194.95.174.230] has joined #shogun16:46
-!- vojtech [9320543b@gateway/web/freenode/ip.147.32.84.59] has joined #shogun16:47
CIA-52shogun: Chiyuan Zhang master * r2590cf5 / (8 files in 4 dirs): added doc to disable warning. - http://git.io/ZhnLIQ16:48
CIA-52shogun: Sergey Lisitsyn master * r4cbdfa4 / (8 files in 4 dirs): Merge pull request #750 from pluskid/multiclass - http://git.io/KVKzpQ16:48
-!- pluskid [72f6b389@gateway/web/freenode/ip.114.246.179.137] has quit [Quit: Page closed]16:49
wikingalexlovesdata: hey hey16:50
-!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has joined #shogun16:53
-!- vojtech [9320543b@gateway/web/freenode/ip.147.32.84.59] has quit [Quit: Page closed]17:13
-!- uricamic [~uricamic@2001:718:2:1634:e560:dc03:82b4:748f] has quit [Quit: Leaving.]17:22
-!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has quit [Quit: Page closed]17:44
n4nd0see you later guys18:24
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving]18:25
-!- yoh_ is now known as yoh18:25
-!- blackburn [~blackburn@83.234.54.182] has joined #shogun18:48
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun19:37
n4nd0hey guys, one question about the final evaluation: is there a part to submit by the mentors too?19:39
blackburnooooohh nice you mentioned that i totally forgot19:42
blackburnyes mentors do submit it too19:42
@sonney2kyoh, https://buildd.debian.org/status/package.php?p=shogun looks good - powerpc still has the clang error but otherwise all good19:42
@sonney2kyoh, so now where do you think shall I ask?19:43
@sonney2kn4nd0, I can tell that it is important to have BMRM based SO stuff... otherwise we won't have many users...19:44
n4nd0sonney2k: ok, I understand19:44
blackburnn4nd0: if you want I could help you somehow with it19:45
n4nd0blackburn: sure19:45
@sonney2kn4nd0, it is like one is invited to that *great* party but one needs a car to get there (or boat if you want :)19:46
blackburnn4nd0: for example I could introduce a risk for your HM model19:46
blackburnand you could test it19:46
blackburnI mean compare with that G&G :D implementation19:46
n4nd0blackburn: I think that the generic risk function in the StructuredModel should work19:46
n4nd0blackburn: otherwise, what risk would you like to introduce in the HM model?19:47
blackburnwell the same but not generic19:47
blackburnI could add generic if you want :)19:49
blackburnand then you could test19:49
blackburnis that better for you?19:50
n4nd0ok19:50
blackburnsonney2k: hey we have a shogun party on mars19:50
blackburnbut no curiosities left though19:51
blackburnso you have to swim19:51
-!- octopine is now known as audy19:59
blackburnn4nd0: did you ever eat cat food?20:14
blackburnI did but nobody else did :D20:15
blackburndidn't*20:15
blackburnsonney2k: may be you? :D20:15
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 244 seconds]20:16
-!- alexlovesdata [~binder@194.95.174.230] has left #shogun []20:34
yohsonney2k: well -- #debian-release or whatever it is ... ?  or the mailing list... or 'reportbug release.debian.org' although that one probably would be stale for a while and IRC or ML might be a better choice for the discussion20:46
-!- blackburn1 [~blackburn@188.168.13.81] has joined #shogun21:13
blackburn1sonney2k: any idea what is wrong wtih gp regression and apply_regression extend ?21:14
-!- blackburn [~blackburn@83.234.54.182] has quit [Ping timeout: 240 seconds]21:14
CIA-52shogun: Sergey Lisitsyn master * r6d6770b / (2 files): Removal for last multiclass warnings - http://git.io/aFKt-Q21:17
blackburn1n4nd0 hmm multiclass risk is more efficient as it is now21:38
blackburn1it uses no compute joint feature vector21:39
-!- yoo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has joined #shogun21:55
yoohi all21:55
blackburn1yoo: hey21:56
yooblackburn1: hey! how is it going ?21:56
blackburn1pretty busy last days because of getting back to job I had before gsoc :)21:56
blackburn1lets do that multiclass output stuff you wanted tonight21:58
yoonice21:58
yooI am figuring out stuff with yaml format21:58
yoosince today I ahve only work with shogun c++21:58
blackburn1so you switched to C++?21:58
yoobut plotting is a pretty hard task, using boost python ..21:59
yooI try to switch to python modular21:59
blackburn1okay - what makes it hard to use in python?21:59
yoosorry, you didnt understand, I usually use c++ but now I try python22:00
blackburn1oh :D22:00
blackburn1I mix things now because quite a few users use python or C++22:00
yoowhat did people use usually ?22:00
blackburn1I use python mainly22:01
blackburn1but recently I've used C++ for problem where opencv is required22:01
blackburn1opencv's python interface is crappy22:01
yooI am from image processing then I have opencv routines as well22:01
yooyep it is !22:01
yoothats why I am trying to export xml or yml (opencv) format to python22:02
blackburn1recently I switched to vlfeat because it has quite efficient and clear phow (dense sift aka hog) implementation22:02
blackburn1opencv's hog is something like WTF22:02
yooI have coded my own HOG last year22:03
blackburn1it is much easier to try to convert your problem to pedestrian detection than extract HOG features from it :D22:03
yooopencv's hog uses some weird buffer etc ..22:03
yooI/O in opencv is crappy for everythg except c++ and I must admit that the opencv community is not as "kind" as shogun one :p22:05
blackburn1yoo: there is http://www.vlfeat.org/api/dsift.html I like most22:05
blackburn1I never talked to any of them - what is wrong?22:05
yoomaybe that because opencv community is very big ^^22:05
blackburn1yeah we have only a few developers22:05
yooit seems that the bindings for python are not very well update22:06
yooFile storage is very messy for python22:06
blackburn1why?22:06
yoodo you know YML format ?22:10
blackburn1no22:10
blackburn1what is it for?22:10
yooit permits to store data like matrix (data, size, type) and sequences .. in a hash talbe style22:10
yooformat22:11
yoovery efficient to parse22:11
yooBUT python bindings are not up-to-date then ndoes are not well recognize etc ..22:12
blackburn1I see22:12
blackburn1sonney2k: okay if everything is correct HM SVM should work with BMRM now22:13
blackburn1I will commit in a min22:13
yooBMRM ?22:13
blackburn1bundle method for risk minimization22:13
blackburn1currently HM works only with mosek22:13
yooyes I have seen that22:14
yooI try to keep an eye on whats new here every weeks :p22:14
blackburn1things will stop now unfortunately22:14
blackburn1however I am thinking about CRF and other stuff like that research so may be I will extend it22:15
yooyet, YML format is a right way to store data for large scale machine learning.22:16
yooI was quite surprise that shogun dont have is own i/O format22:16
yoofor now it is only libsvm style right ?22:17
blackburn1for serialization? we have quite a few22:17
blackburn1json, xml, pure ascii22:17
blackburn1hdf522:18
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun22:19
yoo<blackburn1> things will stop now unfortunately : what do you mean ? End of gsoc ?22:19
blackburn1yeah exactly22:19
blackburn1n4nd0: I am done with generic risk probably - will commit in a minute22:20
n4nd0blackburn1: I have just seen logs about the BMRM22:20
n4nd0blackburn1: nice!22:20
blackburn1waiting it to finish compilation22:20
blackburn1resultset you added is pretty useful22:20
blackburn1implementation is really straightforward with it - I thought of something more difficult22:21
n4nd0blackburn1: well after that conversation we had, it seemed it was going to be something easy22:21
blackburn1lets check if everything is correct22:21
blackburn1in a minute more22:21
blackburn1:D22:21
n4nd0maybe we have misunderstood something22:22
n4nd0blackburn1: let me know when you commit please22:23
blackburn1I am pretty sure we didn't do any mistake out there22:25
blackburn1at least it fits with multiclass things just perfectly22:25
n4nd0yeah22:25
blackburn1n4nd0: generic risk in multiclass is really bad22:26
blackburn1it would make very sparse vectors go all around22:26
blackburn1so specific should stay22:26
n4nd0blackburn1: what do you mean with very sparse vectors go all around?22:27
n4nd0what is the difference between making it generic and specific?22:27
blackburn1n4nd0: okay imagine we have 5000 dimensional vectors and 50 classes22:28
blackburn1what is dim of joint feature vector22:28
blackburn1and how many zeros out there?22:28
blackburn1current multiclass risk do not allocate anything22:28
blackburn1generic will allocate two 250000 dimensional vectors for each feature vector22:29
blackburn1and will add that to subgrad22:29
n4nd0I understand what you mean22:30
n4nd0although using sparse data structures that shouldn't be a problem, right?22:31
blackburn1yeah but currently we are not using that22:32
n4nd0ok22:32
n4nd0do you want to carry out the test for HM-SVM in any case?22:33
n4nd0using the StructuredAccuracy class it shouldn't take much to get the feeling it the idea is correct at least22:33
n4nd0just as proof of concept22:33
blackburn1so you want me to test it too? :D22:35
blackburn1I don't mind but I don't know how22:35
n4nd0I can test it22:35
CIA-52shogun: Sergey Lisitsyn master * r530155d / src/shogun/structure/StructuredModel.cpp : Added generic risk function implementation for structured model class - http://git.io/96jvMg22:35
n4nd0I was not saying it like "do this"22:35
blackburn1do we have an example?22:35
n4nd0now that is commited I can make it ;)22:35
blackburn1oh we need to add an example I guess22:36
blackburn1hmm22:36
blackburn1is HMSVMModel guarded for mosek?22:36
blackburn1oh that's cool22:37
blackburn1nothing to change22:37
n4nd0if a bundle method is used, of course22:37
blackburn1n4nd0: I will add an example right now22:37
n4nd0blackburn1: I am trying to work out one too :)22:37
blackburn1n4nd0: what is accuracy in that example you did add to python modular?22:38
n4nd099%22:38
n4nd0do we have BMRM in modular interfaces?22:38
blackburn1http://www.reactionface.info/sites/default/files/images/YvEN9.png22:39
blackburn1yes22:39
n4nd0haha22:40
blackburn1ohhh22:41
blackburn1matrix features issue22:41
blackburn1michal's svm wants to have dot features22:41
n4nd0game over22:41
blackburn1nope22:42
n4nd0any good guess for lambda?22:42
blackburn1I don't have any22:42
n4nd0let's change DotFeatures -> Features?22:42
blackburn1yeah seems that is required22:43
n4nd0yes22:44
n4nd0in the DualLibQPBMSOSVM it seems that it doesn't affect much22:44
n4nd0btw, CDualLibQPBMSOSVM::train_machine looks weird to me22:46
blackburn1n4nd0: problem is that dlqpbmsosvm is linear SO machine22:46
n4nd0blackburn1: what problem do you find with it?22:46
blackburn1n4nd0: why?22:46
blackburn1ahh I though linear SO machine needs dot features22:47
blackburn1why do we need to have features in svm at all?22:47
blackburn1if we have model that have features?22:47
n4nd0I was wondering that right now too22:48
n4nd0and this is actually a decision I made...22:48
n4nd0I don't think we need them there too22:50
n4nd0it is like the labels22:51
n4nd0probably they shouldn't be given in the constructor of SOMachine22:52
n4nd0but just take the ones that are in the model22:52
n4nd0blackburn1: around22:53
n4nd0?22:53
blackburn1sorry got interrupted by email22:53
blackburn1okayy22:53
blackburn1I don't know to be sure22:53
blackburn1to be honest :D22:53
n4nd0hehe22:54
blackburn1n4nd0: we could remove it completely - they are not needed for training22:54
n4nd0I agree22:54
n4nd0blackburn1: do you want to do the change or do you want me to do it?22:54
blackburn1I will do22:54
n4nd0ok22:55
blackburn1one thing is unclear here22:55
blackburn1n4nd0: take a look at22:55
blackburn1apply_structured22:55
blackburn1in linear so machine22:56
blackburn1n4nd0: we have to set features for model, rgiht?22:56
blackburn1currently it looks like something wrong for me22:57
n4nd0set_features sets the ones for the model too22:57
blackburn1ah got it22:57
n4nd0blackburn1: is sonney2k still around?22:58
blackburn1n4nd0: okay I think best way is to emulate it has m_features22:58
blackburn1but handle model's features22:58
n4nd0blackburn1: ok22:59
blackburn1good for you?22:59
n4nd0blackburn1: so you are just planning to remove them from the constructor?22:59
n4nd0the features param I mean22:59
blackburn1from class22:59
n4nd0the member too?22:59
blackburn1??23:00
blackburn1oops23:00
blackburn1yes I mean23:00
n4nd0ok23:01
n4nd0so get_features and set_features are still there23:01
n4nd0just calling the ones of the model23:01
n4nd0is that what you meant with emulate?23:01
-!- blackburn [~blackburn@37.61.181.133] has joined #shogun23:02
blackburnhere23:02
blackburnokay23:04
blackburnn4nd0: so we don't need features in dual lib qp bm so svm too, right?23:04
n4nd0blackburn: if they go out of the LinearSOMachine, they go out of this one too23:04
n4nd0dual lib qp bm so svm gets it by inheritance23:04
blackburnyes23:05
yoore23:05
-!- blackburn1 [~blackburn@188.168.13.81] has quit [Ping timeout: 244 seconds]23:05
blackburnokay lets see if that works23:05
n4nd0blackburn: how does it look like?23:09
blackburncompiled lets check23:10
blackburnI need to adjust examples now23:10
blackburnuh23:10
blackburnhm svm objective grows exponentially :D23:10
n4nd0wow, really?23:11
n4nd0I am not sure if that is good23:11
blackburnyeah I think it is caused by some error23:11
n4nd0blackburn: let me know if I can help with something23:13
blackburnn4nd0: you could check a risk function I added23:15
blackburnare signs of psi_pred and psi_truth I have added correct?23:16
n4nd0http://git.io/96jvMg?23:16
blackburnyes23:16
blackburnPsi(x,hat y) - Psi(x,y), right?23:17
n4nd0yeah, I think so23:18
n4nd0blackburn: I think there might be a problem with the definition though23:20
blackburnn4nd0: definition of?23:21
n4nd0http://pastebin.com/PUfyBp9P23:21
n4nd0the risk function23:21
blackburnahhhhh23:22
n4nd0it is the max of that thing there23:22
blackburnI forgot delta23:22
n4nd0but are you sure that using the argmax is correct?23:22
blackburnyes, pretty sure23:22
blackburnI forgot delta - that is the bug for sure23:23
n4nd0shouldn't it be checked for every y which one is it that maximizes l + <w, psi(x,y) - psi(x,y)>23:23
n4nd0blackburn: mmmm why are you so pretty sure?23:23
n4nd0I mean23:23
blackburnI can't see nothing wrong here23:23
n4nd0the argmax will tend to minimize the first term23:24
n4nd0Delta23:24
n4nd0while maximize the second23:24
n4nd0right?23:24
blackburnI think I got what you mean23:25
blackburnso you mean we minimize only <w,predict-truth>23:25
blackburnnot delta?23:25
n4nd0look that in his code Michal is actually maximizing the whole thing23:25
n4nd0what I mean is that23:25
n4nd0what you did for the risk function assumes23:26
n4nd0that max_y [Delta(yi, y) +  <w, Psi(xi, y) - Psi(xi,yi)>]23:26
n4nd0is always given by the argmax23:26
n4nd0and I think that is not true23:27
blackburnare you going to say delta changes the game?23:27
n4nd0blackburn: do you understand what I mean?23:27
n4nd0yes, it is Delta what changes it :)23:28
blackburnI have no proof for that but it is true for multiclass and I induce it for other so stuff23:28
n4nd0blackburn: what is true for multiclass?23:29
n4nd0it is not true that the max of the equation23:29
n4nd0\max_{y \in \mathcal{Y}} \left[ \ell(y_i, y) + \langle {\bf w}, \Psi(x_i, y) - \Psi(x_i, y_i)  \rangle  \right]23:30
blackburnargmax_y [ <w, Psi(xi,y) - Psi(xi,yi) > ] = argmax_y [ Delta(yi,y) + <w, Psi(xi,y) - Psi(xi,yi) > ]23:30
n4nd0that is not true23:30
n4nd0because the term Delta(yi,y) is going to be minimized by the argmax_y23:31
blackburnno it is maximized23:31
n4nd0mmm why?23:31
wikingmmm woah i need to read this conversation :D23:31
n4nd0wiking: haha23:32
blackburnbecause you maximize difference between dot products23:32
wikingbtw: anybody knows why tar -t <tarfile> doesn't work :)23:32
blackburnyou make them farther from each other23:32
blackburnand loss grows23:32
blackburnsomething like that in my mind23:32
n4nd0blackburn: look just at the term delta23:32
n4nd0Delta(yi,yi) = 023:32
n4nd0right?23:32
blackburnyes23:32
n4nd0ok, now Delta(yi,y)23:32
n4nd0the closer y is to yi -> the smaller Delta(yi,y) is23:33
n4nd0ok?23:33
blackburnyes23:33
n4nd0when you have a good model23:33
n4nd0i.e. a good w23:33
n4nd0you will get good predictions23:34
blackburnyes23:34
-!- yoo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has quit [Ping timeout: 245 seconds]23:34
n4nd0i.e argmax_y for the example xi will be close to yi23:34
blackburnyes23:34
n4nd0then Delta(yi,y) is minimized for y = argmax23:34
n4nd0?23:34
blackburnno, it is maximized by argmax of <w,Psi(x_i,y_i)-Psi(x_i,y)>23:35
n4nd0wait23:35
n4nd0we are not talking about that term now23:35
blackburnDelta is max then y_i is far away from y23:35
n4nd0we are just seeing how Delta(yi,y) behaves23:35
n4nd0exactly23:36
blackburnI see no problem out there23:36
n4nd0the contrary happens for the other term23:36
blackburnthe second term is zero then y_i is close to y23:36
blackburnand vice versa if y_i is far away from y23:37
@sonney2kwiking, tar -tf23:37
n4nd0blackburn: I am writing it down and thinking of it :D23:38
blackburnokay I will try to find true bmrm minimizer meanwhile23:38
blackburnokay unstoppable grow of objective was caused by too small lambda23:40
blackburnthat happened to me with other method so I guess that is ok23:40
n4nd0blackburn: ok23:40
blackburnno I get reasonable w23:40
blackburnbut I broke apply probably23:40
blackburnoh yes I did23:41
blackburnsonney2k: captain we are fixing SO!23:41
blackburnoh segfault now cool23:42
blackburnokay fixed23:43
n4nd0blackburn: look at this23:43
n4nd0blackburn: in your reasoning23:43
blackburnAccuracy = 0.994823:43
n4nd0blackburn: you say that when y_i is far away from y23:43
blackburnn4nd0: my reasoning works :D23:43
n4nd0blackburn: hahaha23:43
n4nd0blackburn: c'mon ... it cannot be true!23:44
blackburnn4nd0: they have the same interface for increasing/decreasing23:44
n4nd0blackburn: I need you to tell this23:44
blackburnboth decrease when y_i is close23:44
blackburnyes tell me23:44
n4nd0both are zero when y_i = y23:44
n4nd0I am ok with that23:44
n4nd0but when y_i is very different from y, what happens then?23:44
n4nd0Delta is maximized23:45
blackburnboth delta and loss are big23:45
n4nd0but what about the second term?23:45
n4nd0delta and loss are the same, aren't they?23:45
blackburnthe same?23:45
blackburnwhy the same?23:45
n4nd0what about < w, Psi(xi, y) - Psi(xi,yi)>23:45
n4nd0let's sync notation23:45
n4nd0for me the loss is Delta(y,yi) -> that's why I said that Delta and the loss are the same :D23:46
n4nd0I guess that you meant the dot product with delta23:46
blackburnah23:46
blackburnyeah23:46
n4nd0ok23:46
n4nd0so now23:46
blackburnokay let delta be 1st23:46
blackburnand w, psi the second23:46
n4nd0good23:46
n4nd0then23:46
n4nd0what happens with the second when yi and y are very far from each other23:47
n4nd0?23:47
blackburnsecond is really big then23:47
n4nd0why so?23:47
-!- yooo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has joined #shogun23:48
blackburnbecause they are far away in feature space23:48
n4nd0well23:48
n4nd0it could be that by chance, this Psi(xi,y) ~ 023:48
n4nd0then23:48
n4nd0<w, psi(xi,y) - psi(xi,yi)> ~ -<w, psi(xi,yi)>23:49
blackburnit is not the case to help the understanding23:49
blackburn:)23:49
n4nd0hahaha23:49
n4nd0:D23:49
blackburnimagine classes23:49
blackburnto the left23:49
blackburnand to the right23:49
blackburnwhat happens with the second term23:49
blackburnthen one is to the left23:50
n4nd0yes23:50
blackburnbut it should be to the right23:50
n4nd0but you are assuming that23:50
n4nd0y very different from yi => psi(x,y) very different from psi(x,yi)23:50
blackburnyes, it is23:50
n4nd0there is no condition for that23:50
blackburntry to get back to typical svm23:51
n4nd0let me give you a more formal argument a moment23:51
-!- yoo [~eric@bdv75-2-87-91-8-203.dsl.sta.abo.bbox.fr] has joined #shogun23:51
-!- yoo [~eric@bdv75-2-87-91-8-203.dsl.sta.abo.bbox.fr] has quit [Client Quit]23:52
n4nd0blackburn: let's go23:53
n4nd0so we have23:53
n4nd0max_y [ Delta(yi, y) + < w, Psi(xi,y) - Psi(xi,yi) > ]23:53
n4nd0now, the second Psi doesn't depend on y, then23:53
n4nd0max_y [ Delta(yi, y) + < w, Psi(xi, y) > ] - Psi(xi, yi)23:54
n4nd0agree?23:54
blackburnseems to be23:55
n4nd0ok23:55
n4nd0by definition23:55
n4nd0the argmax_y maximizes < w, Psi(xi, y) >23:55
n4nd0AND23:55
n4nd0providing that we have a good model23:55
n4nd0the argmax_y makes y close to yi; therefore, minimizes Delta(yi, y)23:56
blackburnno, it is a loss term23:56
blackburnit finds most violating y23:57
blackburnnot most complying one23:57
n4nd0http://www.cs.cornell.edu/people/tj/publications/tsochantaridis_etal_04a.pdf23:57
n4nd0equations (1) and (2)23:57
n4nd0I think that the concept of finding the mos violating one is related to the way of optimization23:58
n4nd0cutting plane algorithm23:58
n4nd0it is not related to the argmax definition23:58
blackburn(1) and (2) are decision functions23:58
blackburnthey are not related with risk23:59
n4nd0it was just to tell you that the idea of mos violated doesn't appear there23:59
n4nd0most*23:59
--- Log closed Wed Aug 22 00:00:05 2012