Open in new window / Try shogun cloud
--- Log opened Mon Jun 16 00:00:34 2014
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has quit [Quit: Konversation terminated!]00:56
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has joined #shogun00:58
shogun-buildbotbuild #29 of nightly_fedora is complete: Failure [failed git]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_fedora/builds/29  blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com>, Parijat Mazumdar <mazumdarparijat@gmail.com>03:00
-!- khalednasr [~k.nasr92@41.69.233.193] has quit [Ping timeout: 272 seconds]04:16
shogun-buildbotbuild #1794 of rpm1 - libshogun is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/rpm1%20-%20libshogun/builds/1794  blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com>04:21
-!- zxtx_ [~zv@37.205.63.158] has joined #shogun04:49
-!- abinash_panda [~abinash@182.66.31.108] has quit [Read error: Connection reset by peer]04:56
-!- abinash_panda [~abinash@182.66.18.16] has joined #shogun05:00
-!- zxtx_ [~zv@37.205.63.158] has quit [Ping timeout: 245 seconds]05:07
-!- zxtx_ [~zv@37.205.63.158] has joined #shogun05:09
-!- zxtx_ [~zv@37.205.63.158] has quit [Ping timeout: 240 seconds]05:30
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has quit [Remote host closed the connection]05:37
-!- abinash_panda [~abinash@182.66.18.16] has quit [Ping timeout: 264 seconds]06:27
-!- lisitsyn [~qdrgsm@85.114.187.192] has quit [Quit: Leaving.]06:56
-!- abinash_panda [~abinash@117.200.235.81] has joined #shogun07:28
-!- abinash_panda [~abinash@117.200.235.81] has quit [Ping timeout: 240 seconds]07:45
-!- Saurabh7 [~Saurabh7@117.222.13.170] has joined #shogun08:42
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has joined #shogun08:48
thoralfHeyho.08:48
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has quit [Quit: Konversation terminated!]09:03
-!- abinash_panda [~abinash@117.200.238.190] has joined #shogun09:43
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun10:09
shogun-notifier-shogun-data: Parijat Mazumdar :master * 8384def / multiclass/categorical_dataset/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/8384def9f612372b070d9f310f51856e8274e20210:09
shogun-notifier-shogun-data: servo dataset added10:09
shogun-notifier-shogun-data: Fernando Iglesias :master * 2838474 / multiclass/categorical_dataset/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/28384743f9903745cec24caca8df901e46daa50010:09
shogun-notifier-shogun-data: Merge pull request #52 from mazumdarparijat/master10:09
shogun-notifier-shogun-data:10:09
shogun-notifier-shogun-data: servo dataset added10:09
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun11:00
-!- mode/#shogun [+o HeikoS] by ChanServ11:00
-!- khalednasr [~k.nasr92@41.69.130.43] has joined #shogun11:11
@wikingHeikoS: btw cloud is back online for a week now but it's outdated since we dont have a way yet to generate nightly packages :S11:22
-!- soumyaC [uid15286@gateway/web/irccloud.com/x-rsvcwvepjgaihcpc] has joined #shogun11:26
-!- zxtx_ [~zv@95-91-192-118-dynip.superkabel.de] has joined #shogun11:32
-!- zxtx_ [~zv@95-91-192-118-dynip.superkabel.de] has quit [Ping timeout: 255 seconds]11:41
@HeikoSwiking: at least it runs! nice!11:43
@HeikoSwiking: what did you do?11:43
-!- abinash_panda [~abinash@117.200.238.190] has quit [Ping timeout: 255 seconds]11:48
@wikingHeikoS: just fixed some minor bug in the website's code and updated some components on fatbot11:49
@HeikoSwiking: cool!11:53
@HeikoSwill check soon11:53
@wikingkk11:53
@HeikoSnow meeting a ML meetup person who wants to do a shogun hands on workshop here soon11:54
@wikingbtw we need to do something about our preprocessors11:55
-!- HeikoS [~heiko@90.201.233.229] has quit [Ping timeout: 264 seconds]11:58
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun11:59
-!- mode/#shogun [+o HeikoS] by ChanServ11:59
@wikingHeikoS: almost all our preprocessors are dense feature preprocessors12:02
@wikinglike PNorm12:02
@wikingthat should be available for sparse case as well :S12:02
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun12:10
@HeikoSwiking: yep! also feature selection as lambday currently builds12:11
lambdayHeikoS: hey!12:11
@wikingmmm feature selection12:11
@wikinghow where?12:11
lambdaywhat did I miss ... checking log12:11
@wikingsince now we have random forest and that could be part of feature selection as well12:11
@wikingjesus i'm going to benchmark this fucking scikit-learn as it's amazing how long it does take to train a liblinear...12:13
lambdaywiking: what's your idea about preprocessor?12:14
@HeikoSlambday: hey! I currently am waiting for a call, but will be there afterwards.-12:14
lambdaywiking: I'm planning to add some feature selection algorithms under CDimensionReductionPreprocessor12:14
lambdayHeikoS: alright :)12:15
@wikingmmm shit12:15
@wikingCDimensionReductionPreprocessor is based on CDensePreprocessor12:15
@wiking:(12:15
@wikingso actually we cannot do either pca nor kpca on sparse matrix12:15
lambdaywiking: oh... well it doesn't necessarily be under dense per se...12:17
lambdayI mean what I was planning...12:17
-!- Saurabh7 [~Saurabh7@117.222.13.170] has quit [Remote host closed the connection]12:18
@wikinglambday: yeah no i mean it's not your fault12:18
@wikingi mean that we should refactor12:18
@wikinga bit the preprocessing12:18
@wikingas it's quite bad that we cannot do atm pca on sparse matrix12:19
@wikinglambday: i'll need another month and i'll be back in shogun full time :P12:19
lambdaywiking: hehe yeah.. :)12:19
lambdaywiking: I was also thinking to work on the plugins project with lisitsyn and make it ready before next year gsoc12:20
@wikinglambday: i'm up for it12:20
@wikingafter mid july12:20
lambdaywiking: yeah I am hopefully will be a bit free after this gsoc...12:20
lambdaycurrently juggling many balls at once :/12:21
@wikingoh woah12:22
@wikingIn [4]: f = SparseRealFeatures()12:22
@wiking[1]    13334 segmentation fault  PYTHONPATH=/Users/wiking/shogun-latest-build/lib/python2.7/site-packages/ =12:22
@wiking:DDD12:22
@wikinggit HEAD12:22
lambdaywtf12:23
lambdayno unit-tests?12:24
@wikingdunno12:25
@wikingdo we support scipy.sparse -> shogun sparse?12:25
lambdaywiking: I just checked... works fine here :/12:25
lambdaywiking: yep12:26
lambday>>> from modshogun import SparseRealFeatures >>> f=SparseRealFeatures() >>> f SparseFeatures >>>12:26
@wikinglambday: how do i create SparseRealFeatures from <60000x80371 sparse matrix of type '<type 'numpy.float64'>' with 426868 stored elements in Compressed Sparse Row format>12:27
@wiking?12:27
lambdaywiking: wait checking12:28
lambdaywiking: check this examples/undocumented/python_modular/mathematics_logdet.py12:29
@wikingmmm python is constantly crashing on osx :(12:31
@wikingwith shogun12:31
lambdaythat sucks!...12:34
lambdayit works on linux.. just tested12:35
@wikingX_train, X_test, y_train, y_test = cross_validation.train_test_split(feats_norm, labels, test_size=0.4, random_state=0)12:36
@wikingb_train = BinaryLabels(y_train)12:36
@wikingthis should work afaik12:36
@wikingas y_train is a simple np array12:36
@wikingi.e. SGVector12:36
@wikingbut it segfaults12:36
@wikingfuck no more gdb on osx12:37
@wikinglet's see what lldb can do12:37
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds]12:41
@wikingoh woah this crashes: b_train = BinaryLabels(np.array([1.0, -1.0]))12:43
@wiking:D12:43
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun12:44
-!- lisitsyn [~lisitsin@mxs.kg.ru] has joined #shogun12:48
-!- khalednasr [~k.nasr92@41.69.130.43] has quit [Quit: Leaving]12:48
@wikingbtw we need to start generating python api docs as well12:49
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]13:09
@wikingppl we need osx buildbot ASAP13:14
lambdaywiking: well this may seem naive but how travis works actually? they run on our machines or on machines they provide?13:19
-!- zxtx_ [~zv@g225077069.adsl.alicedsl.de] has joined #shogun13:37
@wikinglambday: they provide the machines13:37
lambdayokay..13:38
-!- kislay [~Abhijeet@103.240.205.204] has joined #shogun14:04
@HeikoSlambday:  hey14:06
lambdayHeikoS: hi14:06
@HeikoSlambday: how are things?14:06
lambdayHeikoS: ran into a problem while fixing streaming mmd cov bug... - https://github.com/shogun-toolbox/shogun/pull/230914:06
@wikingHeikoS: we need a sponsor that gives us an osx buildbot14:07
lambdayHeikoS: but I think its minor issue and I spent too much time on figuring out what's wrong :(14:07
@HeikoSwiking: what exactly does that mean? a physical machine? or some money to pay for a cloud VM?14:08
@HeikoSlambday: checking the PR14:08
lambdayHeikoS: okay14:08
@HeikoSlambday: so its about the burst mode?14:09
@HeikoSlambday: since now there is lots of overhead?14:09
@wikingHeikoS: it's either $$ or just to run a buildslave on an osx machine that is online 24/714:09
@HeikoSwiking: ok, the latter seems more tricky actually14:09
@HeikoSwiking: how much is that?14:09
lambdayHeikoS: yeah - so instead of streaming one block at a time, I changed it to stream many such blocks... but computation works blockwise14:10
@HeikoSlambday: ok yes, thats what I did before, right?14:10
lambdayHeikoS: this way it speeded up like before14:10
@HeikoSlambday: and the problem now are integration tests?14:10
lambdayHeikoS: exactly14:10
lambdayHeikoS: so I think it won't be hard to accomodate earlier implementation within this for testing purpose14:10
@wikingHeikoS: a lot ~80 usd/month14:11
lambdayHeikoS: yeah but I tried with more samples/changing the seed and it seems to work just fine14:11
lambdayas in, gives reasonable values14:11
@HeikoSlambday: so precomputing the fill kernel doesnt make sense you are right, but rather do it block wise14:13
lambdayHeikoS: yeah that sounded reasonable to me so did it this way14:14
@HeikoSlambday: well then its fine, currently reading but your summary sounds totally reasonable14:14
@HeikoSwiking: thats too much for us to pay14:14
@HeikoSwiking: why is it so expensive?14:14
@wikingHeikoS: i know that's why i said that we need a sponsor14:14
@wikingHeikoS: dunno.. i couldn't find cheaper14:14
@HeikoSwiking: why can't we put OSx on fatbot?14:14
lambdayHeikoS: I am sending data PR then...14:14
@HeikoSwiking: okok, I will ask for that next time I meet people.14:15
@HeikoSwiking: and about 24/7 macos14:15
@HeikoShow reliable must tha tbe?14:15
@wikingHeikoS: because a) fatbot is already full with other tasks b) running osx in a VM is pretty fucking hard...14:15
@HeikoScan I put one here? like an old laptop or so?14:15
@wikingyeah anything is ok14:16
@wikingjust to have it online 24/714:16
@HeikoSwiking: how reliable?14:20
@HeikoSwiking: sometimes internet fails here etc14:20
@HeikoScleaner might unplug14:20
@HeikoSwiking: not for long, but might happen once in a while14:20
@HeikoSwiking: and how old can the laptop be maximum?14:20
@HeikoSlambday: patch seems fine though I might be missing details14:21
@HeikoSlambday: so what about the integration problems?14:21
@HeikoSlambday: before you send the PR14:21
@HeikoSlambday: what exactly fails?14:22
@wikingHeikoS: well at least it should be intel14:22
@wiking:P i.e. not powerpc14:22
@wikingand that it should have at least 5G free mem14:22
@wikingbecause of the modular interface14:22
@HeikoSlambday: so the nice thing about the null approximations is that we have ground truth (permutation test)14:22
@HeikoSwiking: ok, will ask my friends for one14:22
@HeikoSas every is using max that should be possible14:22
lambdayHeikoS: well, it was giving weird results.. in the statistic_linear_time_mmd.py test, p-value was coming out to be weird (more than 0.5)...14:23
@HeikoSlambday: that should be checked14:24
lambdayHeikoS: and the test failed to detect that p!=q14:24
@HeikoSlambday: which nul approximation was that?14:24
lambdayHeikoS: with both14:24
lambdaybut I changed the seed and it was fine14:24
lambdayin fact, with more samples, it always succeeds14:24
@HeikoSlambday: what about a unit (or even python) test that compares the null approximations?14:25
@HeikoSto ground truth (permutation) one14:25
@HeikoSlambday: since as long as the statistic is computed correctly, te permutation test is correct14:25
@HeikoSand this p-value issue seems werd14:25
@HeikoSweird14:25
@HeikoSthough, if changing the seed solves the issue, this might be a problem of not having enough data14:26
lambdayHeikoS: yep.. I was stuck with this for quite a long time but didn't understand what was going so fishy... I mean,... I am just streaming more blocks at once14:26
lambdayshouldn't make such difference.14:26
@wikingwtf is this: SGSparseVector.cpp line 395: Computing sparse_dot(a,b) on unsorted vectors is very expensive: O(n^2)14:26
lambdayyes that's what I figured finally14:26
lambdaycompared streamed features by both earlier method and current method14:27
@wikinglambday: ^14:29
lambdayHeikoS: and what you mean by comparing null approximation.. I guess its already done in existing python tests14:30
lambdaywiking: umm.. I did that? let me check14:30
lambdaywiking: sorry man I didn't write this code... :(14:32
-!- HeikoS1 [~heiko@90.201.233.229] has joined #shogun14:33
lambdaywiking: but having a hashmap it can be improved I guess :/14:34
HeikoS1lambday: ah yeah, that might be true. I think I did that before14:34
HeikoS1wiking: ask thoralf about that, we introduced it14:34
lambdaywiking: HeikoS1: at least a break statement could be introduced after SGSparseVector.cpp line 40814:35
-!- HeikoS [~heiko@90.201.233.229] has quit [Ping timeout: 240 seconds]14:35
HeikoS1lambday, wiking I think this method is fallback and should be avoided14:36
HeikoS1thats why the warning14:36
@wikingHeikoS1: :)))) but afaik it's wrong because the features are sorted14:36
@wikingor i have to figure out wtf is happening14:36
HeikoS1wiking: I see, well that might be true.14:39
HeikoS1wiking: would be good to find the reason14:39
HeikoS1wiking: did you get the scripts from sonne|work or besser82 yet?14:43
@wikingnada14:44
HeikoS1rrr14:45
HeikoS1wiking: btw whats the state of you currently?14:46
lisitsynohmy I was reading last two lines and wondering what's the language you're speaking in :D14:46
@wikingHeikoS1: mmm moving14:46
@wikingHeikoS1: off to .sg within a month14:46
lisitsynwiking: how long will you stay there btw?14:47
@wiking3-4 years14:47
-!- Saurabh7 [~Saurabh7@117.222.13.170] has joined #shogun14:47
lisitsynwah wah14:47
lisitsynthat's quite a while!14:48
@wikingyeah ...14:48
lambdayHeikoS1: sent a data PR... https://github.com/shogun-toolbox/shogun-data/pull/53/files14:48
lambdayalso updated PR with data14:49
@wikingbtw i'm planning to create a wheel pacakge of shogun14:49
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun14:49
shogun-notifier-shogun-data: lambday :master * b65fecd / testsuite/ (12 files): https://github.com/shogun-toolbox/shogun-data/commit/b65fecdac7d15f961868fb11b5517414882dddf814:49
shogun-notifier-shogun-data: added data for PR #230914:49
shogun-notifier-shogun-data: Heiko Strathmann :master * 6df1281 / testsuite/ (12 files): https://github.com/shogun-toolbox/shogun-data/commit/6df12810fb0a2a806b5fd47354d8d086ccfbf0f514:49
shogun-notifier-shogun-data: Merge pull request #53 from lambday/master14:49
shogun-notifier-shogun-data:14:49
shogun-notifier-shogun-data: added data for PR #230914:49
HeikoS1lambday: feel free to merge the main one once travis says ok14:49
@wikinghttp://wheel.readthedocs.org/en/latest/14:49
HeikoS1wiking: whats weheel?14:49
lambdayHeikoS1: alright :)14:49
@wikingHeikoS1: http://wheel.readthedocs.org/en/latest/14:50
HeikoS1wiking: so pip install14:50
HeikoS1wiking: nice!14:50
HeikoS1super nice!14:50
lambdayHeikoS1: btw did you happen to take a look at the class diagram for feature selection? it works as a subclass of CDimensionReductionPreprocessor14:50
HeikoS1lambday: not yet but about to14:51
lambdayHeikoS1: Dino suggested that we should also think about three-variable interaction ... but I think we need another subclass for that14:51
HeikoS1lambday: one thing: do we want to use DenseFeatures as subclass?14:51
HeikoS1lambday: yes, thats subclass14:51
lambdayHeikoS1: that's the point - we don't need to :/14:51
lambdaywell, no14:51
HeikoS1lambday: I think it would be cooler to allow doing this for other features types14:51
HeikoS1lambday: at least sparse14:51
HeikoS1lambday: though I dont know of any algorithms for that :;D14:52
HeikoS1wiking: sparse feature selection?14:52
HeikoS1wiking: or more precise: feature selection on sparse data14:52
@wikingHeikoS1: well i would need a lot of shit sparsed14:52
@wiking:))14:52
lambdayHeikoS1: wiking ws worried about that :D14:52
@wikingHeikoS1: like why p-norm is only working currently in shogun for densefeatures?14:53
@wikingtotally bullshit14:53
@wikingHeikoS1: btw what's the eta for a gaussian binary GP for a sparse feature dataset with 60k training examples? :)14:54
@wikingshould i try even? :)14:54
@wikinglibsvm with gaussian kernel is running for ages already14:54
HeikoS1wiking: 60k14:54
HeikoS1crash :)14:54
@wikinghehehe ok14:55
lambdayHeikoS1: errr... travis failed in git submodule update :/ could you please restart the build? :(14:55
HeikoS1wiking: GP need to invert the covariance matrix which in this case doesnt fit into memory14:55
HeikoS1lambday: I think you can do that too14:55
HeikoS1no?14:55
lambdayHeikoS1: via web interface?14:55
HeikoS1lambday: yes in travis, press "sign in with github" and then you can start/stop jobs14:55
lambdayHeikoS1: alright.. cool14:56
HeikoS1wiking: for such a large amount of data, I would go for SVM or even KRR14:56
@wikingHeikoS1: is it possible to horizontally scale the inversion of the cov matrix ?14:56
HeikoS1wiking: horizontally?14:56
HeikoS1wiking: one can scale up these things quite a bit14:57
@wikingi.e. run it distributed14:57
HeikoS1wiking: but not in the naive (current) implementation14:57
HeikoS1wiking: there are SGD variants of this14:57
-!- Saurabh7 [~Saurabh7@117.222.13.170] has quit [Ping timeout: 240 seconds]14:57
HeikoS1and variational methods, but not in Shogun14:57
HeikoS1wiking: Wu is working on nice things for that14:58
HeikoS1wiking: but all not really large14:58
HeikoS1wiking: maybe next year :)14:58
@wikingthat's why i'm asking14:58
@wikingwould it be possible to run that distributed14:58
@wikinglike MR (map-reduce)14:58
HeikoS1wiking: probably there exist some things for that14:58
HeikoS1this one here is also very interesting:14:58
HeikoS1wiking: http://arxiv.org/abs/1305.502914:59
HeikoS1super nice14:59
HeikoS1we should have that14:59
HeikoS1a few things can be done map-reduce like in Shogun14:59
HeikoS1grid-search for SVM14:59
HeikoS1x-validation14:59
HeikoS1this KRR15:00
HeikoS1maybe a GP15:00
@wikingoh fuck15:00
@wikingthis scipy.sparse -> SparseRealFeatures conversion does not work the best15:00
HeikoS1wiking: haha ,whenever one starts using Shogun, this is what happens ;)15:01
@wikingbut WTF15:01
@wikingit's just unacceptable that we cannot mix scipy.sparse and shogun's sparse15:01
lambdaywiking: it works fine for logdet examples :/15:02
@wikinglambday: RealSparseMatrixOperator ?15:02
@wikinglambday: how should i feed that for GaussianKernel? :)15:02
lambdaywiking: well you don't need to use that15:02
lambdaywait I'm checking15:02
lambdaywiking:15:05
lambdayyou need SparseRealFeatures working, right?15:05
lambdayso this works -15:05
lambdayfrom modshogun import SparseRealFeatures15:05
lambdayfrom scipy.sparse import eye15:05
lambdayA=eye(10,10)15:05
lambday f=SparseRealFeatures(A.tocsc())15:06
@wikinglambday: A.T.tocsc() i suppose15:08
@wikingi mean in case of eye it doesn't matter15:08
@wikingbut any other sane dataset would require to have the .T15:08
lambdaywiking: well, yes..15:08
lambdaybut this should work, right?15:08
@wikingyeah seems liek that15:08
@wikingit seems fucking sklearn is making crazy sutff15:09
@wiking*stuff15:09
lambdaycrazy good or...?15:09
@wikingit reverts the order of the sparse vector values15:09
lambdayreverts as in?15:09
@wikingwell i have 0:1 123:1 222:2 ...15:10
@wikingand when i do this15:10
-!- Saurabh7 [~Saurabh7@117.200.185.14] has joined #shogun15:10
@wikingcross_validation.train_test_split(..)15:10
@wikingthe created sparse matrices' value are going to be reverted15:10
@wikingi.e. 222:2 123:1 0:115:10
lambdaywtf :/15:10
@wikingdunno why15:11
lambdayshouldn't matter though15:11
lambdaybut weird15:11
@wikingyeah it does matter15:11
@wikinglambday: as you see from the warning :P15:11
@wikingO(n^2)15:11
@wiking;P15:11
lambdayoh that! :D15:12
* lambday afk15:14
-!- HeikoS1 [~heiko@90.201.233.229] has quit [Quit: Leaving.]15:55
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has joined #shogun16:03
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has quit [Read error: Connection reset by peer]16:03
@wikinglisitsyn: terminate called after throwing an instance of 'tapkee::not_enough_memory_error'16:30
@wiking:DDDD16:30
@wikinglisitsyn: what do i dooooo? :)16:30
lisitsynwiking: whoa16:33
lisitsynwiking: how did you do that?16:33
-!- PirosB3 [~pirosb3@host80-223-dynamic.246-95-r.retail.telecomitalia.it] has joined #shogun16:33
@wikinglisitsyn: sparse matrix16:37
@wikinglisitsyn: with 60k elements16:37
@wikingand about 80k different values16:37
@wikingi.e. if you convert that matrix to dense then matrix.shape = (80k, 60k)16:38
@wikingand i guess that's what happens16:38
@wikingor?16:38
@wikingi.e. sparse -> dense matrix conversion16:38
lisitsynwiking: what method?16:44
lisitsynit is quite possible it does this conversion16:44
lisitsynif you use something linear16:44
@wikingTDistributedStochasticNeighborEmbedding16:44
lisitsynahh16:45
lisitsynyes I didn't patch it enough16:45
lisitsynto work on distances or whatever16:45
@wiking?16:45
lisitsynyou know it is just ported code16:45
lisitsynwhich takes float* Y16:45
@wikingah ok16:45
@wikingyeah16:45
lisitsynit is possible to change it though16:45
lisitsynbut involves 90% rewrite of code :D16:46
@wikingheheheh16:46
lisitsynwiking: it is quite ready to do that stuff though16:48
-!- zxtx_ [~zv@g225077069.adsl.alicedsl.de] has quit [Ping timeout: 245 seconds]16:49
lisitsynI mean it already uses vantage point tree to find neighbors16:49
@wikingi need to b able to do dim reduction16:49
@wikingon a very very sparse matrix16:49
@wiking:)16:49
lisitsynwell try KLLE16:49
lisitsynit won't fail on that matrix as you just need a kernel16:50
lisitsynthough 80k eigenproblem could be quite slow to solve16:50
lisitsynerr 60k16:50
lisitsynwiking: you'd also try landmark isomap or landmark mds16:50
lisitsynthough this is very gaussian shit :D16:51
lisitsynif you reduce to 2d you probably will see gaussian blob16:52
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds]17:04
-!- Anand__ [2a683cfa@gateway/web/freenode/ip.42.104.60.250] has joined #shogun17:05
Anand__Hi! I am new here and need some help regarding the Shogun Naive Bayes classifier.17:07
Anand__Does Shogun have a method in NBC to get the probabilities of a particular test instance lying in all the classes?17:09
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun17:14
-!- mode/#shogun [+o HeikoS] by ChanServ17:14
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun17:15
-!- HeikoS [~heiko@90.201.233.229] has quit [Quit: Leaving.]17:25
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds]17:44
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]17:49
-!- khalednasr [~k.nasr92@41.69.237.204] has joined #shogun17:53
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun17:58
-!- mode/#shogun [+o HeikoS] by ChanServ17:58
-!- khalednasr [~k.nasr92@41.69.237.204] has quit [Quit: Leaving]18:08
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 240 seconds]18:11
-!- Saurabh7 [~Saurabh7@117.200.185.14] has quit [Ping timeout: 264 seconds]18:17
-!- abinash_panda [~abinash@117.200.225.204] has joined #shogun18:17
-!- zxtx_ [~zv@f052018195.adsl.alicedsl.de] has joined #shogun18:18
-!- zxtx_ [~zv@f052018195.adsl.alicedsl.de] has quit [Ping timeout: 272 seconds]18:26
-!- HeikoS [~heiko@dab-yat1-h-70-3.dab.02.net] has joined #shogun18:28
-!- mode/#shogun [+o HeikoS] by ChanServ18:28
-!- Saurabh7 [~Saurabh7@117.200.180.114] has joined #shogun18:28
-!- abinash_panda [~abinash@117.200.225.204] has quit [Read error: Connection reset by peer]18:40
-!- Saurabh7 [~Saurabh7@117.200.180.114] has quit [Quit: Leaving]18:42
-!- HeikoS [~heiko@dab-yat1-h-70-3.dab.02.net] has quit [Ping timeout: 240 seconds]18:43
-!- abinash_panda [~abinash@117.200.235.238] has joined #shogun19:00
-!- abinash_panda [~abinash@117.200.235.238] has quit [Client Quit]19:05
-!- abinash_panda [~abinash@117.200.235.238] has joined #shogun19:05
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun19:06
-!- mode/#shogun [+o HeikoS] by ChanServ19:06
-!- Anand__ [2a683cfa@gateway/web/freenode/ip.42.104.60.250] has quit [Quit: Page closed]19:09
-!- abinash_panda [~abinash@117.200.235.238] has quit [Ping timeout: 252 seconds]19:12
-!- abinash_panda [~abinash@182.66.24.191] has joined #shogun19:17
-!- abinash_panda [~abinash@182.66.24.191] has quit [Ping timeout: 244 seconds]19:31
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 245 seconds]19:44
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun19:55
shogun-notifier-shogun-data: Heiko Strathmann :master * 4fc0eb7 / testsuite/python2-tests/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/4fc0eb7698f3b2b0303b5eaeb9a59a5020aeccd919:55
shogun-notifier-shogun-data: added registered parameter19:55
-!- HeikoS [~heiko@dab-yat1-h-70-1.dab.02.net] has joined #shogun20:01
-!- mode/#shogun [+o HeikoS] by ChanServ20:01
@HeikoSwiking: around?20:14
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun20:18
-!- abinash_panda [~abinash@223.176.23.125] has joined #shogun20:28
@HeikoSlambday: hi!20:36
lambdayHeikoS: hey20:37
lambdayHeikoS: integration tests again failed20:37
lambdaychecking some more20:37
-!- PirosB3 [~pirosb3@host80-223-dynamic.246-95-r.retail.telecomitalia.it] has quit [Quit: PirosB3]20:38
lambdayHeikoS: regarding the complex wrapper thing, I noticed that last year.. but don't know what do we have to do there to avoid this warning.. :/20:39
@HeikoSlambday: I see, well its not important for now20:41
@HeikoSlambday: integration tests are more important20:41
lambdayHeikoS: yeah... running locally20:41
@HeikoSand?20:41
lambdayHeikoS: yep.. worked fine..20:42
lambdayHeikoS: made some changes20:42
@HeikoSlambday: cool! :)20:42
@HeikoSlambday: will soon take a much closer look to the feature selection stuiff you wrote down20:43
lambdayHeikoS: just one test was failing in python modular.. generated data for that20:43
lambdayHeikoS: yeah.. I should try to finish that before mid term20:43
lambdayb-test will be finished by today20:43
lambday(just copy paste CLinearTimeMMD and change two lines :D)20:43
@HeikoSnice!20:45
@HeikoSlambday: do you have any idea how to start the demos locally?20:57
@HeikoSshogun-demo?20:57
@HeikoSits django, but I dont know about that stuff20:57
lambdayHeikoS: no man! :(20:58
lambdaythere's no README?20:58
@HeikoSlambday: there is, but I know too little about django to make sense of it :(21:00
lambdayI know nothing about django :(21:00
lambdayHeikoS: btw did you watch the match? :D germany 4 portugal 0 :D21:01
lambdayHeikoS: do you know how to run ctest locally for python3?21:03
@HeikoSlambday: did not watch it, but not surprised haha :)21:05
@HeikoSno idea :)21:05
@HeikoSlambday: man I just want to run demos .....21:05
@HeikoScant be so hard21:05
lambdayHeikoS: the only demo I know is these ones - http://shogun-toolbox.org/page/documentation/demo21:07
lambdayHeikoS: which ones are you trying to run?21:07
@HeikoSsaurabh added much more stuff to them21:07
@HeikoSbut the latest on the website dont include this21:07
@HeikoSso I wanna run locally to see21:07
lambdaykevin is the guy :/21:08
lambdayHeikoS: I sent a new data PR for fixing the integration tests... shall I merge it?21:11
-!- HeikoS [~heiko@dab-yat1-h-70-1.dab.02.net] has quit [Ping timeout: 240 seconds]21:13
shogun-notifier-shogun-data: lambday :master * e8b5520 / testsuite/ (6 files): https://github.com/shogun-toolbox/shogun-data/commit/e8b5520b8ef7946863cf84719b8b8e0e51a7caf421:16
shogun-notifier-shogun-data: added data for PR #230921:16
shogun-notifier-shogun-data: Soumyajit De :master * b6d9995 / testsuite/ (6 files): https://github.com/shogun-toolbox/shogun-data/commit/b6d99954619587cfa7db68d4631108141274fd2a21:16
shogun-notifier-shogun-data: Merge pull request #54 from lambday/master21:16
shogun-notifier-shogun-data:21:16
shogun-notifier-shogun-data: added data for PR #230921:16
-!- abinash_panda [~abinash@223.176.23.125] has quit [Quit: leaving]21:26
-!- abinash_panda [~abinash@223.176.23.125] has joined #shogun21:29
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun21:32
-!- mode/#shogun [+o HeikoS] by ChanServ21:32
@HeikoSsonney2k:  around?21:34
@HeikoSsonne|work: arpound?21:35
@HeikoSwiking: around?21:35
@HeikoSbesser82:  around?21:35
@HeikoSlisitsyn: around?21:35
-!- lisitsyn1 [~qdrgsm@80.252.20.67] has joined #shogun21:36
lambdaymulticlass-libsvm fails - unrelated... phew!!21:36
-!- lisitsyn1 is now known as nystisil21:36
nystisilhey lambday21:37
-!- lambday is now known as yadbmal21:37
nystisil:D21:37
yadbmalney nystisil21:37
yadbmal:D21:37
shogun-notifier-shogun: khalednasr :develop * f018255 / src/ (3 files): https://github.com/shogun-toolbox/shogun/commit/f018255fd2008ae5de398f3bb27ca5c542c80dda21:37
shogun-notifier-shogun: Added support for softmax visible units and sampling with evidence to CRBM21:37
shogun-notifier-shogun: Sergey Lisitsyn :develop * 31f5609 / src/ (3 files): https://github.com/shogun-toolbox/shogun/commit/31f5609f7a7345ca05b5c1f8c7425236da2270df21:37
shogun-notifier-shogun: Merge pull request #2315 from khalednasr/rbms21:37
shogun-notifier-shogun:21:37
shogun-notifier-shogun: Added support for softmax visible units and sampling with evidence to CRBM21:38
nystisilI have no idea what I merged21:38
nystisil:D21:38
nystisilgonna talk to khaled about it :D21:38
@HeikoSyadbmal: nice one :)21:39
yadbmalHeikoS: nystisil started it :D21:39
@HeikoSyadbmal: haha :)21:40
@HeikoSnystisil: you should read the code! ;);)21:40
nystisilHeikoS: what code?21:40
nystisilahh I've read it21:40
@HeikoSnystisil: the one your merge21:40
yadbmalHeikoS: reading the documentation of CStreamingMMD... reads nice now21:40
nystisilit is not about how code looks21:40
@HeikoSnystisil: not that I get you, how to start the shogun-demo django stuff locally21:41
@HeikoSnystisil: thats why I suggest to read it haha ;)21:41
nystisilHeikoS: doesnt' help21:41
nystisilHeikoS: khaled is deep into contrastive divergence of restricted boltzmann machines21:41
@HeikoSnystisil: anyways, demo stuff, any idea?21:41
nystisiland other smart words21:41
nystisildemo hmm21:41
nystisilI didn't try to run it21:41
nystisilI am not sure I am of any help with that :D21:42
@HeikoSnystisil: ok21:42
nystisilHeikoS: I bet wiking is the man21:43
shogun-buildbotbuild #2377 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/2377  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>, khalednasr <k.nasr92@gmail.com>21:44
-!- pickle27f [~pickle27@216.191.231.218] has joined #shogun21:57
-!- pickle27f [~pickle27@216.191.231.218] has quit [Client Quit]21:58
-!- pickle27 [~pickle27@216.191.231.218] has joined #shogun21:59
@HeikoSpickle27: hey!22:16
pickle27hey!22:16
pickle27did you get the demos working?22:16
@HeikoSpickle27: nope22:16
@HeikoSthe server runs22:16
@HeikoSbut it just echos my browsers requests22:16
pickle27hmm what is it complaining about22:16
@HeikoSpickle27: see pm22:17
@HeikoShow do I get a list of demos, or link to find them behind?22:17
@HeikoSthe ones in the readme dont work22:17
pickle27so the program boots the django web server22:17
pickle27if you want to use it you need to go to localhost:8000 I think22:17
pickle27oh except you don't know what the urls are22:18
@HeikoSyep22:18
@HeikoSpickle27: so as said, my terminal prints my browser requests22:18
@HeikoSbut I dont know what to do now22:18
pickle27the urls.py maps urls to python functions22:18
pickle27so if you look at the first one22:19
pickle27regression/svr/22:19
pickle27localhost:8000/regression/svr/ should run the code in the regression.svr.handler function22:19
pickle27man I hate django22:19
pickle27so ugly22:19
pickle27is that working for you?22:20
@HeikoSpickle27: kind of22:21
@HeikoSit doesnt look right22:21
@HeikoSnot like on our website22:21
@HeikoSand buttons dont do things22:21
@HeikoSpickle27: could you try on your machine?22:22
@HeikoSmaybe something is wrong with my setup ?22:22
pickle27I don't have django setup on here unfortunately22:25
pickle27I haven't ever played with the demo app22:26
pickle27for starters it should be flask too and not django ...22:26
pickle27maybe I'll do that some day soon22:26
pickle27I *really* hate how hard it is to clone any of our web repos and work on them22:27
pickle27waaaaay to much config specific to the exact server running those apps22:27
pickle27are you trying to add a new demo?22:27
@HeikoSpickle27: I agree with you22:32
@HeikoSpickle27: I try to inspect what Saurabh did22:32
pickle27ahh gotcha22:32
@HeikoSHe added some really nice stuff22:32
@HeikoSbut I cannot check it22:32
pickle27maybe he can help you more?22:32
pickle27I'll port that app to flask someday soon22:32
@HeikoSpickle27: yes I asked him already but just had some time22:32
pickle27things will be better22:32
pickle27flask == simple django == gross22:33
@HeikoSthe application basically doesnt run here22:33
@HeikoSjust prints the text and renders all buttons/boxes22:33
@HeikoSbut nothing happens in the back22:33
@HeikoSpickle27: cool sounds good. I mean this should be easy to use22:33
pickle27yeah exactly22:35
pickle27flask is the perfect framework for our stuff22:36
pickle27it has almost zero conceptual overhead to deal with in order to work with it22:36
@HeikoSok22:37
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.]22:38
-!- chakri [srichakrad@gateway/shell/bnc4free/x-jqfqzylcqabpvnjk] has quit [Remote host closed the connection]22:51
-!- besser82 [quassel@fedora/besser82] has quit [Remote host closed the connection]22:51
-!- travis-ci [~travis-ci@ec2-54-83-107-201.compute-1.amazonaws.com] has joined #shogun23:21
travis-ci[travis-ci] it's Sergey Lisitsyn's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/2771141123:21
-!- travis-ci [~travis-ci@ec2-54-83-107-201.compute-1.amazonaws.com] has left #shogun []23:21
-!- khalednasr [~k.nasr92@41.69.237.204] has joined #shogun23:27
-!- chakri [srichakrad@gateway/shell/bnc4free/x-msubycjynfywgrma] has joined #shogun23:29
-!- yadbmal [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds]23:34
-!- pickle27 [~pickle27@216.191.231.218] has quit [Quit: This computer has gone to sleep]23:58
--- Log closed Tue Jun 17 00:00:36 2014